Algorithmic bias occurs when an AI or automated system produces systematically unfair outcomes for certain groups, usually because the training data reflected existing human biases. Bias often comes from the training data, labeling decisions, or optimization goals inside the system.
Use cases
Bad systems lose trust fast and can exclude entire groups of users.
A hiring platform uses an algorithm that downranks resumes containing words like "women's" or "maternity leave" after training on historical hiring data skewed toward male candidates.
A photo app's auto-cropping feature fails to consistently recognize faces with darker skin tones, causing users' friends and family members to get cropped out of profile images and shared photos.
How it's used in practice
Diversify your data: Insist on analyzing and diversifying the training data before design sprints. If you can’t see the data sources, you can’t design for the outcomes.
Build 'friction for fairness': Design interaction points that ask the user for confirmation when an algorithm makes a high-stakes decision (like suggesting a loan denial).
Design for transparency: When an algorithm provides a recommendation, explain why. Add a small "Why am I seeing this?" tooltip or text block.
🪄
Pro-tip: Focus less on the 'math' of the algorithm and more on its inputs and outputs. As a UXer, your primary job is to test for harmful effects during user testing with edge cases and diverse user groups.
Challenges & limitations
Complete neutrality doesn’t exist: Every dataset reflects human decisions, omissions, and priorities.
Some fixes require engineering changes: UI improvements can reduce confusion, but ranking models, training data, and moderation rules often need deeper system changes too.
