AI Bias: How It Happens and What to Do About It

AI bias is one of the most discussed topics in AI ethics, but it is also one of the most misunderstood. This lesson clarifies what it is, how it occurs, and what can actually be done about it.

What Bias in AI Actually Means

In everyday language, "bias" means unfair prejudice. In AI, it has a more specific meaning: systematic, predictable errors that affect certain groups differently than others.

1. 1.AI bias is not usually the result of an AI being malicious or having prejudices in any conscious sense. It is the result of:
2. 2.Patterns in training data that reflect historical human biases
3. 3.Optimisation targets that correlate with protected characteristics
4. 4.Representation gaps in training data
5. 5.Deployment contexts that differ from training contexts

The Training Data Problem

If an AI system is trained on historical data, and that historical data reflects historical discrimination, the AI will learn to replicate those patterns.

Classic example: An Amazon recruiting tool trained on historical hiring data learned to downgrade resumes from candidates who attended all-women's colleges — because Amazon had historically hired fewer women, and the model learned to replicate that pattern. Amazon scrapped the tool.

The problem: even removing protected characteristics (race, gender) does not solve this, because other variables (zip code, school attended, gap years) can serve as proxies.

Representation Gaps

AI systems perform worse on groups underrepresented in training data.

Early facial recognition systems had significantly higher error rates for darker-skinned faces — because training datasets overrepresented lighter-skinned faces. This is not malice; it is a straightforward consequence of imbalanced training data.

In language models, topics, dialects, and languages that appear less frequently in training data are handled less well. This can systematically disadvantage non-native English speakers or people from regions less represented in online text.

What Individuals and Organisations Can Do

• As a user:
• Be skeptical of high-stakes AI decisions about people, particularly in hiring, lending, or healthcare
• Ask what the error rate is for the group you are a member of or are making decisions about
• Maintain meaningful human oversight of consequential decisions

• As a decision-maker evaluating an AI system:
• Ask vendors to provide disaggregated performance data by relevant demographic groups
• Test the system on representative data from your actual use case before deploying
• Define what "fairness" means in your context — there are multiple competing definitions, and choosing between them is a value judgment

• As an organisation:
• Conduct bias audits on high-stakes AI systems
• Build diverse teams who can identify blind spots
• Create mechanisms for affected people to report problems

The Hard Truth About Bias

There is no such thing as a "bias-free" AI system. Every system makes trade-offs. Optimising for one definition of fairness can conflict with another. The goal is not to eliminate all bias but to make deliberate, transparent choices about trade-offs and to build systems that are fair enough for their intended use.