Recognizing and Managing AI Bias (Topic 2) in Module 3 – AI-at-Work (BG)

Recognizing and Managing AI Bias

What Is AI Bias?

AI models learn patterns from training data. When that training data contains historical patterns that reflect societal biases — in hiring, lending, criminal justice, healthcare — the model can learn to replicate and amplify those patterns.

Types of bias in AI outputs:

• Training data bias: The model reflects the demographics, assumptions, and viewpoints most represented in its training corpus (primarily English-language, Western, and post-war content).
• Representation bias: AI-generated images of doctors default to male; AI-generated images of nurses default to female. AI generates names and examples that skew demographically.
• Performance disparities: AI models often perform less accurately on language from underrepresented communities, non-native speaker syntax, or regional dialects.
• Recency bias: AI may over-represent ideas and worldviews prevalent at the time of its training cutoff.

Where Bias Matters Most in Professional Settings

• Hiring: AI screening tools that weight certain resume patterns can perpetuate demographic bias. Amazon famously decommissioned an AI hiring tool for this reason.
• Credit and lending: AI scoring models trained on historical approval data can encode demographic disparities.
• Content generation: AI-generated visuals, names, or examples for communications may reinforce stereotypes.
• Customer service: AI performance may vary by customer dialect, language proficiency, or communication style.

Practical Mitigation

1. Audit AI outputs for patterns: If AI generates lists of examples, names, or images, check whether results are demographically balanced.
2. Provide diverse examples in prompts: Including diverse examples in few-shot prompts can shift AI output distribution.
3. Recognize when your domain has high-stakes bias risk: Hiring, lending, and legal decisions require extra scrutiny.
4. Maintain human oversight for consequential decisions: Automated decisions that affect people's opportunities should have human review.