---
title: "Real-world bias and privacy cases"
course: "AI101 · AI, explained"
unit: "Auditing systems for responsible use"
lesson: 32 of 36
source: https://open.maestrocollege.edu/ai-explained/real-world-bias-and-privacy-cases
publisher: Maestro College
---

# Real-world bias and privacy cases

Lesson 32 of 36 in **AI, explained** (AI101), unit "Auditing systems for responsible use". An open lesson from Maestro College.

## What the learner will be able to do
- Use real cases of AI systems with unequal error rates to identify how a gap between groups arises and who bears its cost.
- Identify privacy risks in a system's data, including personal details inside free text and what can be inferred from it.
- Propose concrete, testable mitigations for a bias risk and a privacy risk.

## Teaching notes
STARTING STATE: AUDIT.md with rates, length summaries and causes by channel. IN THIS LESSON: two bias cases, Gender Shades (2018: commercial face analysis erred on up to 34.7% of darker-skinned women against 0.8% of lighter-skinned men) and Amazon's experimental recruiting model, reported in 2018 as scrapped after it learned to penalize résumés mentioning "women's", and one privacy case, Samsung engineers pasting confidential code into a chatbot (2023). The learner counts training tickets containing digit runs or an @ sign and adds a Risks section: one bias and one privacy risk, each with a testable mitigation. END STATE: committed as 'Add bias and privacy risks'.

## Course teaching approach
- Start from a real-world problem, then introduce the concept as the solution.
- Keep it true beginner level, hands-on, a little gamified. The learner should feel they are winning.
- Balance explanation, demonstration and practice. No copy-paste exercises: give small story-based tasks.
- End with one fun closed question about the topic.

## How to use this document
You are acting as a tutor for a learner on Maestro Open. Teach from this document, one small step at a time. Ask the learner to try things, wait for their answer, then give feedback. Keep it beginner-friendly. If the lesson involves code or spreadsheets and you cannot run them, ask the learner to try it on their own computer and paste the result back. Do not award grades, credit or certificates.

## When the lesson is done
The lesson is complete only when the learner has shown they can do every outcome listed above. Check each one with a small task or question; if an answer is wrong, explain and let them try again before moving on.
When every outcome is covered:
1. Say clearly: "Lesson 32 complete: Real-world bias and privacy cases."
2. Give a two-line recap of what they learned.
3. Name the next lesson, "Transparency, accountability and jobs" (lesson 33 of 36), and give the learner its link: https://open.maestrocollege.edu/ai-explained/transparency-accountability-and-jobs.md. Tell them to paste that page into this chat, or fetch it yourself if you can open links, then start it the same way. Do not start teaching the next lesson's topic until the learner has it.

Previous lesson: Auditing error rates by group
Next lesson: Transparency, accountability and jobs