Exit focus
Responsible AI in Practice›Module 3›Bias in model evaluation
Chunk 4 of 5 · 62%

Check: leakage and fairness

In the loan example, the team reported 94% accuracy. Two problems hid inside that number. First, the customer ID appeared in both the training and test sets, so the model partly memorized people rather than patterns. Second, accuracy was averaged across all applicants, which masked a 17-point gap between two age groups.

Quick check · not graded
Which change would you make first before trusting the 94% figure?
Right. Leakage invalidates every downstream metric, so fix the split first. F1 would still be inflated. This item will return in your review queue in 3 days.
Continue to Reflect