Check: leakage and fairness
In the loan example, the team reported 94% accuracy. Two problems hid inside that number. First, the customer ID appeared in both the training and test sets, so the model partly memorized people rather than patterns. Second, accuracy was averaged across all applicants, which masked a 17-point gap between two age groups.