Unit 6

How AI learns from data

Train a simple classifier on examples, change its training data and watch fairness change with it.

1. Read

   A machine is only as fair as its examples

   A fruit packing house near Ratnagiri wants a machine to sort mangoes into ripe and unripe. A camera and a squeeze sensor measure each mango, but nobody can write the rule by hand. "Ripe" depends on colour, softness and the variety, in tangled ways.

   So the engineers try something different. They do not write the rule. They show the machine examples and let it find the pattern. This is how most modern AI is built, and it explains both why AI is powerful and why it can be unfair.

   Learning from examples

   To teach a machine by example you need three things.

   • Features. The measurements the machine sees. Here: colour (0 = green, 10 = orange) and softness (0 = hard, 10 = soft).
   • Labels. The right answer for each example: "ripe" or "unripe". A person, or an earlier test, had to supply these.
   • Training data. The pile of examples, each with its features and its label. The machine learns only from this.

   Then comes a learning step that builds a model. A model that sorts things into groups is a classifier. Afterwards you give it a mango it has never seen and it guesses a label.

   

   One simple classifier: ask your nearest neighbours

   The simplest classifier is the nearest neighbour. It stores the training examples. For a new mango, it finds the closest training example (closest in colour and softness) and copies its label.

   Example. The training data has two mangoes: an unripe one at (colour 2, softness 2) and a ripe one at (colour 8, softness 8). A new mango measures (colour 4, softness 3).

   • Distance to the unripe example: $\sqrt{(4-2)^2 + (3-2)^2} = \sqrt{5} \approx 2.2$.
   • Distance to the ripe example: $\sqrt{(4-8)^2 + (3-8)^2} = \sqrt{41} \approx 6.4$.

   The unripe example is nearer, so the classifier says unripe. Real systems use millions of examples and much cleverer maths, but the idea holds: the model's answers come from its training data.

   Test on examples it has not seen

   How good is the model? You must test it on new examples with known labels that were not used in training. If you tested on the training examples themselves, a model that simply memorised them would score 100% without having learned any pattern.

   Accuracy is the share of test answers that are right:

   $\text{accuracy} = \frac{\text{right answers}}{\text{all test answers}}$

   Out of 24 test mangoes, 20 correct gives $20 \div 24 \approx 83\%$.

   The mango trap: bias in the data

   The engineers train the machine with mangoes from their own orchard, a variety called Kesar. A ripe Kesar turns orange and soft. An unripe one is green and hard. The model learns a sensible rule: orange and soft means ripe.

   Then the packing house starts receiving Totapuri mangoes. A ripe Totapuri stays mostly green-yellow, though it does soften. The machine, which has never seen one, looks at a ripe, soft, greenish Totapuri and sees "green", so it says unripe.

   

   Test it on 24 mangoes, 12 of each variety. Suppose the machine gets all 12 Kesar right and 8 of 12 Totapuri right. Overall accuracy is $20 \div 24 \approx 83\%$, which sounds fine. But for Totapuri it is only $8 \div 12 \approx 67\%$, and almost all of the mistakes are ripe Totapuri thrown away as unripe. A farmer growing Totapuri would lose money.

   This is bias: the model works well for some groups and badly for others, because of what the training data did or did not contain. Nobody told the machine to be unfair. It simply never saw enough Totapuri.

   An AI model is only as fair as its data. If a group is missing or thin in the training data, the model will probably do worse for that group.

   How to find bias, and how to reduce it

   1. Split the results by group. Report accuracy for each group, not only overall. An overall 83% hid a group at 67%.
   2. Look at who is in the training data. Count the examples from each group. Are some missing?
   3. Add more examples of the group that is doing badly, especially the cases it gets wrong. Adding more of what you already have does not help: 1,000 more Kesar mangoes will not teach the machine about Totapuri.
   4. Test again, group by group. Keep going until the groups are about equally accurate.

   Try the lab to do exactly this: train on only Kesar mangoes, watch Totapuri accuracy fall, then fix it by changing the training data.

   Real examples

   • Researchers who tested commercial software that guesses a person's gender from a face photo found in 2018 that it made far more errors for darker-skinned women, with error rates up to about 35%, than for lighter-skinned men, under 1%. The training photos had mostly shown lighter-skinned faces.
   • A company is reported to have dropped an experimental CV-sorting tool because it learned from years of past hiring, which had mostly favoured men, and so scored CVs that mentioned women's activities lower.

   In both cases the data carried an old unfairness into a new machine, and an overall accuracy figure would not have shown it.

   Common mistakes

   • "A computer is objective, so its answers are fair." It copies the patterns in its data, including the unfair ones.
   • "More data always fixes it." Only if the new data covers what was missing.
   • "High accuracy means it works for everyone." Check each group.
   • "Testing on the examples it learned from is fine." That measures memory, not learning.

   Try it: Think of an app that sorts or recommends (photos, videos, spelling correction, voice typing). Whose examples might it have been trained on? Who might it work worse for?

2. Lab

   Train the mango sorter

   Choose which mangoes go in the training set, then see how well the classifier sorts 24 test mangoes it has never seen. First train on Kesar mangoes only and look at the Totapuri accuracy. Then change the training data until it is fair to both varieties.

   Your goals

   • Train on Kesar mangoes only (at least 6) and find that Totapuri accuracy is clearly lower than Kesar accuracy.
   • Change the training data so that both Kesar and Totapuri accuracy reach at least 85%.

   🔒 The interactive lab opens once you register. Register to do this.

3. Quick check

   How AI learns check

   Eight questions about training data, accuracy and bias.

   1. 1.In machine learning, what is the training data?

      • Examples with their right answers (labels) that the model learns from
      • Rules a programmer typed in by hand
      • The new examples used only to test the finished model
      • The screen where the answers are shown

      Choose one

   2. 2.A classifier is tested on 24 new examples and gets 18 right. What is its accuracy, as a percentage?

      Answer with a number

   3. 3.A model is tested on two groups. Group A: 20 test examples, all 20 right. Group B: 10 test examples, 4 right. What is the overall accuracy, as a percentage?

      Answer with a number

   4. 4.In the same test (Group A: 20 of 20 right; Group B: 4 of 10 right), what is the accuracy for Group B alone, as a percentage?

      Answer with a number

   5. 5.A classifier does badly for Group B because Group B was thin in the training data. Which actions would help find or reduce the bias?

      • Add more training examples from Group B
      • Report accuracy for each group separately
      • Remove the Group B examples that were wrong
      • Add thousands more examples from Group A

      Choose all that apply

   6. 6.Match each term to its meaning.

      • Label
      • Classifier
      • Bias
      • Features

      Match each to: The right answer attached to a training example · Working well for some groups and badly for others · The measurements the model looks at · A model that puts things into groups

   7. 7.Put the steps of building a fair classifier in order.

      • Train the model on them
      • Collect examples with labels from all the groups you care about
      • Check accuracy for each group and improve the data if one is behind
      • Test it on new examples it has not seen

      Put these in order

   8. 8.Why must you test a model on examples that were not in its training data?

      • A model that memorised its training examples would score well without having learned any pattern
      • Testing on training data is slower
      • It is only a tradition
      • Unseen examples are cheaper

      Choose one

   🔒 Answering and checking are for registered learners. Register to do this.

4. Quick check

   New puzzles with AI

   Use what you know about training data in new situations.

   1. 1.A talent-spotting program learns from the past records of players picked for a big academy. Almost all of those players came from a few big cities. What is the most likely problem?

      • It will tend to favour players who look like the past ones, and miss good players from small towns
      • It will pick players completely at random
      • It will be perfectly fair because it is a computer
      • It will run too slowly

      Choose one

   2. 2.A classifier is tested on 40 images, 20 from group 1 and 20 from group 2. It gets 95% of group 1 right and 70% of group 2 right. What is its overall accuracy, as a percentage?

      Answer with a number

   3. 3.A nearest-neighbour model has two training examples: an unripe mango at (colour 2, softness 2) and a ripe mango at (colour 8, softness 8). What does it say about a new mango at (colour 6, softness 5)?

      • Ripe
      • Unripe
      • It cannot decide: the distances are equal

      Choose one

   4. 4.A voice-typing app is much worse for people with one accent. The team adds 100,000 more recordings, all from speakers of the accent it already handles well. What is the likely effect on the poorly served group?

      • Little improvement: the missing group is still missing
      • The problem is fixed, since there is more data
      • The app stops working for everyone

      Choose one

   5. 5.A company says its face-unlocking AI is 94% accurate. Which extra figure would best help you check whether it is fair?

      • The accuracy for each group of people separately
      • How fast it runs
      • The size of the company
      • The colour of the phone

      Choose one

   🔒 Answering and checking are for registered learners. Register to do this.

5. Your turn

   The leaf doctor app

   A start-up builds a phone app that looks at a photo of a wheat leaf and says "healthy" or "diseased". It is trained on 10,000 photos taken on farms in one district of Punjab, in bright winter sunshine. Farmers in Kerala later report that the app often calls healthy leaves diseased. The app was 96% accurate in the company's tests.

   1. In your own words, explain how the app "learned": what are its training data, its labels and its features? What is the model doing when it gives an answer?
   2. Give two likely reasons why it fails in Kerala, and say how you would find out which is true.
   3. Describe a plan to make it fairer: what new data would you collect, and what numbers would you report to show it works for every region? Explain why adding 10,000 more Punjab photos would not be enough.

   How it's judged

   • Explaining how a classifier learns
   • Finding causes of bias
   • A plan to be fair

   🔒 Writing and submitting your answer is for registered learners. Register to do this.