Unit 1

Measuring what matters

Turn a fuzzy goal like "fun" into something you can count, and learn what every count leaves out.

1. Read

   How do you measure "fun"?

   The school wants to know: is the new playground fun? The head teacher asks the class to find out. Nobody can hold a ruler up to "fun". So how can you measure something you can't measure directly?

   Goals, metrics and proxies

   A goal is the thing you really care about: fun, a healthy canteen, a popular library, a clean river. Many goals are fuzzy. They have no unit, like centimetres or kilograms.

   A metric is a number you can actually collect. A proxy is a metric that stands in for a fuzzy goal.

   Goal	Possible proxies
   The playground is fun	number of children playing at break; how long children stay; smiles per minute counted by an observer
   The library is popular	books borrowed per week; visitors per day
   A river is healthy	clarity of the water; number of fish species

   A good proxy is measurable (you can count or time it), linked to the goal (when the goal changes, the number changes), and hard to cheat.

   Observation versus self-report

   There are two main ways to collect data.

   • Observation: you watch what people do. "Twelve children were on the climbing frame at 11:05."
   • Self-report: you ask people what they say. "How much did you enjoy the playground, from 1 to 5?"

   

   People are not always accurate about themselves. They may want to please the person asking, forget, or answer differently when friends are listening. What people do is often better evidence than what they say. But observation misses feelings you cannot see. The best studies use both and see whether they agree.

   Example. In a survey, 9 out of 10 children say they "love" the playground (self-report). But a counter on a tally sheet finds that only 3 children are on it at break and most stand by the wall (observation). The two measures disagree, which tells you to look closer.

   Collecting and summarising

   Suppose you tally the number of children on the playground at 11:00 on five days:

   Mon	Tue	Wed	Thu	Fri
   14	9	16	11	14

   • Total: 14 + 9 + 16 + 11 + 14 = 64
   • Mean (average): 64 ÷ 5 = 12.8 children
   • Range: 16 − 9 = 7

   A summary like "about 13 children a day, between 9 and 16" is easier to read than five raw numbers.

   Worked example: choosing a proxy

   Example. Your class wants to know whether the new water fountain is "well used". Follow the three tests.

   1. Measurable? "Number of cups of water drunk" is hard to count, but "number of children who use the fountain between 10:00 and 10:15" is easy to tally.
   2. Linked? If more children are thirsty or like the fountain, the tally goes up. If the fountain is broken, it falls to zero. So the number follows the goal.
   3. Hard to cheat? A child could press the button again and again to push the tally up. The counter can solve this by counting each child only once.

   Now ask what it leaves out. The tally does not show children who brought bottles from home and do not need the fountain. So "well used" is only partly captured. Say that in your report: "The fountain was used by 23 children in 15 minutes, but this does not count children with their own bottles."

   Try it: Pick a fuzzy goal from your own day (a quiet classroom, a tidy bag, a good bus ride). Write one proxy, say how you would collect it, and write one thing it leaves out.

   Every metric leaves something out

   Measuring changes what people do. If the head teacher says "the best playground is the one with most children", the playground might be crowded and noisy and still not fun for everyone. Counting children says nothing about whether the quiet ones feel left out.

   Misconception. "If it can't be measured, it isn't real." Fun is real even if no number captures it fully. And the opposite trap: "if we measure it, we know it." Every metric is a window, and every window shows only part of the view.

   Whenever you choose a metric, ask:

   1. What does it capture?
   2. What does it leave out?
   3. Could someone make the number go up without improving the goal? (For example, adding a loud speaker to get more children to the playground.)

   Summary

   • A proxy is a measurable stand-in for a fuzzy goal.
   • Observation (what people do) and self-report (what people say) can disagree. Doing is often stronger evidence.
   • Summarise data with totals, means and ranges.
   • Every metric leaves something out. Say what.

2. Lab

   The proxy picker

   For each fuzzy goal, sort the candidate measures into good proxies and weak proxies, and read what each good proxy captures and leaves out.

   Your goals

   • Sort the candidate measures for all three goals into good and weak proxies.
   • Mark the observation and the self-report that are good proxies for each goal, and read what each leaves out.

   🔒 The interactive lab opens once you register. Register to do this.

3. Quick check

   Measuring what matters check

   Eight questions on proxies, observation, self-report and summaries.

   1. 1.What is a proxy?

      • A measurable stand-in for something that is hard to measure directly.
      • A person who answers a survey for someone else.
      • A way to make results look better.

      Choose one

   2. 2.The goal is "the school library is popular". Which is the best proxy?

      • How tall the bookshelves are
      • Books borrowed per week
      • How many chairs there are

      Choose one

   3. 3.A researcher watches a canteen and counts how many children choose fruit. What kind of data is this?

      • Observation
      • Self-report

      Choose one

   4. 4.Children on the playground at 11:00 over five days: 14, 9, 16, 11, 14. What is the mean?

      Answer with a number

   5. 5.What is the range of 14, 9, 16, 11, 14?

      Answer with a number

   6. 6.9 in 10 children say they love the playground, but only 3 are using it at break. What is the best conclusion?

      • The two measures disagree, so we should look closer, for example by asking why children are not using it.
      • The children are lying, so ignore what they say.
      • The tally must be wrong, so ignore it.

      Choose one

   7. 7.A school measures "healthy eating" by counting apples sold. Which things does this metric leave out? Pick all that apply.

      • Fruit brought from home
      • Whether the apples were eaten or thrown away
      • Whether apples were sold

      Choose all that apply

   8. 8.Match each item to what it is.

      • Counting visitors at the door
      • Asking visitors to rate their visit
      • Visitors per day
      • Mean visitors per day over a week

      Match each to: Self-report · A metric · A summary · Observation

   🔒 Answering and checking are for registered learners. Register to do this.

4. Quick check

   New puzzles with measuring

   Choose and judge metrics in new situations.

   1. 1.A town wants to know whether its new bus route is "useful". Which metric is the most direct?

      • Number of passengers boarding per day
      • Number of bus stops
      • Length of the route in km

      Choose one

   2. 2.A school judges teachers only by pupils' test marks. One teacher teaches only what will be on the test. Which problem is this?

      • The metric can be raised without improving what we really care about (learning).
      • The test marks were collected by observation.
      • There is no problem.

      Choose one

   3. 3.A class measures water used by the school garden over four days, in litres, and gets 38, 42, 40, 44. What is the mean, in litres?

      Answer with a number

   4. 4.A canteen manager asks children, in front of the cook, "Do you like the lunch?" Almost all say yes. What is the most likely issue?

      • Children may answer to please the cook, so a self-report may not match what they really think.
      • The question uses numbers.
      • Observation is always wrong.

      Choose one

   🔒 Answering and checking are for registered learners. Register to do this.

5. Your turn

   Choose and defend a metric

   Your school wants to know: "Is our school garden a good place to spend break?"

   1. Choose two proxies (at least one based on observation). For each, say exactly how you would collect it and how often.
   2. Say what each proxy captures and what it leaves out.
   3. Imagine your data show 12 children a day in the garden, but a short survey finds that most children rate the garden 4 or 5 out of 5. Describe one explanation for how both could be true, and what you would do next.

   How it's judged

   • Choosing proxies
   • Limits of each metric
   • Reconciling two measures

   🔒 Writing and submitting your answer is for registered learners. Register to do this.