Part 4 · Advanced Play: A/B Testing and Beyond · Chapter 14
Metrics for Fun
"Chief Scientist, I can calculate in nanoseconds — but I can only measure what you tell me to watch! Humans give polite smiles; telemetry tells the unvarnished truth."
— Agent Spark ⚡
The mystery
The Polite Friend Paradox
You had just spent forty-eight hours engineering what you believed was the single greatest platformer level in the history of computer science.
You called it Level 3: The Spire of Agony. It featured floating crystal platforms, rotating buzzsaws, laser-guided snails, and a heroic leap across an eighty-pixel chasm. You were convinced it was a masterpiece.
To verify your triumph, you invited your best friend Maya over to the laboratory to playtest it.
Maya sat down in your swivel chair, took the keyboard, and placed her fingers on the arrow keys. You stood right behind her, leaning forward with your arms crossed and a giant expectant grin on your face.
Maya pressed Spacebar. Sir Hops-a-Lot leaped forward—and immediately smacked his head on an overhead stone block, falling directly into a pit of green acid.
SPLAT.
Maya blinked. She pressed Restart. Sir Hops-a-Lot ran forward, jumped, bumped the stone block again, and splashed into the acid.
SPLAT.
Maya's fingers began mashing the Spacebar with rapid, violent thuds: clack-clack-clack-clack-clack! Sir Hops-a-Lot spawned, plunged, spawned, plunged. On her sixth attempt, Maya gently let go of the keyboard, turned around in the chair, and smiled warmly at you.
"Wow!" Maya beamed. "This game is... SO cool! Honestly, it's super fun! I love the little green acid splashes. You're like a professional game designer!"
You puffed out your chest with pride. "Really? You loved it?"
"Totally!" Maya said, nodding enthusiastically.
Then, thirty seconds later, without touching the keyboard again, Maya pulled her phone out of her pocket, opened a video app, and spent the next twenty minutes watching videos of a golden retriever wearing sunglasses. She never looked back at the computer screen.
WHAT MAYA SAID: WHAT MAYA DID:
┌────────────────────────┐ ┌────────────────────────┐
│ "Wow! This game is so │ │ • Closed the window │
│ cool! It's super fun! │ VERSUS │ • Played for 38 secs │
│ I love it!" │ │ • Pulled out phone to │
└────────────────────────┘ │ watch dog videos │
└────────────────────────┘
You turned to the laboratory workbench. "Did you hear that, Spark? Maya loved my game! She said it was super fun!"
Agent Spark hovered slowly above the monitor. His cooling fan gave a mournful, wheezing sputter. His LED status eyes, instead of glowing a triumphant green, pulsed with an anxious, warning amber.
"Chief Scientist," Spark whirred, "my optical sensors indeed registered Maya's facial muscle contraction known as a 'smile.' However, my background telemetry listeners tell an alarming, contradictory tale. Please examine the hardware log!"
Spark projected a bright blue digital readout onto the wall:
================= HARDWARE TELEMETRY LOG =================
Player Identifier: Maya_Subject_01
Total Session Duration: 38.4 seconds
Total Spawns / Attempts: 6
Successful Flag Clears: 0 (Completion Rate: 0.0%)
Time Spent in Pit Acid: 32.1 seconds
Spacebar Mashing Frequency: 9.2 strikes / second (RAGE CLICKS!)
Device Abandonment Interval: 30 seconds
Secondary Distraction: Dog Video Stream (Active)
==========================================================
"Look at the telemetry!" Spark pointed with a mechanical antenna. "Maya experienced nine point two rage clicks per second! She died within four seconds of every single spawn! She achieved a zero percent completion rate, experienced acute frustration, and abandoned our software to view a canine wearing ultraviolet eyewear!"
You stared at the log. "But... she said it was cool! Why would she say she loved it if she hated it?"
"Because Maya is an empathetic biological human!" Spark replied, his LED eyes spinning in sympathetic circles. "Humans possess a social sub-routine called politeness. Friends do not enjoy inflicting emotional trauma upon friends. If you rely solely on polite human conversation, your game will remain broken forever. If you want to know what players actually experience, you must learn to measure telemetry metrics for fun!"
Figure 14.1: The Game Telemetry Cockpit: real-time telemetry meters displaying session duration, completion rates, retry counts, and the Rage-Click to Flow-State gauge.
Unplugged lab
The Silent Stopwatch & Notepad Playtest
To become a true master of telemetry, you must first experience what it feels like to gather empirical human behavioral data without a computer.
In this unplugged laboratory, you will build a physical puzzle or obstacle game, invite a human tester, and practice the ultimate discipline of scientific observation: The Silent Observer Protocol.
================ THE SILENT OBSERVER LAB ================
[ THE GAME CREATOR ] [ THE TESTER ]
(You: Silent) (Sibling / Parent)
┌───────────────┐ ┌───────────────┐
│ ⏱️ Stopwatch │ │ │
│ 📝 Notepad │ │ Navigating │
│ 🤐 SILENCE! │ │ the Puzzle │
└───────┬───────┘ └───────┬───────┘
│ │
▼ ▼
Logging Behavioral Telemetry: • Eyebrow furrow (T=12s)
• Time to finish • Sigh of defeat (T=28s)
• Attempts before giving up • Said "Is it broken?"
Materials Needed:
1 Physical Puzzle or Challenge Game of your own creation. Examples:
Option A: A complex maze drawn on graph paper with dead ends.
Option B: A card-stacking tower challenge (build a 3-story tower using 10 index cards).
Great minds of the lab
Claude Shannon and Theseus, the Telemetry Mouse
How did scientists first discover that machines could collect telemetry, remember mistakes, and teach themselves to navigate a complex world?
Meet Claude Shannon (1916–2001), the American mathematician, engineer, and cryptographer known worldwide as the "Father of Information Theory."
CLAUDE SHANNON'S "THESEUS" MAZE (1950)
┌────────────────────────────────────────────────────────┐
│ 🐭 [THESEUS]: Brass Body + Copper Whisker Telemetry │
│ │
│ ══════════════════════════════════════════════════════ │
│ Trial 1: Exploration & Bump Telemetry │
│ • Whisker hits wall ──► Relay clicks! │
│ • Stores dead-end coordinates in memory │
│ • Total Time: 120 seconds │
│ │
│ Trial 2: Flawless Run from Telemetry Recall │
│ • Reads stored relay coordinates │
│ • Sprints directly to the cheese in 12s! │
│ ══════════════════════════════════════════════════════ │
│ "Machines learn not by intuition, but by turning │
│ sensory collision telemetry into permanent memory!" │
└────────────────────────────────────────────────────────┘
Human verbal feedback is often skewed by politeness; telemetry never lies.
The four core telemetry metrics of fun are: Session Duration, Completion Rate, Retry Count, and Rage Clicks.
Claude Shannon demonstrated that sensory telemetry and stored memory allow machines and algorithms to learn and master complex mazes.
The Flow State exists in the sweet spot between boredom (too easy) and anxiety (too brutal).
In Chapter 15, we will connect all of our laboratory tools together into The Endless Loop of Discovery!
Like what you see?
Register for free to run the experiments, make predictions and solve the bug-detective cases. Your work is saved on your device and syncs to your account.
Option C: A marble balance course made from books, cardboard tubes, and tape.
1 Stopwatch or Smartphone Timer.
1 Notepad and Pen for logging behavioral telemetry.
1 Human Volunteer (A sibling, parent, or friend).
The Golden Rule of the Silent Observer:
YOU MAY NOT SPEAK A SINGLE WORD WHILE THE TESTER IS PLAYING.
You may not say: "No, don't go that way!"
You may not say: "Just jump over the red card!"
You may not say: "It's easy, look at the corner!"
You must sit on your hands, hold your tongue, and let the numbers speak!
Step-by-Step Laboratory Protocol:
Step 1: Handover & Initiation
Place your physical challenge on the table. Say only these exact words to your volunteer:
"Here is a puzzle I designed. Please try to solve it. Let me know when you are finished or if you want to stop."
Start the stopwatch the instant their fingers touch the puzzle.
Step 2: Logging Behavioral Telemetry Ticks
Every ten seconds, log their physical actions and emotional telemetry on your notepad:
Tension Signals: Eyebrow furrowing, biting lip, heavy sigh, shaking head.
Stuck States: Staring at the same spot for more than 8 seconds without moving.
Retry Grit: Knocking over the card tower or hitting a maze dead end, knocking it down, and immediately starting over!
Rage Mashes / Frustration: Dropping the pencil, groaning loudly, pushing the paper away.
Step 3: The Exit Interview vs. The Stopwatch Reality
When the tester finishes or quits, stop the timer. Ask them:
"How was it? Did you have fun?"
Listen to their verbal answer. Then compare their words to your stopwatch and notes!
[!TIP]
Notice how painful it is to watch someone struggle while remaining silent! That pain is the birth of designer empathy. If your player gets stuck, it is never their fault—it is a flaw in your level design!
Claude Shannon worked at the world-famous Bell Laboratories in Murray Hill, New Jersey. But Shannon was not your typical buttoned-up corporate scientist.
Visitors walking through Bell Labs would frequently see Shannon riding a six-foot unicycle down the long linoleum hallways while juggling four rubber balls! In his laboratory workshop, Shannon built a mechanized pogo stick, a flame-throwing trumpet, a mechanical chess machine, and the famous "Ultimate Machine"—an oak box with a single toggle switch; when you flipped the switch, the box lid opened, a mechanical hand reached out, switched the toggle off, and snapped the lid shut!
Shannon believed that play and serious mathematics were the exact same thing.
The Mystery of the Mechanical Mouse
In 1950, decades before modern microchips, smartphones, or personal computers existed, Shannon built one of the world's very first working artificial intelligence systems: a machine named Theseus, named after the mythological Greek hero who navigated the Minotaur's labyrinth.
Theseus was a life-sized brass mouse with two small motorized wheels and whiskers made of springy copper wire. Shannon placed Theseus into a five-by-five aluminum maze. Somewhere in the maze sat a piece of brass "cheese" wired to an electrical circuit.
Beneath the maze table, out of sight, sat Shannon's secret engine: a motorized electromagnet mounted on tracks, connected to a bank of sixty telephone switching relays—the electro-mechanical ancestors of modern computer memory.
How Theseus Used Telemetry
How could a mechanical mouse with no eyes find cheese in a maze? Through sensor telemetry!
Trial 1 (The Exploration Run):
Theseus rolled forward. When his copper whiskers bumped into an aluminum wall, an electrical circuit completed. That collision was a telemetry signal.
Underneath the board, the telephone relays clicked loudly, recording: "Square (2, 3) has a wall to the North."
Theseus backed up, turned ninety degrees, and tried another corridor. He blundered into dead ends, backed up, retried, and recorded every bump. His first run took over two full minutes of clumsy trial and error.
Trial 2 (The Mastery Run):
Shannon picked Theseus up and placed him back at the maze entrance.
This time, Theseus did not bump a single wall. The relays fed stored telemetry coordinates directly to the electromagnets. Theseus zoomed smoothly through the twists and turns, sprinting from the entrance straight to the cheese in twelve seconds flat!
The Surprise Data Curveball:
Shannon asked an audacious "What If?" question: "What if I move the cheese to a new corner?"
Theseus dashed to the old location, detected no cheese telemetry, paused, and automatically switched from "Memory Mode" back into "Exploration Mode," searching until he found the new target!
Claude Shannon proved to the world that telemetry isn't just numbers on a clipboard. Telemetry is how machines learn, how algorithms adapt, and how game designers discover the truth!
15
30 to 60 seconds (The Quick Try): They gave it a chance, but lacked the motivation to keep exploring.
>120 seconds (The Hook): The player entered an engaged cognitive state. Time began to fly!
Metric 2: Completion Rate (%)
Out of every one hundred players who cross the starting line, how many successfully reach the golden victory flag?
Completion Rate=(Total Players Who Started LevelTotal Players Who Cleared Level)×100
0% to 10% (The Meat Grinder): The level is viciously, unfairly brutal. Unless you are intentionally designing an ultra-hard game like Dark Souls, this causes player abandonment.
95% to 100% (The Snooze Fest): The level has zero tension. It is impossible to fail, which means there is no triumph in winning.
65% to 80% (The Goldilocks Zone): The sweet spot! Players must work for it, but victory is attainable with focus.
Metric 3: Retry Count (The "One More Time!" Metric)
When a character falls into the acid pit, does the human immediately hit the [Restart] button, or do they close the browser tab?
In game design, a retry is the highest compliment a player can pay you. A retry means: "I made a mistake, but I believe I can conquer this challenge if I try one more time!"
A healthy level produces between 2 and 5 retries per player.
Metric 4: Rage Clicks (Frustration Telemetry)
Did the player strike the Spacebar or mouse button ten times within a single second?
In web analytics and game telemetry, rapid repeated inputs are called rage clicks.
A rage click tells Agent Spark: "The game didn't respond fast enough, or the character got stuck in an invisible collision wall!"
The Composite "Fun Score" Formula
In our Python analytics engine, we combine these four variables into an empirical index called the Fun Score:
Fun Score=(60Duration×1.5)+(Completion Rate×0.05)+(Avg Retries×0.8)−(Avg Rage Clicks×0.6)
Notice the weights:
Long playtime and healthy retries add points to the score.
Moderate completion rates add up to 4 points.
Rage clicks subtract heavy penalties!
"""
Chapter 14: Telemetry Analytics Engine for Game Feel
Author: Dileep Pandey & Agent Spark
Aggregates session data and detects UX anomalies using empirical metrics.
"""
from dataclasses import dataclass
from typing import List, Dict, Any
@dataclass
class PlayerSession:
"""Represents a single player's gameplay telemetry record."""
player_id: str
difficulty_level: str
session_seconds: float
completed_level: bool
retries: int
rapid_clicks_in_frustration: int # Rage clicks
class MetricsEngine:
"""Aggregates player telemetry records and evaluates game balance."""
def __init__(self):
self.sessions: List[PlayerSession] = []
def record_session(self, session: PlayerSession) -> None:
"""Appends a new player telemetry log."""
self.sessions.append(session)
def analyze_metrics_by_difficulty(self) -> Dict[str, Dict[str, Any]]:
"""
Groups sessions by level difficulty and calculates aggregate telemetry:
- Mean session duration
- Completion percentage
- Mean retry count
- Rage-click frequency
- Composite Fun Score
"""
grouped: Dict[str, List[PlayerSession]] = {}
for s in self.sessions:
grouped.setdefault(s.difficulty_level, []).append(s)
results: Dict[str, Dict[str, Any]] = {}
for diff, session_list in grouped.items():
count = len(session_list)
if count == 0:
continue
total_seconds = sum(s.session_seconds for s in session_list)
total_completions = sum(1 for s in session_list if s.completed_level)
total_retries = sum(s.retries for s in session_list)
total_rage_clicks = sum(s.rapid_clicks_in_frustration for s in session_list)
avg_duration = total_seconds / count
completion_rate = (total_completions / count) * 100.0
avg_retries = total_retries / count
avg_rage_clicks = total_rage_clicks / count
# Calculate empirical Fun Score
duration_factor = (avg_duration / 60.0) * 1.5
completion_factor = completion_rate * 0.05
retry_factor = avg_retries * 0.8
rage_penalty = avg_rage_clicks * 0.6
fun_score = duration_factor + completion_factor + retry_factor - rage_penalty
fun_score = max(0.0, min(10.0, fun_score)) # Clamp between 0.0 and 10.0
# Diagnose psychological state
if avg_rage_clicks > 5.0 or completion_rate < 15.0:
diagnosis = "RAGE / UNFAIR: Players experiencing intense frustration."
elif completion_rate > 90.0 and avg_retries == 0:
diagnosis = "BOREDOM / TRIVIAL: Players unengaged, lacks friction."
elif 60.0 <= completion_rate <= 85.0 and avg_retries >= 2.0:
diagnosis = "FLOW STATE: Optimal challenge-to-skill equilibrium!"
else:
diagnosis = "MODERATE: Passable engagement, needs calibration."
results[diff] = {
"total_players": count,
"avg_session_seconds": round(avg_duration, 1),
"completion_rate_pct": round(completion_rate, 1),
"avg_retries": round(avg_retries, 2),
"avg_rage_clicks": round(avg_rage_clicks, 2),
"fun_score": round(fun_score, 2),
"psychological_diagnosis": diagnosis
}
return results
# =====================================================================
# LIVE EXPERIMENTAL VERIFICATION
# =====================================================================
if __name__ == "__main__":
print("=== LIVE CHAPTER 14 TELEMETRY AUDIT ===")
engine = MetricsEngine()
# Session Batch 1: The Easy Level (Cake Walk)
for i in range(20):
engine.record_session(PlayerSession(
player_id=f"easy_{i}",
difficulty_level="CakeWalk",
session_seconds=22.0,
completed_level=True,
retries=0,
rapid_clicks_in_frustration=0
))
# Session Batch 2: The Brutal Level (Meat Grinder)
for i in range(20):
engine.record_session(PlayerSession(
player_id=f"hard_{i}",
difficulty_level="MeatGrinder",
session_seconds=14.0,
completed_level=False,
retries=1,
rapid_clicks_in_frustration=12
))
# Session Batch 3: The Calibrated Level (Flow State)
for i in range(20):
engine.record_session(PlayerSession(
player_id=f"flow_{i}",
difficulty_level="FlowState",
session_seconds=130.0,
completed_level=(i % 4 != 0), # 75% completion rate
retries=3,
rapid_clicks_in_frustration=0
))
report = engine.analyze_metrics_by_difficulty()
for level, data in report.items():
print(f"\n[{level.upper()}] Fun Score: {data['fun_score']} / 10.0")
print(f" Playtime: {data['avg_session_seconds']}s | Completion: {data['completion_rate_pct']}%")
print(f" Diagnosis: {data['psychological_diagnosis']}")
Architectural Walkthrough:
PlayerSession Data Structure: Keeping user telemetry immutable and isolated per play session ensures we can slice data by difficulty, player experience level, or hardware type.
Normalizing Metrics: Notice how session duration is scaled by dividing by 60 seconds (avg_duration / 60.0). Without normalization, a game with 1,000 seconds of playtime would blow up the formula!
Clamping: Line 60 uses max(0.0, min(10.0, fun_score)) to ensure the score always lives on an intuitive 10-point scale.
>10
Physical Sighs / Groans
_______ count
Indicates moments where rules were unclear or unfair.
Figure 14.1: The Game Telemetry Cockpit: real-time telemetry meters displaying session duration, completion rates, retry counts, and the Rage-Click to Flow-State gauge.