Twelve real windows from the evaluation, chosen to span weak and strong readers. Each page shows the printed text, the child's clip, the speech-to-text transcript, the enumerator's live marks, the round-1 alignment verdicts, and every model's per-word verdicts, then the counts side by side and the second human's review. Nothing is summarised away: what you see is exactly what the scorer saw and what it returned.
Take Example 3, a grade-4 child reading the Urdu story. Section 1 is the printed text: the 60 words the child had in front of them, numbered in reading order. Section 2 links to the 60-second clip, timed from the child's first word. Section 3 is what the speech-to-text service typed from that clip; the microphone hears the enumerator too, so prompts like "shabash" appear between the child's words.
Section 4 has one row per scorer with a colour chip for every printed word. The enumerator row is the live tablet marking, the official score: 57 green, 3 red. The baseline row is last round's automatic method, which matches transcript words to printed words by spelling; here it reaches only 14 green because the transcript is rough. Each model row is that model's own judgement, from the audio or from the transcript.
Over-flagging shows as red where the enumerator has green: DeepSeek's row has 21 red chips against the enumerator's 3. Under-flagging shows as green where the enumerator has red: three audio models mark all 60 green, waving through the three words the enumerator rejected. Grey means the scorer thinks the child skipped or never reached the word; faded means the scorer gave no answer.
Section 5's "difference from enumerator" column is each scorer's correct count minus the enumerator's 57. Near zero is good; a large positive number means too lenient, a large negative number means too harsh.
Two failure shapes repeat. Some audio models say "all correct" whatever they hear: Gemini 2.5 Flash, MiMo 2.5, Nemotron and Voxtral give full marks to children the enumerator scored 0 of 5 (Examples 1, 5, 9). Others flag almost everything: the round-1 baseline, GPT audio mini (0 of 59 on Examples 7 and 8, yet 60 of 60 on Example 1) and, on transcripts, DeepSeek 3.2 and GPT-5 mini. Gemini 3.1 Pro and Gemini 3.8 Flash listening to audio track the enumerator most closely, within one to three words on Examples 2, 3, 10, 11 and 12. Transcript-based scorers are capped by the transcript: on Urdu pages it is fragmentary, so no model can recover the read from it. Two pages need a human ear: Example 6 (enumerator 1 of 6, every audio model 4 to 6) and Example 7 (second human marked the enumerator "incorrect"; models range 0 to 59).
| Page | Subtask | Grade | Enumerator: correct / attempted | Scorers shown |
|---|---|---|---|---|
| Example 1 | Urdu story (60 s) | grade 2 | 0 / 5 | 13 models (1 gave no usable answer) |
| Example 2 | Urdu story (60 s) | grade 4 | 22 / 23 | 10 models (1 gave no usable answer) |
| Example 3 | Urdu story (60 s) | grade 4 | 57 / 60 | 10 models (2 gave no usable answer) |
| Example 4 | Urdu story (60 s) | grade 4 | 60 / 60 | 13 models |
| Example 5 | English story (60 s) | grade 2 | 0 / 5 | 13 models (1 gave no usable answer) |
| Example 6 | English story (60 s) | grade 5 | 1 / 6 | 10 models |
| Example 7 | English story (60 s) | grade 3 | 36 / 43 | 13 models (3 gave no usable answer) |
| Example 8 | English story (60 s) | grade 5 | 53 / 57 | 10 models |
| Example 9 | Urdu word list (60 s) | grade 3 | 0 / 5 | 10 models |
| Example 10 | Urdu word list (60 s) | grade 2 | 27 / 32 | 12 models (1 gave no usable answer) |
| Example 11 | Urdu word list (60 s) | grade 5 | 48 / 50 | 13 models |
| Example 12 | Urdu word list (60 s) | grade 4 | 49 / 50 | 13 models |