Worked example 7: English story (60 s)

Recording AA_eaf07843-… · grade 3 · age 9 · enumerator #35 · window 986–1053 s of the session, 60-s clock from the child's first word at 986.4 s

How to read this page. Section 1 is the printed text the child had in front of them, numbered in reading order. Section 2 is the 60-second clip, timed from the child's first word. Section 3 is what the speech-to-text service typed from that clip; the microphone hears the enumerator too, so prompts like "shabash" can appear between the child's words. Section 4 has one row per scorer with a colour chip for every printed word: the enumerator row is the live tablet marking, the official score; the baseline row is the first automatic method, which matches transcript words to printed words by spelling; each model row is that model's own judgement from the audio or from the transcript. Red where the enumerator has green means the scorer flagged too much; green where the enumerator has red means it waved a mistake through; grey means skipped or never reached; faded means no verdict. "Attempted" is the furthest word reached. Section 5's "difference from enumerator" is each scorer's correct count minus the enumerator's: near zero is good, a large positive number is too lenient, a large negative number too harsh. Section 6 is the second human's re-listen.

1. What the child was supposed to read

1. imran 2. woke 3. up 4. early 5. for 6. school 7. today 8. his 9. class 10. was 11. planting 12. trees 13. he 14. took 15. a 16. small 17. plant 18. in 19. a 20. bag 21. the 22. teacher 23. asked 24. imran 25. to 26. show 27. the 28. class 29. how 30. to 31. plant 32. it 33. imran 34. dug 35. a 36. hole 37. put 38. the 39. plant 40. in 41. the 42. soil 43. and 44. put 45. water 46. in 47. it 48. the 49. teacher 50. was 51. happy 52. with 53. his 54. work 55. his 56. friends 57. clapped 58. for 59. him

The study export holds only right/wrong flags, not the text itself, so this text was reconstructed from 30 children's reads with an LLM; a word or two of it may itself be wrong, which would show up as the same "mistake" in every scorer's row.

2. The child's audio

Play the 60-second clip (Drive, Taleemabad accounts)
8 kHz classroom recording from the May 2026 study, upsampled. Folder: worked-examples clips

3. What the speech-to-text service heard

perform miracles today as plants are planting a child reads a short english story about imran and a plant aloud a child reads a short english story about imran and a plant aloud

whisper-1 forced to English · 33 words inside the 60-s window. The enumerator's own words (prompts, "shabash") are included when they fall inside the window.

4. Every scorer's verdict on every word

read correctly wrong skipped or not reached no verdict

Enumerator (tablet, live)
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
36 correct of 43 attempted in 60 s (words after the 60-second point are grey: the export records 0 for both "correct" and "never reached", so they are only known up to the attempted count)
Baseline: speech-to-text + spelling alignment
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
7 correct of 59 attempted; 0 extra words the child said are ignored
Gemini 3.1 Pro · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
The model returned no usable answer for this window (its reply was not in the requested format).
Gemini 3.8 Flash · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
41 correct of 56 attempted · model note: The child skipped several short phrases and mispronounced 'his' as 'has'.
Gemini 3.5 Flash-Lite · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
56 correct of 57 attempted · model note: The child read almost the entire text correctly, missing only the final words.
Gemini 2.5 Flash · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
59 correct of 59 attempted · model note: The child read all words correctly.
GPT audio · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
The model returned no usable answer for this window (its reply was not in the requested format).
GPT audio mini · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
0 correct of 59 attempted · model note: The child’s pronunciation did not match the expected words.
MiMo 2.5 · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
59 correct of 59 attempted · model note: Child read all words correctly with clear pronunciation and no skips.
Nemotron omni · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
The model returned no usable answer for this window (its reply was not in the requested format).
Voxtral small · audio
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
59 correct of 59 attempted · model note: The child read the entire text correctly.
Claude Sonnet 5 · transcript
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
0 correct of 0 attempted · model note: Transcript content (a jumbled description mentioning 'imran','plant','planting','today') does not correspond in sequence to the printed pass…
GPT-5 mini · transcript
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
0 correct of 33 attempted · model note: Spoken tokens were aligned sequentially to the printed words; only the first 33 printed words were attempted (all mismatched).
Gemini 3 Flash · transcript
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
0 correct of 0 attempted · model note: The transcript contains meta-commentary and unrelated phrases; none of the printed text words were identified as read.
DeepSeek 3.2 · transcript
imranwokeupearlyforschooltodayhisclasswasplantingtreeshetookasmallplantinabagtheteacheraskedimrantoshowtheclasshowtoplantitimrandugaholeputtheplantinthesoilandputwaterinittheteacherwashappywithhisworkhisfriendsclappedforhim
0 correct of 0 attempted · model note: Child did not read the printed story; transcript appears to be a description of the activity.

5. The counts

ScorerWords correctWords attempted (furthest word reached)Difference from enumerator (correct)Difference (attempted)
Enumerator3643+0+0
Baseline: speech-to-text + alignment759-29+16
Gemini 3.1 Pro · audio——no usable answer—
Gemini 3.8 Flash · audio4156+5+13
Gemini 3.5 Flash-Lite · audio5657+20+14
Gemini 2.5 Flash · audio5959+23+16
GPT audio · audio——no usable answer—
GPT audio mini · audio059-36+16
MiMo 2.5 · audio5959+23+16
Nemotron omni · audio——no usable answer—
Voxtral small · audio5959+23+16
Claude Sonnet 5 · transcript00-36-43
GPT-5 mini · transcript033-36-10
Gemini 3 Flash · transcript00-36-43
DeepSeek 3.2 · transcript00-36-43

6. The second human

The second human (laraib) re-listened: on the correct count the reviewer disagreed with the enumerator; on the attempted count the reviewer disagreed with the enumerator; on orf_reading_sentences_eng the reviewer disagreed with the enumerator. Reviewer comment: “Not given time to attempt.”