Worked example 1: Urdu story (60 s)

Recording AA_b68b3349-… · grade 2 · age 7 · enumerator #49 · window 586–601 s of the session, 60-s clock from the child's first word at 586.1 s

How to read this page. Section 1 is the printed text the child had in front of them, numbered in reading order. Section 2 is the 60-second clip, timed from the child's first word. Section 3 is what the speech-to-text service typed from that clip; the microphone hears the enumerator too, so prompts like "shabash" can appear between the child's words. Section 4 has one row per scorer with a colour chip for every printed word: the enumerator row is the live tablet marking, the official score; the baseline row is the first automatic method, which matches transcript words to printed words by spelling; each model row is that model's own judgement from the audio or from the transcript. Red where the enumerator has green means the scorer flagged too much; green where the enumerator has red means it waved a mistake through; grey means skipped or never reached; faded means no verdict. "Attempted" is the furthest word reached. Section 5's "difference from enumerator" is each scorer's correct count minus the enumerator's: near zero is good, a large positive number is too lenient, a large negative number too harsh. Section 6 is the second human's re-listen.

1. What the child was supposed to read

1. آج 2. بلال 3. اپنے 4. والد 5. کے 6. ساتھ 7. دریائے 8. جہلم 9. کے 10. کنارے 11. گیا 12. وہاں 13. ٹھنڈی 14. ہوا 15. چل 16. رہی 17. تھی 18. اور 19. پانی 20. آہستہ 21. بہہ 22. رہا 23. تھا 24. بلال 25. نے 26. کشتی 27. کو 28. حرکت 29. کرتے 30. دیکھا 31. اس 32. کے 33. والد 34. نے 35. اسے 36. بتایا 37. کہ 38. دریا 39. ہمارے 40. ملک 41. کے 42. لیے 43. اہم 44. ہے 45. واپسی 46. پر 47. بلال 48. نے 49. خوبصورت 50. پتھر 51. جمع 52. کیے 53. اور 54. مسکراتا 55. رہا 56. وہ 57. بہت 58. خوش 59. بھی 60. تھا

The study export holds only right/wrong flags, not the text itself, so this text was reconstructed from 30 children's reads with an LLM; a word or two of it may itself be wrong, which would show up as the same "mistake" in every scorer's row.

2. The child's audio

Play the 60-second clip (Drive, Taleemabad accounts)
8 kHz classroom recording from the May 2026 study, upsampled. Folder: worked-examples clips

3. What the speech-to-text service heard

ٹہیک ہی شاباش پڑہتی جاییں پڑہتی جاییں ہمم ٹہیک ہی یس پڑہتی جاییں پڑہتی جاییں

Soniox stt-async-v3, Urdu hint · 15 words inside the 60-s window. The enumerator's own words (prompts, "shabash") are included when they fall inside the window.

4. Every scorer's verdict on every word

read correctly wrong skipped or not reached no verdict

Enumerator (tablet, live)

آجبلالاپنےوالدکےساتھدریائےجہلمکےکنارےگیاوہاںٹھنڈیہواچلرہیتھیاورپانیآہستہبہہرہاتھابلالنےکشتیکوحرکتکرتےدیکھااسکےوالدنےاسےبتایاکہدریاہمارےملککےلیےاہمہےواپسیپربلالنےخوبصورتپتھرجمعکیےاورمسکراتارہاوہبہتخوشبھیتھا

0 correct of 5 attempted in 60 s (words after the 60-second point are grey: the export records 0 for both "correct" and "never reached", so they are only known up to the attempted count)

Baseline: speech-to-text + spelling alignment

0 correct of 60 attempted; 0 extra words the child said are ignored

Gemini 3.1 Pro · audio

0 correct of 4 attempted · model note: The child only spells out letters (e.g., 'wao, alif, lam, dal' for 'walid') and does not read any words correctly before the teacher stops t…

Gemini 3.8 Flash · audio

0 correct of 6 attempted · model note: The child only spelled out individual letters rather than reading words before the teacher stopped the reading.

Gemini 3.5 Flash-Lite · audio

0 correct of 0 attempted · model note: The child did not read the Urdu text at all during the recording.

Gemini 2.5 Flash · audio

60 correct of 60 attempted · model note: The child read all 60 words correctly within the 60-second timeframe.

GPT audio · audio

The model returned no usable answer for this window (its reply was not in the requested format).

GPT audio mini · audio

60 correct of 60 attempted · model note: All words were read correctly.

MiMo 2.5 · audio

60 correct of 60 attempted · model note: Child read all words fluently and correctly with no mispronunciations or skips.

Nemotron omni · audio

60 correct of 60 attempted · model note: All words read correctly.

Voxtral small · audio

60 correct of 60 attempted · model note: The child read the entire text correctly.

Claude Sonnet 5 · transcript

0 correct of 0 attempted · model note: Transcript only contains teacher's encouragement phrases (e.g., 'ٹھیک ہے شاباش پڑھتی جائیں'); no words from the printed text were actually r…

GPT-5 mini · transcript

0 correct of 15 attempted · model note: Child spoke unrelated words; first 15 printed words were attempted but read incorrectly, remaining 45 were skipped.

Gemini 3 Flash · transcript

0 correct of 0 attempted · model note: The transcript contains only teacher encouragement and no words from the printed text.

DeepSeek 3.2 · transcript

0 correct of 0 attempted · model note: Child did not read any words from the given text; transcript contains only encouragement and procedural phrases.

5. The counts

Scorer	Words correct	Words attempted (furthest word reached)	Difference from enumerator (correct)	Difference (attempted)
Enumerator	0	5	+0	+0
Baseline: speech-to-text + alignment	0	60	+0	+55
Gemini 3.1 Pro · audio	0	4	+0	-1
Gemini 3.8 Flash · audio	0	6	+0	+1
Gemini 3.5 Flash-Lite · audio	0	0	+0	-5
Gemini 2.5 Flash · audio	60	60	+60	+55
GPT audio · audio	—	—	no usable answer	—
GPT audio mini · audio	60	60	+60	+55
MiMo 2.5 · audio	60	60	+60	+55
Nemotron omni · audio	60	60	+60	+55
Voxtral small · audio	60	60	+60	+55
Claude Sonnet 5 · transcript	0	0	+0	-5
GPT-5 mini · transcript	0	15	+0	+10
Gemini 3 Flash · transcript	0	0	+0	-5
DeepSeek 3.2 · transcript	0	0	+0	-5

6. The second human

The second human (fareeda) re-listened: on the correct count the reviewer agreed with the enumerator; on the attempted count the reviewer agreed with the enumerator; on orf_reading_sentences_urd the reviewer agreed with the enumerator; the enumerator followed the protocol.