Worked example 2: Urdu story (60 s)

Recording AA_bd2f2910-… · grade 4 · age 11 · enumerator #35 · window 407–556 s of the session, 60-s clock from the child's first word at 406.5 s

How to read this page. Section 1 is the printed text the child had in front of them, numbered in reading order. Section 2 is the 60-second clip, timed from the child's first word. Section 3 is what the speech-to-text service typed from that clip; the microphone hears the enumerator too, so prompts like "shabash" can appear between the child's words. Section 4 has one row per scorer with a colour chip for every printed word: the enumerator row is the live tablet marking, the official score; the baseline row is the first automatic method, which matches transcript words to printed words by spelling; each model row is that model's own judgement from the audio or from the transcript. Red where the enumerator has green means the scorer flagged too much; green where the enumerator has red means it waved a mistake through; grey means skipped or never reached; faded means no verdict. "Attempted" is the furthest word reached. Section 5's "difference from enumerator" is each scorer's correct count minus the enumerator's: near zero is good, a large positive number is too lenient, a large negative number too harsh. Section 6 is the second human's re-listen.

1. What the child was supposed to read

1. آج 2. بلال 3. اپنے 4. والد 5. کے 6. ساتھ 7. دریائے 8. جہلم 9. کے 10. کنارے 11. گیا 12. وہاں 13. ٹھنڈی 14. ہوا 15. چل 16. رہی 17. تھی 18. اور 19. پانی 20. آہستہ 21. بہہ 22. رہا 23. تھا 24. بلال 25. نے 26. کشتی 27. کو 28. حرکت 29. کرتے 30. دیکھا 31. اس 32. کے 33. والد 34. نے 35. اسے 36. بتایا 37. کہ 38. دریا 39. ہمارے 40. ملک 41. کے 42. لیے 43. اہم 44. ہے 45. واپسی 46. پر 47. بلال 48. نے 49. خوبصورت 50. پتھر 51. جمع 52. کیے 53. اور 54. مسکراتا 55. رہا 56. وہ 57. بہت 58. خوش 59. بھی 60. تھا

The study export holds only right/wrong flags, not the text itself, so this text was reconstructed from 30 children's reads with an LLM; a word or two of it may itself be wrong, which would show up as the same "mistake" in every scorer's row.

2. The child's audio

Play the 60-second clip (Drive, Taleemabad accounts)
8 kHz classroom recording from the May 2026 study, upsampled. Folder: worked-examples clips

3. What the speech-to-text service heard

اج دن ہوا بلال بلال الف الف نون نی اپنی والوال ل ل ل ل کی ساتہ ساتہ ساتہ در در دریا دریا دریا دریا جی جی ج ج ج ج ج ج ج ج ج ج نیکسٹ پڑہتی جاییں پڑہتی جاییں کی سامنی ا گیا ر ر ر ر ر ر ر ر ر ر جلدی ٹایم تہوڑا ہی و و و و و و و و و و د د د د د د د د د د و و و و و و و و و و چل چل ر ر ر ر ر ر ر ر ر ر اور پانی کی پار پ پ پ پ پ پ پ پ پ پ بہ بہ بہ رہ رہ رہ رہ رہ رہ رہ رہ رہ رہ ب ب

Soniox stt-async-v3, Urdu hint · 131 words inside the 60-s window. The enumerator's own words (prompts, "shabash") are included when they fall inside the window.

4. Every scorer's verdict on every word

read correctly wrong skipped or not reached no verdict

Enumerator (tablet, live)

آجبلالاپنےوالدکےساتھدریائےجہلمکےکنارےگیاوہاںٹھنڈیہواچلرہیتھیاورپانیآہستہبہہرہاتھابلالنےکشتیکوحرکتکرتےدیکھااسکےوالدنےاسےبتایاکہدریاہمارےملککےلیےاہمہےواپسیپربلالنےخوبصورتپتھرجمعکیےاورمسکراتارہاوہبہتخوشبھیتھا

22 correct of 23 attempted in 60 s (words after the 60-second point are grey: the export records 0 for both "correct" and "never reached", so they are only known up to the attempted count)

Baseline: speech-to-text + spelling alignment

5 correct of 60 attempted; 71 extra words the child said are ignored

Gemini 3.1 Pro · audio

23 correct of 24 attempted · model note: The student reads slowly, spelling out most words. He misreads 'ہوا' as 'ووا'.

Gemini 3.8 Flash · audio

22 correct of 23 attempted · model note: Child sounds out/spells each word aloud before reading it, reaching word 23.

Gemini 3.5 Flash-Lite · audio

8 correct of 60 attempted · model note: The child read a few initial words correctly and then struggled and made numerous errors through the rest of the text.

Gemini 2.5 Flash · audio

23 correct of 23 attempted · model note: The child read the first 23 words correctly and then stopped.

GPT audio mini · audio

6 correct of 60 attempted · model note: The child read the first six words correctly, then struggled with the rest of the text.

MiMo 2.5 · audio

26 correct of 44 attempted · model note: Child read first ~44 words with several mispronunciations (e.g. جہلم, ٹھنڈی, آہستہ, بہہ, کشتی); teacher intervened twice; child stopped befo…

Claude Sonnet 5 · transcript

The model returned no usable answer for this window (its reply was not in the requested format).

GPT-5 mini · transcript

13 correct of 22 attempted · model note: Based on the transcript: child attempted 15 words (13 judged correct, 2 wrong); remaining words were not attempted/skipped.

Gemini 3 Flash · transcript

12 correct of 22 attempted · model note: The child struggled significantly with stuttering and phonetic decoding, skipping several words while attempting to move forward.

DeepSeek 3.2 · transcript

7 correct of 22 attempted · model note: Speech-to-text transcript is fragmented, repetitive, and contains many non-words; child appears to struggle with fluency and decoding, makin…

5. The counts

Scorer	Words correct	Words attempted (furthest word reached)	Difference from enumerator (correct)	Difference (attempted)
Enumerator	22	23	+0	+0
Baseline: speech-to-text + alignment	5	60	-17	+37
Gemini 3.1 Pro · audio	23	24	+1	+1
Gemini 3.8 Flash · audio	22	23	+0	+0
Gemini 3.5 Flash-Lite · audio	8	60	-14	+37
Gemini 2.5 Flash · audio	23	23	+1	+0
GPT audio mini · audio	6	60	-16	+37
MiMo 2.5 · audio	26	44	+4	+21
Claude Sonnet 5 · transcript	—	—	no usable answer	—
GPT-5 mini · transcript	13	22	-9	-1
Gemini 3 Flash · transcript	12	22	-10	-1
DeepSeek 3.2 · transcript	7	22	-15	-1

6. The second human

The second human (kainat) re-listened: on the correct count the reviewer agreed with the enumerator; on the attempted count the reviewer agreed with the enumerator; on orf_reading_sentences_urd the reviewer agreed with the enumerator; the enumerator followed the protocol.