The corpus, the 21-minute enumerator timing baseline, the QA review layer
Written: 2026-09-15 · Bead: bd-gqoz1 · Source data: the shared Drive folder 19tfSEeBaOchNwSB2VNZd9s9PmEpDAIkG (downloaded into audio/ by download_drive_audio.py)
| Item | What it is | Count |
|---|---|---|
AA_<uuid>_enumerator.m4a |
SurveyCTO audio-audit recording of one enumerator administering the full battery to one child (whole session, not per subtask) | 207 unique recordings (331 Drive entries, 124 are "Copy of" duplicates) |
EGRA_EGMA client N=2233.xlsx |
The SurveyCTO export of the study: one row per child, 1,364 columns — every item response, per-subtask start/end timestamps, 60-second timer state, scores | 2,233 children · 28 enumerators · 99 schools (Rawalpindi EMIS codes) · 27 Apr – 21 May 2026 |
Data.xlsx → Reviews |
The coach QA layer: a reviewer re-listened to an enumerator recording and, per subtask, marked the enumerator's score correct / incorrect / unknown plus protocol compliance flags and comments |
1,462 reviews (874 ASER instrument, 187 EGRA/EGMA, rest unlabelled) by 7 reviewers |
Join keys: unique_id_calc (= emis-section-?-child) is on both the child row and the review row; audio_filename is on the review row. audio/_audio_index.csv has the join done for the 206 recordings that match a child.
| Block | Subtask (column prefix) | Items | Timed | Score fields |
|---|---|---|---|---|
| Urdu | Listening comprehension listcomp_*_urd |
4 literal + 2 inferential | no | numcorrect / percorrect |
| Urdu | Letter identification letterid_*_urd |
100 letters | 60 s | attempted, correct, time_remain, score (clpm) |
| Urdu | Familiar word reading idwrd_*_urd |
50 words | 60 s | attempted, correct, cper |
| Urdu | Oral reading fluency orf_*_urd |
~70-word passage | 60 s | attempted, correct, sentences, time_remain |
| Urdu | Reading comprehension rdcomp_*_urd |
4 literal + 2 inferential | no | numcorrect (gated on ORF: continue_rc_urd) |
| English | Listening comprehension | 4 + 2 | no | |
| English | Letter identification | 100 | 60 s | |
| English | Pseudoword decoding pw_*_eng (English only) |
50 | 60 s | |
| English | Familiar word reading | 50 | 60 s | |
| English | ORF passage + comprehension | 60 s | ||
| Math | Number identification idnummag (identify_1a..4b) |
8 | no | |
| Math | Number representation / discrimination numrep (represent_*) |
8 | no | |
| Math | Basic-level fluency blfluency (blfl1, blfl4 = two timed operations grids, 60 s each) |
~20 each | 60 s | |
| Math | Computation bleffcomp (computation_*) |
8 | no | |
| Math | Word problems wrdpblm |
8 | no | |
| Math | Patterns patterns |
8 | no |
Stop rules are visible as *_firstline_* (zero correct in the first line → discontinue) and no_orf / continue_rc gates.
Whole SurveyCTO form, first tap to submit: median 21.1 min, mean 22.5, p25 17.4, p75 25.9, p90 32.3. First subtask to last subtask (excludes consent/roster): median 18.0 min, p90 26.5.
| Block | Median | Mean | p75 | p90 |
|---|---|---|---|---|
| Urdu reading (5 subtasks) | 5.7 min | 6.2 | 7.2 | 9.2 |
| English reading (6 subtasks) | 6.9 min | 7.5 | 8.6 | 11.5 |
| Math (6 subtasks) | 6.2 min | 6.5 | 7.8 | 9.6 |
Per subtask (seconds): see baseline_subtask_timings.csv. The three timed 60-second items each cost ~68 s wall-clock (timer + instructions). Listening comprehension is the slowest untimed item (~80 s) because the enumerator reads the story aloud twice. The math fluency grid is the single longest subtask (~90 s median).
Recorded audio (209 files): median 22.5 min, mean 24.5, p10 14.3, p90 37.6, max 62. Total 85 hours. All files are AAC, 8 kHz mono, ~12 kbps — SurveyCTO's audio-audit setting. This matters: every pronunciation/phoneme API expects ≥16 kHz, so a coach-recorded WhatsApp voice note (16 kHz Opus) will be a better input than this baseline corpus.
| Today (enumerator, tablet) | Target (coach, phone + models) | |
|---|---|---|
| Per child, full battery | 18–21 min | 5 min |
| Per 5 children | ~105 min | 30 min |
| Compression needed | 3.6–4.2× |
Where the time goes today: 17 subtasks × (instruction script + 60-second timer or item-by-item marking) with the enumerator tapping each item on the tablet. The compression has to come from (a) fewer, shorter, adaptive subtasks, (b) removing item-by-item tapping — the coach just records and the model scores, (c) the timed tasks stopping early on stop-rule hits. It cannot come from speeding up the child.
1,462 human re-listens with per-field verdicts on whether the enumerator's score was right. Median compliance 92%, but 35% of reviews are flagged and comments show the failure modes any AI scorer must also survive: background noise from an uncontrolled class, enumerator prompting or correcting the child, story read once instead of twice, translating English items into Urdu, inappropriate tone. The 187 EGRA/EGMA-instrument reviews plus the 206 matched recordings are the seed of a blind human-anchored evaluation set — the thing the July SpeechAce-vs-Azure study said was missing.
Grades 1–5 roughly equal (43/43/34/33/34), ages 6–14 (mode 10), 23 enumerators, 57 schools. Home language codes: 1, 5, 2 dominate in the full dataset (codebook not in the export — treat as Urdu/Punjabi/Pothwari until confirmed).
download_drive_audio.py — reproduces the downloadbaseline_timings.py → baseline_subtask_timings.csv, audio/_child_index.csvaudio_durations.py → audio/_audio_index.csv (duration + child + reviewer per file)audio/_coach_reviews.csv, audio/_audio_to_child.csv, audio/_drive_manifest.json, audio/_xlsx_headers.jsonmeetings/ — Extended Mind exports of the adjacent meetings (the 15 Sep alignment call itself was not captured by the recorder; nothing exists in the meetings DB between 08:00 and 10:30 UTC that day)