Subtasks, timing, stop rules, Urdu adaptations, benchmarks, automated-scoring literature
Purpose: reference document for the AI-assisted, coach-administered EGRA (English + Urdu) and EGMA design. Target: whole battery per child in ~5 min, 5 children in 30 min. Prepared: 2026-09-15. Every factual claim carries a source URL. Items not confirmed against a primary source are marked unverified.
Primary sources used (read directly, text-extracted): RTI EGRA Toolkit, Second Edition (USAID, 2016) — https://ierc-publicfiles.s3.amazonaws.com/public/resources/EGRA%20Toolkit%20V2%202016.pdf ("Toolkit"); RTI EGMA Toolkit (USAID, March 2014) — https://ierc-publicfiles.s3.amazonaws.com/public/resources/EGMA%20Toolkit_March2014.pdf ("EGMA Toolkit"); Dubeck & Gove (2015), The EGRA: its theoretical foundation, purpose, and limitations, IJED — http://aserpakistan.org/document/learning_resources/2015/Dubeck_Gove_IJED_2015_EGRA_article.pdf; Sindh SELECT EGRA Baseline Assessment Report (2024, Grade 2, Sindhi + Urdu) — https://rsusindh.gov.pk/contents/SELECT/EGRA_Baseline_Assessment_Report_Sindh.pdf ("Sindh 2024"); ACR-Asia EGRA Benchmarks and Standards Research Report (RTI, Dec 2017) — https://ierc-publicfiles.s3.amazonaws.com/public/resources/Asia%20ACR%20Benchmarking%20Report_FINAL%20REVISED_21%20Dec%2017.pdf ("ACR-Asia"); Tanzania National EGRA 2016 Final Findings Report (ABE-ACR) — https://ierc-publicfiles.s3.amazonaws.com/public/resources/FdngsRpt_TZNtnlEGRA_08-Nov2016_FNL.pdf ("TZ 2016").
The Toolkit's common instrument for letter-based orthographies has six subtasks, four of them core: (1) listening comprehension, (2) letter identification (names or sounds), (3) nonword reading, (4) oral reading fluency (ORF) with reading comprehension. The two "most often included" additions are phonemic awareness (initial-sound identification) and familiar-word reading. Dictation, maze and cloze are optional; vocabulary is optional and untimed and "has not yet been through the same expert-panel review and validation process as the other subtasks" (Toolkit pp. 39–41). The Toolkit recommends keeping the count "around six" to control fatigue.
Subtask_per_minute = Subtask_score / (Time given for subtask − subtask_time_remain) × 60
where Subtask_score = items correct, Time given is normally 60 s, and subtask_time_remain is seconds left if the child finished early (0 if timed out). Only children who finish early have a rate different from their raw count. Units: clpm (letter names), clspm (letter sounds), cnwpm (nonwords), cwpm (familiar words or passage), csspm (syllables).
| # | Subtask | What the child sees / hears | Timed | Items | Stop rule | Score | Source |
|---|---|---|---|---|---|---|---|
| 1 | Listening comprehension (core) | Assessor reads a ~30-word locally familiar passage aloud once; child may not see it; 3–5 questions (mostly literal, 1–2 inferential; no yes/no items); child may answer in any language | No | 3–5 Q | none (10 s no-response per Q) | correct / total (%) | Toolkit p. 42; protocol |
| 2a | Letter-sound identification (core variant) | Grid of 100 letters (10 rows × 10), random order, upper+lower case, frequency-weighted; child says each letter's sound, pointing as they go; 2 practice items | 60 s | 100 | no correct in first line (10) | clspm; also attempted, incorrect, time remaining | Toolkit pp. 45–48 |
| 2b | Letter-name identification (alternative) | Same 100-letter grid; child says names. Often hits ceiling by G2 in many countries | 60 s | 100 | first 10 | clpm | Toolkit p. 48 |
| 3 | Initial-sound identification (phonemic awareness; common) | Oral only: assessor says a word twice ("What is the first sound in mouse?"); 10 words from G1–2 word lists. Alternative design: 10 sets of 3 words, child names the one with a different initial sound | No | 10 | none in Toolkit script (Dubeck & Gove: stop if 0 in first 5) | correct / 10 | Toolkit p. 55; D&G Table 1 |
| 4 | Nonword (invented-word) reading (core) | Grid of 50 one- and two-syllable pronounceable nonwords, 5 per row, legal orthographic patterns, no homophones of real words, clear well-spaced font | 60 s | 50 | first line (5) | cnwpm | Toolkit p. 49 |
| 5 | Familiar-word reading (common) | Grid of 50 high-frequency grade-level words (from textbooks/word lists), 5 per row, random order, lowercase | 60 s | 50 | first line (5) | cwpm | Toolkit p. 56 |
| 6 | Oral reading fluency (passage) (core) | Grade-level narrative of about 60 words, familiar setting, one-way plot; child reads aloud; assessor marks incorrect words; passage removed before questions | 60 s | ~60 words | no correct word in first line | cwpm + attempted + time remaining | Toolkit pp. 50–52 |
| 7 | Reading comprehension (core, paired with 6) | 3–5 questions, literal + ≥1 inferential; only questions covering text the child actually read are asked (software prompts this); answers may be in any language; 10 s no-response | No | 3–5 | not asked past point reached | correct / asked, and / total | Toolkit p. 51; protocol |
| 8 | Dictation (optional) | Assessor reads a sentence once whole, then in chunks (~1 word/s), waits 15 s, reads again; child writes on a lined form; scored per word (partial credit possible) plus capitals/punctuation. "Proven challenging to score in a consistent manner" | No (15 s pauses) | 1 sentence | none | words correct (partial) | Toolkit p. 41; protocol; D&G |
| 9 | Syllable identification (for syllabic orthographies e.g. Kiswahili) | Grid of 50–100 syllables | 60 s | 50–100 | first 10 | csspm | D&G Table 1; TZ 2016 Table 1 |
| 10 | Phoneme/syllable segmentation (optional) | Oral; child says the sounds of a spoken word; 10 items; used when initial-sound ceilings | No | 10 | 0 in first 5 | correct / 10 | Toolkit p. 41; D&G |
| 11 | Vocabulary (optional, unvalidated) | Receptive: point to body parts / objects / spatial relations | No | ~10–20 | none | correct | Toolkit p. 41 |
| 12 | Orientation to print (optional) | Directionality, where to start | No | ~3 | none | correct | D&G Table 1 |
| 13 | Maze / cloze (optional) | Choose the word completing a sentence | No | — | none | correct | Toolkit p. 41 |
EGRA 2.0 (2016–17): RTI piloted four additional subtasks in Accra (spelling, oral language / vocabulary, silent reading comprehension, etc.); the English versions related strongly to existing comprehension measures, the Akuapem Twi versions did not perform well — https://shared.rti.org/content/development-and-pilot-testing-additional-subtasks-early-grade-reading-assessment-egra-20 and brief https://ierc-publicfiles.s3.amazonaws.com/public/resources/10745_EGRA_Subtasks_brief_02-15-17_r5.pdf. Exact subtask list not extracted here (unverified).
Every subtask follows the same script skeleton in the Toolkit annex: (a) show stimulus / read directions; (b) two worked practice items with corrective feedback ("Good, the sound of this letter is /s/" / "The sound of this letter is /s/"); (c) "When I say Begin, start here [point], go across the page, point to each letter and tell me…"; (d) start timer on first response; (e) mark with slash, bracket after last item at 60 s, record time remaining; (f) early-stop line. Prompts such as "Please tell me the sound, not the name" may be given only once. Full Timor-Leste (Tetum) and English scripts are reproduced in the Toolkit annex.
The Toolkit gives no per-subtask timings, only totals (see §3). Working estimate used in this document: each timed grid ≈ 60 s test + 30–45 s instructions/practice ≈ 90–105 s; ORF+comprehension ≈ 2–2.5 min; listening comprehension ≈ 1.5 min; initial sounds ≈ 1–1.5 min; dictation ≈ 2 min. These are estimates, not Toolkit figures.
Core EGMA = six subtests (eight instruments because addition/subtraction each have Level 1 and Level 2). All are orally administered; "no student writing is required. Answers to all items are orally transmitted" — but paper/pencil and counters are available and the assessor records the strategy used (EGMA Toolkit pp. 12, 20–24). For untimed subtests, the assessor prompts the child to move on after 5 s of no response and covers not-yet-reached items with a sheet of paper. Children are reminded to point at the item they are answering so the assessor can align tablet and stimulus. Approximate administration length: 20 minutes, realistic field planning 25 min per child including swap-over (EGMA Toolkit pp. 12, 50).
| Subtest | Stimulus & response | Timed | Items | Stop rule | Score | Response modality (capture implication) |
|---|---|---|---|---|---|---|
| Number identification | 20 numerals in a 4-row grid: 3 single-digit (incl. 0 and 9), 12 two-digit (10–99), 5 three-digit (100–999); child says each number aloud | 60 s | 20 | none | correct per minute (rate) | Audio (child points + says) |
| Number discrimination (quantity comparison) | 10 pairs; child says which number is larger. 1 single-digit pair, 5 two-digit, 4 three-digit. "Pointing at the correct number is insufficient evidence" — child must say the number | No | 10 | 4 successive errors | correct / 10 | Audio (spoken numeral); pointing alone not accepted |
| Missing number | 10 items, four horizontal boxes with one empty; 8 increasing, 2 decreasing; steps of 1, 2, 5, 10, 100; child says the missing number | No | 10 | 4 successive errors | correct / 10 | Audio |
| Addition Level 1 | 20 items increasing in difficulty; addends ≤10, sums ≤19; presented on stimulus sheet; child answers orally; assessor notes strategy (fingers/tick marks, paper-pencil, mental) | 60 s | 20 | none | correct per minute | Audio + observed strategy (vision or assessor tap) |
| Subtraction Level 1 | Inverse of Addition L1 items | 60 s | 20 | none | correct per minute | Audio + strategy |
| Addition Level 2 | 5 items, two-digit, increasing difficulty; paper & pencil and counters available | No | 5 | 4 successive errors | correct / 5 | Audio; optional written working (image) |
| Subtraction Level 2 | 5 items, inverse | No | 5 | 4 successive errors | correct / 5 | Audio; optional written working |
| Word problems | 6 items read aloud by the assessor (to avoid confounding with reading); 3 mirror A/S L1 items; counters, pens and paper provided; strategy recorded | No | 6 | 4 successive errors | correct / 6 | Audio (assessor speech + child answer); counters (vision) |
Sources: EGMA Toolkit §2.4.1–2.4.6 (pp. 20–24) and §3 (scoring: timed subtests reported as rate, untimed as proportion correct; results "are not interpreted in reference to a criterion or normative range").
Older/expanded EGMA variants (EGMA Toolkit pp. 25–27): the 2008–09 "EGMA 5" and pilot versions included Counting fluency (60 s), One-to-one correspondence, Shape recognition, Shape attributes, Pattern/number extension, Multiplication/Division; the pilot "took much longer than anticipated (well over 20 minutes)", which is why the Core EGMA was cut to the six subtests above. Shape/pattern items are therefore not in the 2014 Core; some country instruments retain them. Tanzania's national EGMA (2016) used five subtests: A/S Level 1 (timed), A/S Level 2, Missing Number, Word Problems (TZ 2016 p. 18).
| Instrument | Time per child | Source |
|---|---|---|
| Full EGRA (≈6 subtasks) | "about 15 to 20 minutes per child"; a team of 3 assessors ≈ 9–10 instruments/hour, ~30 children in 3 h | Toolkit §9.2, p. 96 |
| EGRA, subtask-dependent | "the entire battery could be administered in 10–20 min with the assessor actively aiming to maintain the child's attention" | Dubeck & Gove 2015 |
| Kiswahili national EGRA, 5 subtasks (syllables, nonwords, ORF, comprehension, dictation) | "approximately 10 to 15 minutes" | TZ 2016 p. 15 |
| Core EGMA | "approximately 20 minutes"; plan 25 min incl. swap-over | EGMA Toolkit pp. 12, 50 |
| ASER reading + math (India) | "about 10 minutes" for both tools | Vagh 2012 (ASER validation) |
| Pratham/J-PAL Fluency Battery (8 EGRA/DIBELS-style subtests, Hindi) | "about 10 minutes" | Vagh 2012 |
| Tanzania self-administered SA-EGRA (tablet, no ORF) | mean 51.8 ± 18.2 min at time 2 — "still very long" | TZ SA-EGRMA 2025 p. 10, https://www.imagineworldwide.org/wp-content/uploads/Tanzania_Kiswahili_SA-EGRMA_Final-Pilot-Report_2025.pdf |
Abbreviated / core subsets in practice - Toolkit "core four": listening comprehension, letter identification, nonword reading, ORF + comprehension (Toolkit p. 39). - Sindh 2024 (Grade 2, N = 7,248): five subtasks — phoneme awareness (10 untimed), nonword reading (60 s), expressive vocabulary (10 pictures, untimed), passage reading (60 s) + 5 questions, listening comprehension (3 questions). Letter naming (ceiling), syllables, phoneme segmentation (floor), familiar words, dictation and maze were deliberately dropped (Sindh 2024 pp. 6–7). - Tanzania national: 5 Kiswahili subtasks, 10–15 min (above). - Kenya Tusome / Tanzania benchmarking use only ORF + comprehension (and nonwords) as "benchmark subtasks" (TZ 2016 Table 5; ACR-Asia p. 17). - ASER (Pakistan/India) is the shortest proxy (see §4.4): 5 letters, 5 words, one paragraph, one story; ~10 min with math.
Implication for the 5-minute target: timed EGRA subtasks alone are 60 s each; three timed reading grids + ORF + comprehension + a two-item EGMA timed pair already exceed 5 min before instructions. A 5-min battery must be a subset (e.g., ORF+comprehension, one decoding grid, Number ID, A/S Level 1) or use shortened time windows — which breaks comparability with published cwpm benchmarks unless re-normed.
Set under the USAID project (PRP) by grade and language, three bands (ACR-Asia Table 2, p. 16):
| Grade | Urdu: does not meet / meets / exceeds | Sindhi: does not meet / meets / exceeds |
|---|---|---|
| 1 | <30 / 30–60 / >60 cwpm | <30 / 30–50 / >50 |
| 2 | <60 / 60–90 / >90 | <50 / 50–80 / >80 |
| 3 | <70 / 70–100 / >100 | <60 / 60–90 / >90 |
At the time of the ACR-Asia compilation only 20% of Urdu and 24% of Sindhi students met the grade-2 benchmark (ACR-Asia Table 5, p. 18). Sindh 2024 uses the same "50–80 cwpm" standard for Sindhi and quotes it for Urdu too; comprehension standard = 4 of 5 questions. Note these Urdu targets (60–90) are far above the 40–50 cwpm typical of Latin-script benchmarks; ACR-Asia (p. 8, citing Abadzi 2011) argues unvowelled scripts (Arabic, Farsi, Pashto, Urdu) may need a higher speed to comprehend, because readers hold alternative pronunciations in working memory.
Tanzania's 2014 national Standard-2 benchmarks: ORF 50 cwpm, nonwords 40 cnwpm, reading comprehension 80%; EGMA: A/S Level 2 80%, Missing Number 60% (TZ 2016 Table 5). 2016 national results (Std 2, 650 schools): ORF mean 23.6 cwpm (2013: 17.9), 16.1% zero; nonwords 15.7 cnwpm, 17.9% zero; syllables 39.9 per min; 5.3% "proficient" (met both ORF and comprehension benchmarks); proficient readers averaged 57.9 cwpm vs 18.8 for beginners (TZ 2016 pp. ii–iv, 31). Kenya's Tusome set two Kiswahili benchmarks, 17 cwpm (emergent) and 45 cwpm (fluent, 75% comprehension) (ACR-Asia p. 17). Kiswahili instruments use a syllable grid instead of letters, 5 comprehension questions (4 literal + 1 inferential), and dictation (TZ 2016 Table 1).
Open-source (GPL), offline-first Android/web app launched 2011 by RTI (Carmen Strigel, Amber Gove) to replace paper EGRA; used in 65+ countries; management transferred to Tangerine Central (women-owned) in 2025 (https://www.tangerinecentral.org/about; https://github.com/Tangerine-Community/Tangerine; https://www.rti.org/impact/tangerine-mobile-learning-assessments-made-easy). It implements the Toolkit protocol: assessor taps incorrect items, timer with red flash at 60 s, automatic time-remaining capture, auto-stop prompts, and prompts to ask only comprehension questions covered by text read (Toolkit protocol annex, tablet column). It does not score speech; a human still marks every item. Used in Sindh 2024 and Tanzania 2016.
Tablet, child-driven, no assessor scoring. Ghana English pilot (Sept–Oct 2022, 279 G2 pupils): "the tool does not currently have speech recognition capacity", so ORF is proxied by the Spelling subtask or a composite (https://shared.rti.org/content/report-self-administered-egraegma-pilot-ghana-english; https://www.tangerinecentral.org/impact/egra-egma-ghana-2022-analysis). Tanzania Kiswahili pilot (2025, 600 pupils G2–4, 15 schools): SA-EGRA/SA-EGMA "do not assess oral reading fluency … nor numeric automaticity", correlations with traditional EGRA/EGMA "reduced but generally acceptable" (threshold r ≥ .5), Cronbach α .87/.90, mean duration 51.8 min (TZ SA-EGRMA 2025 pp. ES-1, 10). Lesson: without ASR, the fluency construct is lost and the child-driven format is slow.
| Study | Language / setting | Task | Model | Result vs humans | Failure modes |
|---|---|---|---|---|---|
| Henkel et al. 2023/2025 (Oxford/Jacobs), https://arxiv.org/pdf/2310.17606 ; journal https://eric.ed.gov/?id=EJ1461166 | Ghana, 162 students aged 13–18, PrePIRLS ~100-word passages, classroom recordings, English L2 with regional accents | ORF (cwpm) | Whisper V2 out of the box, no fine-tuning; wav2vec2 | WER 13.5 (preprint; 10.3 in journal version) vs 25.2 for wav2vec2; automated ORF r = 0.96 (0.98 journal) with expert raters | wav2vec2 transcribes accent phonetically; ambient noise; not tested on early-grade children |
| Chevtchenko et al. 2025 (EGRA-Xhosa-14.9k), https://arxiv.org/pdf/2505.17371 | South Africa, grades 1–4, 14,971 recordings (~4 s each) of 10 EGRA letters/words, app-recorded in classrooms, 3 markers each | Letter-sound & word correct/incorrect | Fine-tuned wav2vec2, HuBERT, Whisper as binary classifiers | Diagnostic efficiency ≈ 91–92%, FPR ~3–5%, FNR ~3–5% with 200–300 samples per class; markers agreed with an expert EGRA reviewer only 71% (85% on consensus subset) | Off-the-shelf multilingual ASR (MMS-1B) matched only 6.9% of items; short sounds (hl, ng, v, n) <1% — "direct multilingual ASR usage in classroom conditions may be less effective for short word- or letter-level inputs" |
| Ardington / UCT, Gates Grand Challenges + EdTech Hub, https://gcgh.grandchallenges.org/grant/automating-early-grade-reading-assessments-egra-african-languages-using-voice-recognition-ai ; https://docs.edtechhub.org/lib/HYK9N7TW | Sepedi, isiXhosa; ~3,000 G2–3 children; validating EGRA-AI inside a 120-school trial | Letter sounds, words, passage, comprehension | wav2vec → IPA transcription → marking models | Design described; no accuracy results published in accessible sources | — |
| Zhang et al. 2025 (Interspeech), https://arxiv.org/pdf/2505.19671 | Singapore, Tamil & Malay children, classroom recordings, 27-sentence reading + picture Q&A | Fluency level (3-class) | Whisper fine-tuned on child speech + augmentation; metrics (WER/CER/PER, pauses, rate) → GPT meta-evaluator | Base Whisper WER 56–86% on children; after fine-tuning 7.5% (Malay), 24–40% (Tamil); fluency accuracy 0.91 Malay, 0.75 Tamil | Very-low-resource language (Tamil) stays poor; audio-LLM (gpt-4o-audio) worse than pipeline |
| Diarra et al. 2026 (RobotsMali), https://arxiv.org/pdf/2606.31508 | Mali, Bambara, 60 children aged 9–18, 55 h phone-app recordings | Reading ASR benchmark | Fast-Conformer (Soloni) vs QuartzNet, on-device (<300 ms per 8 s) | WER 0.42 → 0.22 after fine-tuning | "Children under 10 remain the main source of residual errors" |
| Molenaar et al. 2023 (Interspeech), https://arxiv.org/pdf/2306.03444 | Dutch, 71 children aged 7–11 (JASMIN) | Word-level accuracy | Kaldi (prompt-aware LMs) vs Whisper | Best MCC .63 with humans; LM containing reading errors helps | ASR over-flags errors; skewed classes |
| SERDA validation 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC12686063/ | Dutch, 653 children G2–3, 176 h | Word lists + passages | ASR + IRT | Word-level MCC .43 (passage .55); sensitivity .93 but precision .31; speed r = .57–.61 with human timing | "ASR somewhat overestimates the number of errors"; adult-trained ASR |
| Nese & Kamata 2020 (ERIC ED607983), https://eric.ed.gov/?id=ED607983 | US English, N = 650, grades 2–4, CORE/easyCBM passages | ORF WCPM | Custom ASR | Word-score agreement significantly lower for students with disabilities; WCPM differences not exacerbated for SWD/EL when aggregated to passage level | Word-level errors for atypical speakers |
| Vaidya, Sahoo & Rao 2024 (IIT Bombay), https://arxiv.org/pdf/2405.19426 | India, children 10–14, English L2, 50–70-word passages rated 0–5 by teachers | Holistic fluency incl. prosody | wav2vec2 end-to-end | Prior hand-crafted system r = .794 with manual scores; wav2vec2 variants comparable (exact figure not extracted) | WCPM "necessary but not sufficient" |
| SoapBox Labs (Interspeech 2020; MetaMetrics), https://www.isca-archive.org/interspeech_2020/kelly20b_interspeech.html ; https://metametricsinc.com/wp-content/uploads/2022/05/SoapBox-MetaMetrics-Webinar-QA.pdf | English, child-specific ASR, commercial | ORF | Proprietary | "very high correlation … no meaningful difference" vs human annotators (vendor claim); a widely quoted "96%" figure is unverified | — |
| Amira Learning, https://explore.amiralearning.com/hubfs/Amira-Speech-Recognition-Contributes-to-Reading-Development-for-Young-Learners-of-English.pdf | English, US classrooms | ORF vs DIBELS | Proprietary (Project LISTEN lineage) | Gains study, not an accuracy-vs-human study | — |
| Google Read Along / Bolo, https://readalong.google/impact/ | Hindi, English, Urdu among 10+ languages; on-device ASR | Reading tutor; "beginner <45 cwpm" internal metric | Proprietary | 64% of India-pilot children improved on ASER; internal claim beginners' ORF 38%→88% after 100 min (https://blog.google/intl/en-in/products/google-companies/reinforcing-our-commitment-to/); no published ASR-vs-human accuracy | — |
| Curious Learning | Feed the Monster / Curious Reader apps; no published ASR scoring evaluation found (https://www.curiouslearning.org/resources) | — | — | — | — |
| Kaldi-based Kiswahili ORF scoring | Not found in this session's searches; Kiswahili ASR work exists for adults (https://arxiv.org/pdf/2210.16537) — unverified for children/EGRA | — | — | — | — |
Urdu-specific ASR baseline: Whisper's training set contains only ~104 h of Urdu; small models are a "tipping point" for usability; Urdu ASR is hampered by dialect diversity and Urdu–English code-switching (survey, https://arxiv.org/pdf/2411.14493). Child Arabic (a related script/phonology) shows Whisper large-v3 WER 66%, 4.1× its adult WER (https://arxiv.org/pdf/2510.23319). No Urdu child-speech reading corpus was found — the Xhosa recipe (record every item as a ~4 s clip, triple-mark, fine-tune a classifier) is the closest published path.
Assessor subtask score (%) = agreements with Gold Standard / items in subtask; item-level agreement across assessors is also computed; for timed subtasks an assessor is "in agreement" on time-remaining if within 2 seconds of the gold standard; overall percent agreement = mean of subtask and time-remaining agreements. Report by assessor, by subtask, and overall.Implications for validating an AI scorer 1. The AI must be treated as one more assessor: score the same gold-standard scripted child performances and the same field double-scored children, and report percent agreement per subtask, per item, and on time-remaining (±2 s) in the Toolkit's Exhibit J format — this is the format ministry and donor teams already accept. 2. Pass bar: ≥90% item agreement with the gold standard per subtask (the enumerator bar), and field agreement in the 88–98% band observed for humans; for cwpm, agreement within the noise of human timing (±2 s ≈ ±3–4% of a 60 s window) and a mean bias near zero (SERDA-type error over-detection would fail a bias check). 3. Build the gold standard the way RTI does — scripted, error-laden performances, not only clean readings — so the AI is tested on hesitations, self-corrections, letter names given for sounds, skipped lines and auto-stop cases. 4. Multiple human markers disagree substantially at letter level (Xhosa: 71% expert agreement on all clips, 85% on consensus); use consensus-of-three labels for training/validation and report agreement against an expert EGRA reviewer, not against a single marker. 5. Keep the coach-in-the-loop marking as the primary record until the AI meets the bar per subtask and per language; publish disagreement rates by student subgroup (rural/urban, gender, weak readers) since ASR degrades on atypical speakers (Nese & Kamata).
Modality = what the phone must capture to score the item. Audio = child's speech (plus assessor prompts); Image = photo of the child's written/ worked output or of stimulus/counters; Tap = coach marks on screen (Tangerine-style) as the reference/fallback. Seconds are working estimates (test time + instructions/practice); Toolkit/EGMA Toolkit give only totals.
| Subtask | Language | Input modality | Timed? | Stop rule | Score produced | Est. s |
|---|---|---|---|---|---|---|
| Listening comprehension | English / Urdu | Audio (assessor reads ~30-word story; child answers orally) + Tap | No; 10 s per Q | none | correct / 3–5 (%) | 90 |
| Letter-sound ID (100 grid) | English | Audio + Tap (fallback) | 60 s | 0 correct in first 10 | clspm | 100 |
| Letter-sound / letter-name ID (isolated forms, RTL grid) | Urdu | Audio + Tap; Urdu variant choice (name vs sound; aerab) unverified | 60 s | 0 correct in first 10 | clspm / clpm | 100 |
| Initial-sound identification (10 words) | English / Urdu | Audio (oral only; no stimulus) | No; 3 s no-response | 0 in first 5 (D&G) | correct / 10 | 75 |
| Nonword reading (50 grid) | English / Urdu | Audio + Tap | 60 s | 0 correct in first 5 | cnwpm | 100 |
| Familiar-word reading (50 grid) | English / Urdu (dropped in Sindh 2024) | Audio + Tap | 60 s | 0 correct in first 5 | cwpm | 100 |
| Oral reading fluency (~60-word passage) | English / Urdu | Audio + Tap | 60 s | 0 correct in first line | cwpm, words attempted, time remaining | 105 |
| Reading comprehension (3–5 Q, only text reached) | English / Urdu | Audio (spoken answers; any language) + Tap | No; 10 s per Q | not asked beyond text read | correct / asked; / total | 60 |
| Dictation (1 sentence) | English / Urdu (optional) | Image of child's writing (+ audio of assessor) | No; 15 s pauses | none | words correct (partial credit) | 120 |
| Expressive vocabulary (10 pictures; Sindh 2024) | Urdu | Audio (child names picture) | No | none | correct / 10 | 60 |
| Number identification (20 items, 0–999) | Math | Audio (child says numeral) + Tap | 60 s | none | correct per minute | 90 |
| Number discrimination (10 pairs) | Math | Audio — child must say the larger number; pointing alone insufficient | No; 5 s prompt | 4 successive errors | correct / 10 | 75 |
| Missing number (10 items) | Math | Audio + Tap | No; 5 s prompt | 4 successive errors | correct / 10 | 100 |
| Addition Level 1 (20) | Math | Audio + strategy observation (Tap or Image of fingers/paper) | 60 s | none | correct per minute | 90 |
| Subtraction Level 1 (20) | Math | Audio + strategy | 60 s | none | correct per minute | 90 |
| Addition Level 2 (5) | Math | Audio; optional Image of written working | No; 5 s prompt | 4 successive errors | correct / 5 | 90 |
| Subtraction Level 2 (5) | Math | Audio; optional Image | No | 4 successive errors | correct / 5 | 90 |
| Word problems (6, read aloud by assessor; counters + paper) | Math | Audio (assessor + child) + Tap; counters visible (Image optional) | No; 5 s prompt | 4 successive errors | correct / 6 | 150 |
| Shape recognition / pattern extension | Math (non-core; pre-2014 EGMA) | Tap / pointing (child points to shapes) | No | per-sheet stop at first error (2009 pilot) | correct | 90 |
Totals implied: full EGRA (7 rows, no dictation) ≈ 10.5 min; full core EGMA (8 rows) ≈ 13 min — consistent with the toolkits' 15–20 + 20 min once transitions are added, and 2–4× the 5-minute target.
score / (60 − time_remaining) × 60; the app must capture time-remaining to the second, and the human/AI must be within ±2 s of a gold standard to be "in agreement".