All files

Every document, script and table behind the plan, plus private team links to the recordings and the study export.

Documents

Investigation index
What was asked, what is in the folder, headline findings, open decisions · INVESTIGATION.md

Baseline: the May 2026 study
The corpus, the 21-minute enumerator timing baseline, the QA review layer · BASELINE.md

The evaluation
Every model we tested, one table per subtask, the human floor and the baseline beside every number, and the stack we recommend · harness/HARNESS_RESULTS.md

EGRA and EGMA structure
Subtasks, timing, stop rules, Urdu adaptations, benchmarks, automated-scoring literature · research/01_EGRA_EGMA_structure.md

Model landscape
Every pronunciation, ASR, phone-recognition, audio-LLM and vision option checked, with locale lists and prices · research/02_model_landscape.md

What the main bot already has
The reading pipeline and the live ASER tool, file by file · research/03_current_reading_pipeline.md

The NIETE observe HITL flow
How the AI pre-fill and coach correction work today; hook points; reuse map · research/04_niete_observe_hitl.md

What a recording can yield
Constructs recoverable from a read-aloud beyond WCPM, with evidence and phase · research/05_what_a_recording_can_yield.md

Assessment batteries walkthrough
EGRA, EGMA, ICAN, TIP, ASER, ELANA, DIBELS, MELQO, Literacy Boost, Pakistan incumbents, AI readers · research/06_assessment_batteries_walkthrough.md

Model deep dive
Child-speech ASR, Urdu models, phone/GOP stacks, prosody, audio LLMs, maths ITN, vision; the eval shortlist · research/07_model_deep_dive.md

Probe 1: an audio LLM on a whole session
Gemini 2.5 Pro and Flash segment and score one 17-minute enumerator recording · probes/PROBE1_FINDINGS.md

Probe 2: production STT on the Urdu block
Soniox timestamps, diarisation and the "STT cuts, LLM labels" rule · probes/PROBE2_FINDINGS.md

The TIP diagnostic (LEAPS/CERP)
What the written placement test contains and how it relates to ours · tips/TIP_NOTES.md

Alignment call, 15 Sep 2026
Decisions with timestamps; how the transcript was recovered · meetings/2026-09-15_alignment-call.md

Harness scripts (run in order to reproduce every number)

audio_durations.py — ffprobe every recording, join to the child index + coach reviews, write audio/_audio_index.csv and print baseline stats. (raw)
baseline_timings.py — Baseline: how long enumerators took per subtask and per child, from the SurveyCTO export. (raw)
build_site.py — Build the self-contained GitHub Pages site for the EGRA-in-five-minutes plan into ./site. (raw)
download_drive_audio.py — Download every file in the shared EGRA/EGMA enumerator-recordings Drive folder into ./audio. (raw)
download_tips_folder.py — Download the TIPS assessment Drive folder into ./tips (Google-native files exported). (raw)
harness/01_soniox_batch.py — Harness step 1: Soniox stt-async-v3 (diarised, hints ur+en) over every recording → harness/soniox/<file>.json (raw)
harness/02_ground_truth.py — Harness step 2: extract per-child ground truth from the SurveyCTO export for the 206 matched recordings. (raw)
harness/03_segmenter.py — Harness step 3: rule-based segmenter over Soniox diarised tokens. (raw)
harness/03b_segmenter_llm.py — Harness step 3b: LLM-labelled segmentation on STT timestamps ("STT cuts, LLM labels"). (raw)
harness/04_reference_reconstruct.py — Harness step 4: reconstruct the instrument's reference texts by consensus across children. (raw)
harness/04b_reference_llm.py — Harness step 4b: reconstruct the reference stimuli with an LLM instead of positional voting. (raw)
harness/05_human_floor.py — Harness step 5: the human floor. For every subtask, how often did the QA reviewer (second human, re-listening (raw)
harness/06_reading_score.py — Harness step 6: Urdu + English reading scores from STT alone (the "cheap stack"). (raw)
harness/07_maths_grade.py — Harness step 7: maths from speech. For each child, give a TEXT LLM the diarised transcript of the maths block (raw)
harness/08_soniox_english_pass.py — Harness step 8: second Soniox pass on the ENGLISH block only, language hint 'en' only. (raw)
harness/08b_whisper_english_pass.py — Harness step 8b: English block via OpenAI whisper-1 with language forced to 'en' + word timestamps. (raw)
harness/09_english_azure_speechace.py — Harness step 9: English passage + word list — Azure Pronunciation Assessment vs SpeechAce vs Soniox-diff, (raw)
harness/09b_speechace_eval.py — Re-score the cached SpeechAce responses properly: per-word verdict (quality_score < 60 = wrong) vs the enumerator's (raw)
harness/10_window_scorers.py — Harness round 2, step 10: score the 60-second reading windows with the models we recommended but had not tested. (raw)
harness/11_comprehension_grade.py — Harness round 2, step 11: grade the Urdu reading-comprehension answers with text LLMs. (raw)
harness/12_round2_summary.py — Harness round 2, step 12: one table per experiment with the human floor and the Levenshtein baseline beside every (raw)
harness/13_round2_charts.py — Harness round 2, step 13: house-style charts of the round-2 results for the board post and the site. (raw)
harness/14_worked_examples.py — Harness step 14: worked examples — one page per scored window showing every artefact behind the score. (raw)
harness/15_evaluation_doc.py — Harness step 15: write the ONE canonical evaluation record (harness/HARNESS_RESULTS.md) from every cached result, (raw)
harness/_chain_after_segmenter.sh — (raw)
harness/_notion_flow_section.py — One-off: swap the 'The checking screen' section on PROJ-068 for the accurate WhatsApp Flow design. (raw)
harness/_patch_plan_eval.py — One-off: rewrite PLAN.html's evaluation and stack sections into the single combined record. (raw)
harness/_patch_plan_flow.py — One-off: replace the invented checking-screen mockup with the accurate WhatsApp Flow design. (raw)
harness/_patch_speechace.py — One-off: fold the corrected SpeechAce numbers and the Azure-vs-SpeechAce recommendation into the record. (raw)
harness/_r2_audio.sh — (raw)
harness/_r2_scale.sh — (raw)
harness/_r2_seg_maths.sh — (raw)
harness/_r2_text.sh — (raw)
probes/probe1_segment.py — Probe 1: can an audio LLM segment an enumerator EGRA session into subtasks and extract the child's responses? (raw)
probes/probe2_soniox.py — Probe 2: our production STT (Soniox stt-async-v3, Urdu hint, diarization) on the Urdu letter-ID slice. Run: uv run --with requests python probe2_soniox.py <wav> <lang> (raw)

Result tables

baseline_subtask_timings.csv — Per-subtask enumerator timings (seconds) (csv)
harness/human_floor.csv — Human floor: reviewer agreement with the enumerator per subtask (csv)
harness/reading_summary.csv — Reading counts: machine vs enumerator per subtask, with window-failure split (csv)
harness/round2_windows.csv — Sixty-second windows: every audio and text model vs the enumerator, with baseline and human floor (csv)
harness/round2_comprehension.csv — Comprehension graders vs the enumerator (csv)
harness/round2_segmenter.csv — Segmenter labellers vs the tablet timestamps (csv)
harness/round2_maths.csv — Maths graders vs the enumerator, by subtask (csv)
harness/window_scores.csv — Per-window rows behind the sixty-second-window table (file ids only) (csv)
harness/comprehension_scores.csv — Per-child rows behind the comprehension table (csv)
harness/reading_scores.csv — Reading counts per recording window (file ids only) (csv)
harness/reading_items.csv — Per-word machine vs enumerator flags (subtask, machine_wrong, enum_wrong) (csv)
harness/segmenter_llm_google_gemini-2.5-pro.csv — Segmenter agreement with tablet timestamps per recording (csv)
harness/maths_scores_google_gemini-2.5-pro.csv — Maths from speech: machine vs enumerator counts per recording (csv)
harness/english_results.csv — English windows: SpeechAce and STT-diff vs enumerator per recording (csv)
harness/segmenter_results.csv — Rule-based segmenter (v1) results (csv)

The checking form

egra-review-flow.json: the coach's checking screens as a WhatsApp Flow JSON draft (version 7.0, endpoint data model), built on the same pattern as the Islamabad observation form. The three screen pictures in the plan are rendered from this file's example data.

Worked examples

Twelve real windows, each showing the printed text, the clip (Drive, team accounts), the transcript, the enumerator's marks and every model's per-word verdicts side by side.

Private team data (Google Drive, Taleemabad accounts only)

The May 2026 study folder — the 207 enumerator recordings (AA_<uuid>_enumerator.m4a), the SurveyCTO export EGRA_EGMA client N=2233.xlsx (one row per child, 1,364 columns), and the reviewer workbook Data.xlsx (1,462 QA re-listens). These hold child data and are not copied into this repository.
The TIP assessments folder — the LEAPS/CERP written diagnostic PDFs (English/Urdu all classes; maths Class 1–5). The read-through is in the TIP notes above.
The working folder with every intermediate (transcripts, per-child JSON, reconstructed stimuli, the full alignment-call transcript): 06_Logs & Misc/Reports/Active/EGRA Student Assessment - Sep 2026/ in the Rumi workspace.