← EGRA Test Build Review · Plan conformance and coach experience

EGRA/EGMA staging build: plan conformance and coach experience

Review lens: product lead with EGRA/EGMA background, checking the build against our plan and against the coach's day. Reviewed: 27 Sep 2026 · bead bd-ak1go · read-only. What was built: the frozen snapshot in ../src/ (8 commits, 783116cc → e81dbc03). File references below are relative to ../src/. The plan it should serve: ../../PLAN.html (sections named by their ids, e.g. #battery), ../../meetings/2026-09-15_alignment-call.md, ../../harness/HARNESS_RESULTS.md §8, ../../research/04_niete_observe_hitl.md, ../../harness/flow/egra-review-flow.json, ../../BASELINE.md. Live evidence, also read-only: the shared Supabase schema and rows, queried through the analyst role, and the staging bot's logs in Axiom (digital-coach-logs, service digital-coach-staging, 17 to 24 Sep). The exact queries are in the appendix.


The short version

  1. This build puts the May 2026 enumerator test on WhatsApp almost in full. It does not build the plan's five-minute test. It keeps 14 of the 17 May subtests. Those 14 took an enumerator about 14 minutes of a 21-minute session. On WhatsApp my estimate is about 22 minutes per child (range 18 to 25) and about 1 hour 50 minutes for five children. The target is 5 minutes and 30 minutes, so the build runs 3 to 4 times over and saves no time against today's enumerator.
  2. The subtests are close to the reverse of the plan's list. The build keeps what the plan drops: listening comprehension, the 100-letter grids, the 50-word list and patterns. It leaves out what the plan keeps: made-up words, first sounds, reading numbers aloud, one-minute quick sums, and the photo of written sums. The Urdu and English stories and their comprehension questions are the only overlap.
  3. The coach marks every item, one WhatsApp message per item. That is 64 Correct/Incorrect taps plus 10 typed answers per child. On 11 Sep the field team rejected exactly this one-question-per-message design, and PLAN.html#today says the plan removes it. The AI does nothing. The new table even has a database rule (a check constraint) that stops it from ever storing an AI score.
  4. Audio is recorded for the listening-comprehension questions only, the one subtest the plan drops, and only after the child has already answered. The timed reading minutes, which the evaluation showed the AI can count to within 5 words, are never recorded.
  5. Things are broken on staging right now, and TESTING.md does not say so. Every stimulus picture sits on a private storage address that returns an error. All 3 live attempts to send the passage picture failed (19, 21 and 24 Sep). The letter, word and story grids use the same address, so the coach would get a "Ready? ▶️ Start" button with nothing to show the child. - The listening-comprehension Flow is registered on staging. - The database columns that would hold the voice note were never added. So if a coach finishes that subtest with a voice note, saving the row fails and the coach's own marks are lost.
  6. No child has been tested yet. Across 9 staging sessions: - 1 subtest was ever saved. - 0 voice notes were captured. - 0 sessions finished. - The Flow was "completed" 10 to 15 seconds after the child was picked. That is a tester tapping through; a real passage read twice would take far longer.
  7. Children are only partly random. The coach picks the class, and each tap draws one child with replacement. Across five draws from a class of 9 there is a 74% chance of drawing someone twice. Cancelling and re-drawing is a way to choose the child. The child's database ID is never saved, so repeat tests of the same child cannot be linked, and linking them over time is the whole point of testing often.
  8. Some pieces are worth keeping: - the switch-plus-region gate - the real item bank with answer keys - stimulus pictures drawn by code, not by AI, with a correct Nastaliq font - picking the child at random on the server from the real roster - uploading audio to our own storage straight away - the careful routing and "never hijack" checks - the chat-loop fix (bd-eg7st)

These are good parts to build the plan's version from.


0. What actually ran on staging (live evidence)

TESTING.md was last updated 19 Sep. The live system has moved on since then.

Claim in TESTING.md What is live Evidence
"The database migration has NOT been applied anywhere" (TESTING.md:221-227) egra_egma_ict_test_attempts exists on the shared database with the base migration's 23 columns and 1 row information_schema.columns query, 27 Sep
"Meta Flow registration: NOT done" and "no coach sees any different behaviour today" (TESTING.md:153-161, :239-244) The Flow is registered and the Flow ID is set on staging. Axiom logs listening_flow_completed on 18 Sep 13:55, 19 Sep 04:49, 21 Sep 18:57 and 24 Sep 07:06 Axiom, service digital-coach-staging
The voice-note migration is an optional extra step (TESTING.md:144-151) Not applied: the live table has no audio_url, audio_whatsapp_media_id or audio_recorded_at. The handler always fills the media ID and timestamp (handler.js:786-791), and results.service.js:57-59 sends them. So the save for any Flow-path listening subtest fails and the coach's six marks are lost without the coach being told Code plus live schema
Pictures are "working end-to-end" (TESTING.md:246-276) 3 of 3 live passage-picture sends failed with ❌ Error in sendImageFromUrl, "Request failed with status code 400", and an XML error body. The pictures sit on …r2.cloudflarestorage.com, the private storage address, which answers 400 InvalidArgument: Authorization to any unauthenticated fetch (checked with curl on 27 Sep). whatsapp.service.js on staging already records this failure in its bd-1736 comment ("R2 rejected with 400 (InvalidArgument: Authorization) — the object isn't publicly readable") Axiom, 19/21/24 Sep · shared/config/egra-egma-grid-images.json, shared/config/egra-egma-passage-images.json
A missing picture falls back to text Only when no address is configured (handler.js:153-176). When the send fails, sendImageFromUrl returns false and the handler ignores it (handler.js:163, :276). The coach gets nothing. The lost caption also carried the subtest name, the "60s cap — please self-time" line and the stop rule Code
Usage 9 sessions started (17 to 24 Sep). 1 subtest saved (Urdu listening comprehension, 18 Sep; the database row runs 06:28:30 to 06:30:33 with student_id null). 0 voice notes. 0 sessions finished. That one session got past the first subtest at 06:30 UTC, before the picture commit (b1a2513e, 06:47 UTC). No session has reached a grid since, so there is no log line for the grid pictures, but they use the same private address Axiom plus the live row

Note: the staging service's log rows carry env: "production". I identified staging by service name.


1. Conformance table

Legend: Built · Partly · Not built · Contradicts (the build does the opposite, or rules the plan's version out).

# Plan element (source) Verdict In plain words Evidence
1 5 minutes per child (#idea, call item 2) Contradicts About 22 minutes per child (section 2), the same as today's enumerator §2 arithmetic
2 5 children in 30 minutes per coach per day (call item 2) Contradicts About 110 minutes for five children §2
3 The computer marks, the coach checks and corrects (#idea, #form, HITL) Contradicts 100% coach-marked. The table's check constraint pins the AI-score field to 'not_attempted_this_build', so an AI score cannot be stored without changing the schema database/migrations/pending_egra_egma_ict_test_attempts.sql:76-77
4 Keep the AI's answer, the coach's answer and the recording side by side (#journey "Saved twice", #form) Not built Only the coach's answer is stored. The recording exists for 2 of 14 subtests, and those saves currently fail results.service.js:33-60; §0
5 Humans mark only letters and first sounds; AI does counts, comprehension, spoken maths, photo and the coach audit (HARNESS §8) Contradicts Humans mark everything, including the reading counts, where AI came within 5 words for 74 to 97% of children handler.js:458-617
6 Islamabad first (call item 2) Built Gated to users.region ict, islamabad or federal access-gate.js:22,45-47
7 All three subjects, Urdu first, comprehension stays (call item 3) Built Urdu, then English, then maths, with comprehension in both languages item-bank.service.js:19-34
8 Based on EGRA, not ASER or TIP (call item 5) Built Uses the real fielded EGRA/EGMA content shared/data/egra-egma-real-items.json:1-5
9 An algorithm picks the children, not the coach (call item 4, #call) Partly The child is random within a class, but the coach picks the class from every ICT class (at most 9 shown). The draw is with replacement, and cancel-and-redraw lets the coach choose. The coach flagged the class choice as a bias (TESTING.md:233-238) handler.js:121,648-686; roster.service.js:51-62,106-124
10 Children pre-selected and sent before or at the visit (#journey "The offer") Not built The child's name appears only after the coach taps a class, so the coach cannot fetch the next child while the current one works handler.js:668-685
11 Offered after the lesson recording (#journey; roadmap step 4, NIETE) Not built (acceptable for now) Started by typing /egra. The roadmap puts the after-lesson offer at step 4 in the NIETE bot, so a command is fine for step 2 handler.js:64-68; PLAN.html#roadmap
12 Built in the main bot on staging first (roadmap step 2) Built Yes. research/04 §8 recommended NIETE-Rumi, but the canonical plan says main bot first PLAN.html#roadmap
13 Not released until under 30 minutes (call item 2) Built (gate) Off by default, and inert for everyone else access-gate.js:25-27
14 Printed child card; the phone only records (#journey) Contradicts The child reads a picture on the coach's phone. That picture is the enumerator's version, with an item number in every cell grid-image.service.js:63-68; handler.js:162
15 One voice note per block, covering the whole block (#journey, #pipeline) Contradicts One voice note, only for listening comprehension, recorded after the child has answered handler.js:269-287; Flow JSON :113,:125
16 Works with poor signal: notes queue offline (#journey, roadmap step 2) Not built Every tap needs a live round trip to the bot before the next item appears handler.js:223-230,401-415
17 Per-child checking Flow: pre-filled counts, chips, a radio per question, dropdowns, photo, in Urdu (#form, egra-review-flow.json) Not built Her Flow is 2 read-only English instruction screens that collect nothing but confirmed: "true" docs/flows/egra-egma-listening-comprehension-flow.json:42-141
18 Audio saved to our storage straight away, with school, session, child and block in the file name (#pipeline, research/04 §3) Partly Saved straight away (good), but the file name holds only session and subtest, and the database row cannot hold the link yet handler.js:778-779; §0
19 Speech-to-text, window finder, AI word marks, comprehension grader, coach audit, confidence (#pipeline, #models) Not built None of it is built TESTING.md:207-215
20 Urdu story, 60 s, then 3 questions, with a fallback to 10 letters and 10 words (#battery) Partly Urdu story present, but shown as a numbered word grid; 6 questions; the 100-letter and 50-word grids are the default for every child, not a fallback item-bank.service.js:19-34,50-51
21 Urdu first sounds (5) and made-up words (5) (#battery) Not built Neither exists item bank
22 English story, 60 s, then 2 questions, plus 8 made-up words (#battery) Partly Story present, but as a grid that counts punctuation as words; 6 questions; made-up words excluded §3
23 Maths: read 8 numbers aloud, one-minute quick sums spoken, written sums photographed (#battery, #maths, D2) Not built Instead: 40 untimed coach-marked items (compare/next number, place value, computation, word problems, patterns). No timed maths and no photo item-bank.service.js:79-83
24 Drop listening comprehension, the 100-letter grids for readers, and the 50-word lists (#battery) Contradicts All kept. Listening comprehension is where the only Flow and the only audio went item-bank.service.js:19-34
25 Stop rules keep each child at their own level (#battery) Partly A Discontinue button on the grids, applied loosely (§3c). Reading comprehension is not gated on the story; maths is not laddered; the plan's branch rules are absent handler.js:590-601; egra-egma-real-items.json:473; TESTING.md:228-232
26 Every timed window a full 60 s, kept comparable (#battery) Partly The coach taps Start and Stop, the server measures the gap, and there is no countdown or alert handler.js:573-588; TESTING.md:201-206
27 Zero new tables; reuse aser_sessions, reading_assessments, numeracy_assessments (#data) Contradicts New table. The migration says aser_sessions is "NOT YET SHIPPED" after checking migration files only. The live database has aser_sessions (with student_id, school_id, grade_level), and reading_assessments already has rumi_level, coach_level, aser_session_id, audio_url, word_timestamps and student_id pending_egra_egma_ict_test_attempts.sql:11-19; live schema, 27 Sep
28 Link each result to the child, school and grade, so growth curves and class views work (#yield "The bigger point") Not built The handler knows the child's database ID (handler.js:669), but the save never writes it (results.service.js:34-60). The column exists and is documented "always NULL in this build". No school, class or grade is saved pending_…attempts.sql:43-44
29 Coach copy in Urdu through the string catalog (#form draft is Urdu; root Rule 20) Contradicts All coach copy is hard-coded English TESTING.md:216-220; handler.js throughout
30 Instrument choices fixed with Sabeena and Sameer before the marker is built (D3, roadmap step 1) Not built D3 is still open; the build uses the May instrument as-is PLAN.html#decisions
31 Use recordings to check the tester (call item 8) Not built Not built —
32 Field team, 11 Sep: all of a child's questions in one form, not one per message (meetings/2026-09-11_ASER-roll-out.md:40) Contradicts 64 one-item messages per child §2
33 Field team, 11 Sep: the teacher or coach has the final say (human toggle) Built The coach's mark is the only mark handler.js:340-342

2. Time on task: one child, message by message

The walk-through below uses staging as it is configured today (Flow switched on), and follows the code, not TESTING.md.

Step Bot sends Coach does Notes
Start intro message; class list types /egra; opens the list, taps a class, sends (about 3 taps) handler.js:477-492,120-134
Child picked "Selected name (class) — starting the assessment." goes to find this child; the name only arrives now handler.js:678-681
Urdu listening comprehension Flow card taps Start; reads the passage twice; taps Next; asks the 6 questions; taps "Ready to record" (3 taps) Flow JSON screens. The child answers inside the Flow, with no recording
passage picture (fails live); "record a voice note of the child's actual spoken response" records and sends a voice note (the child has already answered, so either the questions are asked again or the recording holds nothing useful) handler.js:269-287. The questions are not re-sent, so the coach has to recall them
"Voice note received…", then item 1 of 6 6 × taps Correct or Incorrect, each followed by the next item handler.js:795-801,223-230. No answer key is shown
Urdu letters (100) picture plus caption (fails live); "Ready? ▶️ Start" taps Start handler.js:147-182
"Timer running…" with Stop / Discontinue child reads for up to 60 s from the coach's phone; coach keeps time with no clock; taps Stop handler.js:184-192,573-588
"Which item numbers did the child get WRONG?" types e.g. 3,7,15,22 (from memory, or a paper sheet the build does not provide) handler.js:194-199
"What was the LAST item number the child reached?" types e.g. 47 handler.js:201-203
"Recorded: 43/47 correct, 41.3 items/min." — handler.js:394-397
Urdu words (50), Urdu story, English letters, English story same 6-message pattern each 2 taps and 2 typed answers each 5 grids in all
Urdu reading comprehension, English reading comprehension intro, then 6 items each 6 taps each Asked even when the child read nothing of the story
English listening comprehension same as the Urdu one 9 taps plus 1 voice note
5 maths subtests intro, then 8 items each 8 taps each (40) The child sees nothing unless the coach shows the phone
End per-subtest summary types /egra to start the next child handler.js:328-349

Counts per child: - Bot messages: 113 designed. 106 arrive today, because all 7 picture sends fail (2 passage pictures and 5 grids). - Coach actions: 96 in all: - 83 taps: 3 at start, 9 + 9 in the two listening subtests, 10 on the grids, 12 in reading comprehension, 40 in maths - 11 typed answers: /egra plus 2 per grid - 2 voice notes - Per-item marking taps: 64 (6+6+6+6 comprehension, plus 40 maths). - For comparison, the plan's journey is about 3 voice notes, 1 photo and one 3-screen form per child, so roughly 10 coach actions.

Minutes per child: arithmetic and assumptions.

Part Minutes How it was reached
A. Time with the child for the same 14 subtests, as an enumerator does them on a tablet 13.8 Sum of the May medians in baseline_subtask_timings.csv: 78+68+50+67+32+80+68+76+36+52+69+34+67+51 = 828 s. These 14 are 82% of the full 17-subtest time (1,008 s)
B. Per-child setup: /egra, class list, the draw, fetching the child just named 1 to 2 The plan itself allows about 1 minute of moving children, with the list known in advance; here the name arrives only after the draw
C. Waiting for the bot after each action (a tablet has no wait) 1.6 to 4.8 96 actions × 1 to 3 s per round trip. This is an assumption: Axiom batches its timestamps in 5-second steps, so it cannot measure this precisely
D. Typing the wrong-item lists and last items for 5 grids 1.3 to 2.5 10 typed answers × 8 to 15 s. On the tablet the enumerator taps wrong items during the minute
E. Re-asking for the listening voice notes (Flow path) 0 to 1.5 2 × about 45 s: about 35 s to re-ask 6 questions (compare the May reading-comprehension medians of 32 to 36 s) plus record, send and upload
Total per child 17.7 to 24.6 (about 22) 13.8 + 1.5 + 3.2 + 2 + 1.5 at the midpoints
Five children 89 to 123 (about 110) × 5

Cross-checks: - The August WhatsApp ASER tool took 10 to 15 minutes per child, according to the field team on 11 Sep (meetings/2026-09-11_ASER-roll-out.md:38). The plan puts paper ASER at about 10 minutes. So one message per question costs about 1.0 to 1.5 times the paper time. Applying that to A: 13.8 × 1.0 to 1.5, plus 1 to 2 minutes of setup, gives 15 to 23 minutes. That range is consistent with the table. - One real staging run on 18 Sep (plain path, a tester tapping, no child) went from the child being picked at 06:28:50 to the Urdu listening subtest being saved at 06:30:35: 1 minute 45 seconds. The May enumerator median for that subtest with a real child is 78 s.

Against the targets:

Per child Five children
Target (call, 15 Sep) 5 min 30 min
Today's enumerator (BASELINE.md:40) 21.1 min (median, whole form) about 105 min
This build (estimate) about 22 min (18 to 25) about 110 min

The build lands at about today's enumerator time, 3 to 4 times over the target.

Left out of the estimate (each would make it slower): - There is no answer key on screen, so the coach works out every answer live. - 3 maths items refer to pictures that do not exist. - The voice-note step has no skip. - With today's broken pictures, the 5 timed subtests cannot be given at all.


3. The battery

3a. What is in it

Skill Plan's five-minute test (#battery) This build Full EGRA/EGMA (May 2026)
Listening comprehension dropped Urdu + English, 6 questions each; the only subtests with a Flow and audio yes
Letter naming, 100-letter grid fallback only: 10 letters, 20 s every child, Urdu + English yes
First sounds Urdu, 5 items none Sindh variant
Made-up words Urdu 5, English 8 excluded English, 50
Familiar words fallback: 10 words, 30 s Urdu 50-word grid, every child Urdu + English, 50
Story, 60 s Urdu + English Urdu + English, shown as a numbered word grid yes
Story questions Urdu 3, English 2, only if a line was read 6 each, always asked 6 each, gated on the story
Reading numbers aloud 8 items, stop after 4 wrong in a row none. "Identify" here means which is bigger and what comes next yes
Quick sums, timed 1 minute spoken, timed excluded yes (blfl)
Computation 4 written sums, photographed 8 untimed, coach-marked 8
Word problems spoken, 1 written 8 spoken, coach-marked 8
Place value / quantity — 8, of which 3 need pictures that do not exist 8
Patterns "photo or drop" 8 8

The build's reason for dropping made-up words is SpeechAce's poor automatic scoring (egra-egma-real-items.json:6-20). That reason does not apply here, because nothing is scored automatically, and TESTING.md says so at :172-178. It also runs against our own evaluation: English made-up words by transcript plus alignment came within 5 for 91% of children, marked "counts ready" (PLAN.html#experiment).

3b. Are the timed subtests really timed?

Real EGRA avoids this with a printed card for the child. The plan does the same.

3c. Stop rules

In May, only 1,382 of 2,233 children (62%) have a timing for the Urdu story, and 952 (43%) for the English one (baseline_subtask_timings.csv). The May form gated these with no_orf and continue_rc (BASELINE.md:36). Here, every child is asked all 12 comprehension questions, including children who read nothing of the story. - Maths laddering and the "continue" gates are not built (TESTING.md:228-232). - None of the plan's stop rules exist: the fall-back branch, "questions only if a line was read", and "4 wrong in a row stops maths".

3d. Language and order

3e. What the child sees versus what the coach sees


4. Coach experience

Copy and language. - Every coach-facing line is English and hard-coded: prompts, buttons, Flow screens and the end summary (TESTING.md:216-220). - There is no preferred_language, no string catalog, and no Urdu. Our checking-form draft and the ICT observation form are in Urdu. - Some wording is internal or confusing for a coach: - "Discontinue rule: If the child scores 0/10 correct on the first row…" appears in the caption. - The Flow says "WhatsApp will ask you to record…"; it is Rumi that asks, not WhatsApp. - "items/min" is jargon.

Clarity. - There is no overall progress. The coach sees "Item 3/8" but never "subtest 6 of 14", or any sign of how much of a roughly 20-minute session is left. - The 5 grid subtests depend on a caption for their name and instructions. With the picture failing, the coach sees only "Ready?" and a Start button.

Number of taps. 83 taps, 11 typed answers and 2 voice notes per child (§2). That is about ten times the plan's journey, and it is the pattern the field team rejected on 11 Sep.

Recovering from mistakes. - There is no undo. One tap on Correct or Incorrect records the mark and moves on. - Buttons carry no item number (handler.js:205-212), and any Correct/Incorrect/Pass tap is accepted while marking is open (handler.js:603-607). So a double tap, or a tap on an older item's button further up the chat, is recorded against whichever item is current. With 64 item messages in the chat, this will happen. - /cancel only works in button-driven steps. Typed as the child's name, it becomes the name (handler.js:506-511). While typing wrong items or the last item it gets "not a number" (:513-538). TESTING.md:69 promises "/cancel at any point". - The voice-note step has no skip. Anything typed there gets "Please use the buttons above" (handler.js:539-546), but that step has no buttons. The only way out is a voice note or cancelling the whole child. - Cancelling loses the child's progress. If the child is absent or the recording fails, the coach can only cancel the whole session, which throws away the on-screen summary, and draw again. - The access gate fails silently. A coach without the right region who types /egra falls into general chat. Commit 3c024544 (bd-eg7st) traces the 18 Sep "send me the story" loop to exactly this, and 7 /egra messages on staging that day started no session. The fix sends such coaches to /reading; it does not tell them the test is not switched on for them. - The session expires 1 hour after the last step saved (session-state.service.js:21).

What the coach sees at the end. - One plain text list per subtest, for example "• Letter Identification (Urdu): 43/47 correct (41.3/min)", plus "no AI … is used" (handler.js:328-343). It arrives only if all 14 subtests are finished. - There is no level, no benchmark and no TIP group (D6). - There is nothing for the teacher and no class view. - There is no portal page. - There is no "child 2 of 5" and no list of who is left today.

How it is started. By typing /egra or four other phrases (handler.js:64-68), not offered after the observation. That is fine for roadmap step 2.

The 11 Sep field team said something relevant here: in NIETE, ASER is run separately from observations (meetings/2026-09-11_ASER-roll-out.md:40). The plan's "offer after the lesson" (#journey) conflicts with that feedback. This is for the plan owner to settle, not a fault in this build.


5. Strategic read

This is a different path from the plan, not an early step along it. Phase 0 of the plan would test the risky parts: - one recording per block, taken while the child reads from a printed card - the per-child checking form - the storage shape that keeps the AI's answer and the coach's answer side by side - the 5-minute clock

This build tests something else: a WhatsApp chat version of the full 21-minute enumerator battery. The differences are built in, not surface-level:

Area The plan This build
What is tested the five-minute test (D1, D2) the May instrument (D3 not decided)
How it is marked recording → AI pre-fills → coach corrects in one form coach taps each item in chat; a schema rule forbids AI scores
Audio every block, while the child performs one subtest the plan drops, recorded after the answers
Storage existing aser_sessions, reading_assessments and numeracy_assessments, which exist live with AI and coach columns new table, with no child ID, school or grade saved
Sampling pre-selected list, algorithm decides coach picks the class; each child drawn with replacement at tap time
Language Urdu form English everywhere

The build could still be useful as a tool for putting the item bank on screen, and as a coach-only fallback while the AI is not ready. But it cannot reach 5 minutes in its current shape. Removing a few subtests will not fix that, because the one-message-per-item design adds minutes of its own.

What Sabeena should do next, in order:

  1. Stop staging from failing, even before any redesign. - Send the pictures from memory, the way the bd-1736 fix does for coaching cards, or from a public storage address. - Check the send result and fall back to text when it fails. - Apply the voice-note migration, or stop sending the audio fields, so the coach's marks are not lost. - Update TESTING.md: the Flow is live and the base table exists.
  2. Settle D1 and D3 before any more building. Agree the five-minute test and the Urdu instrument choices with Haroon and Sameer. The plan says the marker cannot be built until D3 is fixed.
  3. Cut the battery to the plan's blocks: - Urdu story, 60 s, plus 3 questions, with the fall-back rungs of 10 letters and 10 words. - English story plus 2 questions plus 8 made-up words. - Maths: 8 numbers read aloud, one-minute quick sums, and the photo strip.

Reuse her item bank's stories and answer keys. Print the child cards without numbers, and keep stories as connected text. 4. Swap the one-item-per-message design for record-then-check: - one voice note per block, starting on the cue phrase - then one per-child Flow, reusing harness/flow/egra-review-flow.json and the NIETE /observe pattern from research/04 §2, with the answer keys shown

This works even before any AI: a blank form the coach fills from memory already turns 64 messages and 10 typed answers into 3 screens. That is the field team's first request. Then add the AI pre-fill per HARNESS §8: Gemini 3.8 Flash for counts, a Gemini 3 Flash-tier grader for comprehension, and the coach for letters and first sounds. 5. Store it where the plan says: - aser_sessions plus reading_assessments and numeracy_assessments, with student_id, school and grade filled in - the AI's answer and the coach's answer as separate fields - audio file names that include school, session, child and block

Drop the new table, or keep it only as a scratch table. 6. Sampling: a pre-selected list of 5 per visit, drawn without replacement and sent at the start of the visit. Add a "child absent → next on list" option that is logged. Decide D5. 7. Urdu coach copy through the string catalog (Rule 20). 8. Then run the step-3 stopwatch pilot (2 or 3 coaches, 25 children) with a real child in front of the coach. So far on staging nobody has got past the first subtest with the Flow switched on.


6. What was done well


Appendix: how the live evidence was gathered (read-only)