ICT child test · v2 · live on the NIETE sandbox

Run a visit yourself

Tap through a visit the way a coach does. The phone shows the bot's own messages (rendered by the live conversation code), with the real buttons. Beside it, the table: the card the child is holding at that moment (click it to see it full size) and the exact words to say, with an English translation. On a wide screen, a third column explains why each step works the way it does. The recorder runs fast (8×), so a full child takes about a minute. Real screen opens the real WhatsApp screenshot of that moment.

The simulator

The print pack

Every side of every card, and the coach cards

The PDF is 20 A4 pages, printed double-sided (flip on the long edge) and laminated: one set per coach for the term. Each child card is one sheet, front and back; Grade 3 is green, Grade 5 blue; "Set A · Oct–Dec 2026" is on every side, and no child card shows an answer, a question or an item number. The coach card (one per grade, back blank) holds the questions and word problems, as EGRA's assessor sheet does. Click any page to see it full size.

For the coach

Coach card · Grade 3 (don't show the child)

Front · what to say, the 3 Urdu and 3 English questions with the answers that count, the pairs and sums with answers, the 2 word problems to read aloud with answers. The EGRA assessor sheet, simplified; the paper twin of the phone message

Coach card · Grade 5 (don't show the child)

Front · the same for Grade 5

Coach page

Front · English: where to sit, what to bring, five rules
Back · the same in Urdu

Cover

Front · what to print, how many, what to laminate
Back · blank, so the cards start on a fresh sheet

Grade 3 child cards (green)

Grade 3 · Urdu

Front · the story only (May 2026 passage)
Back · 10 letters, then 10 words: only for a child who can't read the story. Faces up when the card is turned over for the questions

Grade 3 · English

Front · the story only
Back · letters and words, the same rule

Grade 3 · Maths

Front · four "which is bigger" pairs, four sums
Back · blank on purpose: when the coach turns it over to read the word problems, the child has nothing to read

Grade 5 child cards (blue)

Grade 5 · Urdu

Front · the longer story (May passage + continuation)
Back · letters and words

Grade 5 · English

Front · the longer story
Back · letters and words

Grade 5 · Maths

Front · four pairs, four harder sums (8 × 9, 72 ÷ 6)
Back · blank on purpose
Fidelity, time and calibration

How close to EGRA/EGMA, and what is missing

Time, measured on real WhatsApp on 4 Oct: 5.0–5.7 minutes per child, so about 27 minutes for five plus a 1-minute review (before: 35.4). The items: the Grade 3 stories and all of Set A's maths come word for word from the May 2026 instrument, which 443 Grade 3 and 431 Grade 5 children took. The Grade 5 story continuations and all of Set B were written later, and no child has read them yet.

Partv2EGRA / EGMATried on children?
Grade 3 stories (Set A)60 words each, the May textAbout 60 words, 60 sYes, May: 33% (Urdu) and 16% (English) finished inside the minutecalibrated
Grade 5 stories (Set A)The May 60 words plus a continuation (Urdu +116, English +106 words)Long enough that few finishFirst 60 words only. The lengthening was needed (76% of May Grade 5 finished 60 Urdu words inside a minute) but the new part is untriedpartly
Set B (next term), all partsWritten to match Set APiloted, then equated to the first formNonot yet
Questions per story3 (Grade 3: 2 literal + 1 inferential)3–5, mostly literal; Pakistan asks 5, May asked 6Grade 3 Set A: yes (May items)below
Which questions are askedAll 3, however far the child readOnly questions about text the child read; scored out of those asked—gap
Maths: which is bigger4 pairs, the same in Grade 3 and 510 pairs, 1–3 digitsYes. In Grade 5, 62% got all 4 right (too easy)short form
Maths: sums4. Grade 5 swaps in 8 × 9 and 72 ÷ 65 + 5 items. × and ÷ are not in Core EGMA (built for Grades 1–3)Yes. Grade 3 sums 3 and 4 (92 − 41, 85 + 37): 6% right eachshort form
Maths: word problems2, read aloud by the coach6, read aloudYes (May items)kept
Listening, letter sounds, made-up wordsLetters and words only for a non-readerEvery child—dropped
Maths overall10 items, about 105 s6 tasks, 96 items, about 20 min—short form

What we are missing, and what it costs

1 · Questions past where the child stoppedEGRA asks only about the part a child read. The bot asks all three. The Grade 5 Urdu question 3 sits at word 148, which 9% of May Grade 5 children reach in a minute. A slow reader loses marks twice, for speed and again for comprehension. The bank already records each question's line; the bot doesn't use it yet.
2 · Untried text: Set B and the Grade 5 continuationsUntil they are piloted and equated, a change from this term to next mixes real learning with a harder or easier card.
3 · No listening, letter-sound or made-up-word tasks for every childA low English comprehension score can't be split into "can't read" and "doesn't understand spoken English": in May, Grade 3 readers averaged 1 of 6 on English listening. For a slow Urdu reader we can't see whether decoding is the problem.
4 · Maths at the floor and the ceiling10 of EGMA's 96 items. Grade 3 sums 3–4 are right for 6% of children, so they barely separate anyone; 38% score 0 of 4 sums. The Grade 5 pairs are at the ceiling. Number identification, missing number and timed facts are absent.
5 · One child's score is roughFrom the May data: one child's maths score is uncertain by about ±2 of 10, and Grade 3 Urdu comprehension by about ±1 of 3. Read results by class or school, not child by child.

The three cheapest fixes

Needs a decisionAsk only what the child reached; 5 questionsThe bot skips questions past the child's last word and scores out of those asked. Grade 3 Set A takes the May instrument's other questions; the rest need 2 new questions each. About 2 days. Cost: about 50 s more per child, 4 minutes per visit. Changes the coach cards.
Needs a decisionRe-level maths from May's own resultsNo fieldwork. Grade 3: swap 92 − 41 and 85 + 37 (6% right) for two easier May sums (37% and 33% right). Grade 5: swap the two easiest pairs for May's "what comes next" items. About 1 day. Changes the maths cards and coach cards.
Next termOne pilot that does two jobsAt least 150 children per grade read both Set A and Set B, in alternating order. That equates Set B and tests the Grade 5 continuations in one go: about 60–80 coach-hours, then 2–3 days of analysis.
Still open

Before a real classroom

DonePrinted coach cardOne A4 side per grade with the questions, accepted answers, word problems and sum answers: in the print pack (pages 5 and 7).
Needs youA 15-minute test on a real phoneLock a voice note and count aloud; then scroll, leave the phone 2.5 minutes, let a message arrive. Did it keep recording?
Needs a decisionConsent before any real child is recordedWho agrees, how parents are told, how long audio is kept.
NextOral maths on real children89–91% right on test voices; real audio not yet tested.
NextQuality without coach countingA trained person re-scores a sample of recordings.
FAQ

Your questions, answered

Are the sums, stories and comprehension questions calibrated to EGRA/EGMA? What are we missing?

Partly. The Grade 3 stories and all of Set A's maths are the May 2026 instrument, tried on 874 children, so their difficulty is known. Not calibrated yet: the Grade 5 story continuations and all of Set B (no child has read them), and the way questions are asked (EGRA asks only about what the child read; the bot asks all three).

What it costs: comprehension that punishes slow readers twice; Grade 3 sums 3–4 that only 6% get right and Grade 5 number pairs most get all right; no listening task, so a low English score can't be split into reading and spoken English; scores rough for one child (maths ±2 of 10). The full table, and the three cheapest fixes.

The coach needs the child's section and class. One grade, or a mix? How do they find the children?

One grade per visit (Grade 3 or Grade 5, picked by the bot). The list is grouped by classroom, with the class teacher, in the roster's own words: "Grade 3 - A · Teacher: Saima Bibi". From the NIETE roster (924 Grade 3/5 classes, 31,826 children): every child has a full name, 92% of classes have a section letter, 99% a reachable class teacher; two in three visits need only one classroom. One button sends each class teacher only their own room's children, in order.

Drop roll numbers.

Done. No roll number in any message, the form or the portal. Two classmates with the same name (2% of children) get the father's name when the roster has it, or "2 in this class" when it doesn't.

If passages are printed and rotated, how do we know which variant the child has? Variant number on the card? PDF to print, A4? Is that how EGRA does it?

There is no rotation any more. EGRA gives every child in a round the same form; forms rotate between children only in pilots and group tests. So each coach carries one card set per grade for the whole term, colour-coded (Grade 3 green, Grade 5 blue) with "Set A · Oct–Dec 2026" printed on every side, and the bot names the card in every step. The print pack is A4, laminated, one task per side, as EGRA's are: the print pack above, PDF sent to you.

"First sounds from the coach's copy": what does that mean?

EGRA's initial-sound task: the adult says a word ("machhli") and the child says its first sound ("m"). It is a Grade 1–2 task, Pakistan's Grade 3 tests drop it, and our AI could only guess at it. The "coach's copy" was a 5-page paper sheet the bot never mentioned. Both are gone.

The copy should be much simpler. And my language is English: shouldn't the messages be English?

Your sandbox account had been switched to Urdu for the first demo recording and never switched back; it is back to English. Every message now follows the coach's language (try both buttons above). Only the words the coach says to the child stay in Urdu, and English for the English story. Each step is now numbered actions: which card, what to say, when to stop, when to send.

Why did it tell me to give the maths to "the other kid" when Usman was my first child?

The old design had the next child write maths while the current child read, to save time; that child wasn't even in the room. Removed. One child at a time, start to finish; maths is spoken, no strip, no photo, no child number.

Maths: does the coach read the sums, or the child? Two maths pages?

The child reads the four number pairs and the four sums on one maths card and says the answers; the coach points to each and says "which is bigger?" / "tell me the answer". The coach reads the two word problems aloud with the card face down. Try the maths step above: the table shows both.

The child wrote sums on the strip while the child before read; if it's my first child, then what? Do both maths pages go to the child?

That whole arrangement is gone: no strip, no second page, nothing handed to another child. The child in front of the coach does Urdu, English, maths; then the next child comes.

Does the coach read the word problem, explain it, or just give the paper?

Reads it aloud, slowly, once, as EGMA does (EGMA reads word problems to the child so a weak reader isn't marked down in maths; the child may use paper and pencil; no explaining). The two problems are in the maths message and on the coach card.

The Urdu card had only the story; why did the form ask Urdu comprehension questions? Should Urdu have them?

Yes: Pakistan's Urdu EGRA asks 5 questions; we ask 3. In EGRA the questions are never on the child's card; the assessor asks them aloud after taking the passage away. The old message never told you to ask them. Now step 4 of the Urdu message lists them, the coach turns the card face down and asks; the table above shows them with translations.

"This child's key says 56": what is a key? How does the coach count words per minute, in their head?

They don't count. The "key" was the test child's answer sheet in my demo; real coaches have none. EGRA assessors train 5+ days to mark every word live and still disagree. Here the recording is the score sheet: the AI counts from it, and on the May Rawalpindi children it agreed with the enumerators more often than a second trained person did (Urdu 74% vs 58% within ±5 words). The coach only answers the few things the recording couldn't settle, at the end.

Doesn't EGRA print the questions and word problems on the card the coach carries?

Yes: the EGRA/EGMA assessor holds their own sheet (paper or tablet) with the questions and accepted answers and the word problems to read; only the child's card is bare. In v2 the phone message is that sheet, and a printed coach card per grade ("don't show the child": questions with accepted answers, word problems, sum answers) is in the print pack (pages 5 and 7), as the paper backup.

Coaches go to many schools. Does EGRA/EGMA change the papers from school to school?

No. In a data-collection round every child in every school reads the same form; that is what makes schools comparable. Forms change between rounds (baseline, endline) using parallel forms checked to be equally hard. So: one set for every school this term, Set B next term. EGRA rounds last weeks, our term three months, so the cards stay with the coach (never left at a school or photographed), the questions are never printed for children, and we can watch for leakage: a passage whose scores climb through the term while letters and words don't.

The rotation was unclear. With rotation the coach's attention leaves the child; without it we exceed 30 minutes.

Two things had been called rotation: two card forms in one visit, and the next child writing maths while the current one read. Both are gone. The time came back from elsewhere: the coach no longer counts or fills a form per child (about 2 minutes each), first sounds, made-up words and the 60-second quick sums are dropped, and maths is one short spoken set. Measured on real WhatsApp: about 5 minutes per child, about 27 minutes for five plus a 1-minute review.