← EGRA Test Build Review · Design

EGRA/EGMA ICT test build: design review (child cards, Listening Flow, coach messages)

Reviewed: 2026-09-27 · Bead: bd-ak1go · Build: main-bot staging @ 554c6a8d (the snapshot handler in ../src/ is byte-identical to staging head) Scope: everything a coach or child sees: the 7 stimulus images, the Listening Comprehension WhatsApp Flow, and every coach-facing string in src/shared/handlers/egra-egma-ict-test.handler.js. Method: downloaded all 7 real images, looked at each one full-size, and measured glyph heights in pixels, converted to millimetres on a 6.1" phone (64.9 × 140.7 mm screen). I rendered the Flow and the coach's chat as phone mockups using the real strings, and checked staging logs (Axiom) for what actually happened when testers ran it. Nothing in git, R2, Meta or Railway was changed. R2 access was GET-only.

The short version

Blocker: no stimulus image has ever reached a coach. All 7 image URLs point at R2's private storage address, which refuses anonymous downloads (HTTP 400). The bot downloads by URL without credentials, so every send fails. Staging logs show 3 of 3 attempts failing, and the handler then carries on as if the send had worked.
The failure is silent and takes the instructions with it. The fallback to plain text only runs when no URL exists, not when the send fails. The coach gets "Ready? ▶️ Start" with no grid, no 60-second rule and no discontinue rule.
The images themselves are well made (clean, correct Unicode, correct Nastaliq, correct right-to-left order) but they are designed as a score sheet, not a child's card. Every item carries a printed number (1., 2., … 100.). A real EGRA child card has no numbers; numbers belong only on the assessor's sheet.
Oral Reading Fluency is shown as 68 numbered word boxes, not as a story. The child is no longer reading connected text, so this is not an ORF measure. In English, full stops and commas get their own numbered boxes, and the scorer counts each one as a correctly read word. That adds up to 8 words to every English ORF score (+13% for a child who finishes).
At phone size the print is small. English ORF words are about 2 mm tall even with the phone turned sideways (about 8-point print) and 1.1 mm upright. English letter-grid capitals are 3.5 mm. On a printed A4 card, each of the 10 columns gets about 27 mm; on the phone it gets about 6 mm.
One phone cannot be both the child's card and the coach's score sheet. The coach has to hold the phone up to the child, keep track of which item numbers were wrong (up to 100), and tap Start and Stop on the same screen.
Everything the coach reads is English-only, hardcoded in the handler rather than routed through the ux-strings catalog. ICT coaches are Urdu-first.
Two instructions are wrong for the reading subtests. "You will read out each item to the child" and "(or read it out per the item type)" would make a coach read the letters and words aloud for the child, which invalidates those subtests.
The coach never sees an answer key. correct_answer_note exists for every question but no code shows it. Three EGMA items ask "How many bananas are there?" about pictures that were never made, and the coach is not told this.
The Listening Flow is within Meta's limits but has problems. The steps are in the wrong order: the child answers before the recording starts. Item-bank typos ("On what did did Sara…", "WHo walked…") appear verbatim. The heading mixes literal text with a ${data…} binding, which Meta's spec says needs backticks.

1. Blocker: the images never arrive

What happens. sendGridSubtestIntro (handler.js:151-163) and sendVoiceNoteRequest (handler.js:274-276) call WhatsAppService.sendImageFromUrl(url, …). On staging, that function does a plain axios.get(imageUrl) with no credentials, then uploads the bytes to Meta (shared/services/whatsapp.service.js on origin/staging, sendImageFromUrl at ~line 1117). The URLs in src/shared/config/egra-egma-grid-images.json:3-7 and egra-egma-passage-images.json:3-4 are all https://58a6fb2a…r2.cloudflarestorage.com/digital-coach-audio/…. That is R2's private S3 API address, not a public one.

Evidence. - curl of all 7 URLs returned 400 <Code>InvalidArgument</Code><Message>Authorization</Message>. - Staging Axiom, 15–27 Sep: the only 3 EGRA image sends (19, 21 and 24 Sep, all listening_comprehension_urd.png) each log Fetching image from URL for WhatsApp upload → ❌ Error in sendImageFromUrl → whatsapp.message.failed "Request failed with status code 400". They then log egra_egma_ict_test.listening_flow_completed and send the voice-note request. The traces are corr-1789793344332-r6dner1ty, corr-1790017018281-ap2iptcms and corr-1790233612977-s9fje057m. - No grid-image send appears in staging logs at all in that window, so no tester has yet seen a letter/word/ORF card on a phone.

Why tests are green. Both handler test files replace the send with an always-successful fake: shared/handlers/__tests__/egra-egma-ict-test.handler.test.js:15 (sendImageFromUrl: jest.fn().mockResolvedValue(true)) and tests/egra-egma-listening-flow-completion-routing.test.js:33. The URL test (grid-image-urls.test.js:13-15) only checks that each URL is a string starting with https://.

What the coach sees. img/chat_3_grid_as_delivered.png compared with img/chat_3_grid_as_intended.png, and img/chat_2_after_flow.png (the passage never arrives).

Fix direction. The code already has the right tools: getPresignedUrl() (shared/storage/r2.js:545) or downloadFromR2() (r2.js:366). Separately, check the true/false that sendImageFromUrl returns (handler.js:163, :276) and send the text fallback when it is false.

2. The child's cards (stimulus images)

I looked at all 7 at full resolution (copies in img/, ≤1600 px viewing copies in img/view/).

2a. What is right

Content matches the item bank. Spot-checked against src/shared/data/egra-egma-real-items.json:
English letters: rows 1, 9 and 10.
Urdu letters: rows 1, 2, 4 and 10.
Urdu words: rows 1, 2 and 5.
English and Urdu ORF: all tokens.
Every item is present and in order, and nothing is cropped (100 / 100 / 50 / 68 / 60).
Urdu is correct. Nastaliq joining is right in every word checked (کھڑکی، مچھلی، گھڑی، چائے، بچہ). Isolated letters render correctly (ؤ ئ ں ے ہ آ ء). The right-to-left order is correct, with item 1 top-right and 10 top-left.
Black on white, crisp, small files. Rendered at 2× and 120–220 KB each, far under WhatsApp's 5 MB image limit.

2b. Item numbers are printed on the child's card

Every cell has a grey "N." label (grid-image.service.js:67, .num style :115-126). In a real EGRA the child's card is plain letters or words, and only the assessor's sheet is numbered. Numbers on the child's card distract a beginning reader, invite the child to read the numbers aloud, and make the page look like a test. They are also the reason the cells are tall and narrow (see 2d). The bidi fix that keeps "1." from flipping to ".1" (grid-image.service.js:120-125) is careful work, but the labels should not be on this card at all.

2c. Oral Reading Fluency is not a passage any more

orf_eng.png and orf_urd.png put each word in its own numbered box, 10 per row (img/orf_eng.png, img/orf_urd.png). ORF measures reading a connected story. A box-per-word grid turns it into a word-list task: the child loses the sentence flow, and the results cannot be compared with ORF norms or with the paper instrument.

In English, punctuation is tokenised separately: boxes 7, 14, 23, 36, 41 (","), 54, 62 and 68 are just "." or ",". The item bank has token_count: 68 against word_count_excl_punctuation: 60. The handler uses 68 as the item count (item-bank.service.js:50), and the scorer counts every attempted box that is not marked wrong as a correct item (rate-scorer.js:54: correct = attempted - incorrect). So a child who reads all 60 English words correctly scores 68 per minute, not 60. Urdu keeps "۔" attached to the word (60 tokens = 60 words), so the two languages are not scored the same way.

Recommendation: render ORF as a normal paragraph, left-aligned for English and right-aligned for Urdu, in large type with no numbers. Put the numbered word list only on the coach's side. Score against the 60 words.

2d. Size at phone distance

Measured from the real PNGs on a 6.1" phone, using WhatsApp's full-screen viewer (the image is fitted to the screen). The in-chat preview is smaller still.

Card	Image shape	Best way to hold the phone	Letter / word height on screen	Roughly like printed text of
Letters, English	920×1652, tall	upright (sideways makes it smaller)	capitals 3.5 mm, lower-case x-height 2.6 mm	~14 pt
Letters, Urdu	1368×2672, tall	upright	median 3.3 mm (1.4–5.0)	—
Words, Urdu	1880×1222, wide	sideways	median 4.3 mm (2.8 mm upright)	—
ORF, English	1754×954, wide	sideways	capitals 2.0 mm (1.1 mm upright)	~8 pt (~4 pt upright)
ORF, Urdu	2254×1332, wide	sideways	median 3.6 mm (2.1 mm upright)	—
Item-number labels	—	—	1.3 mm (letters), 0.7 mm (ORF)	—

See img/viewer_letter_id_urd.png, img/viewer_orf_eng_portrait.png and img/viewer_orf_eng_landscape.png. These PNGs are drawn at 2× so they look bigger on a monitor than on a real phone; the millimetres above are the true sizes.

The letter grids are tall because each cell stacks a number above the letter, and Nastaliq's tall line height adds more. A 10×10 card with no numbers, laid out wide, would roughly double the letter size on the same screen. The English ORF card is the worst: an 8-year-old reading 2 mm text against a clock is measuring eyesight and phone size as much as reading.

2e. Smaller issues on the cards

Capital I vs lower-case l. English letter item 82 is capital "I", which in Arial/system sans is a bare vertical bar identical to lower-case "l" (img/crops/letter_eng_row9.png). A child who says "l" is reading what they see. Use a school-style font where I and l differ (for example Andika, which is built for early literacy), or confirm the paper card's font. The two-storey "a" in Arial is also not the "a" most Grade 1 books teach.
Wide Urdu letters touch the cell borders. Measured per cell: the long top stroke of ک (items 4, 47, 72) and گ (14, 54, 100) runs into the box edge. These 6 of 100 cells have no clearance; ے, ض and ص come close (img/crops/letter_urd_row10.png shows item 100). It is legible, but it looks cramped. More cell padding would fix it.
Urdu listening passage. The English header "(FOR ENUMERATOR: READ THE FOLLOWING PASSAGE)" falls back to a condensed serif inside the Nastaliq image (img/view/listening_comprehension_urd.png). It is coach-facing and readable, but it looks unfinished; drop the line or render it in Urdu.
Stale comment. handler.js:165-167 says ORF uses the text fallback "by design", but ORF now has images.

2f. The bigger design question: one phone, two jobs

The grid caption tells the coach to "Show this to the child … then tap ▶️ Start" (handler.js:162). With one phone, the coach has to: 1. open the image full-screen and turn it to the child, 2. come back to the chat to tap Start (the child now sees the chat, not the card), 3. watch the clock, because nothing stops at 60 s, 4. remember the numbers of every wrong item while not looking at the screen, and 5. type them in afterwards ("Which item numbers did the child get WRONG? … e.g. 3,7,15", handler.js:197).

Paper EGRA and Tangerine both split this: the child reads from a printed card with no numbers, and the assessor marks a numbered copy. Recommendation: a printed, laminated child card set (letters, words, ORF story, EGMA numbers), with the phone as the coach's numbered marking sheet only. If the phone must be the child's card, the images need to lose the numbers and be laid out for the phone turned sideways, and the coach needs a separate way to record errors.

3. The Listening Comprehension Flow

Renders (Meta Flow chrome, real text from docs/flows/egra-egma-listening-comprehension-flow.json and the item bank, filled exactly as sendListeningComprehensionFlow does at handler.js:238-256): img/flow_read_passage_urd.png, img/flow_ask_questions_urd.png, img/flow_read_passage_eng.png, img/flow_ask_questions_eng.png. The chat message that opens it is at the bottom of img/chat_1_start.png. Every screen fits without scrolling.

Live status. Staging has EGRA_EGMA_LISTENING_FLOW_ID set (181656…), and the logs show 3 real submissions, so this Flow is published and in use. That goes beyond TESTING.md:153, which says it was never registered. I could not pull the published Flow JSON from Meta to diff it against the repo file, because that needs the staging WhatsApp token and was blocked.

3a. Meta constraints: all fine except one

Check	Result
Component types for version 6.3 (TextHeading, TextSubheading, TextBody, TextCaption, Footer)	valid
One Footer per screen	yes (flow.json:62, :127)
Footer labels ≤35	"Next — go to questions" 22, "Ready to record" 15
TextHeading/Subheading ≤80	33 / 19 / 7 / 9
TextCaption ≤409	243
Whole passage in one TextBody ≤4096	Urdu 307, English 360 characters; questions 205 / 223
Audio assumed inside the Flow	no. The Flow only confirms, and the voice note happens in chat (flow.json:8 states this correctly)
Mixed literal text + binding	flow.json:47 `"Listening Comprehension — ${data.language_label}"`. Meta's Flow JSON rule (as documented, e.g. Kapso's Flow JSON spec summary) is that `"Hello ${data.name}"` is invalid unless written as a backtick nested expression (v6.0+): "`'Listening Comprehension — ' ${data.language_label}`". No other Flow in `docs/flows/` mixes the two. Unverified on a phone: check whether the heading shows "— Urdu" or the literal `${data.language_label}`.

3b. Design problems in the Flow

The steps are in the wrong order for the recording. Screen 2 says "Ask the child each question below, one at a time" (flow.json:113). The child answers while the Flow is open, and the coach cannot record a WhatsApp voice note while a Flow is on screen. Only after "Ready to record" does the bot ask for "a voice note of the child's actual spoken response" (handler.js:284). By then the answers have already been given, so the recording is either a second asking (a different measurement) or nothing. Once the Flow closes, the questions are no longer on screen: sendVoiceNoteRequest re-sends only the passage (which currently fails, section 1), not the questions. Fix: ask the coach to start the voice note first ("press and slide up to lock"), then read the passage twice and ask the questions from a list that stays visible in chat.
No answer key. The questions screen and the later marking buttons show only the question. correct_answer_note (for example "Correct answer: Sunday") is never displayed anywhere in production code. The screen already says "Do not show this list to the child", so it is the natural place to show accepted answers.
Typos reach the coach verbatim. "1. On what did did Sara wake up early?" and "3. WHo walked with Sara to the shop?" (img/flow_ask_questions_eng.png). The Flow's __example__ (flow.json:27, :89) has the corrected wording, so anyone previewing in Flow Builder will not see the typo.
The same instruction twice. The passage data begins "(FOR ENUMERATOR: READ THE FOLLOWING PASSAGE)", right under "Read the passage below ALOUD to the child TWICE" (img/flow_read_passage_*.png).
"WhatsApp will ask you to (1) record…" (flow.json:125). It is Rumi, not WhatsApp, that asks.
English screens around an Urdu passage. One Flow serves both languages, but every label is hardcoded English. The labels could be passed in as ${data.*} values so the Urdu run gets Urdu instructions.

4. Coach-facing messages

Chat mockups with the handler's exact strings (placeholder teacher and student names): img/chat_1_start.png (/egra → intro → class list → pick → Flow message) and img/chat_2_after_flow.png (Flow sent → voice-note ask → voice note → "Item 1/6" marking).

Language. Every coach string is English and hardcoded in the handler (handler.js:124-132, 160-162, 180-202, 245-246, 284-285, 340-342, 477-479, 545, 798). The handler does not use shared/config/ux-strings.js, which is where other features now keep their en/ur strings. The questions are Urdu for the Urdu subtests, so a coach sees "Item 1/6:" in English above an Urdu question. Rule 20 (language protocol) applies before any translation work.
Instructions that would invalidate subtests.
The intro says "You will read out each item to the child" (handler.js:479). In the 5 reading subtests (letters, words, ORF) the child reads and the coach must not.
The grid caption says "Show this to the child (or read it out per the item type)" (handler.js:162). The same problem, and "per the item type" is not something a coach can act on.
Timer and discontinue copy.
"(60s cap — please self-time this)" (handler.js:149) sits next to the bot's own "Timer running" (handler.js:186), but nothing stops or warns at 60 s, and the rate is computed over whatever time passes before Stop is tapped.
The Discontinue button text says "0 correct on the first row of 10" for every grid subtest. The Urdu word list's own rule is "0-1 correct". Tapping Discontinue always records items 1–10 as wrong (handler.js:596), so a child who read 1 of 10 is stored as 0.
ORF has no discontinue rule in the item bank but still gets the button.
EGMA has no child-facing stimulus at all. Items like "Choose the larger number: 11 or 16" or the 🔺🔵 pattern exist only in the coach's marking message, next to the ✅/❌ buttons. Three represent items ("How many bananas are there?", "What box has more apples?", "What number does the picture below show?") are flagged needs_real_image with a note that "the coach is shown the item's text description". But sendCurrentQaItem sends only item.prompt (handler.js:223-230), and no production code reads correct_answer_note, placeholder_note or needs_real_image. The coach gets a question about a picture that does not exist.
Error recovery wipes the session. Any error sends "❌ Something went wrong with the assessment. Please type /egra to restart." and clears state (handler.js:551-552, 613-614, 693-694). The coach loses everything done so far with that child.
Small copy points.
The list header says "Pick a class", but each row's title is the teacher's name (handler.js:115).
"Recorded: 45/68 correct, 44.2 items/min" uses internal words ("items/min") and the 68 includes punctuation (section 2c).
A randomly picked child who is absent has no "pick another child" option.
Length caps: all within limits (measured in code points):
Buttons: ▶️ Start 8, ⏹ Stop 6, 🚫 Discontinue 13, ✅ Correct 9, ❌ Incorrect 11, ⏭ Pass 6 (cap 20).
List: header "Pick a class" 12 / 60, footer 36 / 60, button "Select class" 12 / 20, section "Classes" 7 / 24, row "Type a name instead" 19 / 24, its description 52 / 72.
Flow message: header "📋 Listening Comprehension" 25 / 60, CTA "Start" 5.
Dynamic row titles and descriptions are cut to 24 / 72 by truncateForWhatsAppRow (handler.js:95-98). That counts UTF-16 units, which is safe for Urdu names and only over-trims emoji.
Gender-neutral: yes. The copy says "the child", "their answer", "you, the coach". The only she/her are in the stories (Sara, Ayesha), which is content, not how we address anyone.
Emoji: functional, not decorative. Emoji are used on buttons and as subtest markers (📋 📖 🎙️). There are no emoji sign-offs. "⏹" (U+23F9, sent without the emoji variation selector) rendered as a plain black square in my mockup render; worth checking on a real Android phone, or just use the word "Stop".

5. What is done well

Deterministic HTML→PNG instead of AI-generated stimulus (grid-image.service.js:7-10). This is the right call: one mis-drawn letter would corrupt a test item, and the Urdu came out correct.
Embedded Nastaliq font as a base64 data URI (grid-image.service.js:17-20, 71-77), so there is no network fetch and no silent font fallback. Every Urdu word and letter I checked is correctly shaped and joined.
Round-trip check: extractItemsFromHtml diffs the rendered HTML against the item bank before upload (grid-image.service.js:144-166).
Careful bidi handling of number labels inside right-to-left grids (grid-image.service.js:120-125). The technique is correct, even though the numbers should go.
Correct reading of Meta's limits: no audio inside the Flow, with the voice note in chat (flow.json:8). It is a static Flow with no endpoint, so there is no server to go down. "Do not show this screen to the child" is on both screens.
Honest scoring framing. "your own marking is the official score" and "no AI or audio auto-scoring is used" set the right expectation.
Button labels are short, clear and within limits, and the flow degrades to free-text name entry when the roster is empty.
Re-sending the passage after the Flow closes (bd-2712) is the right instinct. It just needs to work (section 1) and to include the questions (section 3b).

6. What to do, in order

Make the images arrive. Presign or download with credentials (r2.js:545 / :366), check the return value, and fall back to text on failure. Add a test that calls the real download step against a mocked HTTP 400, not a mocked sendImageFromUrl → true.
Remove the item numbers from child-facing cards. Render ORF as a paragraph, and score English ORF on 60 words, not 68 tokens.
Decide the child-card medium. Printed card plus phone as score sheet is recommended. If it stays on the phone, lay cards out for sideways viewing at roughly twice today's type size.
Fix the two "read it out" instructions, and show accepted answers to the coach for every question.
Reorder the Listening step (record first, then administer), keep the questions visible in chat, and fix the item-bank typos at the source.
Move all coach strings to the strings catalog with Urdu, following the language-protocol skill.
EGMA: make the child-facing number and pattern cards, or pull the 3 picture items until the pictures exist.
Smaller fixes: - Discontinue copy per subtest. - A 60-second reminder. - A school font with a distinct I/l. - Cell padding for wide Urdu letters. - Backtick form for the Flow heading. - "Rumi" rather than "WhatsApp" in the caption.

Files

Images reviewed (downloaded from R2, read-only): img/letter_id_eng.png, img/letter_id_urd.png, img/familiar_word_urd.png, img/orf_eng.png, img/orf_urd.png, img/listening_comprehension_eng.png, img/listening_comprehension_urd.png (viewing copies in img/view/, zoomed crops in img/crops/).

Renders produced (all looked at): img/flow_read_passage_urd.png, img/flow_ask_questions_urd.png, img/flow_read_passage_eng.png, img/flow_ask_questions_eng.png, img/chat_1_start.png, img/chat_2_after_flow.png, img/chat_3_grid_as_delivered.png, img/chat_3_grid_as_intended.png, img/viewer_letter_id_urd.png, img/viewer_orf_eng_portrait.png, img/viewer_orf_eng_landscape.png.

Generator: render_design_review.py (re-run: python3 render_design_review.py; it uses the html-mockups chat rig; HTML intermediates are in html/).

Not verified: - The published Flow JSON on Meta (see 3a), so the heading binding is unconfirmed. - The in-chat image preview crop in chat_3_grid_as_intended.png, which is an approximation. The full-screen viewer renders are exact fits.