The reading pipeline and the live ASER tool, file by file
Written: 2026-09-15 (traced by hand from origin/main @ 2d5a3925, after the agent trace hit a session limit). Repo: hyasin270/whatsapp-ai-bot (02_Main Rumi Bot). Note the local main checkout is behind origin — the shared/services/aser/ directory only exists on origin/main.
| Stack | Entry | Unit | Scoring | HITL | Status |
|---|---|---|---|---|---|
| Reading assessment v3 (PROJ-047) | menu → WhatsApp Flow READING_ASSESSMENT_FLOW_V3_ID |
one child, one generated passage (letters / words / sentences / paragraph / story) | Soniox STT → FluencyService alignment → WCPM/LCPM → ASER level; Azure PA for English pronunciation |
none — teacher sees the report | live PK-wide since Jul 2026 |
| ASER assessment tool (bd-2716 → bd-2767, Aug–Sep 2026) | /asertest text command, or the coach region-flow (aser-coach-region-flow.json v6.3: REGION → MARKAZ → SCHOOL → REVIEW_STUDENTS) |
one child, three subjects (local literacy, English literacy, numeracy), each an adaptive ladder | same reading pipeline on fixed ASER content + numeracy vision/STT | Auto (Rumi) or Manual (enumerator) per subject, plus a "Correct it" coach-level override; both values persisted | live on main; field team rejected it on 2026-09-11 |
voice note → R2 reading_audio/{userId}/{assessmentId}_{ts}.{ext}
→ TranscriptionService.transcribeReading(rowId, audioUrl, lang, passageText) shared/services/reading/transcription.service.js:70
Soniox stt-async-v3, diarization ON, language hint, passage-biased context (BUG-076 / bd-1150 / bd-2709)
→ FluencyService.calculateFluencyMetrics(...) shared/services/reading/fluency.service.js:40
cleanTranscriptForAlignment → collapseSpelledWords → tokenize (Arabic-block punctuation stripped, bd-1935)
alignWords = Levenshtein alignment → omissions / substitutions / insertions (:366)
timeElapsed from word_timestamps (:463); self-corrections (:502)
WCPM for connected text, LCPM for letter grids (:139-153); accuracy = correct / passage words
→ PronunciationService.assessPronunciation shared/services/reading/pronunciation.service.js:55
English: Azure Pronunciation Assessment, Phoneme granularity, NBestPhonemes=5, prosody+miscue (:107-181)
Urdu / other: GPT-4o-audio word-level fallback (Azure has no ur locale)
→ ComprehensionService (story questions, LLM-graded) comprehension.service.js
→ aser-level.service PASS_THRESHOLD = 80 % accuracy → level
→ reading_assessments row (+ report.service PDF, voice-feedback mp3)
reading_assessments already holds what an EGRA row needs: passage_type, passage_text, audio_url, audio_duration_seconds, num_speakers_detected, transcript_text, word_timestamps JSONB, words_read, words_correct, wcpm, accuracy_percentage, time_elapsed_seconds, pronunciation_data JSONB, errors JSONB (omission/substitution/insertion/repetition with position), self_corrections_count, grade benchmarks, on_track. Migration bd-2716 added aser_session_id, content_source ('generated'|'aser_fixed'), and the HITL pair rumi_level / rumi_label / rumi_comprehended vs coach_level / coach_user_id / coach_reviewed_at / rumi_coach_delta; bd-2735 added aser_scored_by ('rumi'|'enumerator'); bd-2720 added aser_ladder_node.
numeracy_assessments (migration 063) already has the same two-column discipline for math: auto_grade_result / auto_grade_confidence / auto_grade_extracted_answer vs human_confirmed_result / human_confirmed_by / human_confirmed_at, discrepancy_flag, recognition_audio_url, computation_photo_url, cost columns.
aser_sessions (bd-2716, bd-2732) = user_id (coach), student_id, student_name, grade_level 1–5, school_id, region, status, completed_at. aser_results_flat (bd-2746) is the analyst view: one row per session with the final level per subject = COALESCE(coach_level, rumi_level).
shared/services/aser/aser-ladder.js)Per subject a directed graph of nodes with pass/fail targets, fixed content (aser-content-bank.js), PASS_THRESHOLD = 80:
- Local literacy (Urdu): LOCLIT_LETTERS → LOCLIT_WORDS → LOCLIT_SENTENCES → LOCLIT_STORY (story has three outcomes: struggled / fluent-no-comprehension / fluent-comprehended). Kiswahili variant starts at Sentences.
- English literacy: ENGLIT_CAPITAL → ENGLIT_SMALL → ENGLIT_WORDS → ENGLIT_SENTENCES.
- Numeracy: NUM_DIGIT_1_9 → 10_99 → 100_999 → NUM_SUB_2D → NUM_SUB_3D → NUM_DIVISION → NUM_FRACTIONS (bd-2743), numbers by audio, computation by photo (NumeracyVisionService), batteries of 3 problems (pickBattery).
- isOnTarget(subject, grade, level, comprehended) gives the grade-target verdict.
shared/handlers/aser-flow.handler.js, Redis key aser:state:{phone})awaiting_region (auto-filled from users.region, bd-2737) → awaiting_school (free text, or the coach region-flow / AEO roster picker, bd-2734/2739)
→ awaiting_grade (picker) → awaiting_name → awaiting_subject_choice (LIST: each subject × Auto|Manual, bd-2735)
→ awaiting_reading_audio | awaiting_numeracy_audio | awaiting_numeracy_photo | awaiting_story_comprehension_audio
→ awaiting_coach_review (Auto path: Rumi's level shown, "Correct it" re-enters live administration, bd-2761; coach_level validated against the subject ceiling, bd-2747)
| awaiting_manual_outcome (Manual path: coach taps pass/fail; audio still captured; background eval-only re-score fills rumi_* for comparison, bd-2735)
→ loop to next subject → final report (bd-2750)
The coach region-flow samples 3–5 students round-robin across a school's student_lists (aser-student-sampling.service.js, default 5) and requires a real class list (bd-2764/2766). Roles allowed: aeo, training_manager, coach, cluster_coordinator (bd-2749/2762).
| Complaint | On main today |
|---|---|
| "type the school name every time" | region auto-filled (bd-2737); school picker exists only where an AEO roster or the region-flow class list exists (bd-2734/2739) — free text elsewhere |
| "one question per message, 10–15 min per child" | still one prompt → one voice note per ladder rung; the ladder is adaptive (stops on first fail) but each rung is a WhatsApp round trip |
| "AI unreliable on Pakistani accents; give us a human toggle" | Manual mode per subject (bd-2735) + "Correct it" override (bd-2747/2761) — shipped |
| "no internet in schools; upload later" | not addressed — every rung is online |
| "photo of written work is a hassle" | numeracy computation still needs a photo |
| "ASER is separate from observation in NIETE" | tool is standalone; no observe hook |
aser_sessions + reading_assessments + numeracy_assessments already carry session, per-child identity, audio URL, transcript, word timestamps, per-word errors, the AI-vs-coach pair, and who-scored. An EGRA subtask = one reading_assessments row with a new passage_type value (letter_grid, nonword_grid, word_grid, orf, listening_comp) and content_source='egra_fixed'; math subtasks = numeracy_assessments rows. Per-item coach corrections fit errors JSONB / pronunciation_data JSONB with a coach_verdict field, or one new JSONB column coach_item_edits (Rule 15: defend it).FluencyService.alignWords + calculateTimeElapsed from Soniox word timestamps). What is missing is (a) a 60-second window cut from timestamps with the toolkit formula score / (60 − time_remaining) × 60, (b) stop-rule logic, (c) per-item confidence for Urdu letters/nonwords (needs the forced-aligner/GOP layer from research/02), (d) a per-item HITL form instead of a per-subject level override.sequential-reading.service.js, pick / auto-pick / attendance / next-or-end, bd-2350 fix) and the ASER sampler (aser-student-sampling.service.js) are the "5 children" loop already built; the EGRA flow reuses them.../probes/) shows a single continuous recording per block can be segmented by STT timestamps and labelled by the LLM, so the coach records once per block, not once per item.Orenda-Project/NIETE-Rumi, which has the observe stack but not the ASER tool or the sequential-reading loop — those must be ported (see research/04 §8 and the "code ported but dependency wasn't" warning). For Rawalpindi/PK it lands in the main bot where the ASER tool already lives.