Soniox timestamps, diarisation and the "STT cuts, LLM labels" rule
Probe 2 — production STT (Soniox stt-async-v3) on the same session (2026-09-15)
Input: the Urdu block of the Probe-1 recording, 01:20–08:30, 16 kHz WAV from the 8 kHz source. Language hint ur, diarization on (2–3 speakers). Script probe2_soniox.py; outputs probe2_soniox_probe1_urdu_block.wav.{json,md}. 12 s to transcribe 7 minutes.
Findings
Timestamps are exact and the LLM's were not. Soniox places the enumerator's "start" at 03:09 and "bas" (stop) at 04:09 — a 60.0-second window, matching the tablet timer to the second. Gemini 2.5 Pro had put the letter task at 04:22–05:39 (≈75 s late); Gemini Flash put listening comprehension at 01:30 (≈20 s early). The slice I cut from Gemini's timestamps for the "letters" actually contained the familiar-word list. Design rule: STT word timestamps cut the audio; the LLM labels and scores the cuts. Never trust an audio LLM's mm:ss.
Diarization separates child from enumerator on 8 kHz classroom audio.spk1 = enumerator, spk2 = child, consistently through story, questions, timed reads and comprehension. This is what lets us score only the child's tokens and audit the enumerator's protocol from the same file.
Urdu letter names transcribe usably. The child's 60-second run comes back as a letter-name string (دال رے دال میم کاف شین جیم پے تے بے الف …), ~80 tokens including repeats (ذال ذال ذال) — the tablet says 74 attempted / 70 correct. Counting against the printed grid needs alignment, but the raw material is there. Note the protocol in this study is letter names, not letter sounds.
Familiar words and the ORF passage come back nearly verbatim (بچہ دروازہ پانی کتاب … ; the Bilal/Jhelum passage with the child's self-corrections — والد والد, جمع کے جمع کے — preserved). Self-corrections and repeats are exactly what a WCPM scorer must handle; Soniox keeps them rather than smoothing them away.
Comprehension answers are short and separable ("ٹیبل", "پتھر", "اپنے والدین کے ساتھ") — trivially gradeable by an LLM against the answer key.
Subtask boundaries recovered from STT (this session)
Subtask
Start
End
Length
Urdu listening comprehension (story ×2 + 6 Qs)
01:51
03:01
70 s
Urdu letter identification (timed)
03:09
04:09
60 s
Urdu familiar words
04:21
05:13
52 s
Urdu ORF passage
05:19
06:27
68 s
Urdu reading comprehension
06:27
07:00
33 s
English listening comprehension
07:00
…
Implication for the stack
A two-model pipeline covers reading end-to-end on our existing vendors: Soniox (timestamps, diarization, Urdu/English transcript) → Gemini-Pro-class audio LLM or text LLM (segment labelling, per-item verdicts, comprehension grading, protocol audit), with Azure Pronunciation Assessment / SpeechAce only for the English phoneme layer and an open forced-aligner for Urdu item confidence. Everything in this probe ran on 8 kHz audio-audit quality; WhatsApp voice notes are 16 kHz Opus and should only improve it.