Sixty-second windows: every audio and text model vs the enumerator, with baseline and human floor
| arm | model | subtask | n | count_r | MAE | bias | within5 | word_precision | word_recall | baseline_r | baseline_MAE | human_floor_agree | cost_per_window |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| audio | google/gemini-3.1-pro-preview | idwrd_urd | 33 | 0.99 | 1.7 | 0.1 | 97 | 0.75 | 0.79 | 0.96 | 6.6 | 50.8 | 0.0566 |
| audio | openai/gpt-audio | idwrd_urd | 12 | 0.93 | 3.3 | 1.3 | 92 | 0.32 | 0.44 | 0.98 | 6.8 | 50.8 | 0.0294 |
| audio | google/gemini-3.8-flash | idwrd_urd | 149 | 0.93 | 3.8 | 0.2 | 86 | 0.68 | 0.59 | 0.86 | 9.0 | 50.8 | 0.0098 |
| audio | google/gemini-3.5-flash-lite | idwrd_urd | 35 | 0.76 | 8.0 | -0.9 | 69 | 0.26 | 0.52 | 0.9 | 8.6 | 50.8 | 0.0032 |
| audio | openai/gpt-audio-mini | idwrd_urd | 35 | 0.6 | 11.9 | -2.1 | 57 | 0.19 | 0.64 | 0.9 | 8.6 | 50.8 | 0.003 |
| audio | xiaomi/mimo-v2.5 | idwrd_urd | 34 | 0.6 | 11.9 | 7.3 | 62 | 0.25 | 0.31 | 0.9 | 8.6 | 50.8 | 0.0011 |
| audio | google/gemini-2.5-flash | idwrd_urd | 35 | 0.56 | 12.8 | 10.9 | 57 | 0.3 | 0.17 | 0.9 | 8.6 | 50.8 | 0.0043 |
| audio | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | idwrd_urd | 15 | 0.52 | 15.3 | 15.3 | 47 | 0.0 | 0.0 | 0.94 | 8.5 | 50.8 | 0.0 |
| audio | mistralai/voxtral-small-24b-2507 | idwrd_urd | 10 | 22.8 | 22.8 | 40 | 0.0 | 0.0 | 0.94 | 9.5 | 50.8 | 0.0065 | |
| text | anthropic/claude-sonnet-5 | idwrd_urd | 27 | 0.93 | 4.2 | -3.7 | 89 | 0.46 | 0.83 | 0.94 | 6.1 | 50.8 | 0.0379 |
| text | google/gemini-3-flash-preview | idwrd_urd | 35 | 0.9 | 6.7 | -6.1 | 63 | 0.31 | 0.86 | 0.9 | 8.6 | 50.8 | 0.0025 |
| text | openai/gpt-5-mini | idwrd_urd | 34 | 0.81 | 8.9 | -8.4 | 65 | 0.23 | 0.85 | 0.92 | 8.0 | 50.8 | 0.0077 |
| text | deepseek/deepseek-v3.2 | idwrd_urd | 35 | 0.86 | 11.5 | -11.1 | 40 | 0.2 | 0.92 | 0.9 | 8.6 | 50.8 | 0.0008 |
| audio | openai/gpt-audio | orf_eng | 11 | 0.94 | 4.2 | -4.0 | 73 | 0.18 | 0.33 | 0.71 | 13.8 | 61.5 | 0.0293 |
| audio | google/gemini-3.5-flash-lite | orf_eng | 25 | 0.95 | 5.1 | -2.4 | 72 | 0.25 | 0.38 | 0.86 | 11.3 | 61.5 | 0.0033 |
| audio | google/gemini-3.8-flash | orf_eng | 104 | 0.89 | 5.8 | -0.2 | 74 | 0.31 | 0.38 | 0.8 | 11.2 | 61.5 | 0.0092 |
| audio | google/gemini-3.1-pro-preview | orf_eng | 24 | 0.94 | 6.1 | -1.5 | 71 | 0.19 | 0.31 | 0.88 | 10.4 | 61.5 | 0.0573 |
| audio | google/gemini-2.5-flash | orf_eng | 25 | 0.86 | 7.7 | 5.2 | 68 | 0.22 | 0.13 | 0.86 | 11.3 | 61.5 | 0.0046 |
| audio | xiaomi/mimo-v2.5 | orf_eng | 23 | 0.82 | 9.1 | 4.2 | 52 | 0.29 | 0.28 | 0.85 | 11.7 | 61.5 | 0.0011 |
| audio | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | orf_eng | 12 | 21.5 | 21.3 | 42 | 0.0 | 0.0 | 0.88 | 10.7 | 61.5 | 0.0 | |
| audio | mistralai/voxtral-small-24b-2507 | orf_eng | 10 | 22.2 | 22.0 | 30 | 0.0 | 0.0 | 0.89 | 12.8 | 61.5 | 0.0065 | |
| audio | openai/gpt-audio-mini | orf_eng | 24 | -0.05 | 31.7 | -17.0 | 29 | 0.12 | 0.77 | 0.85 | 11.6 | 61.5 | 0.0033 |
| text | anthropic/claude-sonnet-5 | orf_eng | 20 | 0.86 | 9.2 | -8.4 | 55 | 0.2 | 0.42 | 0.88 | 9.2 | 61.5 | 0.033 |
| text | google/gemini-3-flash-preview | orf_eng | 25 | 0.86 | 11.3 | -10.8 | 44 | 0.18 | 0.49 | 0.86 | 11.3 | 61.5 | 0.0027 |
| text | deepseek/deepseek-v3.2 | orf_eng | 23 | 0.84 | 11.6 | -10.7 | 43 | 0.19 | 0.48 | 0.85 | 11.7 | 61.5 | 0.0016 |
| text | openai/gpt-5-mini | orf_eng | 24 | 0.47 | 20.7 | -19.0 | 42 | 0.16 | 0.77 | 0.85 | 11.3 | 61.5 | 0.0082 |
| audio | google/gemini-3.1-pro-preview | orf_urd | 28 | 0.97 | 2.6 | -2.0 | 89 | 0.59 | 0.81 | 0.93 | 15.5 | 54.3 | 0.0551 |
| audio | google/gemini-3.8-flash | orf_urd | 129 | 0.9 | 5.5 | -1.2 | 74 | 0.48 | 0.52 | 0.87 | 15.7 | 54.3 | 0.0097 |
| audio | google/gemini-3.5-flash-lite | orf_urd | 34 | 0.84 | 7.0 | -0.1 | 62 | 0.39 | 0.68 | 0.91 | 15.4 | 54.3 | 0.0037 |
| audio | openai/gpt-audio | orf_urd | 15 | 0.64 | 13.1 | -9.2 | 27 | 0.13 | 0.81 | 0.9 | 20.0 | 54.3 | 0.0302 |
| audio | google/gemini-2.5-flash | orf_urd | 34 | 0.49 | 13.6 | 10.1 | 62 | 0.57 | 0.25 | 0.91 | 15.4 | 54.3 | 0.0047 |
| audio | xiaomi/mimo-v2.5 | orf_urd | 33 | 0.51 | 13.8 | 7.7 | 58 | 0.18 | 0.15 | 0.91 | 15.6 | 54.3 | 0.0014 |
| audio | nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free | orf_urd | 15 | 17.8 | 17.8 | 53 | 0.0 | 0.0 | 0.94 | 17.1 | 54.3 | 0.0 | |
| audio | mistralai/voxtral-small-24b-2507 | orf_urd | 11 | 20.0 | 20.0 | 45 | 0.0 | 0.0 | 0.91 | 17.8 | 54.3 | 0.0065 | |
| audio | openai/gpt-audio-mini | orf_urd | 34 | 0.15 | 25.9 | -11.1 | 29 | 0.1 | 0.72 | 0.91 | 15.4 | 54.3 | 0.0036 |
| text | anthropic/claude-sonnet-5 | orf_urd | 24 | 0.95 | 8.5 | -8.1 | 46 | 0.19 | 0.78 | 0.97 | 12.6 | 54.3 | 0.0418 |
| text | openai/gpt-5-mini | orf_urd | 31 | 0.85 | 12.8 | -12.4 | 26 | 0.18 | 0.89 | 0.94 | 14.1 | 54.3 | 0.0081 |
| text | google/gemini-3-flash-preview | orf_urd | 34 | 0.87 | 14.9 | -14.8 | 29 | 0.16 | 0.91 | 0.91 | 15.4 | 54.3 | 0.0029 |
| text | deepseek/deepseek-v3.2 | orf_urd | 34 | 0.8 | 21.5 | -21.5 | 24 | 0.13 | 0.97 | 0.91 | 15.4 | 54.3 | 0.0007 |