# Parity against the ONNX reference

A format conversion is only worth publishing if it is provably the same model.
Comparing interval *counts* catches gross breakage and nothing else — every
precision setting tested here reproduces all 830 intervals. So the check goes
three levels deep: interval counts, then every interval boundary, then the
downstream windows those boundaries actually feed.

## Method

- **Reference**: the upstream ONNX model via `onnxruntime` 1.18, CPU execution
  provider, 4 worker threads, batch 1 (the export pinned its output to batch 1).
- **Candidate**: `vad_torch.py` from this repo loading `model.safetensors`,
  CUDA, batch 8.
- Both backends are fed the **identical preprocessed waveform**, so the
  comparison isolates the model. Chunking (30 s), threshold (0.4, in logit
  space) and interval merging are the same code path for both.
- **Material**: 7 tracks, 6129.9 s (102.2 min) of 16 kHz audio, Japanese
  close-mic speech — much of it quiet or whispered, which is the acoustic case
  this model was tuned for and the one where a VAD's boundaries are least
  stable. 2299.1 s of it is speech, so the set is 37.5% active.
- Hardware: RTX 3070 Laptop (8 GB), Windows, torch 2.2.2+cu121.

Track identities are not published; they are numbered in source order. The set
is a fixed, ordered corpus so the run is repeatable on the machine that has it.

## Result: fp32, TF32 off

| track | audio | mix route | L/R corr | intervals (ref → cand) | boundaries exact |
|---|---|---|---|---|---|
| 01 | 333.7 s | mid | 0.78 | 70 → 70 | 140 / 140 |
| 02 | 728.6 s | mid-blend | 0.49 | 141 → 141 | 282 / 282 |
| 03 | 860.8 s | max-side | 0.10 | 142 → 142 | 284 / 284 |
| 04 | 1187.2 s | max-side | 0.28 | 130 → 130 | 260 / 260 |
| 05 | 621.3 s | mid-blend | 0.59 | 80 → 80 | 160 / 160 |
| 06 | 1151.1 s | max-side | 0.05 | 134 → 134 | 268 / 268 |
| 07 | 1247.3 s | max-side | 0.12 | 133 → 133 | 266 / 266 |
| **total** | **6129.9 s** | | | **830 → 830** | **1660 / 1660** |

```
interval counts : identical on all 7 tracks (830 intervals)
boundaries      : 1660 compared, 1660 exact (100.0%), worst 0.000s
ASR windows     : 559 built, 559 identical (100.0%)
```

Bit-identical, not merely close. Worst boundary delta is 0.000 s — every one of
the 1660 interval endpoints lands on the same 20 ms frame with the same value.

The L/R correlation column spans 0.05 to 0.78, so the set exercises genuinely
different stereo material rather than seven takes of one condition.

## Result: fp16

fp16 is 2.1× faster and is *not* the default. Why:

| track | boundaries moved | worst delta |
|---|---|---|
| 01 | 2 / 140 | 0.020 s |
| 02 | 3 / 282 | 0.060 s |
| 03 | 1 / 284 | 0.020 s |
| 04 | 1 / 260 | 0.040 s |
| 05 | 1 / 160 | 0.100 s |
| 06 | 2 / 268 | 0.220 s |
| 07 | 2 / 266 | 0.220 s |
| **total** | **12 / 1660 (99.3% exact)** | **0.220 s** |

```
interval counts : identical on every track (830 intervals)
boundaries      : 1660 compared, 1648 exact (99.3%), worst 0.220s (11 frames)
ASR windows     : 559 built, 8 of 559 moved (98.6% identical), worst 1.620s
```

Read the three rows in order and note that only the last one is alarming.
Interval counts are perfect. Boundaries are 99.3% exact. But 12 moved
boundaries become **8 changed ASR windows**, because a boundary that shifts
0.220 s can cross a 10 s window-splitting threshold and hand ~1.6 s of audio to
a different window — which changes what a downstream ASR model transcribes.

This is the argument for checking the consumer of the intervals rather than the
intervals themselves. If nothing downstream of your VAD is sensitive to window
placement, fp16 is a good trade.

TF32 is the other half of the default. Ampere and later run fp32 matmul and
convolution through TF32 unless told not to, dropping the mantissa to 10 bits.
With TF32 left on, fp32 lands at 99.8% exact boundaries and 2 of 559 windows
moved — better than fp16, still not exact. `load_vad` disables it when fp32 is
requested. It measured the same speed either way, so the fidelity is free.

## Timing

VAD stage only, same 6129.9 s of audio, same machine:

| backend | VAD stage | speedup | vs. realtime |
|---|---|---|---|
| ONNX, CPU, 4 threads, batch 1 | 88.56 s | 1.0× | 69× |
| torch, CUDA, fp32, batch 8, TF32 off | 8.26 s | **10.7×** | 742× |
| torch, CUDA, fp16, batch 8 | 3.89 s | 22.8× | 1572× |

Per-track VAD seconds, ONNX → fp32 → fp16:

| track | ONNX | fp32 | fp16 |
|---|---|---|---|
| 01 | 5.12 | 0.99 | 0.78 |
| 02 | 10.68 | 0.97 | 0.44 |
| 03 | 12.28 | 1.07 | 0.47 |
| 04 | 16.89 | 1.45 | 0.61 |
| 05 | 9.11 | 0.79 | 0.33 |
| 06 | 16.66 | 1.44 | 0.62 |
| 07 | 17.82 | 1.55 | 0.64 |

Two caveats on these figures. The ONNX side is CPU, because the dependency
conflict described in the README is the reason this conversion exists — it is
not a GPU-to-GPU comparison and does not claim to be. And track 01 barely
improves (5.12 → 0.99) compared to the rest, because at 333.7 s it is short
enough that fixed costs dominate; the ratio grows with track length.

## Reproducing this

`model.safetensors` is checked against its source on conversion: all 136
tensors are read back out of the written file and compared bit-for-bit against
the arrays extracted from the ONNX graph. sha256 of the published file:

```
39e5e22218d89647afb926163a4ee5f7989e1cfe840d657c7cb226997b9e48e0
```

To re-derive the weights from scratch rather than trusting this repo, run
`extract_weights.py` against the upstream `model.onnx` and compare.