smart-turn-tamil

Semantic end-of-turn detection for Tamil — given the last 8 seconds of a speaker's audio, predicts whether they have finished their turn.

Smart Turn v3's architecture and training recipe, trained on real Tamil telephone conversations (116 two-party calls). ONNX, CPU-only, drop-in for Pipecat and LiveKit.

Variants

tiny int8 is the recommended variant. Held-out split: 4,168 clips from 30 unseen calls, threshold 0.5.

variant	precision	accuracy	ROC-AUC	FP/N	size	p50
tiny	int8	83.71%	0.905	7.94%	8.7 MB	83 ms
tiny	fp32	83.35%	0.904	7.73%	32 MB	133 ms
base	int8	86.13%	0.921	9.17%	21 MB	143 ms
base	fp32	86.23%	0.922	8.95%	81 MB	232 ms

For reference on the same split: majority class 63.08%, and smart-turn-v3.2 int8 zero-shot 70.30% / AUC 0.751.

tiny matches upstream's architecture, so it is a true drop-in replacement. base is +2.4 points for 2.4× the size and 1.7× the latency.

int8 is dynamic, not static — static was measured and rejected (−4.03 at tiny, −12.76 at base). Latency is 1 thread, batch 1, idle i5-12450H, inference only; add ~12 ms for mel.

FP/N is Smart Turn's convention (FP/N + FN/N = error rate), not the standard FP/(FP+TN). The two differ by roughly 3×.

Usage

pip install 'smart-turn-livekit[livekit]'

LiveKit Agents — weights download on first use and are cached.

from smart_turn_livekit import SmartTurnDetector
turn_detection = SmartTurnDetector(model="smart-turn-tamil-tiny")

Pipecat — no adapter needed; its built-in analyzer takes any Smart Turn ONNX.

from smart_turn_livekit import resolve_model
from pipecat.audio.turn.smart_turn.local_smart_turn_v3 import LocalSmartTurnAnalyzerV3

turn_analyzer = LocalSmartTurnAnalyzerV3(
    smart_turn_model_path=str(resolve_model("smart-turn-tamil-tiny")),
)

Direct ONNX — input input_features, shape (batch, 80, 800), log-mel at 16 kHz. The int8 graphs need onnxruntime>=1.24: they carry ConvInteger, which the CPU provider has no kernel for before that, and raise NOT_IMPLEMENTED on the first call. The smart-turn-tamil.onnx fp32 graphs carry no quantised ops and load on any runtime.

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("smart-turn-tamil-tiny/smart-turn-tamil-int8-dynamic.onnx")
p = sess.run(None, {"input_features": feats})[0].reshape(-1)   # (batch,) in [0,1]
complete = p > 0.5

Two things that fail silently if you get them wrong:

• The output is named logits but is already a sigmoid. Applying another maps [0,1] onto [0.5, 0.73] and every clip predicts complete.
• Features must match WhisperFeatureExtractor(chunk_length=8) with do_normalize=True — not the Whisper default.

Threshold

0.5 is inherited from the training loop and neither variant peaks there, but tuning it for accuracy on dev lost base 1.13 points on test — so 0.5 ships as the default. Targeting a false-positive rate transfers better than targeting accuracy:

operating point	tiny	base	effect
inherited (default)	0.50	0.50	figures above
polite	0.75	0.92	FP/N roughly halves, costs ~3 points of accuracy

Both picked on dev, then measured once on test.

For scale

Smart Turn v3.2's published per-language figures. These come from a different benchmark on TTS-generated audio; this model is scored on real narrowband telephone conversation, so the two are not directly comparable:

	accuracy	FP/N
Hindi	90.11%	8.57%
Bengali	83.80%	10.90%
Marathi	82.43%	15.12%

What serving costs

The figures above score pre-cut clips, ending 0.20 s past the speech offset. A production endpointer cuts a different window, so the same labelled boundaries were replayed through the real Silero VAD and the real streaming adapter and scored both ways in one run:

	pre-cut clip	live window
accuracy	86.12%	83.51%
FP/N	5.21%	6.51%

−2.60 points, identical verdict on 90.9% of boundaries (n=461, McNemar p=0.09 — consistent in direction, not formally significant). The cause is a 90 ms window shift: training clips end 0.20 s past the speech offset, the VAD closes at 0.29 s.

Coverage 92.2%. LiveKit will not request a prediction below min_silence_duration + 50 ms, so the model is never consulted on the shortest pauses. Those sit outside the product rather than being errors — which is also why the pre-cut column reads 86.12% against 83.35% for the full split: the boundaries a VAD surfaces are the easier ones.

Verified end-to-end as a Tamil voice agent on LiveKit Agents 1.7 with a Sarvam STT/LLM/TTS stack, and on Pipecat 1.7 via LocalSmartTurnAnalyzerV3. Adapter latency on live audio is ~120–155 ms — mel plus ONNX plus the thread handoff — against the 83 ms inference-only figure above.

Training data

santhosh-005/tamil-eot — 18,485 labelled turn boundaries from 116 Tamil telephone conversations, split by call. Labels validated at 97.5% agreement with a human listener before use.

Encoder initialised from openai/whisper-tiny / openai/whisper-base with randomly initialised heads, matching upstream Smart Turn's own training script; attention pooling, binary head, 6 epochs, lr 5e-5, batch 32.

Limitations

• Narrowband telephony, one corpus, one domain. Wideband or close-mic speech is out of distribution.
• Tamil only.
• 8 s window — only the last 8 seconds are read.
• Splits are call-disjoint, not speaker-disjoint. The corpus ships no speaker labels, and agent voices are likely shared across splits.
• Labels are machine-produced and human-validated, not human-produced.
• Run-to-run spread at identical config is ~0.9 points; treat smaller differences as noise.

Files

smart-turn-tamil-{tiny,base}/smart-turn-tamil-int8-dynamic.onnx	int8 dynamic
smart-turn-tamil-{tiny,base}/smart-turn-tamil.onnx	fp32 reference
smart-turn-tamil-{tiny,base}/best.pt	torch checkpoints
mel_filters.npz	80-mel filterbank for the numpy feature path
config.json	thresholds and metadata

Licence and citation

BSD-2-Clause, matching pipecat-ai/smart-turn, whose architecture and training code this reuses. Encoder weights are initialised from openai/whisper-tiny / openai/whisper-base (MIT at OpenAI's release). Training data derives from SPRING_INX Tamil R1 (CC BY 4.0), SPRING Lab, IIT Madras.

@article{tamileot,
  author  = {Santhoshkumar V},
  title   = {TamilEOT: A Dataset and Model for Semantic End-of-Turn
             Detection in Tamil Telephone Speech},
  journal = {arXiv preprint arXiv:2609.05631},
  year    = {2026},
  url     = {https://arxiv.org/abs/2609.05631}
}

Paper: arXiv:2609.05631 · Code: https://github.com/santhosh-005/tamil-eot