Tynda-ASR-kk-ru — Kazakh + Russian speech recognition

Tynda (Kazakh тыңда, "listen") is a Kazakh + Russian fine-tune of NVIDIA Nemotron-3.5-Transcribe-30B-A4B. It adds Kazakh to the base model and keeps its Russian, trained only on public data.

Author: Raushan Baisalova, University of Waterloo.

Results

WER / CER in % on the FLEURS test sets, normalized (NFC, lowercase, ё→е, punctuation removed, fillers such as "ыыы" removed, Latin look-alike letters in Cyrillic words mapped to Cyrillic). Greedy decoding, max_tokens capped at 30 × audio seconds + 32 (see Usage).

Test set	WER	CER
FLEURS kk test (856 utterances)	12.23	4.47
FLEURS ru test (775 utterances)	9.21	6.37

FLEURS references write numbers as digits; the model usually writes numbers as words, which adds errors. FLEURS kk train was used in training; the FLEURS test splits were never used for training or model selection.

How it was made

NVIDIA Nemotron-3.5-Transcribe-30B-A4B (revision c176676) fine-tuned on about 1,470 hours of public speech:

Data	Language	Hours	Transcripts
KSC2 train	Kazakh	1,154	95% punctuated (KSC2-Structured, used only where the words match the KSC2 transcript exactly), rest lowercase
FLEURS kk train (×3)	Kazakh	11.7 × 3	punctuated
Golos train	Russian	249	lowercase

• LoRA r=64, α=128 on attention, Mamba and MLP projections, plus the full ASR encoder and audio projection.
• lr 1e-4, 500 warmup steps, cosine; 4,000 steps × 24k tokens; bf16; 1× H200. The released checkpoint is step 3,000, the one with the lowest validation loss.
• Decontamination: KSC2 training rows whose text equals a KSC2 dev/test sentence were removed (KSC2 repeats sentences across speakers), and FLEURS train rows were checked against the FLEURS test and KSC2 dev/test text.
• The CTC word-timestamp head and the diarization branch are NVIDIA's, unchanged, with the CTC timestamp metadata in model.safetensors, so word timestamps work as in the base model.

Usage

Serve with vLLM 0.32 and the NeMo Speech plugin, as for the base model:

export VLLM_PLUGINS=nemo_speechlm VLLM_USE_V2_MODEL_RUNNER=0
vllm serve baisalovar/Tynda-ASR-kk-ru --served-model-name tynda --trust-remote-code --dtype bfloat16 \
  --max-model-len 65536 --enable-chunked-prefill --no-enable-prefix-caching \
  --mamba-cache-mode none --mm-processor-cache-gb 0 --no-async-scheduling \
  --reasoning-parser nemotron_v3 --trust-request-mm-kwargs \
  --middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware

On GPUs with FlashInfer JIT available you can add the base model's --use-replayssm --mamba-backend flashinfer and MTP speculative-decoding flags for speed; the command above needs no CUDA compiler.

import base64
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
audio = base64.b64encode(open("audio.wav", "rb").read()).decode()
duration_s = 12.3                                     # length of the clip
prompt = ("Audio transcription. Language: Kazakh. Wording: verbatim. "
          "Formatting: punctuation and capitalization. Output: transcript only.")
r = client.chat.completions.create(
    model="tynda", temperature=0, max_tokens=min(2048, int(30 * duration_s) + 32),
    messages=[{"role": "user", "content": [
        {"type": "text", "text": prompt},
        {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}]}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False})
print(r.choices[0].message.content)

Prompts used in training (use them verbatim). For Kazakh, use the punctuated prompt: it matches 95% of the Kazakh training data and gives clearly fewer errors on conversational Kazakh than the lowercase prompt. Do not add a repetition penalty; in our tests it made Russian worse and did not help Kazakh.

Language / style	Prompt
Kazakh, punctuated	Audio transcription. Language: Kazakh. Wording: verbatim. Formatting: punctuation and capitalization. Output: transcript only.
Kazakh, lowercase	Audio transcription. Language: Kazakh. Wording: verbatim. Formatting: lowercase, no punctuation. Output: transcript only.
Russian, lowercase	Audio transcription. Language: Russian. Wording: verbatim. Formatting: lowercase, no punctuation. Output: transcript only.

Hardware: BF16 weights are 63 GB; one 80–141 GB GPU (H100 / H200 / RTX PRO 6000).

Limitations

• Conversational Kazakh with many hesitations ("ыыы", false starts) is harder than read speech; on such audio the model can occasionally repeat a word or filler. Keep the max_tokens cap from Usage.
• Russian training data was lowercase (Golos has no punctuation); punctuated Russian relies on the base model.
• Numbers are usually written as words.
• Code-switching (Kazakh–Russian within one utterance) was not specifically trained.

License and attribution

• Model: a derivative of NVIDIA Nemotron-3.5-Transcribe-30B-A4B, distributed under the same OpenMDW-1.1 license. The license text and notices are kept in LICENSE, THIRD_PARTY_NOTICES.md and licenses/. No warranty.
• Training data:
  • KSC2 — Kazakh Speech Corpus 2 by ISSAI, CC BY 4.0, doi:10.48342/m90y-aj02.
  • KSC2-Structured — punctuation-restored KSC2 transcripts by InflexionLab (CC BY 4.0).
  • FLEURS — Google, CC BY 4.0 (Conneau et al., 2022).
  • Golos — Russian speech corpus by SberDevices, CC BY-SA 4.0 (Karpov et al., 2021, arXiv:2106.10161).
• The model weights are released under OpenMDW-1.1. Part of the training data (Golos) is licensed under CC BY-SA 4.0; please respect the terms of the training-data licenses (attribution, and share-alike where it applies) when you use or redistribute this model.

Citation

If you use Tynda-ASR-kk-ru, please cite:

@misc{baisalova2026tynda,
  title        = {Tynda-ASR-kk-ru: Kazakh and Russian Speech Recognition with Nemotron 3.5 Transcribe},
  author       = {Baisalova, Raushan},
  institution  = {University of Waterloo},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/baisalovar/Tynda-ASR-kk-ru}}
}

Please also cite the training data:

@inproceedings{mussakhojayeva22_interspeech,
  title     = {KSC2: An Industrial-Scale Open-Source Kazakh Speech Corpus},
  author    = {Mussakhojayeva, Saida and Khassanov, Yerbolat and Varol, Huseyin Atakan},
  booktitle = {Proc. Interspeech 2022},
  pages     = {1367--1371},
  year      = {2022}
}
@misc{karpov2021golos,
  title         = {Golos: Russian Dataset for Speech Research},
  author        = {Karpov, Nikolay and Denisenko, Alexander and Minkin, Fedor},
  year          = {2021},
  eprint        = {2106.10161},
  archivePrefix = {arXiv}
}
@inproceedings{conneau2022fleurs,
  title     = {FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech},
  author    = {Conneau, Alexis and Ma, Min and Khanuja, Simran and Zhang, Yu and Axelrod, Vera and Dalmia, Siddharth and Riesa, Jason and Rivera, Clara and Bapna, Ankur},
  booktitle = {IEEE Spoken Language Technology Workshop (SLT)},
  year      = {2022}
}