Tintler/drama-3-tr-128e-w4a16

Drama 3 · Turkish · 128 experts · W4A16

Built with Fish Audio.

A Turkish-calibrated, pruned and quantized build of fishaudio/drama-3 that runs on a single 16 GB consumer card (tested on an RTX 5080 that also drives the desktop), about 2x faster than real time.

This is a community derivative, not an official Fish Audio release. Fish Audio does not sponsor or endorse it.

What it is

	original	this build
routed experts per MoE layer	256	128, chosen from Turkish + English speech router statistics
expert weights	FP8	int4 (GPTQ packing, sym, group 128)
attention / linear-attention / shared-expert linears	bf16	int8 (GPTQ, group 128)
text vocabulary	252,288	169,193 (Turkish tokenizes identically)
lm_head	full vocabulary	end-of-speech + 4,096 semantic rows
weights on disk	~37 GB	12.9 GB (+1.1 GB codec)

How the experts were chosen

The original model was run layer by layer over ~4.3 h of FLEURS read speech (1,200 Turkish and 147 English utterances, teacher-forced, with and without a voice reference) plus tagged Turkish prompts, and each layer keeps the 128 experts with the highest weighted router mass (Turkish speech 0.45, Turkish text 0.25, English speech 0.2, English text 0.1).

Router probability mass kept by the 128 experts (mean / worst layer):

Turkish speech	Turkish text	English speech	English text
81.7% / 62.5%	87.1% / 70.6%	79.8% / 62.2%	80.6% / 62.0%

Evaluation

20 held-out Turkish sentences (not in the calibration set), one take each, transcribed with Whisper large-v3-turbo. Default sampling. Speaker similarity: WeSpeaker ResNet34 cosine between output and reference.

CER	WER	sentences with an error
0.62%	2.60%	3/20

One of the three remaining "errors" is Whisper writing iki as 2.

Across six different Turkish reference voices (5-15 s each) the build scored CER 0.26%-3.79% with speaker similarity 0.77-0.90. Results depend strongly on the reference: calm, clean speech that ends on a complete sentence works best.

English is unaffected in spot checks.

Samples

The same Turkish sentence three times, same reference voice, with and without an inline tag: "Sessiz ol, kapının arkasında biri var." ("Be quiet, someone is behind the door.")

input	audio
Sessiz ol, kapının arkasında biri var. (no tag)
[fısıldar] Sessiz ol, kapının arkasında biri var. (Turkish "whispers")
[frustrated] Sessiz ol, kapının arkasında biri var.

Tags can be written in Turkish or English.

Speed and memory (RTX 5080 16 GB, desktop on the same card)

generation	67 frames/s (3.1x real time)
end to end, 7 s sentence	~3.3 s
end to end, 12.5 s sentence	~5.7 s
GPU memory in use	~15.7 GB of 16.3 GB, including ~0.9 GB for the desktop

Requirements

• One NVIDIA GPU with 16 GB. Linux or WSL2.
• SGLang-Omni at commit bcd0d8c6831c146b4f0a5bd726b8062f777543e4, patched with sglang_patch/kur_yama.py.
• Blackwell cards (RTX 50xx, SM 12.0) need the CUDA 13.0 toolkit for FlashInfer / DeepGEMM JIT (CUDA_HOME=/usr/local/cuda-13.0).
• Keep huggingface_hub<2 (transformers requires it).

Setup

git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni
git checkout bcd0d8c6831c146b4f0a5bd726b8062f777543e4
uv pip install --prerelease=allow -e ".[fishaudio]"
cd ..
hf download Tintler/drama-3-tr-128e-w4a16 --local-dir drama-3-tr
python drama-3-tr/sglang_patch/kur_yama.py ./sglang-omni

The patch (safe to re-run, keeps *.orig_tr backups) adds:

1. the W4A16 (+int8 linear) quantization configs;
2. support for the compact lm_head (text_config.lm_head_compact); logits are scattered back to the full vocabulary, sampling is unchanged;
3. a decode-only vocoder (the codec encoder modules are dropped, ~0.39 GB);
4. raw reference audio encoding on the CPU preprocessing stage (it failed on bf16 codec weights);
5. a memory-accounting fix for setups without NVML per-process data (WSL), where a concurrently loading vocoder was charged to the engine.

Run

export CUDA_HOME=/usr/local/cuda-13.0 PATH=/usr/local/cuda-13.0/bin:$PATH   # RTX 50xx
MODEL=$PWD/drama-3-tr PORT=7861 bash drama-3-tr/serve.sh

Voice references are sent as precomputed codes. Encode each voice once on the CPU:

python drama-3-tr/encode_voice.py --audio voice.wav --text voice.txt \
    --codec drama-3-tr --output voice.json

Then:

import json, urllib.request
ref = json.load(open("voice.json", encoding="utf-8"))
body = {"model": "drama-3", "input": "Merhaba! Bugün size kısa bir hikâye anlatacağım.",
        "references": [ref], "response_format": "wav"}
req = urllib.request.Request("http://127.0.0.1:7861/v1/audio/speech",
                             json.dumps(body).encode(), {"Content-Type": "application/json"})
open("out.wav", "wb").write(urllib.request.urlopen(req).read())

You can also send the reference audio itself and let the server encode (and cache) it on the CPU: {"text": "<transcript>", "data": "<base64 WAV>"}. The first request with a new voice then takes a few seconds longer.

Inline tags such as [laughs], [sighs], <whisper>...</whisper> and speaker tags <|speaker:0|> work as in Drama 3. Keep each request under ~14 s of speech (split long text into sentences). Known quirks, shared with Drama 3 itself:

• Occasionally a take ends with no audio (HTTP 500 "generated no audio frames"); retrying once fixes it.
• The model sometimes holds a long silence at a tag or sentence boundary (seen: 6 s after [sighs]) before continuing; the text is still read completely. Shortening silences longer than ~1 s in post-processing hides it. This build has no sampler-side pause fix.

If startup fails with an out-of-memory error, close other GPU programs or use VOC_DEVICE=cpu (slower, but fits).

Reproducing

tools/ holds the scripts used to build this checkpoint from fishaudio/drama-3:

• calib_hazirla.py: encodes FLEURS audio into codec codes (calibration set);
• olc.py: runs the original model layer by layer on one 16 GB GPU and records router statistics;
• buda.py: selects the experts and writes the int4/int8 checkpoint (takes the trimmed tokenizer and base config from a template directory);
• kucuk_bas.py: cuts lm_head to the end-of-speech + semantic rows.

Building took about 30 minutes on an RTX 5080 after the downloads.

License and acceptable use

Licensed under the Fish Audio Community License v1.0. See LICENSE and NOTICE.

• Free for research and non-commercial use. Limited commercial use is allowed below USD 500k annual revenue.
• Offering it as a hosted TTS or voice-cloning service needs a separate license from Fish Audio. Free, non-monetized research demos are allowed.

Do not use this model or its output to:

• imitate the voice of an identifiable person without their consent;
• impersonate anyone to deceive, defraud or harass;
• produce sexual content involving minors or non-consenting real people;
• remove watermarks or provenance signals;
• train or improve other speech or foundation models.

If you offer voice cloning built on this model to others, you must bind them to restrictions at least as protective as these.

Calibration used google/fleurs (CC BY 4.0) only to measure expert usage; no weights were trained on it.

---

Türkçe özet

Fish Audio Drama 3'ün Türkçe için budanmış ve sıkıştırılmış sürümü. 16 GB'lık tek bir ekran kartında (RTX 5080, masaüstü de aynı kartta) gerçek zamanın yaklaşık 2 katı hızında çalışır.

• 256 uzmandan 128'i, Türkçe ve İngilizce konuşma üzerindeki yönlendirici istatistiklerine göre seçildi.
• 20 Türkçe test cümlesinde karakter hata oranı %0,62.
• Ses örnekleri (etiketsiz, [fısıldar], [frustrated]) yukarıdaki "Samples" bölümünde.
• Kurulum: SGLang-Omni'yi yukarıdaki commit'e alın, sglang_patch/kur_yama.py ile yamalayın, serve.sh ile açın.
• En iyi klon sonucu için 8-15 saniyelik, sakin, net ve tam cümleyle biten bir referans kaydı kullanın.
• Lisans: Fish Audio Community License v1.0. Kişisel ve araştırma kullanımı serbest; yıllık geliri 500 bin doların altındaki ticari kullanım serbest; barındırılan ses hizmeti için Fish Audio'dan ayrı lisans gerekir. Rızası olmadan tanınabilir bir kişinin sesini taklit etmek yasaktır.

---

Thanks to asfberlin/drama-3-128e-w4a16, whose 128-expert W4A16 format, quantization configs and vocabulary subset this build reuses.