Index-Echo-S2ST-9B-GGUF

Official GGUF conversion of the text LLM backbone (stlm_llm/) of IndexTeam/Index-Echo-S2ST-9B (v4rft), part of the Index-Echo speech-to-speech translation (S2ST) model family by bilibili.

The v4rft package updates the multilingual translator and covers twelve directions: Chinese, Japanese, or English speech input, dubbed into English, Spanish, Japanese, Korean, or Chinese.

Note: LLM backbone only. The audio tower, connector and speech-synthesis stack are not included - llama.cpp cannot run the full speech pipeline. For full speech-to-speech translation (S2ST) inference, use the original checkpoint or the FP8 build in Index-Echo-S2ST-9B-FP8, which ships the complete pipeline with an FP8-quantized LLM.

Converted with llama.cpp (v0.26.0, 2026-10-15); static post-training quantization, all bit-widths in this single repository.

Provided files

(sorted by size)

File	Size	Notes
Q2_K	3.83 GB	2-bit, significant quality loss
Q3_K_S	4.26 GB	3-bit, noticeable quality loss
Q3_K_M	4.62 GB	3-bit, noticeable quality loss
Q3_K_L	4.93 GB	3-bit, noticeable quality loss
IQ4_XS	5.23 GB	4-bit I-quant, small
Q4_K_S	5.35 GB	4-bit, slightly smaller/faster
Q4_K_M	5.63 GB	4-bit, good balance, recommended
Q5_K_S	6.31 GB	5-bit, low quality loss
Q5_K_M	6.47 GB	5-bit, low quality loss
Q6_K	7.36 GB	6-bit, very low quality loss
Q8_0	9.53 GB	8-bit, near-lossless
f16	17.92 GB	16-bit, lossless conversion baseline

Usage

# Run the recommended Q4_K_M directly:
llama serve -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M
llama cli -hf IndexTeam/Index-Echo-S2ST-9B-GGUF:Q4_K_M

Consistency validation

Before release, each quantization tier was validated against the F16 conversion: per-token KL divergence / RMS delta-p via llama-perplexity (CPU build of llama.cpp), plus greedy-generation spot checks.

Quant	PPL (fixed corpus)	Mean KLD	RMS Δp	Same top-1
f16	3.2502	(reference)	-	-
Q8_0	3.2507	0.001272	1.23%	99.02%
Q6_K	3.2467	0.012566	4.18%	96.61%
Q5_K_M	3.2775	0.016146	4.45%	96.09%
Q5_K_S	3.3047	0.025917	5.45%	95.37%
Q4_K_M	3.2806	0.035922	6.90%	94.39%
Q4_K_S	3.6296	0.090064	10.84%	91.32%
Q3_K_L	3.4872	0.092859	10.21%	91.00%
Q3_K_M	3.8014	0.151914	14.26%	87.87%
Q3_K_S	5.8845	0.535761	30.10%	80.36%
Q2_K	4.6526	0.596711	26.59%	76.32%
IQ4_XS	3.4390	0.081273	10.17%	91.52%

Converted and published by the Index team, 2026-10-16.