Refine model card layout, language coverage, and Preview comparisons

README.md

@@ -40,7 +40,7 @@ tags:
 
 <img src="./audio8-banner.jpeg" alt="Audio8" width="760">
 
-# Audio8-TTS-Realistic-0.6B
+<h1 align="center">Audio8-TTS-Realistic-0.6B</h1>
 
 **More realistic speech. More languages. Still 0.6B.**
 
@@ -52,12 +52,12 @@ tags:
 
 **Audio8-TTS-Realistic-0.6B** builds on [Audio8-TTS-Preview-0.6b](https://huggingface.co/Edge0/Audio8-TTS-Preview-0.6b), delivering **substantially improved speech realism** while retaining its compact **0.6B** architecture and zero-shot voice cloning capabilities.
 
-This release supports **22 languages plus Cantonese**, adding **12 languages** to Preview's coverage. It uses the same Transformers inference interface as Preview. This repository includes the complete model checkpoint, its 44.1 kHz neural audio codec, tokenizer, processor, and custom Transformers code.
+This release supports **23 languages**, adding **12 languages compared with Preview**. It uses the same Transformers inference interface as Preview. This repository includes the complete model checkpoint, its 44.1 kHz neural audio codec, tokenizer, processor, and custom Transformers code.
 
 ## Highlights
 
 - **Improved realism:** more natural prosody and expressive delivery, building on Preview.
-- **Broader multilingual coverage:** 22 languages plus Cantonese, with 12 newly supported languages.
+- **Broader multilingual coverage:** 23 languages, including 12 additions to Preview's 11 supported languages.
 - **Compact model:** 601 million main-model parameters, with the audio codec bundled separately.
 - **Flexible generation:** zero-shot voice cloning from a reference recording and transcript, or speech generation without a reference.
 - **GPU serving:** an SGLang Omni adapter with paged attention, dynamic batching, and an OpenAI-compatible speech API.
@@ -66,11 +66,10 @@ This release supports **22 languages plus Cantonese**, adding **12 languages** t
 
 | Coverage | Languages |
 |---|---|
-| Retained from Preview | Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish |
-| Newly supported | Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, Vietnamese |
-| Also retained | Cantonese |
+| Supported in Preview (11) | Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish |
+| Added in Realistic (12) | Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, Vietnamese |
 
-Language quality varies; see the per-language evaluation below. Benchmark coverage includes some additional languages and is distinct from the supported-language list.
+> **Note:** Language quality varies; see the per-language evaluation below. Benchmark coverage includes some additional languages and is distinct from the supported-language list.
 
 ## Model Details
 
@@ -170,9 +169,9 @@ The [SGLang Omni adapter](https://github.com/Edge0-AI/Audio8_TTS/tree/master/sgl
 | Voice cloning | Reference-audio encoding and waveform decoding |
 | API | OpenAI-compatible `/v1/audio/speech` service |
 
-The adapter targets SGLang Omni revision `68a572348837f7b004857b4b07993c20ade4c017`, with PyTorch 2.9.1, SGLang 0.5.8, and Transformers 4.57.1. Follow the [deployment instructions](https://github.com/Edge0-AI/Audio8_TTS#sglang-omni-serving) for installation, configuration, and API examples, and set the model path to `Edge0/Audio8-TTS-Realistic-0.6b` or its downloaded snapshot. The adapter supports generation without a reference and zero-shot voice cloning.
+Use the **0.6B** configuration, `sglang_omni/configs/audio8_tts_0_6b.yaml`, and set the model path to `Edge0/Audio8-TTS-Realistic-0.6b` or its downloaded snapshot. The adapter supports generation without a reference and zero-shot voice cloning. See the [deployment guide](https://github.com/Edge0-AI/Audio8_TTS#sglang-omni-serving) for compatible dependency versions, installation, server configuration, and API examples.
 
-Use the **0.6B** configuration, `sglang_omni/configs/audio8_tts_0_6b.yaml`. This Realistic checkpoint passed non-streaming `/v1/audio/speech` smoke tests both with and without a reference on an NVIDIA H20, using the versions above and the adapter from Audio8_TTS commit `07e40f5d0b03fc473635ef378654bfb581027ac3`. These checks used BF16 with Torch compilation and CUDA graphs disabled; they validate compatibility, not throughput or streaming performance.
+> **Validation note:** Audio8-TTS-Realistic was tested on an NVIDIA H20 in BF16 for generation with and without a reference through the non-streaming speech API, with Torch compilation and CUDA graphs disabled. These checks validate runtime compatibility; throughput and streaming were not benchmarked.
 
 ## Evaluation
 
@@ -180,43 +179,48 @@ Error rates are percentages (**lower is better**). Speaker similarity (SIM) is o
 
 ### Benchmark summary
 
-Results for **Audio8-TTS-Realistic (0.6B)**:
+**Audio8-TTS-Preview** and **Audio8-TTS-Realistic**, both **0.6B**:
 
-| Benchmark | Coverage | Mean error % ↓ | Mean SIM ↑ |
-|---|---|---:|---:|
-| Seed-TTS | English / Chinese | **1.25** | **0.704** |
-| CV3 | 9 core languages | **4.86** | **0.661** |
-| MiniMax | 24 evaluated languages | **5.89** | **0.754** |
+| Benchmark | Coverage | Preview error % ↓ | Realistic error % ↓ | Preview SIM ↑ | Realistic SIM ↑ |
+|---|---|---:|---:|---:|---:|
+| Seed-TTS | English / Chinese | 1.23 | 1.25 | 0.682 | 0.704 |
+| CV3 | 8 shared languages | 4.80 | 4.56 | — | 0.663 |
+| CV3 | 9 core languages | — | 4.86 | — | 0.661 |
+| MiniMax | 24 evaluated languages | — | 5.89 | — | 0.754 |
 
-All three summary error rates are unweighted language macro averages. Seed-TTS averages English WER and Chinese CER, excluding hard Chinese. CV3 averages Chinese, English, Japanese, Korean, German, French, Spanish, Italian, and Russian, excluding hard subsets and the 13 additional languages. MiniMax averages all 24 evaluated languages, including languages outside the supported-language list.
+> **Note:** Values are unweighted language macro averages. Preview results are calculated from its [published model card](https://huggingface.co/Edge0/Audio8-TTS-Preview-0.6b#evaluation), with SIM converted from percentages to a 0–1 scale. The two releases were evaluated separately. A dash means the corresponding result was not reported on Preview's card.
+>
+> Seed-TTS averages English WER and Chinese CER, excluding hard Chinese. The shared CV3 subset is Chinese, English, Japanese, Korean, German, French, Spanish, and Italian; the 9-language set also includes Russian. CV3 averages exclude hard subsets and the 13 additional languages. MiniMax averages all 24 evaluated languages, including languages outside the supported list.
 
 ### Reference comparison
 
 The following error rates come from the sources used in our technical report. Different evaluation pipelines and text normalization make this a reference comparison rather than a strictly matched ranking. A dash means that a result is unavailable in the comparison, not that the model lacks support.
 
-| Model | Parameters (B) | Seed-TTS EN/ZH error ↓ | CV3 (9) error ↓ | MiniMax (24) error ↓ | Result source |
-|---|---:|---:|---:|---:|---|
-| **Audio8-TTS-Realistic** | **0.6** | **1.25** | **4.86** | **5.89** | Our evaluation |
-| Qwen3-TTS-12Hz-0.6B | 0.6 | 1.12 | — | — | Qwen report |
-| Qwen3-TTS-1.7B | 1.7 | 1.01 | 7.73 | 27.41 | Qwen report / Boson evaluation |
-| Higgs Audio v3 | 4 | 1.11 | 4.41 | 2.74 | Boson evaluation |
-| Higgs Audio v2 | 4.7 | 1.17 | 4.49 | — | Our reevaluation |
-| Fish S2 Pro | 4.6 | 1.32 | 4.49 | 4.40 | Our reevaluation / Fish report |
-| MOSS-TTS-v1.5 | 8.5 | 1.52 | 6.11 | 3.78 | VoxCPM compilation / Boson evaluation |
-| CosyVoice3-1.5B | 1.5 | 1.67 | 6.89 | — | VoxCPM compilation |
-| VoxCPM2 | 2.3 | 1.41 | 5.35 | — | VoxCPM compilation |
+| Model | Parameters (B) | Seed-TTS EN/ZH error ↓ | CV3 (9) error ↓ | MiniMax (24) error ↓ |
+|---|---:|---:|---:|---:|
+| **Audio8-TTS-Realistic** | **0.6** | **1.25** | **4.86** | **5.89** |
+| Qwen3-TTS-12Hz-0.6B | 0.6 | 1.12 | — | — |
+| Qwen3-TTS-1.7B | 1.7 | 1.01 | 7.73 | 27.41 |
+| Higgs Audio v3 | 4 | 1.11 | 4.41 | 2.74 |
+| Higgs Audio v2 | 4.7 | 1.17 | 4.49 | — |
+| Fish S2 Pro | 4.6 | 1.32 | 4.49 | 4.40 |
+| MOSS-TTS-v1.5 | 8.5 | 1.52 | 6.11 | 3.78 |
+| CosyVoice3-1.5B | 1.5 | 1.67 | 6.89 | — |
+| VoxCPM2 | 2.3 | 1.41 | 5.35 | — |
 
-Sources: [Qwen3-TTS report](https://arxiv.org/abs/2601.15621), [Boson AI Higgs Audio v3 benchmarks](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b), [Fish Audio S2 report](https://arxiv.org/abs/2603.08823), and the [VoxCPM repository](https://github.com/OpenBMB/VoxCPM). Fish S2 Pro was reevaluated on Seed-TTS and CV3 because its official evaluation uses a different normalizer; Higgs Audio v2 was evaluated locally. Cross-project SIM values are omitted here because speaker encoders and evaluation conventions differ.
+> **Sources:** Qwen Seed-TTS results are from the [Qwen3-TTS report](https://arxiv.org/abs/2601.15621); Qwen's CV3/MiniMax results, all Higgs Audio v3 results, and MOSS-TTS-v1.5's CV3/MiniMax results are from [Boson AI's evaluation](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b). Fish S2 Pro's MiniMax result is from the [Fish Audio S2 report](https://arxiv.org/abs/2603.08823); its Seed-TTS/CV3 results and Higgs Audio v2's results are our reevaluations. MOSS-TTS-v1.5's Seed-TTS result and the CosyVoice3/VoxCPM2 results are compiled in the [VoxCPM repository](https://github.com/OpenBMB/VoxCPM). Cross-project SIM values are omitted because speaker encoders and evaluation conventions differ.
 
-Parameter conventions also differ: Audio8's 0.6B excludes its separate codec; Qwen and CosyVoice use release labels; Higgs Audio v3 uses its nominal **4B** release label. The remaining sizes are rounded checkpoint counts used in our report (VoxCPM2 excludes its separate AudioVAE). They should not be interpreted as identically counted end-to-end system sizes.
+> **Parameter conventions:** Audio8's 0.6B excludes its separate codec; Qwen and CosyVoice use release labels; Higgs Audio v3 uses its nominal **4B** release label. The remaining sizes are rounded checkpoint counts used in our report (VoxCPM2 excludes its separate AudioVAE). These counts use different conventions across systems.
 
 ### Seed-TTS detail
 
-| Split | Metric | Error % ↓ | SIM ↑ |
-|---|---|---:|---:|
-| English | WER | 1.464 | 0.663 |
-| Chinese | CER | 1.029 | 0.745 |
-| Chinese, hard | CER | 12.797 | 0.702 |
+| Split | Metric | Preview error % ↓ | Realistic error % ↓ | Preview SIM ↑ | Realistic SIM ↑ |
+|---|---|---:|---:|---:|---:|
+| English | WER | 1.506 | 1.464 | 0.632 | 0.663 |
+| Chinese | CER | 0.950 | 1.029 | 0.731 | 0.745 |
+| Chinese, hard | CER | 11.510 | 12.797 | 0.687 | 0.702 |
+
+> **Note:** Preview values are from its [published Seed-TTS table](https://huggingface.co/Edge0/Audio8-TTS-Preview-0.6b#seed-tts), with SIM converted to a 0–1 scale.
 
 ### Multilingual detail
 
@@ -288,9 +292,9 @@ Each language has 100 evaluation items. Rows marked † are evaluated for comple
 
 ### Compared with Preview
 
-In the same archived MiniMax evaluation pipeline, the 24-language mean error rate decreased from **29.637%** for Preview to **5.894%** for Realistic, while mean SIM increased from **0.6932** to **0.7535**. This is an average over the full benchmark, including languages beyond the supported list; improvements are not uniform across every language and split. For example, ordinary Seed-TTS Chinese CER was **0.943% → 1.029%** in that same evaluation series. Preview values in its original public model card were measured in an earlier evaluation and should not be mixed with this comparison.
+Realistic expands Preview's **11-language** coverage to **23 languages**, adding **Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese**.
 
-The main release goal is improved speech realism alongside expanded language coverage. WER/CER and SIM alone do not establish a human preference score.
+In our **private blind listening tests**, Audio8-TTS-Realistic achieved an **overwhelming Elo advantage over Preview in speech realism**.
 
 ## Limitations and Responsible Use