Instructions to use prithivMLmods/Kokoro-82M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prithivMLmods/Kokoro-82M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf prithivMLmods/Kokoro-82M-GGUF:Q4_K_M --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf prithivMLmods/Kokoro-82M-GGUF:Q4_K_M --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf prithivMLmods/Kokoro-82M-GGUF:Q4_K_M --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use prithivMLmods/Kokoro-82M-GGUF with Ollama:
ollama run hf.co/prithivMLmods/Kokoro-82M-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use prithivMLmods/Kokoro-82M-GGUF with Docker Model Runner:
docker model run hf.co/prithivMLmods/Kokoro-82M-GGUF:Q4_K_M
- Lemonade
How to use prithivMLmods/Kokoro-82M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prithivMLmods/Kokoro-82M-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Kokoro-82M-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Kokoro-82M-GGUF
Kokoro-82M is a lightweight, open-weight text-to-speech (TTS) model with 82 million parameters released under an Apache license, making it free for both personal and commercial deployment. Built on a decoder-only architecture based on StyleTTS 2 and ISTFTNet (without diffusion or an encoder release) and utilizing the Misaki grapheme-to-phoneme library, its v1.0 release supports 8 languages and 54 voices while delivering audio quality comparable to much larger models at a fraction of the compute costβaveraging under $1 per million characters of input or $0.06 per hour of generated audio via API. Remarkably efficient to develop, the model was trained for roughly $1,000 across 1,000 hours on A100 80GB GPUs using a few hundred hours of exclusively permissive, public domain, and synthetic audio paired with IPA phoneme labels.
Use F32 for the best quality. Supports
llama-ttsvia llama.cpp.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| Kokoro-82M.F32.gguf | F32 | 23.3 MB | Link | Full 32-bit float weights. Maximum precision, largest file size. |
| Kokoro-82M.BF16.gguf | BF16 | 11.8 MB | Link | Full BF16 weights. High quality, excellent balance of precision and size. |
| Kokoro-82M.Q3_K_L.gguf | Q3_K_L | 3.55 MB | Link | Lower quality but usable, good for very low RAM availability. |
| Kokoro-82M.Q3_K_M.gguf | Q3_K_M | 3.28 MB | Link | Low quality, smallest file size. |
| Kokoro-82M.Q4_K_M.gguf | Q4_K_M | 4.14 MB | Link | Good quality, default size for most use cases, recommended. |
| Kokoro-82M.Q4_K_S.gguf | Q4_K_S | 3.86 MB | Link | Slightly lower quality with more space savings, recommended. |
| Kokoro-82M.Q5_K_M.gguf | Q5_K_M | 4.57 MB | Link | High quality, recommended. |
| Kokoro-82M.Q5_K_S.gguf | Q5_K_S | 4.28 MB | Link | High quality with slight space savings, recommended. |
| Kokoro-82M.Q6_K.gguf | Q6_K | 5.04 MB | Link | Very high quality, near perfect, recommended. |
| Kokoro-82M.Q8_0.gguf | Q8_0 | 6.41 MB | Link | Extremely high quality, generally unneeded but max available standard quant. |
| Kokoro-82M.mmproj-bf16.gguf | mmproj-bf16 | 208 MB | Link | Multimodal projection file in BF16 format. Used for audio/voice capabilities. |
Run with llama.cpp
Install dependencies and clone llama.cpp
!pip install transformers huggingface-hub
!git clone https://github.com/ggml-org/llama.cpp.git
%cd /data/llama.cpp
Download GGUF model files
Download the BF16 model and BF16 multimodal projector:
from huggingface_hub import snapshot_download
# Target local directory
local_dir = "./model/Kokoro-82M-GGUF"
# Download only the specified files
snapshot_download(
repo_id="prithivMLmods/Kokoro-82M-GGUF",
local_dir=local_dir,
allow_patterns=[
"Kokoro-82M.BF16.gguf",
"Kokoro-82M.mmproj-bf16.gguf",
],
resume_download=True,
)
Build llama-tts
Build the TTS CLI with the CUDA backend enabled:
!cmake --build . --config Release -j$(nproc) --target llama-tts
Text-to-speech generation (GPU offloading)
Run inference using llama-tts with GPU offloading enabled:
!/data/llama.cpp/build/bin/llama-tts \
-m /data/llama.cpp/model/Kokoro-82M-GGUF/Kokoro-82M.BF16.gguf \
-mm /data/llama.cpp/model/Kokoro-82M-GGUF/Kokoro-82M.mmproj-bf16.gguf \
--tts-voice am_michael \
-p "Kokoro is an open-weight TTS model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, Kokoro can be deployed anywhere from production environments to personal projects." \
-ngl 99 \
-n 1000 \
--seed 42 \
--output /data/llama.cpp/model/Kokoro-82M-GGUF/out.wav
Output
The generated audio file will be saved at:
/data/llama.cpp/model/Kokoro-82M-GGUF/out.wav
You can play the generated WAV file using an audio player or a notebook audio widget.
llama.cpp
LLM inference in C/C++ β https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.
- Downloads last month
- 549
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
32-bit