Instructions to use KalineZephyr/whisper-small-yoruba-finetuned with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KalineZephyr/whisper-small-yoruba-finetuned with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="KalineZephyr/whisper-small-yoruba-finetuned")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq processor = AutoProcessor.from_pretrained("KalineZephyr/whisper-small-yoruba-finetuned") model = AutoModelForSpeechSeq2Seq.from_pretrained("KalineZephyr/whisper-small-yoruba-finetuned", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Whisper Small Fine-Tuned β Yoruba
This model is a fine-tuned version of openai/whisper-small for Yoruba speech recognition. It was trained on Common Voice 25.0 (yo) augmented with Google Fleurs (yo_ng) for a total of 3,718 training samples.
WER (raw, diacritics preserved):
- Test set (1,071 samples) : 73.67 %
- Validation set (975 samples) : 71.50 %
The WER is computed without any text normalisation in order to preserve Yoruba diacritics (tone marks, sub-dots, etc.). Lowercasing or stripping diacritics would inflate the reported score, but the model genuinely generates them.
Model description
Whisper Small is a transformer-based encoder-decoder model for automatic speech recognition (ASR), pre-trained on 680k hours of multilingual data. It uses 80-channel log-mel spectrograms as input and outputs a sequence of text tokens via autoregressive decoding with cross-attention to the encoder.
This fine-tuned version adapts the model specifically to Yoruba, a Niger-Congo language spoken in Nigeria, Benin and Togo. The model generates Yoruba text with full diacritic marks (Γ , Γ¨, αΊΉ, Γ, Γ², α», ΓΊ, αΉ£, etc.).
Intended uses & limitations
Intended uses:
- Transcribing Yoruba speech from audio recordings
- ASR research for low-resource African languages
- Benchmarking diacritic-preserving ASR systems
Limitations:
- Trained on ~4.3 epochs only (500 steps). More training may improve WER.
- Training data is mostly read-speech from Common Voice; performance on conversational/spontaneous Yoruba may be lower.
- The model retains Whisper's 30-second audio window.
- Diacritic coverage may be imperfect for some Yoruba dialects.
Training and evaluation data
- Common Voice 25.0 (yo) β 1,422 train / 975 validation / 1,071 test samples
- Google Fleurs (yo_ng,
raw_transcriptionfield to preserve diacritics) β 2,339 train samples (+43 removed by the >30 s filter)
Only clips shorter than 30 seconds were kept. Combined training set: 3,718 samples.
Training procedure
Hardware
- 2 Γ NVIDIA Tesla T4 (15.6 GB each)
- Distributed Data Parallel (DDP) via
accelerate launch - Mixed precision:
fp16(Native AMP) - Total training time: ~26 minutes (1,605 seconds)
Training hyperparameters
- learning_rate: 1e-05
- train_batch_size (per device): 16
- eval_batch_size (per device): 16
- gradient_accumulation_steps: 1
- total_train_batch_size (effective): 32
- total_eval_batch_size (effective): 32
- seed: 42
- optimizer: AdamW (torch fused) with betas=(0.9, 0.999) and epsilon=1e-08
- lr_scheduler_type: constant_with_warmup
- lr_scheduler_warmup_steps: 50
- max_steps: 500
- gradient_checkpointing: True
- eval_strategy: steps (every 500 steps)
- save_strategy: steps (every 500 steps, max 2 checkpoints)
- generation_max_length: 225
- predict_with_generate: True
- mixed_precision_training: Native AMP (fp16)
Training results
| Training Loss | Epoch | Step | Validation Loss | Validation WER |
|---|---|---|---|---|
| 0.4559 | 4.2735 | 500 | 0.7070 | 71.50 % |
Qualitative examples (test set, seed=42)
| PrΓ©diction | RΓ©fΓ©rence | WER |
|---|---|---|
| AΜkα»ΜweΜ αΊΉgbαΊΉΜ waΜ niΜ, aΜwα»n α»mα» αΊΉgbαΊΉΜ α»ΜmuΜ ti puΜ t | Akα»ΜwΓ© αΊΉgbαΊΉΜ wa nΓ Γ wα»n α»mα» αΊΉgbαΊΉΜ α»hΓΊn ti kΓΊ tΓ‘n. | 63.6 % |
| LΓ³Γ²Γ³tα»Μ nΓ Γ wα»n alΓ‘boyΓΊ mΓrΓ n ma nΜ kan rΓ‘. | LΓ³Γ²Γ³tα»Μ ni Γ wα»n alΓ‘boyΓΊn mΓrΓ n mΓ‘a Ε kanra | 75.0 % |
| α»kΓΉnrin Γ ti obΓ¬nrin tΓ kΓ² nΓ Γ¬bΓ‘lΓ²pα»Μ kΓ² lΓ© ibi mα»Μ | α»kΓΉnrin Γ ti obΓ¬nrin tΓ kΓ² nΓ Γ¬bΓ‘lΓ²pα»Μ koΜ leΜ biΜmα» | 40.0 % |
Framework versions
- Transformers 5.12.1
- PyTorch 2.10.0+cu128
- Datasets 5.0.0
- Tokenizers 0.22.2
- Accelerate 1.14.0
- Downloads last month
- 14
Model tree for KalineZephyr/whisper-small-yoruba-finetuned
Base model
openai/whisper-smallDataset used to train KalineZephyr/whisper-small-yoruba-finetuned
Space using KalineZephyr/whisper-small-yoruba-finetuned 1
Evaluation results
- Test WER on Common Voice 25.0 (Test)test set self-reported73.670
- Validation WER on Common Voice 25.0 (Validation)validation set self-reported71.500