---
language:
- hi
license: mit
tags:
- automatic-speech-recognition
- whisper
- transformers
- hf-asr-leaderboard
- audio
- speech
- hindi
datasets:
- google/fleurs
metrics:
- wer
- cer
base_model: openai/whisper-small
model-index:
- name: whisper-small-hindi
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: Google FLEURS (hi_in)
      type: google/fleurs
      config: hi_in
      split: test
    metrics:
    - name: Word Error Rate (WER)
      type: wer
      value: 26.87
    - name: Character Error Rate (CER)
      type: cer
      value: 10.41
---

# Whisper Small - Hindi Fine-Tuned

This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) for Hindi Speech-to-Text (ASR) applications. It was fine-tuned on the `hi_in` configuration of the [Google FLEURS](https://huggingface.co/datasets/google/fleurs) dataset. 

The fine-tuning process significantly improved the transcription accuracy, reducing the Word Error Rate (WER) from 68.61% (baseline) down to **26.87%**, and the Character Error Rate (CER) from 34.43% down to **10.41%**.

## Model Details

- **Base Model:** `openai/whisper-small` (244M parameters)
- **Language:** Hindi (`hi`)
- **Task:** Automatic Speech Recognition (ASR)
- **Dataset:** Google FLEURS (Hindi - India)
- **License:** MIT

## Evaluation Results

The model was evaluated on the strictly held-out `test` split (418 samples) of the FLEURS Hindi dataset.

| Metric | Whisper Small (Base) | Whisper Small (Fine-Tuned) | Relative Improvement |
|---|---|---|---|
| **Word Error Rate (WER)** | 68.61% | **26.87%** | ↓ 60.8% |
| **Character Error Rate (CER)** | 34.43% | **10.41%** | ↓ 69.8% |

These results demonstrate a highly successful adaptation to Hindi phonetics and Devanagari script, with a massive reduction in transcription errors.

## Usage

You can use this model directly through the Hugging Face `transformers` pipeline for speech recognition:

```python
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
import librosa

# Load the fine-tuned model and processor
model_id = "rishii100/whisper-small-hindi"
processor = WhisperProcessor.from_pretrained(model_id)
model = WhisperForConditionalGeneration.from_pretrained(model_id)

# Load your Hindi audio file (resample to 16kHz)
audio_path = "path/to/your/hindi_audio.wav"
audio, sr = librosa.load(audio_path, sr=16000)

# Process and generate transcription
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features

with torch.no_grad():
    predicted_ids = model.generate(input_features, max_length=225)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print("Transcription:", transcription)
```

## Training Details

The model was trained using the following hyperparameters:

- **Learning Rate:** `1e-05`
- **Train Batch Size:** `16`
- **Eval Batch Size:** `8`
- **Training Steps:** `2000`
- **Warmup Steps:** `250`
- **Optimizer:** AdamW 
- **Mixed Precision:** FP16
- **Gradient Accumulation:** 1

## Intended Use & Limitations

This model is ideal for transcribing general Hindi speech. However, like all speech models, it may experience performance degradation in the following scenarios:
- High background noise or overlapping speakers.
- Heavy regional dialects not represented in the standard FLEURS corpus.
- Extensive code-switching between Hindi and English (Hinglish) where English words are pronounced with thick accents.