abduaziz/whisper-small-tajik

Whisper Small Tajik

This model is a fine-tuned version of openai/whisper-small on the Google Fleurs dataset. It achieves the following results on the evaluation set:

• Loss: 0.4141
• Wer: 24.2606

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.7687	1.0	79	0.5778	39.6568
0.7193	2.0	158	0.3890	28.3568
0.3659	3.0	237	0.3611	26.0636
0.2021	4.0	316	0.3629	25.1068
0.1099	5.0	395	0.3740	25.3044
0.0597	6.0	474	0.3887	24.3081
0.0339	7.0	553	0.4005	24.6639
0.0213	8.0	632	0.4082	24.3239
0.0158	9.0	711	0.4131	24.2685
0.014	10.0	790	0.4141	24.2606

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0