t5-small-turkish-summarisation-lora

This model is a fine-tuned version of google-t5/t5-small on the mlsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.0317
• Rouge1: 19.4302
• Rouge2: 13.1054
• Rougel: 18.7905
• Rougelsum: 18.8397
• Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.557	1.0	624	2.1852	18.4642	11.9852	17.839	18.0458	20.0
2.364	2.0	1248	2.1269	19.6197	13.135	18.9274	19.0553	19.9913
2.3235	3.0	1872	2.0928	19.5088	13.2191	18.8665	18.9558	20.0
2.3002	4.0	2496	2.0669	19.2649	12.9427	18.6374	18.6937	20.0
2.2803	5.0	3120	2.0542	19.427	13.1022	18.7882	18.8376	20.0
2.2731	6.0	3744	2.0412	19.4496	13.1266	18.8102	18.8668	20.0
2.2655	7.0	4368	2.0338	19.4302	13.1054	18.7905	18.8397	20.0
2.2611	8.0	4992	2.0323	19.4302	13.1054	18.7905	18.8397	20.0
2.2584	9.0	5616	2.0321	19.4302	13.1054	18.7905	18.8397	20.0
2.2533	10.0	6240	2.0317	19.4302	13.1054	18.7905	18.8397	20.0

Framework versions

• PEFT 0.14.0
• Transformers 4.48.0
• Pytorch 2.2.2
• Datasets 3.2.0
• Tokenizers 0.21.0