train_codealpacapy_789_1767686248

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.4767
• Num Input Tokens Seen: 24964664

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6243	1.0	1908	0.5801	1246616
0.5014	2.0	3816	0.5206	2496088
0.4748	3.0	5724	0.5001	3746728
0.6017	4.0	7632	0.4915	4999712
0.4635	5.0	9540	0.4868	6245072
0.5159	6.0	11448	0.4842	7491776
0.4684	7.0	13356	0.4812	8735728
0.4356	8.0	15264	0.4800	9981168
0.5734	9.0	17172	0.4790	11227560
0.444	10.0	19080	0.4781	12474576
0.3451	11.0	20988	0.4774	13719896
0.5939	12.0	22896	0.4774	14970024
0.3983	13.0	24804	0.4771	16222408
0.5125	14.0	26712	0.4770	17474664
0.4654	15.0	28620	0.4771	18722440
0.4001	16.0	30528	0.4768	19970248
0.493	17.0	32436	0.4767	21217120
0.4032	18.0	34344	0.4768	22462808
0.3747	19.0	36252	0.4769	23715264
0.6148	20.0	38160	0.4768	24964664

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1