train_multirc_456_1767293631

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the multirc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3149
• Num Input Tokens Seen: 264580656

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2949	1.0	6130	0.3256	13210560
0.373	2.0	12260	0.3230	26427632
0.3255	3.0	18390	0.3175	39656608
0.3282	4.0	24520	0.3211	52911264
0.3086	5.0	30650	0.3241	66151456
0.4401	6.0	36780	0.3199	79368416
0.3078	7.0	42910	0.3226	92601264
0.3617	8.0	49040	0.3169	105825424
0.322	9.0	55170	0.3192	119051808
0.2832	10.0	61300	0.3191	132282512
0.3129	11.0	67430	0.3181	145503424
0.3616	12.0	73560	0.3156	158718688
0.3026	13.0	79690	0.3163	171968272
0.3285	14.0	85820	0.3208	185192912
0.3308	15.0	91950	0.3165	198406496
0.4025	16.0	98080	0.3157	211654512
0.3549	17.0	104210	0.3156	224875552
0.3564	18.0	110340	0.3156	238097088
0.3044	19.0	116470	0.3153	251336240
0.2477	20.0	122600	0.3149	264580656

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1