Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper
• 2203.05482 • Published
• 7
merged_model
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: flammenai/Mahou-1.3-llama3-8B
parameters:
weight: 1.0
- model: Danielbrdz/Barcenas-Llama3-8b-ORPO
parameters:
weight: 1.0
- model: Weyaxi/Einstein-v6.1-Llama3-8B
parameters:
weight: 1.0
merge_method: linear
tokenizer_source: union
dtype: float16
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 20.69 |
| IFEval (0-Shot) | 41.23 |
| BBH (3-Shot) | 30.87 |
| MATH Lvl 5 (4-Shot) | 7.10 |
| GPQA (0-shot) | 5.37 |
| MuSR (0-shot) | 9.18 |
| MMLU-PRO (5-shot) | 30.42 |
- Downloads last month
- 8
Model tree for Naveenpoliasetty/llama3-8B-V2
Merge model
this model
Paper for Naveenpoliasetty/llama3-8B-V2
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard41.230
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard30.870
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard7.100
- acc_norm on GPQA (0-shot)Open LLM Leaderboard5.370
- acc_norm on MuSR (0-shot)Open LLM Leaderboard9.180
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard30.420