Add Inferno-Exp-1.6 (AlgoDriveAI)
#233
by AlgoDriveAI - opened
Add Inferno-Exp-1.6 (393.6M)
Model: https://huggingface.co/AlgoDriveAI/Inferno-Exp-1.6
Model revision tested: 06f0d479fc7a8a4c4616f7b9bc14aec27945ee4a
Organization: AlgoDriveAI (existing leaderboard organization)
Parameters (counted from unique model parameters): 393,610,240
| Benchmark | Score | Setup |
|---|---|---|
| HellaSwag | 29.80 ± 0.46% | lm-eval, 0-shot, acc_norm, float32 |
| ARC-Easy | 47.01 ± 1.02% | lm-eval, 0-shot, acc_norm, float32 |
| ARC-Challenge | 30.03 ± 1.34% | lm-eval, 0-shot, acc_norm, float32 |
| PIQA | 60.50 ± 1.14% | lm-eval, 0-shot, acc_norm, float32 |
| ArithMark-3 | 83.10% | acc_norm; Official AxiomicLabs ArithMark-3 benchmark script |
Intelligence Index: 26.24
Reproduce lm-eval:
lm_eval --model hf --model_args pretrained=AlgoDriveAI/Inferno-Exp-1.6,trust_remote_code=True,dtype=float32,add_bos_token=False --tasks hellaswag,arc_easy,arc_challenge,piqa --num_fewshot 0 --batch_size 8
Eligibility: public weights, pretrained from scratch (please verify model card), fewer than 400M parameters; zero-shot listed benchmarks.
Only a new model row is added if AlgoDriveAI is already present in ORGS.
Datdanboi25 changed pull request status to merged