Snowball task-arithmetic merge of B, E and E₂ (a = b = 0.5)
Weight merge from the two-expert experiment: B + 0.5·(E − B) + 0.5·(E₂ − B), built with the Marin weight-merging tools (marin PR #9974) with B as anchor and the two experts as donors. No training.
Base model. open-athena/Snowball-67B-A2B-10T-Mixed-RLVR-Sync-Step92 @ fdc3c3c5 (the student B). Tokenizer and chat template are B's, unchanged.
Recipe. Task arithmetic, a = b = 0.5, CPU only (33 min). Router query bias kept in B's float32 (donors export it in bfloat16 with identical values).
Scores (frozen evaluator: Evalchemy ae362881, vLLM H100x8, 32,768-token context, 30,720-token generation cap, thinking on, unseeded sampling at temperature 1.0). CruxEval 76.5 / 67.6; MMLU-Pro subset 63.2, held-out 61.6; GSM8K 93.9; MATH500 92.0; GPQA-D 47.1; IFEval instruction-strict 79.0. The full-strength merge (a = b = 1.0) regressed badly (GSM8K 50.0, GPQA-D 30.0) and is not published.
Provenance. Source export s3://marin-us-east-02a/marin/users/karan/mopd2/merges/ta-a0.5-b0.5/r3/; training job CPU merge job (recipes in experiments/weight_merging/recipes-mopd2/ on PR #9974's branch @ a431b8ce) (Iris, CoreWeave cw-rno2a). Every evaluation record, the paired comparisons and the GPU-hour accounting are in the experiment report: https://github.com/marin-community/marin/issues/9984#issuecomment-6144923850 (issue https://github.com/marin-community/marin/issues/9984).
S3 source deleted on 2026-10-17; weights verified identical to commit 07001c33 by sha256 (see https://github.com/marin-community/marin/issues/9984).
Weights are the HF-format export written by the training job, uploaded unchanged (safetensors shards, config.json, generation_config.json, tokenizer files, chat_template.jinja).
We release our tools to the public with the intent that they are used safely, ethically, and responsibly. They are meant to contribute to the public's understanding of science, and aid in its advancement. By using this Marin model, you agree to comply with all applicable law.
- Downloads last month
- 44