YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
tunekv · Multi-SWE-bench · DeepSeek Harness (dsh) × GLM-5.2-FP8
Frozen GLM-5.2-FP8 (zai-org/GLM-5.2-FP8 @ f33c6dc5, glm_moe_dsa: 78-layer MLA + DSA indexer); only the K/V of the
fixed dsh prompt head (census prefix 1,093 tokens, n_tune 1,024) is trained and served through the vLLM connector
tunekv_vllm.glm_dsa (no indexer retention). DH repo FutureMLS-Lab/differentiable-harness main @ 8b0cc0cf.
Protocol: fixed 50-task Multi-SWE subset (7 languages), 900 s/task, network blocked, dsh defaults
(reasoning_effort high), official multi-swe-bench==1.1.2 judge (zero-error receipts), 4 rounds per arm.
Arms: same-stack base (GLM-5.2), V1 = CE 1.0 + 0.1 forward KL (action segments only), GRPO (clip 0.2 + 0.1 reverse KL).
Layout: configs/ census prefix; stage/ serve / rollout / judge / rows / train scripts; k8s/ pod specs;
judge_runs/<run>/ patches + official receipts; weights_verify/ sha256 checks of the three weight copies.
Judge: research-dev-coder-009 was unreachable (jump host down since 2026-09-23 19:50Z), so all GLM arms are judged on a
cooldog fallback pod running the same official harness (Docker-in-Docker, Docker Hub mswebench/* images). Gold-patch
self-check there: 49/50 resolved, 0 errors (the one miss, vuejs/core#10027, is a timing-sensitive test that fails under load).
Results (resolved / 50, official receipts, 0 judge errors in every round)
| arm | r1–r4 | mean r1–r4 | r5–r8 | mean r1–r8 | vs same-stack base (r1–r8) |
|---|---|---|---|---|---|
| base (GLM-5.2, same stack) | 17 / 16 / 15 / 17 | 16.25 | 14 / 13 / 17 / 19 | 16.00 | — |
| V1 = CE 1.0 + 0.1 forward KL | 19 / 17 / 17 / 18 | 17.75 | 17 / 17 / 23 / 17 | 18.125 | +2.125 (Welch p 0.054, Mann-Whitney p 0.049, per-task 15 better / 6 worse, sign-flip p 0.059) |
| GRPO (clip 0.2 + 0.1 reverse KL, CE 0) | 17 / 15 / 18 / 18 | 17.00 | 16 / 15 / 19 / 17 | 16.875 | +0.875 (Welch p 0.32, per-task 10 better / 7 worse, sign-flip p 0.29) |
Old-stack reference (not paired): GLM-5.3 base 11 / 14 / 12 / 13 = 12.50 (different weights, different serving stack).
Both tuned arms train on the same judged trainpool rollouts (trainpool_250 × 8 passes, 717 / 2000 resolved); V1 uses the
384 winning trajectories (≤ 3 per task), GRPO the 430 rows of the 111 tasks with mixed outcomes (per-task z-scored
advantages). Only post-</think> action tokens are supervised. Artifacts: artifacts/v1_ce01kl/,
artifacts/grpo_clip02_rkl01/ (99 layer files + READY; injected 1,024 rows into 78 MLA + 21 indexer layers).
W&B: training rlzewpro (V1), u1cd96zi (GRPO); eval summary rr1fbtgc (entity zhoumllab, project multi-swe).
Per-round receipts are in judge_runs/glm52dsh_{base,v1,grpo}_r1..r8/; full numbers and paired tests in SCORES.json.