575-lab/kiji-inspector-google-gemma-4-31B-it

kiji-inspector — Gemma-4-E4B-it layer sweep

Mechanistic-interpretability artefacts produced by the kiji-inspector pipeline against google/gemma-4-E4B-it (60-layer dense decoder, d_model = 5376).

Each layer_{N}/ directory contains the trained JumpReLU SAE checkpoint, the per-layer activation shards, contrastive feature analysis, LLM-generated labels, and token-level fuzzing results.

• Branch: feat/new-vllm-gemma4
• Layer sweep: 10, 20, 31, 41, 50, 58 (depths ≈ 17 % / 33 % / 52 % / 68 % / 83 % / 97 %)
• SAE: JumpReLU with d_sae = 16384 (≈ 3.05 × d_model), batch 8192, lr 3 × 10⁻⁴
• Generation / labelling / judging model: Qwen/Qwen3-VL-235B-A22B-Instruct-FP8 (vLLM)

SAE health (per layer)

Metrics computed on 200 batches of held-out activations (sae_checkpoints/feature_health.json).

Layer	Depth	Alive %	Dead %	L0 (mean ± SEM)	Reconstruction MSE
10	17 %	17.9	82.1	2931.7 ± 0.04	47.24
20	33 %	16.6	73.3	2710.5 ± 0.06	45.36
31	52 %	13.2	70.1	2087.2 ± 0.19	20.18
41	68 %	19.8	61.1	618.1 ± 0.78	0.243
50	83 %	18.9	52.7	631.8 ± 0.76	0.093
58	97 %	1.7	81.7	247.5 ± 0.03	0.034

Observations.

• Reconstruction MSE falls ≈ 1400× across depth (47.2 → 0.034). On a dense Gemma backbone, deeper activations are more reconstructable, the opposite of what MoE-based models exhibit.
• A sharp L0 compression event between L31 and L41 (2087 → 618, a 3.4× drop) coincides with the MSE collapse — consistent with decision-relevant features concentrating into a smaller subspace around 65–70 % depth.
• L58 collapses to only 272 alive features (1.7 %), with L0 ≈ 248. Final-layer representations are essentially low-rank and aligned with the unembedding.

Fuzzing evaluation (per layer)

Token-level A/B fuzzing adapted from EleutherAI's autointerp methodology. Combined score = 0.7 · token-level accuracy + 0.3 · prompt-level accuracy. All layers significantly above the 0.5 random baseline (p < 10⁻⁴, one-sample t-test). Source: activations/fuzzing_summary.json.

Layer	Features fuzzed	N examples	Combined (mean ± SEM)	Token-acc (mean ± SEM)	Excellent (> 0.8)	Good (0.6–0.8)	Poor (< 0.6)
10	1961	21 571	0.736 ± 0.003	0.662 ± 0.004	27.1 %	56.0 %	16.9 %
20	1869	20 559	0.713 ± 0.004	0.634 ± 0.004	22.8 %	56.5 %	20.7 %
31	1412	15 532	0.700 ± 0.004	0.624 ± 0.004	19.4 %	58.1 %	22.5 %
41	614	6 754	0.718 ± 0.006	0.643 ± 0.006	22.8 %	58.3 %	18.9 %
50	626	6 886	0.690 ± 0.006	0.617 ± 0.007	19.2 %	55.6 %	25.2 %
58	264	2 904	0.688 ± 0.010	0.619 ± 0.010	19.3 %	56.1 %	24.6 %

Observations.

• Fuzzing quality is roughly flat across depth (combined 0.69 – 0.74). There is no single dominant layer; representations remain interpretable throughout the network.
• L10 is the strongest layer on every fuzzing metric (highest combined, token-acc, and Excellent %). Early-layer features tend to be lexical / surface-form and are easier to label correctly.
• L41 matches L10 on token-accuracy and Excellent % despite having ~3× fewer contrastive features and ~5× lower L0 — i.e., a much sparser representation that is just as well-explained.
• Number of features evaluated drops with depth, reflecting fewer surviving contrastive features after Cohen's d ≥ 0.3 filtering, not lower fuzzing quality.

Per-layer artefacts

Inside each layer_{N}/:

• sae_checkpoints/sae_final.pt — trained JumpReLU SAE weights
• sae_checkpoints/config.json — training config
• sae_checkpoints/feature_health.json — alive %, dead %, L0, reconstruction MSE
• sae_checkpoints/metrics.jsonl — training-step metrics
• activations/shard_*.npy — decision-token activations (float16)
• activations/contrastive_features.json — top contrastive features per contrast type (Cohen's d ranked)
• activations/feature_descriptions.json — LLM-generated label / description / confidence
• activations/decision_report.json — human-readable per-contrast-type decision report
• activations/fuzzing_results.json — raw judge decisions for every (feature, example) pair
• activations/fuzzing_summary.json — aggregated metrics, quality tiers, confidence stratification
• activations/prompts.json, metadata.json — dataset and run metadata

Citation

If you use these artefacts, please cite the kiji-inspector repository: https://github.com/dataiku/kiji-inspector.