Lucebox/gemma-4-26B-A4B-it-DFlash-GGUF

gemma-4-26B-A4B-it DFlash Draft — Q8_0 GGUF

Q8_0 GGUF quantization of the z-lab/gemma-4-26B-A4B-it-DFlash draft model, produced for the Lucebox dflash engine (speculative decoding for google/gemma-4-26B-A4B-it).

• Source: z-lab/gemma-4-26B-A4B-it-DFlash (BF16 safetensors)
• Tool: quantize_gemma_dflash_q8.py (parameterized variant of dflash/scripts/quantize_draft_q8.py)
• Size: 0.46 GB (53% of BF16)
• Arch: gemma4-dflash-draft
• Layers: 5
• Hidden: 2816
• n_head / n_head_kv: 32 / 8
• head_dim: 128
• intermediate_size: 5632
• vocab_size: 262144
• rope_theta: 1e6, rms_eps: 1e-6
• sliding_window: 2048, final_logit_softcapping: 30.0
• DFlash: n_target_layers=30, target_layer_ids=[1,6,11,17,22,27], block_size=16, mask_token_id=4
• Tensors: projection weights → Q8_0, norm weights → F32 (precision-critical, tiny)

Pairs with

• Target: google/gemma-4-26B-A4B-it (run as Q4_K_M GGUF via unsloth/gemma-4-26B-A4B-it-GGUF or equivalent)

Notes

• The dflash-specific singletons dflash.fc.weight and dflash.hidden_norm.weight bridge target hidden states into the draft. Do not re-quantize with stock llama-quantize — it strips these tensors. Use the script above.
• Loader support for gemma4-dflash-draft in lucebox-hub is the next step after PR #232 (gemma4 target adapter).

Usage

# With dflash_server (once gemma4-dflash-draft arch is wired in the loader)
dflash_server gemma-4-26B-A4B-it-UD-Q4_K_M.gguf --draft gemma-4-26B-A4B-it-DFlash-q8_0.gguf

Conversion command

PYTHONPATH=lucebox-hub/dflash/deps/llama.cpp/gguf-py \
python3 quantize_gemma_dflash_q8.py \
  gemma-4-26B-A4B-it-DFlash/ \
  gemma-4-26B-A4B-it-DFlash-q8_0.gguf \
  --name gemma-4-26B-A4B-it-DFlash-Q8_0