nitky/Qwen3.5-397B-A17B-MTP-BF16-mlx

Qwen3.5-397B-A17B MTP BF16 MLX Drafter

This is the native Multi-Token Prediction (MTP) drafter extracted from Qwen/Qwen3.5-397B-A17B. All stored tensors use BF16. The drafter uses the target model's token embeddings and language-model head at runtime.

Configuration

• Model type: qwen3_5_mtp
• Draft block size: 3
• MTP hidden layers: 1
• Quantization: none

Usage

mlx_vlm.generate \
  --model /path/to/Qwen3.5-target-mlx \
  --draft-model /path/to/Qwen3.5-397B-A17B-MTP-BF16-mlx \
  --draft-kind mtp \
  --draft-block-size 3 \
  --prompt "Hello" \
  --max-tokens 128

The target must be compatible with the Qwen3.5-397B-A17B tokenizer and architecture. A modified target can have a low acceptance rate even though speculative verification preserves the target model's output distribution.