Gemma 4 26B-A4B Instruct (MoE) โ€” 8-bit MLX

8-bit MLX quantization of google/gemma-4-26B-A4B-it, for Apple Silicon (~26 GB). Sparse-MoE vision-language model โ€” run it with mlx-vlm, not mlx-lm. The expert router (router.proj) is kept at 8-bit precision.

Usage

pip install -U mlx-vlm
python -m mlx_vlm.generate \
  --model TyKaoz/gemma-4-26B-A4B-it-8bit \
  --prompt "Explique la quantization en une phrase." \
  --max-tokens 200
Base Tool Precision Size
google/gemma-4-26B-A4B-it mlx-vlm 8-bit ยท group 64 (router 8-bit) ~26 GB

By TyKaoz โ€” privacy-first native macOS LLM chat client. Apache 2.0, inherited from the base model.

Downloads last month
15
Safetensors
Model size
26B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for TyKaoz/gemma-4-26B-A4B-it-8bit

Quantized
(441)
this model

Evaluation results