can this model be Quantized?
Your AWQ quantization work is absolutely amazing! It would be incredible if you could work your magic on this model too (cerebras/DeepSeek-V3.2-REAP-345B-A37B) - an AWQ quant of this would be a game-changer for those of us running on limited VRAM!
sure, let me have a look
I tried making an AWQ quant for cerebras/DeepSeek-V3.2-REAP-345B-A37B, but the quality drop was severe — it’s very hard to preserve even basic response quality.
Since this is a heavily pruned MoE (~50% experts removed), I guess it may need a bit of continued training to re-stabilize the model before quantization is viable.
I tried making an AWQ quant for cerebras/DeepSeek-V3.2-REAP-345B-A37B, but the quality drop was severe — it’s very hard to preserve even basic response quality.
Since this is a heavily pruned MoE (~50% experts removed), I guess it may need a bit of continued training to re-stabilize the model before quantization is viable.
Thanks a lot for all your quantization work! Totally get how tough this is. Maybe newer open‑sourced models like GLM‑4.7 or the upcoming Minimax‑M2.1 could be better targets. Thank you for contribution, quantization work is super valuable to the community!