Sakura

Sakura ยท GGUF quantization of Qwen/Qwen2.5-0.5B-Instruct

Independent community quantization. Not an official release of the upstream author.

1. Model Overview

This repository is produced by the local HF Quant Agent control plane. It records only checks that actually ran. Unmeasured quality, speed and compatibility are marked untested.

2. Original Model / Attribution

  • Source: Qwen/Qwen2.5-0.5B-Instruct
  • Publisher of this quantization: webmp3, when a release is actually uploaded
  • Upstream authors retain their license and attribution
  • GGUF source publisher: Qwen/Qwen2.5-0.5B-Instruct-GGUF
  • GGUF source commit: 9217f5db79a29953eb74d5343926648285ec7e67
  • GGUF source filename: qwen2.5-0.5b-instruct-q8_0.gguf
  • GGUF source SHA-256: ca59ca7f13d0e15a8cfa77bd17e65d24f6844b554a7b6c12e07a5f89ff76844e
  • GGUF source provenance verified: True

3. Source Revision

Pinned revision: 7ae557604adf67be50417f59c2c2f167def9a775

Jobs are bound to this SHA. A later upstream commit does not silently change a running or completed job.

4. Supported GGUF Quantizations

Requested: Q3_K_M

File Bytes SHA-256 Quant
Sakura-qwen2.5-0.5b-instruct-q8_0-Q3_K_M-requant-00001-of-00001.gguf 432041920 1a76ce5a1cda6afe188ebbc5de47097890c48539aa4b44e4f5fc87f72117365d Q3_K_M

5. Quantization Methodology

Route: native_gguf_stream

Adapter: standard-llamacpp

This route performs existing-GGUF requantization from the pinned Q8_0/F16/BF16 source GGUF identified above. This job did not convert the upstream BF16/safetensors checkpoint. Source tensor storage measured in the GGUF: F32: 121, Q8_0: 170.

No importance-matrix calibration was run by this job. Requantizing a previously quantized source can add quantization error. Estimated sizes are planning ratios, not measurements, unless a file byte count is listed above.

The target quant label describes the selected stock quantizer preset. Stock llama.cpp may retain floating-point tensors or use fallback tensor types; every tensor is not necessarily stored in the named target type. The table records measured output tensor types and only reports a fallback count when the quantizer printed one.

Source tensor count: 291. Source full tensor-name set SHA-256: 70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0. New and reused outputs must retain the source tensor count and complete name set.

File Measured tensor-type counts Quantizer-reported fallback tensors Full tensor-name set SHA-256
Sakura-qwen2.5-0.5b-instruct-q8_0-Q3_K_M-requant-00001-of-00001.gguf F32: 121, Q4_0: 97, Q4_K: 23, Q5_0: 46, Q5_1: 2, Q5_K: 1, Q8_0: 1 unmeasured; inspect quantizer log 70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0

6. Architecture Information

  • model_type / architecture: qwen2
  • parameters: 494032768
  • experts: n/a or not declared
  • source formats observed: safetensors, bfloat16
  • custom code present: False
  • trust_remote_code executed: False

7. Conversion and Runtime Compatibility

  • GGUF structure valid: True
  • Runtime load tested: True
  • Inference tested: True
File Runtime build reported by executable Runtime executable SHA-256
Sakura-qwen2.5-0.5b-instruct-q8_0-Q3_K_M-requant-00001-of-00001.gguf b11669-620d2225c 486568ae9dd8682e160861a129b70f4a5efa73191542ab0cf80674b8b6d4f814

8. Download and Usage Instructions

Use a llama.cpp build that supports this architecture. Example, after you confirm compatibility:

llama-cli -m <file>.gguf -p "Hello" -n 32

This card does not claim that every runtime can load the file.

9. Validation Results

All selected streamed output compositions passed structural checks and one stock CPU inference per quant. Every composed shard was hashed before and after inference.

10. Limitations

  • No coding benchmark, perplexity or quality score is claimed unless a measured value is copied from this job's own log.
  • Shard resume and upload verification apply only to artifacts listed in this card.

11. License Information

Source license: apache-2.0

Decision: apache-2.0 allows redistribution with attribution

If the license does not allow redistribution, this card must not be uploaded.

12. Reproduction Information

  • Control plane: HF Quant Agent
  • Planner route: native_gguf_stream
  • Workflow: none
  • Blockers: none recorded
Downloads last month
407
GGUF
Model size
0.6B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for webmp3/Sakura-Qwen2.5-0.5B-Instruct-GGUF

Quantized
(403)
this model

Collection including webmp3/Sakura-Qwen2.5-0.5B-Instruct-GGUF