Sakura

Sakura ยท GGUF quantization of meta-llama/Llama-3.3-70B-Instruct

Independent community quantization. Not an official release of the upstream author.

Made with an importance matrix (imatrix). The weights were quantized with importance weights computed from a fixed calibration text; the exact method and the hashes are in the technical details of section 5.

1. Model Overview

Architecture: 70.6 B parameters ยท source formats: safetensors.

Original: 75.0 GB (Q8_0 GGUF, the source of these quantizations).

Quant Size Compared with the original
Q4_K_M 42.5 GB 43 % smaller

2. Original Model / Attribution

  • Original model: meta-llama/Llama-3.3-70B-Instruct
  • Declared base models: meta-llama/Llama-3.1-70B (relation: finetune)
  • Quantized from: bartowski/Llama-3.3-70B-Instruct-GGUF (file Llama-3.3-70B-Instruct-Q8_0.gguf).
  • Quantized by webmp3. Upstream authors retain their license and attribution.

3. Files

File Size Quant
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf 42.5 GB Q4_K_M

4. Download and Usage

llama-cli -m <file>.gguf -p "Hello" -n 32

Needs a llama.cpp build that supports this architecture. If a quantization is split into numbered parts, keep them together and load the first one.

5. Verification

  • GGUF structure valid: True
  • Real inference test (every file loaded by llama.cpp and used to generate text once): yes
  • No coding benchmark, perplexity or quality score is claimed unless a measured value is listed here.
Technical details: exact versions, hashes, how these were made

Exact versions

  • Original model commit: 6f6073b423013f6a7d4d9f39144961bfbfbc386b (model card at that commit)
  • Quantized from: commit b6c5c9f176f3279204034e1d16d393105e95cb88, SHA-256 d7314ee9cab0f263ec470fc984ceba44c0689645d6951f56a6e95f167383cc9c (provenance verified: True)

File hashes

File Bytes SHA-256 Quant
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf 42520398880 fa24f459f08a314d44d5e38df96a4a82083ab4c9500fd794dfbdf8ba2765cb6c Q4_K_M

How these were made

This route performs existing-GGUF requantization from the pinned Q8_0 source GGUF named above. This job did not convert the upstream BF16/safetensors checkpoint.

Quantized using llama.cpp GGML kernels and an importance matrix (imatrix): the matrix was computed once with llama-imatrix from the fixed calibration text hfqa-original-calibration-v2 (6144 bytes of English, German and Chinese text, code and data formats, public domain; 12 chunks of 128 tokens) and applied during streaming with llama.cpp GGML quantization kernels. Matrix SHA-256: 47432094805c0279317c9abec0c50c897b3efbe4c493143e2816a124fac89e1e. Calibration text SHA-256: bc7216ea05c8bd05d6b9e94d813ec9eb651a2375288c08c06135a3f874d886f2. The calibration text is short; no quality improvement is claimed without a benchmark. Requantizing a previously quantized source can add quantization error.

The quant label names the llama.cpp preset. Stock llama.cpp keeps some tensors (for example normalization weights) in higher precision, so not every tensor is stored in the named type; the table shows what each file really contains.

File Tensors by type
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf F32: 162, Q4_K: 441, Q5_K: 40, Q6_K: 81

Test builds

File llama.cpp build Executable SHA-256
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf commit 19ff817fb25e9a1b838b1d7f9f4325f7bf9779c4 323e0b05a9b3e7ed30851d4a393cf457e7a4372dbc0d1be2e93bc16235721fe9

6. License Information

Source license: llama3.3. license llama3.3: not checked automatically (operator decision); the card names the declared license and the original author's terms apply Built with Llama. This model is a derivative of Meta's Llama and is subject to the Llama community license; its text and acceptable use policy are in the original repository: https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/blob/main/LICENSE

ไธญๆ–‡่ฏดๆ˜Ž ยท ๆจฑ่Šฑ Sakura๏ผˆ็‚นๅ‡ปๅฑ•ๅผ€๏ผ‰

1. ๆฆ‚่ฟฐ

็‹ฌ็ซ‹็š„็คพๅŒบ้‡ๅŒ–็‰ˆๆœฌ๏ผŒๅนถ้žไธŠๆธธไฝœ่€…็š„ๅฎ˜ๆ–นๅ‘ๅธƒใ€‚

ไฝฟ็”จ้‡่ฆๆ€ง็Ÿฉ้˜ต๏ผˆimatrix๏ผ‰้‡ๅŒ–ใ€‚ ้‡ๅŒ–ๆ—ถไฝฟ็”จไบ†็”ฑๅ›บๅฎšๆ กๅ‡†ๆ–‡ๆœฌ่ฎก็ฎ—ๅ‡บ็š„้‡่ฆๆ€งๆƒ้‡๏ผ›ๅ…ทไฝ“ๆ–นๆณ•ๅ’Œๅ“ˆๅธŒ่ง็ฌฌ 5 ่Š‚็š„ๆŠ€ๆœฏ็ป†่Š‚ใ€‚

ๆžถๆž„๏ผš70.6 B ๅ‚ๆ•ฐ ยท ๆบๆ ผๅผ๏ผšsafetensorsใ€‚

ๅŽŸๅง‹ๆจกๅž‹๏ผš75.0 GB๏ผˆQ8_0 GGUF๏ผŒ่ฟ™ไบ›้‡ๅŒ–็‰ˆๆœฌ็š„ๆฅๆบ๏ผ‰ใ€‚

้‡ๅŒ– ๅคงๅฐ ็›ธๆฏ”ๅŽŸๅง‹ๆจกๅž‹
Q4_K_M 42.5 GB ๅฐ 43 %

2. ๅŽŸๅง‹ๆจกๅž‹ไธŽ็ฝฒๅ

  • ๅŽŸๅง‹ๆจกๅž‹๏ผšmeta-llama/Llama-3.3-70B-Instruct
  • ๅฃฐๆ˜Ž็š„ๅŸบ็ก€ๆจกๅž‹๏ผšmeta-llama/Llama-3.1-70B๏ผˆๅ…ณ็ณป๏ผšfinetune๏ผ‰
  • ้‡ๅŒ–ๆฅๆบ๏ผšbartowski/Llama-3.3-70B-Instruct-GGUF๏ผˆๆ–‡ไปถ Llama-3.3-70B-Instruct-Q8_0.gguf๏ผ‰ใ€‚
  • ้‡ๅŒ–่€…๏ผšwebmp3ใ€‚ไธŠๆธธไฝœ่€…ไฟ็•™ๅ…ถ่ฎธๅฏ่ฏไธŽ็ฝฒๅใ€‚

3. ๆ–‡ไปถ

ๆ–‡ไปถ ๅคงๅฐ ้‡ๅŒ–
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf 42.5 GB Q4_K_M

4. ไธ‹่ฝฝไธŽไฝฟ็”จ

llama-cli -m <file>.gguf -p "Hello" -n 32

้œ€่ฆๆ”ฏๆŒ่ฏฅๆžถๆž„็š„ llama.cpp ็‰ˆๆœฌใ€‚ๅฆ‚ๆžœๆŸไธช้‡ๅŒ–่ขซๆ‹†ๅˆ†ไธบๅธฆ็ผ–ๅท็š„ๅคšไธช้ƒจๅˆ†๏ผŒ่ฏทๅฐ†ๅฎƒไปฌๆ”พๅœจไธ€่ตทๅนถๅŠ ่ฝฝ็ฌฌไธ€ไธชใ€‚

5. ้ชŒ่ฏ

  • GGUF ็ป“ๆž„ๆœ‰ๆ•ˆ๏ผšTrue
  • ็œŸๅฎžๆŽจ็†ๆต‹่ฏ•๏ผˆๆฏไธชๆ–‡ไปถ้ƒฝ็”ฑ llama.cpp ๅŠ ่ฝฝๅนถ็”Ÿๆˆไธ€ๆฌกๆ–‡ๆœฌ๏ผ‰๏ผšyes
  • ้™ค้žๆญคๅค„ๅˆ—ๅ‡บไบ†ๅฎžๆต‹ๆ•ฐๅ€ผ๏ผŒๅฆๅˆ™ไธๅฃฐ็งฐไปปไฝ•็ผ–็จ‹ๅŸบๅ‡†ใ€ๅ›ฐๆƒ‘ๅบฆๆˆ–่ดจ้‡ๅˆ†ๆ•ฐใ€‚
ๆŠ€ๆœฏ็ป†่Š‚๏ผš็กฎๅˆ‡็‰ˆๆœฌใ€ๅ“ˆๅธŒใ€ๅˆถไฝœๆ–นๅผ

็กฎๅˆ‡็‰ˆๆœฌ

  • ๅŽŸๅง‹ๆจกๅž‹ๆไบค๏ผš6f6073b423013f6a7d4d9f39144961bfbfbc386b๏ผˆ่ฏฅๆไบค็š„ๆจกๅž‹ๅก๏ผ‰
  • ้‡ๅŒ–ๆฅๆบ๏ผšๆไบค b6c5c9f176f3279204034e1d16d393105e95cb88๏ผŒSHA-256 d7314ee9cab0f263ec470fc984ceba44c0689645d6951f56a6e95f167383cc9c๏ผˆๅ‡บๅค„ๅทฒ้ชŒ่ฏ๏ผšTrue๏ผ‰

ๆ–‡ไปถๅ“ˆๅธŒ

ๆ–‡ไปถ ๅญ—่Š‚ๆ•ฐ SHA-256 ้‡ๅŒ–
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf 42520398880 fa24f459f08a314d44d5e38df96a4a82083ab4c9500fd794dfbdf8ba2765cb6c Q4_K_M

ๅˆถไฝœๆ–นๅผ

่ฏฅ่ทฏ็บฟไปŽไธŠๆ–นๆ ‡ๆ˜Ž็š„ๅ›บๅฎš็‰ˆๆœฌ Q8_0 ๆบ GGUF ่ฟ›่กŒ็Žฐๆœ‰ GGUF ็š„ๅ†้‡ๅŒ–ใ€‚ๆœฌไปปๅŠกๆฒกๆœ‰่ฝฌๆขไธŠๆธธ็š„ BF16/safetensors ๆฃ€ๆŸฅ็‚นใ€‚

ไฝฟ็”จ llama.cpp ๆ ‡ๅ‡†ๅทฅๅ…ทๅ’Œ้‡่ฆๆ€ง็Ÿฉ้˜ต๏ผˆimatrix๏ผ‰้‡ๅŒ–๏ผš็Ÿฉ้˜ต็”ฑ llama-imatrix ๆ นๆฎๅ›บๅฎš็š„ๆ กๅ‡†ๆ–‡ๆœฌ hfqa-original-calibration-v2๏ผˆ6144 ๅญ—่Š‚๏ผŒๅซ่‹ฑๆ–‡ใ€ๅพทๆ–‡ใ€ไธญๆ–‡ใ€ไปฃ็ ๅ’Œๆ•ฐๆฎๆ ผๅผ๏ผŒๅฑžๅ…ฌๆœ‰้ข†ๅŸŸ๏ผ›12 ไธชๅ—๏ผŒๆฏๅ— 128 ไธช่ฏๅ…ƒ๏ผ‰่ฎก็ฎ—ไธ€ๆฌก๏ผŒๅœจ้€ๅ—ๆตๅผ้‡ๅŒ–่ฟ‡็จ‹ไธญ๏ผŒ็”ฑ llama.cpp ็š„ GGML ้‡ๅŒ–ๅ†…ๆ ธๅบ”็”จใ€‚็Ÿฉ้˜ต SHA-256๏ผš47432094805c0279317c9abec0c50c897b3efbe4c493143e2816a124fac89e1eใ€‚ๆ กๅ‡†ๆ–‡ๆœฌ SHA-256๏ผšbc7216ea05c8bd05d6b9e94d813ec9eb651a2375288c08c06135a3f874d886f2ใ€‚ๆ กๅ‡†ๆ–‡ๆœฌ่พƒ็Ÿญ๏ผŒๆฒกๆœ‰ๅŸบๅ‡†ๆต‹่ฏ•๏ผŒๅ› ๆญคไธๅฃฐ็งฐ่ดจ้‡ๆๅ‡ใ€‚ ๅฏนๅทฒ้‡ๅŒ–็š„ๆบๅ†ๆฌก้‡ๅŒ–ๅฏ่ƒฝๅขžๅŠ ้‡ๅŒ–่ฏฏๅทฎใ€‚

้‡ๅŒ–ๆ ‡็ญพๆŒ‡ llama.cpp ็š„้ข„่ฎพๅ็งฐใ€‚ๆ ‡ๅ‡† llama.cpp ไผšๆŠŠ้ƒจๅˆ†ๅผ ้‡๏ผˆไพ‹ๅฆ‚ๅฝ’ไธ€ๅŒ–ๆƒ้‡๏ผ‰ไฟ็•™ไธบๆ›ด้ซ˜็ฒพๅบฆ๏ผŒๅ› ๆญคๅนถ้žๆฏไธชๅผ ้‡้ƒฝไปฅๆ‰€ๆ ‡ๆณจ็š„็ฑปๅž‹ๅญ˜ๅ‚จ๏ผ›ไธ‹่กจๅˆ—ๅ‡บๆฏไธชๆ–‡ไปถๅฎž้™…ๅŒ…ๅซ็š„ๅ†…ๅฎนใ€‚

ๆ–‡ไปถ ๆŒ‰็ฑปๅž‹็š„ๅผ ้‡ๆ•ฐ
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf F32: 162, Q4_K: 441, Q5_K: 40, Q6_K: 81

ๆต‹่ฏ•็‰ˆๆœฌ

ๆ–‡ไปถ llama.cpp ็‰ˆๆœฌ ๅฏๆ‰ง่กŒๆ–‡ไปถ SHA-256
Sakura-Llama-3.3-70B-Instruct-Q4_K_M.gguf commit 19ff817fb25e9a1b838b1d7f9f4325f7bf9779c4 323e0b05a9b3e7ed30851d4a393cf457e7a4372dbc0d1be2e93bc16235721fe9

6. ่ฎธๅฏ่ฏไฟกๆฏ

ๆบ่ฎธๅฏ่ฏ๏ผšllama3.3ใ€‚license llama3.3: not checked automatically (operator decision); the card names the declared license and the original author's terms apply

ๅฎŒๆ•ด็‹ฌ็ซ‹็š„ไธญๆ–‡็‰ˆ๏ผšREADME_zh.md

Downloads last month
172
GGUF
Model size
71B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for webmp3/Sakura-Llama-3.3-70B-Instruct-GGUF

Quantized
(173)
this model

Collection including webmp3/Sakura-Llama-3.3-70B-Instruct-GGUF