Sakura

Sakura · GGUF quantization of inception42/Jais-3-32B

Independent community quantization. Not an official release of the upstream author.

1. Model Overview

Architecture: k2_horizon · 34.8 B parameters · source formats: safetensors.

Original: 37.0 GB (Q8_0 GGUF, the source of these quantizations).

Quant Size Compared with the original
Q2_K 13.3 GB 64 % smaller
Q3_K_M 17.1 GB 54 % smaller
Q4_K_M 21.1 GB 43 % smaller
Q5_K_M 24.7 GB 33 % smaller

2. Original Model / Attribution

  • Original model: inception42/Jais-3-32B, commit 170144803abac94000f966ec215596d78a9bbcd8 (model card at that commit)
  • Quantized from: mradermacher/Jais-3-32B-GGUF at 73f86590df20fe5a44fc0e1555fbc8c2b024466a, file Jais-3-32B.Q8_0.gguf (SHA-256 cf654d0797561f15c27e50d85660ce5974be77e33c63e9bfe09565c65fdccd08; provenance verified: True)
  • Quantized by webmp3. Upstream authors retain their license and attribution.

3. Files

File Bytes SHA-256 Quant
Sakura-Jais-3-32B-Q2_K.gguf 13294761920 feb9ef064ec61b9e214e57166463e104ff28659e1700058cae4a67700e770b41 Q2_K
Sakura-Jais-3-32B-Q3_K_M.gguf 17088548800 4c6c60a365cc405c4056cb98a431cec14c2c3726e82f4d3a82094dfef998f8c1 Q3_K_M
Sakura-Jais-3-32B-Q4_K_M.gguf 21075789760 31726468739563143e04e9eefbc3b484d1be0d00b96f7b0e996fcf4a69dc6145 Q4_K_M
Sakura-Jais-3-32B-Q5_K_M.gguf 24696489920 b148ed7dc02f9389e4212a0c2f21a18c8ac50440f6f885177dcbcd0c7e791e86 Q5_K_M

4. How These Were Made

This route performs existing-GGUF requantization from the pinned Q8_0 source GGUF named above. This job did not convert the upstream BF16/safetensors checkpoint.

No importance-matrix calibration was run by this job. Requantizing a previously quantized source can add quantization error.

The quant label names the llama.cpp preset. Stock llama.cpp keeps some tensors (for example normalization weights) in higher precision, so not every tensor is stored in the named type; the table shows what each file really contains.

File Tensors by type
Sakura-Jais-3-32B-Q2_K.gguf F32: 129, Q2_K: 257, Q3_K: 128, Q4_K: 64, Q6_K: 1
Sakura-Jais-3-32B-Q3_K_M.gguf F32: 129, Q3_K: 257, Q4_K: 186, Q5_K: 6, Q6_K: 1
Sakura-Jais-3-32B-Q4_K_M.gguf F32: 129, Q4_K: 385, Q6_K: 65
Sakura-Jais-3-32B-Q5_K_M.gguf F32: 129, Q5_K: 385, Q6_K: 65

5. Verification

  • GGUF structure valid: True
  • Real inference test (every file loaded by llama.cpp and used to generate text once): yes
File llama.cpp build Executable SHA-256
Sakura-Jais-3-32B-Q2_K.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q3_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q4_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q5_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209

6. Download and Usage

llama-cli -m <file>.gguf -p "Hello" -n 32

Needs a llama.cpp build that supports this architecture. If a quantization is split into numbered parts, keep them together and load the first one.

7. Limitations

No coding benchmark, perplexity or quality score is claimed unless a measured value is listed above.

8. License Information

Source license: apache-2.0. apache-2.0 allows redistribution with attribution

中文说明 · 樱花 Sakura(点击展开)

1. 概述

独立的社区量化版本,并非上游作者的官方发布。

架构:k2_horizon · 34.8 B 参数 · 源格式:safetensors。

原始模型:37.0 GB(Q8_0 GGUF,这些量化版本的来源)。

量化 大小 相比原始模型
Q2_K 13.3 GB 小 64 %
Q3_K_M 17.1 GB 小 54 %
Q4_K_M 21.1 GB 小 43 %
Q5_K_M 24.7 GB 小 33 %

2. 原始模型与署名

  • 原始模型:inception42/Jais-3-32B,提交 170144803abac94000f966ec215596d78a9bbcd8(该提交的模型卡)
  • 声明的基础模型:未声明(关系:undeclared)
  • 量化来源:mradermacher/Jais-3-32B-GGUF,提交 73f86590df20fe5a44fc0e1555fbc8c2b024466a,文件 Jais-3-32B.Q8_0.gguf(SHA-256 cf654d0797561f15c27e50d85660ce5974be77e33c63e9bfe09565c65fdccd08;出处已验证:True)
  • 量化者:webmp3。上游作者保留其许可证与署名。

3. 文件

文件 字节数 SHA-256 量化
Sakura-Jais-3-32B-Q2_K.gguf 13294761920 feb9ef064ec61b9e214e57166463e104ff28659e1700058cae4a67700e770b41 Q2_K
Sakura-Jais-3-32B-Q3_K_M.gguf 17088548800 4c6c60a365cc405c4056cb98a431cec14c2c3726e82f4d3a82094dfef998f8c1 Q3_K_M
Sakura-Jais-3-32B-Q4_K_M.gguf 21075789760 31726468739563143e04e9eefbc3b484d1be0d00b96f7b0e996fcf4a69dc6145 Q4_K_M
Sakura-Jais-3-32B-Q5_K_M.gguf 24696489920 b148ed7dc02f9389e4212a0c2f21a18c8ac50440f6f885177dcbcd0c7e791e86 Q5_K_M

4. 制作方式

该路线从上方标明的固定版本 Q8_0 源 GGUF 进行现有 GGUF 的再量化。本任务没有转换上游的 BF16/safetensors 检查点。

本任务没有运行重要性矩阵(imatrix)校准。 对已量化的源再次量化可能增加量化误差。

量化标签指 llama.cpp 的预设名称。标准 llama.cpp 会把部分张量(例如归一化权重)保留为更高精度,因此并非每个张量都以所标注的类型存储;下表列出每个文件实际包含的内容。

文件 按类型的张量数
Sakura-Jais-3-32B-Q2_K.gguf F32: 129, Q2_K: 257, Q3_K: 128, Q4_K: 64, Q6_K: 1
Sakura-Jais-3-32B-Q3_K_M.gguf F32: 129, Q3_K: 257, Q4_K: 186, Q5_K: 6, Q6_K: 1
Sakura-Jais-3-32B-Q4_K_M.gguf F32: 129, Q4_K: 385, Q6_K: 65
Sakura-Jais-3-32B-Q5_K_M.gguf F32: 129, Q5_K: 385, Q6_K: 65

5. 验证

  • GGUF 结构有效:True
  • 真实推理测试(每个文件都由 llama.cpp 加载并生成一次文本):yes
文件 llama.cpp 版本 可执行文件 SHA-256
Sakura-Jais-3-32B-Q2_K.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q3_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q4_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209
Sakura-Jais-3-32B-Q5_K_M.gguf b11669-620d2225c 3b7167843ffb0f3677be964dc80cfc59a0167c82e11eff62f6334f66e831f209

6. 下载与使用

llama-cli -m <file>.gguf -p "Hello" -n 32

需要支持该架构的 llama.cpp 版本。如果某个量化被拆分为带编号的多个部分,请将它们放在一起并加载第一个。

7. 局限

除非上文列出了实测数值,否则不声称任何编程基准、困惑度或质量分数。

8. 许可证信息

源许可证:apache-2.0。apache-2.0 allows redistribution with attribution

完整独立的中文版:README_zh.md

Downloads last month
330
GGUF
Model size
35B params
Architecture
k2-horizon
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for webmp3/Sakura-Jais-3-32B-GGUF

Quantized
(9)
this model

Collection including webmp3/Sakura-Jais-3-32B-GGUF