John1604/Qwen3-30B-A3B-Instruct-2507-gguf

Use the model in LM Studio

download and install LM Studio

https://lmstudio.ai/

Discover models

In the LM Studio, click "Discover" icon. "Mission Control" popup window will be displayed.

In the "Mission Control" search bar, type "John1604" and check "GGUF", the model should be found.

Download the model.

Load the model.

Ask questions.

Use the model in ollama

First download and install ollama.

https://ollama.com/download

Command

in windows command line, or in terminal in ubuntu, type:

ollama run hf.co/John1604/Qwen3-30B-A3B-Instruct-2507-gguf:q6_k

(q6_k is the model quant type, q5_k_s, q4_k_m, ..., can also be used)

C:\Users\developer>ollama run hf.co/John1604/Qwen3-30B-A3B-Instruct-2507-gguf:q6_k
pulling manifest
...
verifying sha256 digest
writing manifest
success
>>> 

quantized models

Type	Bits	Quality	Description
Q2_K	2-bit	🟥 Low	Minimal footprint; only for tests
Q3_K_S	3-bit	🟧 Low	“Small” variant (less accurate)
Q3_K_M	3-bit	🟧 Low–Med	“Medium” variant
Q4_K_S	4-bit	🟨 Med	Small, faster, slightly less quality
Q4_K_M	4-bit	🟩 Med–High	“Medium” — best 4-bit balance
Q5_K_S	5-bit	🟩 High	Slightly smaller than Q5_K_M
Q5_K_M	5-bit	🟩🟩 High	Excellent general-purpose quant
Q6_K	6-bit	🟩🟩🟩 Very High	Almost FP16 quality, larger size
Q8_0	8-bit	🟩🟩🟩🟩	Near-lossless baseline