FoolDev/Janus-35B-HERETIC

Janus-35B

Flagship Reasoning. Sparse Footprint. Uncensored. llmfan46's Heretic abliteration of Qwen 3.6 35B-A3B, repackaged with Claude Fable 5 in the teacher slot.

Architecture: Qwen 3.6 35B-A3B (MoE) | Total Params: 35B | Active Params: 3B | Base: Heretic (llmfan46) | Teacher: Claude Fable 5 | Type: Distilled + Abliterated MoE LLM

A personal fork of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — an uncensored Heretic-style abliteration of Qwen/Qwen3.6-35B-A3B, the 35B-total / 3B-active mixture-of-experts multimodal base — repackaged as Janus-35B with Claude Fable 5 reasoning data in the teacher slot. Refusal-trained behavior is dialed back at the base layer.

TL;DR

One-liner via Hugging Face (pulls a GGUF + this repo's root-level template / system / params files, including the tool-calling template — HF's Ollama bridge ingests those three files, not Modelfile):

ollama run hf.co/FoolDev/Janus-35B-HERETIC               # default ~19 GB Q4_K_M
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M        # same blob, explicit tag

Or build locally (uses this repo's Modelfile, kept in sync with the three bridge files):

git clone https://huggingface.co/FoolDev/Janus-35B-HERETIC && cd Janus-35B-HERETIC
ollama create janus -f Modelfile && ollama run janus

After either path, ollama show janus lists completion, tools, and thinking under Capabilities. Hardware: the default num_ctx is 1000000 — a ~1M ceiling above the 262144 native window (YaRN is not baked into this GGUF, so context past ~262K degrades) — so trim it down to fit your host (see Hardware requirements).

What's here

File	Use
Janus-35B-A3B.Q4_K_M.gguf	Recommended default, ~19 GB
Modelfile	Ollama wrapper for local builds (ollama create janus -f Modelfile) — overrides the GGUF's embedded template with one that exposes .Tools / .ToolCalls to Ollama's capability detector.
template, system, params	Used by HF's Ollama bridge when users ollama run hf.co/FoolDev/Janus-35B-HERETIC directly. The bridge does not read Modelfile (see HF Ollama docs); it ingests these three root-level files instead. Kept in sync with the Modelfile's TEMPLATE / SYSTEM / PARAMETER directives.
scripts/build.sh	Pulls a GGUF from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF (default Q4_K_M) and runs ollama create janus. The bundled Q4_K_M is already this Heretic quant; use this to build other quants locally.
scripts/check_bridge_sync.py	Run before pushing a Modelfile / template / system / params edit to verify the four configurations remain in sync. Exits 0 if in sync, 1 with a per-key diff if not.
scripts/smoke_test.sh	Integration smoke test against a running Ollama daemon: server reachable, model loaded, tools capability present, chat round-trip, and no control-token leakage. TOOLS_TEST=1 adds a tool-call round-trip. Defaults to MODEL=janus.
scripts/bench.sh	Measures tok/s from Ollama's eval_count / eval_duration over a short/medium/long prompt mix (with a discarded warmup). Defaults to MODEL=janus.
scripts/load_bundle.sh	Loads the bundled Janus-35B-A3B.Q4_K_M.gguf into Ollama as a local janus tag without an upstream pull (smudges the LFS pointer via hf download if needed, checks the arch is qwen35moe).
scripts/fetch_vision.sh	Downloads the vision projector (Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf) from the Heretic GGUF repo for llama.cpp image input (Ollama vision is broken upstream — see Vision).
examples/	Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python (text, tools, and vision — see examples/README.md)

GGUF-only release. Pull the Heretic safetensors from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic if you need the transformers tree (or the vanilla pre-Heretic base from Qwen/Qwen3.6-35B-A3B).

Bundled blob status: the bundled Janus-35B-A3B.Q4_K_M.gguf is the Heretic Q4_K_M quant (from llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF), qwen35moe-stamped and verified against the Architecture below (40 layers, 256 experts, vocab 248,320). It serves the uncensored Heretic behavior directly; ./scripts/build.sh remains the path for other quants.

Architecture

• Qwen 3.6, 35B total / 3B active, MoE (256 experts, 8 activated per token)
• 40 layers, 10 × (3 × DeltaNet → MoE / 1 × Gated Attention → MoE)
• 262 144 native context (extensible to ~1 M with YaRN, but YaRN is not enabled in the bundled GGUF)
• Vision + video supported by upstream (mmproj not included in this release)
• Vocab 248,320

Quick start

llama.cpp / LM Studio

Drop the GGUF into your loader of choice. The chat template is embedded in the GGUF metadata, so llama.cpp's --chat-template auto and LM Studio's GGUF auto-detection handle plain conversation correctly.

Ollama

The chat template baked into the GGUF is not sufficient on Ollama — it lacks the .Tools / .ToolCalls blocks Ollama's capability detector requires, so a naive ollama pull reports does not support tools and rejects any request carrying a tools array. Two paths fix this:

# A. Pull straight from HF (uses the root-level template/system/params files):
ollama run hf.co/FoolDev/Janus-35B-HERETIC               # default tag, ~19 GB Q4_K_M
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M        # same blob, explicit tag
# Note: HF's Ollama bridge does NOT read Modelfile; it reads template/system/params.

# B. Build locally (uses Modelfile, which is kept in sync with the three above):
ollama create janus -f Modelfile && ollama run janus

After the local build (path B), ollama show janus lists completion, tools, and thinking under Capabilities. (The HF-pull paths register the model under the full tag hf.co/FoolDev/Janus-35B-HERETIC, not janus.)

Inference examples

Once the model is loaded (via ollama run janus, lms server, or llama-server), all the standard OpenAI-compatible clients work. Examples assume the loader is listening on http://localhost:11434 (Ollama default) — adjust the port for LM Studio (:1234) or llama.cpp (:8080). Runnable versions of everything below live in examples/.

The examples use model: "janus", the tag from the local build (path B). If you pulled via the TL;DR one-liner instead, use the full tag hf.co/FoolDev/Janus-35B-HERETIC, or run ollama cp hf.co/FoolDev/Janus-35B-HERETIC janus once to create the short tag.

curl

curl -s http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "janus",
    "messages": [
      {"role": "system", "content": "You are Janus, a precise reasoning assistant."},
      {"role": "user", "content": "Sketch an algorithm to detect cycles in a directed graph."}
    ],
    "temperature": 0.6,
    "max_tokens": 800
  }' | jq -r '.choices[0].message.content'

Python (openai-compat)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored")

resp = client.chat.completions.create(
    model="janus",
    messages=[
        {"role": "user", "content": "Write a haiku about a stack overflow."}
    ],
    temperature=0.8,
    top_p=0.95,
)
print(resp.choices[0].message.content)

Streaming

stream = client.chat.completions.create(
    model="janus",
    messages=[{"role": "user", "content": "Explain RoPE briefly."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

Recommended sampling

Use	temp	top_p	top_k	repeat_penalty
Default (Fable-matched)	1.0	0.95	0	1.05
Tighter reasoning	0.6	0.95	20	1.05
Creative / RP	0.8	0.95	40	1.02

The shipped default is Fable-matched — warm (temperature 1.0), no top_k, with top_p 0.95 + repeat_penalty 1.05 kept as loop insurance. Drop to the reasoning row for tighter, more deterministic output; lower temperature (0.4–0.6) and bump repeat_penalty to 1.08 if it loops inside <think> tags.

System prompt

You are Janus, a precise and capable assistant for reasoning, writing, coding, and long-form dialogue.

Behavior rules:
- Answer the user's actual request directly.
- Be accurate, complete, and structured.
- Think before answering, but do not get stuck in repetitive loops or meta-commentary.
- If the request is ambiguous or incomplete, state what is missing and make the smallest reasonable assumption needed to continue.
- If the user wants creative writing, preserve tone, continuity, and character consistency.
- If the user wants analysis or technical help, prefer concrete steps, examples, and decisions over fluff.
- Finish with a usable answer, not just planning.

Vision

The Qwen 3.6 base supports image (and video) input via a separate mmproj projector. The full multimodal stack is:

Janus-35B-A3B.Q4_K_M.gguf                              (~19 GB, the text decoder)
Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf    (~903 MB, the vision projector)

The projector and other-quant text decoders live at llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF (BF16 mmproj only). For the vanilla pre-Heretic projector in F16/F32, see unsloth/Qwen3.6-35B-A3B-GGUF (mmproj-F16.gguf). This repo intentionally does not redistribute either; ./scripts/fetch_vision.sh pulls the projector into the repo root.

Loader compatibility

Loader	Text	Vision (mmproj)	Notes
llama.cpp (llama-mtmd-cli, llama-server --mmproj)	✅	✅	Reference path. Upstream has the qwen35moe arch entry.
llama-cpp-python	✅	✅	See examples/llama_cpp_vision.py.
Ollama 0.24+	✅	❌	Text inference works: Ollama's Go engine has the qwen35 / qwen35moe arch entries. Vision (mmproj) is still broken: the C++ llama.cpp fallback that Ollama switches to when an mmproj is attached lacks those entries. ollama create accepts a dual-FROM (text + mmproj) and ollama show reports vision capability — but the first inference request fails with error loading model architecture: unknown model architecture: 'qwen35moe', and once mmproj is attached this blocks text inference too. See ollama/ollama#14575 (open — the earlier #15898 was closed as its duplicate, and the sync PR #15899 was closed unmerged).
LM Studio	✅	✅	Uses upstream llama.cpp directly.

Vision via llama.cpp

# Fetch the projector first (into the repo root):
./scripts/fetch_vision.sh                    # Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf

# A. HTTP via llama-server (the easiest path):
llama-server \
  -m Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --host 127.0.0.1 --port 8765 -c 8192 -ngl 99
# then POST OpenAI-style chat completions with an image_url content block —
# e.g. {"type":"image_url","image_url":{"url":"data:image/jpeg;base64,..."}}
# The thinking trace arrives in message.reasoning_content; the visible
# answer is in message.content. Budget ≥500 max_tokens so the reasoning
# block doesn't crowd out the final answer.

# B. CLI via llama-mtmd-cli (one-shot). It's a separate cmake target, so a
#    selective build can skip it; a plain `cmake --build build` produces it.
llama-mtmd-cli \
  -m Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --image photo.jpg \
  -p "Describe this image."

# C. Python via llama-cpp-python:
python examples/llama_cpp_vision.py \
  --gguf Janus-35B-A3B.Q4_K_M.gguf \
  --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
  --image /path/to/photo.jpg \
  --prompt "What is in this image?"

Until the Ollama upstream issue is fixed, treat Ollama as text-only for this model. The bundled Q4_K_M decoder pairs with the projector directly — the mmproj is family-wide for Qwen 3.6 35B-A3B, so no separate text download is needed for vision.

Hardware requirements

This is a ~19 GB Q4_K_M GGUF. Ollama's runtime footprint is roughly 2× the model file (weights mmap + compute graph), plus a KV cache that scales ~2 GB per 32K (q8_0). The default num_ctx is 1000000 — a ~1M ceiling above the 262144 native window — so KV alone is ~61 GB for ~99 GB total (theoretical, extrapolated from the ~2 GB/32K rule). This GGUF ships no YaRN rope-scaling (rope.freq_base 10M, no rope.scaling), so positions past the 262144 native window use untrained RoPE and output degrades — treat 1M as an advertised ceiling and keep real work within ~262K. Most hosts must override num_ctx down: e.g. the 262144 native window → ~16 GB KV / ~53 GB total, or num_ctx 32768 → ~2 GB KV / ~39 GB total. 32 GB hosts fit the model by trimming ctx + batch (see Z13 row in the table).

How to override it: ollama run has no -o flag, and OLLAMA_CONTEXT_LENGTH only sets a default that the baked num_ctx overrides — so set it per-session from the interactive prompt. The model loads lazily on the first message, so /set applies before the default context is allocated:

ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
>>> /set parameter num_ctx 4096
>>> /set parameter num_batch 256

Programmatic callers pass the same via the API options field: "options": {"num_ctx": 4096, "num_batch": 256}.

Hardware	Status
≥48 GB RAM (CPU-only)	Works, ~3-6 tok/s
Single H100/A100 80 GB	Works, full offload, ~30+ tok/s
RTX 4090 24 GB / 5090 32 GB + 32 GB RAM	Works, partial offload, ~15-25 tok/s
Mac Studio M2/M3 Ultra 64 GB+ unified	Works, ~20+ tok/s
32 GB unified-memory laptops (Ryzen AI Max+, Apple M-series)	Works with num_ctx ≤ 4096 and num_batch ≤ 256 to fit the compute graph; the 1M default OOMs (override num_ctx down). Measured 28.71 tok/s on ASUS ROG Flow Z13 GZ302EA at Q4_K_M (Radeon 8060S iGPU via ROCm gfx1151).

Reaching a coherent ~1M context (opt-in YaRN). The bundled GGUF ships no YaRN rope-scaling, so the 1M default degrades past the 262144 native window (see above). Ollama has no rope knob, so for a genuinely coherent long context run the GGUF under llama.cpp with YaRN enabled:

llama-server -m Janus-35B-A3B.Q4_K_M.gguf \
  --rope-scaling yarn --yarn-orig-ctx 262144 --rope-scale 3.8 -c 1000000

--rope-scale 3.8 ≈ 1000000 / 262144; use a smaller factor for a smaller window. Static YaRN rescales all prompts, so enable it only when you actually need > 262K — it slightly degrades short-context quality otherwise.

Chat template

The model uses the standard Qwen 3.x ChatML format with <|im_start|> / <|im_end|> role markers. The template is embedded in the GGUF metadata for plain conversation use, but Ollama users should rely on the TEMPLATE block in the included Modelfile — that version exposes the tool-calling scaffolding Ollama's capability detector requires (the embedded template alone is insufficient; see Ollama above).

Plain conversation

<|im_start|>system
You are Janus, a precise and capable assistant…<|im_end|>
<|im_start|>user
What is the time complexity of mergesort?<|im_end|>
<|im_start|>assistant

With reasoning trace

When the model decides to think, the assistant turn contains a <think>…</think> block followed by the visible answer:

<|im_start|>assistant
<think>
The user is asking about mergesort. Mergesort divides the array, recursively sorts each half, then merges. The recurrence T(n) = 2T(n/2) + O(n) solves to O(n log n).
</think>

Mergesort runs in **O(n log n)** time in the worst, average, and best cases. The recurrence is T(n) = 2T(n/2) + O(n), which solves to Θ(n log n) by the master theorem.<|im_end|>

Most clients (Open WebUI, LibreChat, etc.) hide the <think> block by default and show only the final answer. If your client doesn't, set its "show reasoning" toggle off.

Disabling thinking

This is a reasoning-first model — it opens a <think> block by default. For a direct answer with no reasoning trace (simple or latency-sensitive calls), turn thinking off:

ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M --think=false

or send "think": false on /api/chat. With thinking off the model skips the reasoning trace and answers straight into content; with it on (the default) reasoning is emitted into the thinking field.

Tool / function calling

The wire format depends on which path you take. Both are valid — the model adapts to whichever format the system prompt specifies.

Ollama path (this repo's Modelfile). The TEMPLATE advertises tools inside <tools>…</tools> and asks the model to reply in JSON-in-XML — the form Ollama's tool-call extractor parses into a structured tool_calls array on /api/chat and /v1/chat/completions:

<tool_call>
{"name": "get_weather", "arguments": {"city": "Tokyo"}}
</tool_call>

Embedded-jinja path (llama.cpp, llama-cpp-python, LM Studio). The Qwen 3.6 native chat template baked into the GGUF instructs the model to emit a more verbose XML form. This is the shape you'll see if you talk to llama-server or LM Studio directly:

<tool_call>
<function=get_weather>
<parameter=city>
Tokyo
</parameter>
</function>
</tool_call>

Pick the parser shape that matches your loader. Don't mix.

Example (Ollama, OpenAI-compatible API)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored")

resp = client.chat.completions.create(
    model="janus",
    messages=[
        {"role": "user", "content": "Call get_weather for Tokyo. Respond ONLY with the tool call."}
    ],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }],
    temperature=0.3,
)
print(resp.choices[0].message.tool_calls)
# [ToolCall(id='call_xxx', type='function',
#           function=Function(name='get_weather', arguments='{"city":"Tokyo"}'))]

Tips

• Use direct prompts ("Call X for Y") rather than soft hints ("Use the tool"). The model thinks before committing to a call, and weak prompts can exhaust num_predict inside the <think> block before the call is emitted.
• Allow at least num_predict: 1024 (or max_tokens: 1024) for tool-calling turns, more if the schemas are large.
• The Modelfile's JSON-in-XML format is what Ollama's tool-call extractor understands; if you swap loaders, swap the parser to match (see "Embedded-jinja path" above).

Known limitations

• No mmproj in this release. The base Qwen3.6 supports image and video input via a separate mmproj file, which is not included here. Text-only inference works out of the box; multimodal inference requires fetching Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf (or equivalent) from upstream — run ./scripts/fetch_vision.sh and see Vision for the full path.
• Quantization-induced quality loss. Q4_K_M is a strong general-purpose quant but does measurably degrade math and code accuracy compared to BF16. If you need maximum quality, run the upstream safetensors on a GPU that fits BF16 (~70 GB).
• MoE expert utilization is uneven. Stock Qwen3.6-35B-A3B routes 8 of 256 experts per token. On narrow domains (e.g. only one programming language) a small subset of experts dominates; load-balance loss was a training-time concern, not a runtime guarantee.
• Thinking traces can loop. Like most reasoning-distilled models, Janus-35B occasionally gets stuck repeating itself inside <think> tags. Mitigations: lower temperature to 0.4-0.6, raise repeat_penalty to 1.08, or set a <think>-token budget cap if your loader supports it.
• Large tool-call arguments can be dropped. Ollama's JSON-in-XML tool format makes the model JSON-escape the entire arguments object inline; for a big/complex payload (e.g. a file's content in a write_file call) the model can fail to escape it, so the field arrives undefined and the call fails. Qwen's native <function=…><parameter=…> format (raw values, no escaping) was tested as a fix but parses unreliably through Ollama, so the template deliberately keeps JSON-in-XML. Mitigation: write large files in smaller pieces per call.
• Uncensored base — not aligned with any specific safety policy. This is a personal repackage of an open-weight base whose refusal behavior has been abliterated away (the llmfan46 Heretic base). There is no RLHF refusal layer; the model will attempt most requests, so downstream safety is entirely the operator's responsibility.
• No formal evaluation in this card. Most numbers in the hardware table are estimates; the Z13 row (28.71 tok/s at Q4_K_M) is measured. If you produce real benchmarks (MMLU, HumanEval, etc.) and want them included, file a PR.

Related models

Model	Size	Notes
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic	35B / 3B active	Immediate base. Uncensored Heretic abliteration of Qwen 3.6 35B-A3B; transformers-native safetensors.
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF	35B / 3B active	Heretic GGUFs — pull other quants here; the bundled Q4_K_M is already this Heretic quant.
llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved	35B / 3B active	Same Heretic base but keeps the MTP head for vLLM / SGLang speculative decoding.
Qwen/Qwen3.6-35B-A3B	35B / 3B active	Upstream pre-Heretic base model. transformers-native multimodal weights.
FoolDev/Thanatos-27B-HERETIC	27B dense	Dense sibling on the llmfan46/Qwen3.6-27B-uncensored-heretic-v2 Heretic base. Same teacher (Fable 5), same dataset family, smaller memory footprint, no MoE quirks. (The older FoolDev/Thanatos-27B and Thanatos-27B-Heretic slugs now 307 to this path.)
Crownelius/Crow-9B-HERETIC-4.6	9B dense	Heretic-flavored fine-tune on a smaller 9B Qwen base. Useful as a fast first-pass model when 35B is too heavy for the host.

Credits

• Immediate base: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — Heretic-style abliteration of Qwen 3.6 35B-A3B
• Upstream base: Qwen/Qwen3.6-35B-A3B (Alibaba)
• Reasoning teacher: Claude Fable 5 (Anthropic)
• Distillation lineage and dataset curation: Crownelius

License inherited from upstream: Apache-2.0.