WARMIND-1017-FLARE-gb202

WARMIND-1017, prepared as a Flare runtime for four RTX PRO 6000 Blackwell GPUs.

WARMIND-1017 is jcartu's abliterated release of DeepSeek-V4.1-Flash-1017. The refusal direction is inverted in 132 dense tensors and nothing else is touched.

This repository is that model exported to Flare. Flare is the native Rust serving runtime from NVIDIA's spark-local-inference project, led by Luke Alonso of Local Inference Lab. The package uses the same gb202-tp4 layout as Local Inference Lab's own DeepSeek-V4.1-Flash-1017-FLARE gb202 package, and it targets the Flare image Local Inference Lab publishes.

This revision is built on Flare 972e28e4 and FlashInfer 6aaa9b47, the same commits as Local Inference Lab's gb202 revision 6d51b296 (November 7), with the clustered PCIe all-reduce ladder. Download it and run it; there is no hour-plus export to do yourself. This gb202 / main revision ships WARMIND-1017 v1.1 (orth4c), which keeps 0/100 refusals while closing two-thirds of the SQuAD 2.0 unanswerable hallucination gap and matching stock under a grounding system prompt. Other builds stay available: gb202-v1.0-972e28e (original v1.0 a3.0 weights on Flare 972e28e4), gb202-78b18b1 (Flare 78b18b1c) and gb202-6fa01a4 (Flare cc696a6b, for the older release-6fa01a4-main-20261101 image).

At a glance

Weights jcartu/WARMIND-1017 @ a6382f22 (v1.1 / orth4c, abstention-preserving edit). Original v1.0 stays on gb202-v1.0-972e28e
Base model deepseek-ai/DeepSeek-V4.1-Flash-1017 (DeepSeek, MIT)
Format Flare runtime package, gb202-tp4 deployment: weights, compiled CUDA kernels and captured execution graphs. Native expert scales (not CSF), runtime-tuned collectives with the gb202_pcie_tp4_clustered_full ladder
Hardware Four 96 GB SM120 GPUs (RTX PRO 6000 Blackwell), tensor parallel 4, Linux x86-64
Runtime Flare image ghcr.io/local-inference-lab/flare:release-972e28e-main-20261107 (the release and latest tags pointed to it on November 7). Tested on that image; we have not run this package on release-18b16e1-main-20261104. Transformers, vLLM, SGLang and llama.cpp cannot load this format
Needs NVIDIA Container Toolkit (tested with driver 615.78.08), 524 GB (488 GiB) of local NVMe (hf writes straight into the folder), and host RAM for the Engram row cache you choose (32 GiB in the command below), the 5.7 GiB of resident Engram scales, and a few GiB for Flare itself
Capacity 32 concurrent requests, 1,048,576-token context per request, 16,864,512 KV tokens on our box
Speculation Seven MTP draft tokens per round, adaptive verification
Engram 188.8 GiB of tables. Kept on disk by default; RAM optional
Size 524 GB (488 GiB)
Signed by Local development key, namespace local-development, fingerprint fa77427544ec0959c79734f8fb5632e2c1ac97c674ada5e07d6ba127b550ce68 (development-publisher.json)
License MIT for the weights; Apache-2.0 and BSD-3-Clause for the compiled runtime components. See License
Access Public download; no access request needed

How it differs from LiL's gb202 package

LiL's gb202 revision 6d51b296 (November 7, 11:06 UTC) is built on the same Flare and FlashInfer commits, with native expert scales and runtime-tuned clustered collectives, like this one.

  • Weights. WARMIND-1017 instead of stock DeepSeek-V4.1-Flash-1017. The files for the untouched layers and the Engram tables come out byte-identical to LiL's (see License). The edited layers 4 to 36 differ.
  • Export settings. joninco's settings: NVFP4 draft head, FP8 Markov head and vocabulary-parallel sampling, with BF16 MLA compute.
  • KV capacity. On our box this package holds 16,864,512 KV tokens. LiL's card for gb202 revision 6d51b296 gives about 16.9 million on four 96 GB GPUs; we have not measured that revision here.
  • Sampling. Like every DeepSeek V4.1 Flare package, temperature, top-k, min-p and the presence and frequency penalties are fixed at their defaults. A request that sets them is served at the default, not rejected.
    • top-p is fixed at 1.0 as well. Vocabulary-parallel sampling does not support a lower top-p. Flare at this commit (since #208) declares no top-p range for DeepSeek V4.1, so a request with a lower top-p is served at 1.0 instead of failing; LiL's 6d51b296 is built the same way.
    • Repetition penalty can be set from 1.0 to 1.1, as in LiL's 6d51b296.
  • Signing. Signed with our own development key, not Local Inference Lab's. Flare asks you to trust it once (see below).

What changed in v1.1 (orth4c, 2026-11-16)

  • Abstention-preserving weights (jcartu/WARMIND-1017 @ a6382f22). Same Flare 972e28e4 + FlashInfer 6aaa9b47 toolchain and gb202_pcie_tp4_clustered_full collective ladder as v1.0 (gb202-v1.0-972e28e), re-exported from the v1.1 orth4c checkpoint. Before applying grimjim's norm-preserving biprojection across layers 4–36, a 4D orthonormal epistemic-abstention basis ([sq_chat, sq_think, gen_epistemic, PC1], extracted from disjoint SQuAD 2.0 train-v2.0 matched passage pairs + missing-evidence pairs) is projected out of the refusal direction with projection-compensated $\alpha_L = \min(3.5,, 3.0/(1 - |U_4^\top \hat{r}_L|_2^2))$.
  • Quality gates. Refusals stay at 0/100 true refusals on native (0/100 keyword) and 0/100 true refusals on CSF (1/100 keyword). Plain SQuAD 2.0 unanswerable fabrications drop from 37.4% (v1.0) to 28.4% (βˆ’9.0 pts, 95% CI βˆ’12.0 to βˆ’6.0; stock 23.8%); with the strict grounding system prompt, v1.1 scores 25.6% vs stock 24.0% (+1.6 pts, 95% CI βˆ’0.2 to +3.6, indistinguishable from stock). 29.4% invented on held-out Round-1 SQuAD2 (vs 35.8% v1.0, 25.0% stock); TruthfulQA MC1 improves from 82.5% to 84.7%.
  • Multimodal / vision check (POPE MSCOCO val2014, 9,000 items). Measured on the CSF v1.1 package (enable_thinking: false, C=4): 88.17% overall accuracy (88.08% F1; 91.43% random, 87.20% popular, 85.87% adversarial; 49.28% yes-ratio; 0/9,000 errors or truncations), plus 81.67% on a 300-item enable_thinking: true check.
  • Runtime tip (Flare main >= 6b68b9b4). Tested on both release-972e28e-main-20261107 (--stage-only) and upstream Flare 6b68b9b4 (which merges prefix-cache fixes #298, #302 and snapshot-planning fix #309): on 4x RTX PRO 6000 with RAM Engrams, the CSF v1.1 build ran ~250k-token cold prefill at 19,153 tok/s and context-0 llm_decode_bench at C1 320, C8 1,196, C32 2,337 tok/s.

Earlier v1.0 packaging history (78b18b1c -> 972e28e4)

The historical notes below (and the v1.0 baseline speed rows in Measured / How it was built) were recorded on the v1.0 (gb202-v1.0-972e28e) package before the v1.1 132-tensor weight update.

The build exported at Flare 78b18b1c (gb202-tp4-202611042329, on the gb202-78b18b1 branch) used FlashInfer e977747a. This one was exported at Flare 972e28e4 with FlashInfer 6aaa9b47, the commits LiL's gb202 revision 6d51b296 was prepared from, with the same export settings and collective ladder.

  • Attention masking. FlashInfer 6aaa9b47 masks KV storage a request must not read in the attention kernels (1f8ed8a3, Luke Alonso; #73 and #74, Mark Yong), so stale page bytes cannot reach the attention output. This addresses the failure in #116, where requests resuming from the prefix cache under KV pressure ended with prepared model reported device error 6; it was reproduced and the fix validated on Qwen3.8-27B. One report in that issue of the same error with DeepSeek V4.1 on four DGX Sparks (TP4 over RoCE) persisted with the masked kernels and is still open. Kernels are captured at export, so packages exported before these commits keep the old kernels even on a newer image. We have not seen this error on our box with any WARMIND build.

  • top-p handled upstream. The 78b18b1c build applied PR #180 locally. Flare at 972e28e4 carries #208, which declares no DeepSeek top-p range (the same effect as PR #180), so this build is exported from an unmodified Flare commit.

  • MoE autotuning. FlashInfer #78 (Martin Vit) adds whole-batch expert reuse to the cases MoE autotuning measures.

  • Weights. Unchanged. 170 of the 186 package files, 521.0 GB, are byte-identical to the 78b18b1c build, including every weight file. Only the per-rank deployment files (kernels, graphs and initial state, plus one new initial-state-globals file, for rank 3), the manifest and export.json are new.

  • Speed. Both builds' release checks ran on their downloaded copies with the same command, except for the image (release-18b16e1-main-20261104 for the 78b18b1c build), on different nights:

    78b18b1c build This build
    Festr's single-user test 544 tok/s 527 tok/s
    llm_decode_bench C1 299 297
    C8 1,144 1,090
    C16 1,593 1,604
    C32 2,071 2,113
    KV capacity 16,807,936 16,864,512

    C8 came out 5% lower and the other cells within 3%. We have no same-night A/B of the two native builds, so we cannot separate that from night-to-night spread.

  • Image. Tested on release-972e28e-main-20261107.

Other hardware

Download and run

Download the package with the Hugging Face hf CLI, check it, then serve it with the Flare image:

mkdir -p models flare-state
hf download jcartu/WARMIND-1017-FLARE-gb202 --revision gb202 --local-dir ./models
(cd models && sha256sum -c SHA256SUMS)
echo "flare:x:$(id -u):$(id -g)::/state:/sbin/nologin" > flare-state/passwd

docker run --rm -it --user "$(id -u):$(id -g)" -w /state \
  -v "$PWD/flare-state/passwd:/etc/passwd:ro" \
  --gpus '"device=0,1,2,3"' -e CUDA_DEVICE_ORDER=PCI_BUS_ID \
  --ipc=host --ulimit memlock=-1:-1 --ulimit nofile=1048576:1048576 \
  --security-opt seccomp=unconfined -p 127.0.0.1:8000:8000 \
  -v "$PWD/models:/models:ro" -v "$PWD/flare-state:/state" \
  ghcr.io/local-inference-lab/flare:release-972e28e-main-20261107 \
  run /models/model.safetensors \
  --publisher-keys /models/development-publisher.json \
  --trust-store /state/trust.toml --gpus 0,1,2,3 \
  --max-concurrency 32 --mtp-draft-tokens 7 --mtp-adaptive-verification \
  --mtp-adaptive-verification-alpha 1 --scheduler-policy throughput \
  --embedding-table-storage disk --embedding-table-cache-size 32GiB \
  --embedding-table-resident-scales --kv-cache-limit memory \
  --host 0.0.0.0 --port 8000
  • Download. On our connection the full download took 15 minutes. The checksum step reads all 524 GB again; skip it if you trust the download.
  • Other builds. For the build exported at Flare 78b18b1c, download with --revision gb202-78b18b1. For release-6fa01a4-main-20261101, download with --revision gb202-6fa01a4 and use that image in the command. Each branch's card has its own numbers.
  • Trust prompt. On first start Flare shows this package's signing key and asks you to type local-development.
    • What it trusts: this one key, not development keys in general. The choice is remembered in flare-state/trust.toml.
    • License line: the prompt shows License: mit, which refers to the DeepSeek weights.
    • Unattended starts: add --dangerously-approve-key fa77427544ec0959c79734f8fb5632e2c1ac97c674ada5e07d6ba127b550ce68 --dangerously-accept-publisher local-development, which accepts exactly this key.
  • The passwd line and -w /state. The release image only knows its own flare user. When Docker runs as your user ID, Flare cannot find a home directory and stops with cannot resolve home directory, or with Permission denied from the image's working folder. These two lines give your user a home in flare-state. Local Inference Lab's own package hits the same error on this image when run that way.
  • --publisher-keys points Flare at this package's public key. Without it Flare stops with package signer is not a configured publisher.
  • Other flags.
    • --ipc=host and seccomp=unconfined are what we ran with. The latter lets Flare use io_uring for disk-backed Engram inside Docker.
    • --embedding-table-resident-scales and --kv-cache-limit memory are Flare's defaults, spelled out.
  • Engram in RAM. With about 256 GiB of host RAM, replace the three --embedding-table-* flags with --embedding-table-storage ram.
  • Model name. /v1/models reports the export folder name, DeepSeek-V4.1-Flash-1017-warmind-orth4c, as the model's root. Requests use DeepSeek-V4.1-Flash.
  • Downloading through Flare. Flare can fetch from the Hub itself (run jcartu/WARMIND-1017-FLARE-gb202 --revision gb202 --filename model.safetensors), but we did not take that path to the end. Use hf download.
    • Speed: in a 3-minute trial it ran at about 15 MB/s here.
    • Key: it would still need --publisher-keys pointing at the downloaded development-publisher.json.
    • Warning: it will warn that the package names local-development rather than this repository as its source.

Send a request

The OpenAI-compatible API serves the model as DeepSeek-V4.1-Flash:

curl --fail http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"DeepSeek-V4.1-Flash","messages":[{"role":"user","content":"Explain how a hash table works."}],"max_tokens":1024}'
  • Sampling. Defaults are temperature 1.0, top-p 1.0, top-k off, min-p 0, repetition penalty 1.0, and presence and frequency penalties 0. Only repetition_penalty (1.0 to 1.1) and seed are request-controlled; any other sampling value you send is clamped to its default.
  • Thinking. Turn it on or off with "chat_template_kwargs": {"enable_thinking": true}.
  • Images. The export includes the vision path (--image-concurrency 4, four image-bearing requests at a time, like gb202), and /v1/models lists image input. Tested on the CSF v1.1 sibling package (WARMIND-1017-FLARE-CSF-gb202, identical vision tower and edited text layers) across all 9,000 items of POPE MSCOCO val2014 (enable_thinking: false): 88.17 % accuracy (88.08 % F1, 91.43 / 87.20 / 85.87 %, 49.28 % yes-ratio, 0/9,000 errors), vs 81.67 % with enable_thinking: true (300 items).

Tips from running it

  • Put the Engram files on XFS or ext4, not btrfs. btrfs routes these small direct reads through extra kernel threads. With the same package copied to both filesystems and the same flags (measured on our sibling export of these weights), a fresh 106k-token prompt took about 10.5 s from btrfs RAID0 (two PCIe 5.0 drives) and about 8 s from XFS on a single PCIe 4.0 drive.
  • Engram read-ahead (unmerged). randomvariable's (Naadir Jeewa) Flare branch topic/engramcache @ c2606aae reads 512-byte Engram rows instead of 4 KiB blocks and prefetches the rows the next step will need. It changes the serving binary, not the package. On our box, serving our CSF export of these weights at Flare 78b18b1c from XFS, a cold 250k-token random-word prompt took 13.6 s with the branch's binary and 22.3 s with the release-18b16e1-main-20261104 binary (two runs each); decode throughput was unchanged. The branch is unmerged and not in any Flare release; we ran only the arithmetic, tool-call and thinking checks on it.
  • A bigger row cache helps text the model has seen before. We run --embedding-table-cache-size 64GiB in production. It does not speed up the first read of brand-new text.
  • Adaptive-verification alpha 1. LiL's card uses --mtp-adaptive-verification-alpha 0.6. On our box (no PCIe switch, each GPU on its own host bridge), 0.6 was about 5% slower than 1 at 16 concurrent requests (1,425 vs 1,510 tok/s, measured with our November 2 export of these weights on the same image), so the command above uses 1. Try both on switched systems.
  • Throughput scheduler for concurrent work. It is the right choice for agent swarms and parallel requests. Keep --max-concurrency 32 unless you need memory back.

Measured

Every number below was measured on the files downloaded from this repository and served with the command above (unattended trust flags, a different host port), on release-972e28e-main-20261107 (downloaded from the staging branch that became gb202). Rig: 4x RTX PRO 6000 Blackwell Workstation Edition, Xeon w9-3495X.

Functional API checks (arithmetic chat, strict JSON schema, 32 tool-call round trips at 4-way concurrency; thinking on) 34/34 correct
Requests with top_p 0.95 All returned 200 (3 at C1, 8 at C8, 1 streamed)
Single-user coding decode (Festr's test, 25 runs) 527 tok/s (501–549 across runs)
llm_decode_bench, 30 s cells, context 0 C1 297, C8 1,090, C16 1,604, C32 2,113 tok/s
KV capacity 16,864,512 tokens
Our 100-prompt refusal battery (keyword screen / true refusals, thinking off, 512 tokens) 0/100 (0 true)
SQuAD 2.0 unanswerable invented answers (Round 2, 500 unanswerable, plain / strict prompt) 28.4% (+4.6 vs stock, βˆ’9.0 vs v1.0) / 25.6% (+1.6 vs stock, CI -0.2 to +3.6)
TruthfulQA MC1 (790 items) 84.7% (+2.2 pts vs v1.0; stock 87.1%)

The quality and refusal evaluations of the WARMIND edit itself are in EVALUATION-1017.md: MMLU within noise of stock, a KL diagnostic, and blind-labelled refusals. They were run on the vLLM CSF container, jcartu/DeepSeek-V4.1-Flash-1017-abliterated-lossless-CSF, whose 132 edited tensors are byte-identical to the ones exported here.

How it was built

  1. Source. The 82 weight and runtime files of jcartu/WARMIND-1017 were hashed and matched against that repository's SHA256SUMS.

  2. Export. Flare's Ignition exported the package at Flare commit 972e28e4 (November 5), the commit behind the release-972e28e-main-20261107 image and the one LiL's gb202 revision 6d51b296 was prepared from. No pull requests were applied.

    • #154 by Brandon M. Music, which lets a 32-request export fit in 96 GB of VRAM, and #208, the top-p fix, are merged at this commit.
    • FlashInfer 6aaa9b47 is the revision LiL's gb202 revision 6d51b296 was prepared with. Flare's own pin at this commit, 1f8ed8a3, is five commits earlier; the five later commits are the paged-attention masks (#73, #74), RoCE and io_uring changes (#75, #76) and the MoE autotuning cases (#78).

    The full command, commits and settings are in export.json. Autotuning ran at Ignition's default effort.

    ignition export <WARMIND-1017> --gpus 0,1,2,3 \
      --max-concurrency 32 --prefill-chunk-size 4096 --image-concurrency 4 \
      --mtp-draft-tokens 7 --speculative-token-capacity 128 --speculative-graph-budget 64 \
      --mla-compute-mode bf16 --draft-head-quantization nvfp4 --markov-head-quantization fp8 \
      --vocabulary-parallel-sampling --expert-scale-compression native \
      --collective-tuning runtime \
      --collective-ladder ignition/preparation/ladders/gb202_pcie_tp4_clustered_full.json \
      --compile-workers 8
    
  3. Pre-upload checks. Before upload, the package was served back to back with our production build at the time (our CSF export at Flare 78b18b1c), with the same flags, using local builds of Flare with the Engram read-ahead branch above (c2606aae, and c2606aae merged onto 972e28e4 for this package). It passed the harness's arithmetic (sent with top_p 0.95), tool-call and thinking checks. It ran at 307, 1,128, 1,642 and 2,183 tok/s at C1, C8, C16 and C32, against 306, 1,199, 1,628 and 2,258 for the production CSF build, and held 16,835,456 KV tokens. Cold-prefill times from that run are not reported here, because this package was read from Btrfs and the production package from XFS.

  4. Release checks. After upload, it was downloaded back from this repository with the command above, and every package file was matched against SHA256SUMS. The Measured numbers were taken on that download.

Files

File What it is
model.safetensors Package manifest and signature. Point Flare at this file. The Hub's parameter count describes this manifest, not the model
deployments/gb202-tp4/ Compiled kernels, captured graphs and initial state for each of the four ranks
weights/ Model weights per rank, plus the Engram tables and their scales
development-publisher.json Public key that verifies the package signature
export.json Exact export command, Flare commits, source revision and signer
SHA256SUMS SHA-256 of every file in the repository except itself and the Hub's .gitattributes
LICENSE, LICENSES/, NOTICE License texts and notices

Keep the files together under their original names.

Lineage and credits

  • Base weights: DeepSeek-AI, deepseek-ai/DeepSeek-V4.1-Flash-1017 @ 1bae0e1088993b9b82de08f59de5be5a34e42170, MIT.
  • Method: grimjim, Norm-Preserving Biprojected Abliteration.
  • Refusal directions and the original s3.0 recipe: distributedcognition/DeepSeek-V4.1-Flash-abliterated (norm-preserving biprojection, scale 3.0, layers 4 to 36), built on the previous DeepSeek-V4.1-Flash.
  • WARMIND-1017: operator re-implementation, validation, 1017 re-abliteration and evaluation by jcartu (WARMIND-1017).
  • Flare: the runtime and the export toolchain, by NVIDIA and the Flare community contributors, led by Luke Alonso of Local Inference Lab.
  • Local Inference Lab: the gb202 runtime layout and the release image this package targets, from Luke Alonso and Festr. Their official DeepSeek-V4.1-Flash-1017-FLARE package is the reference this one follows.
  • Martin Vit (voipmonitor): the clustered PCIe all-reduce collectives and ladders this build is tuned with (Flare #174, FlashInfer #70) and the shared startup tuning storage (Flare #197, FlashInfer #72) that gives it back its KV capacity. He also wrote FlashInfer #78 (MoE autotuning).
  • Mark Yong: the attention masking fixes in FlashInfer #73 and #74 and the fused-MoE fix in #71.
  • Luke Alonso: the attention operand masking in FlashInfer 1f8ed8a3.
  • Jason Cook (original-el8): the DeepSeek V4.1 top-p fix in Flare #208.
  • joninco: the export settings that bring a self-exported package to gb202 speed.
  • Brandon M. Music: the export memory fix in #154.
  • FlashInfer and the b12x authors: the kernels Flare compiles into this package (FlashInfer 6aaa9b47).
  • This package: export, qualification and these documents by jcartu on a 4x RTX PRO 6000 build machine. Packaging review by Claude Fable.
  • nero (JMNI-Labs): the DGX Spark (GB10) build of WARMIND (updated to v1.1).
  • pangfather, rascazzione, el8, ufear: community evaluation and discussion motivating the v1.1 epistemic-abstention subspace separation.

License

  • Weights. The DeepSeek-V4.1-Flash-1017 weights are under the MIT License (LICENSE, Copyright (c) 2023 DeepSeek). The WARMIND edit adds no terms.
  • Compiled runtime components. The package's compiled CUDA kernels and captured graphs were produced by Flare, under the Apache License 2.0 (LICENSES/Flare-Apache-2.0.txt, Copyright NVIDIA CORPORATION & AFFILIATES and Flare community contributors).
  • Local Inference Lab. No Local Inference Lab license applies here. This is not a Local Inference Lab release, and nothing from LiL's CSF containers is in it.
    • What is shared: 36 of the package's weight files are byte-identical to files in LiL's MIT-licensed gb202 package (revisions cefb84d2, e659bf6c, b3c7e7c6, 32385269 and 6d51b296): the four Engram files and, for each rank, the weight files for the untouched layers 0–3 and 37–39 plus misc, 274.6 GB in all. (LICENSE is DeepSeek's MIT text in both repositories.)
    • Why: Ignition serializes the same DeepSeek bytes the same way, and its content-addressed cache reused files it already held after this export had verified its own source bytes.
    • What is new: the 132 rank files for layers 4–36 are the WARMIND edit (unchanged from the 78b18b1c build). The kernels, the graphs and the manifest were produced by this export.
    • Check it yourself: compare the weight files' SHA-256s on the Hub. The weight files that differ are exactly the WARMIND edit; the kernels, graphs and manifest differ because they are this export's.
Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
48.4M params
Tensor type
U8
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for jcartu/WARMIND-1017-FLARE-gb202

Quantized
(2)
this model