DaisyRose-27B β€” NInfer NVFP4

An unofficial conversion of RadixArk/DaisyRose-27B-NVFP4 into a single NInfer v3 artifact. It runs DaisyRose-27B on one 24 GB Blackwell (SM120) GPU with a long context and MTP speculative decoding.

All credit for the model goes to RadixArk (DaisyRose-27B, an RL-trained agentic fine-tune) and the Qwen team (Qwen3.8-27B). This repository only changes the file format. See the original model card for training details, intended use and benchmarks.

Files

File Purpose
daisyrose_27b_nvfp4.ninfer The NInfer v3 artifact: text, vision and MTP
daisyrose_27b_nvfp4.ninfer.conversion.json Per-tensor conversion report
daisyrose_recipe.py Conversion recipe
converter.patch Two small converter fixes needed to read this ModelOpt checkpoint
SHA256SUMS Checksums
LICENSE Apache-2.0, unchanged from the original

Precision

Part Format
MLP, all 64 layers NVFP4, imported as shipped
Full-attention and Gated DeltaNet projections FP8 E4M3, imported as shipped (codes bit-exact)
lm_head FP8 row-wise, quantized from the original BF16 weights
Token embedding FP8 row-wise, from the shipped BF16 weights
Vision encoder, MTP head Q8/Q6/Q5/Q4 group formats, NInfer defaults
Norms, GDN conv/a/b, dt_bias, A_log BF16/FP32, unchanged

Weights occupy about 18.5 GiB on the GPU.

What differs from the source checkpoint

  1. FP8 scales. The source stores one FP32 scale per FP8 tensor, while NInfer expects one BF16 scale per row. The single scale is repeated on every row. The FP8 codes are unchanged. The only difference is the BF16 rounding of the scale, at most about 0.2% per tensor.
  2. lm_head. NInfer has no NVFP4 kernel for the vocabulary-width output head, so the head was re-quantized to FP8 from the original BF16 lm_head.weight of RadixArk/DaisyRose-27B. This matches the precision of the official Qwen3.8-27B NInfer NVFP4 recipe.

The runtime is stock NInfer. The patches are only needed if you want to re-run the conversion.

Requirements

  • NVIDIA Blackwell GPU (SM120: RTX 5090 / RTX PRO 6000 / DGX Spark class), 24 GB or more
  • NInfer at commit 4de9e42599b3f1fb893cac64a063e6f7d2581515
git clone https://github.com/Neroued/ninfer.git && cd ninfer
git checkout 4de9e42599b3f1fb893cac64a063e6f7d2581515
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120a
cmake --build build -j --target ninfer-serve

Download and serve

hf download artisticalien17/DaisyRose-27B-NInfer-NVFP4 --local-dir ./daisyrose-ninfer
cd daisyrose-ninfer && sha256sum -c SHA256SUMS && cd ..

./build/apps/ninfer-serve ./daisyrose-ninfer/daisyrose_27b_nvfp4.ninfer \
  --host 127.0.0.1 --port 8081 --model-id daisyrose \
  --max-context 180224 --kv-capacity 180224 --kv-dtype nvfp4 \
  --max-concurrency 4 --pending-timeout-ms 600000 --device-state-slots 0 \
  --prefill-chunk 1024 --spec mtp --draft-tokens 3 \
  --preserve-thinking --default-max-tokens 32768 --default-thinking-budget 16384
  • This serves an OpenAI-compatible API at http://127.0.0.1:8081/v1, with model set to daisyrose.
  • The KV pool (180,224 tokens) is shared by the 4 lanes. One chat on its own can use all of it.
  • The thinking budget is set server-wide. NInfer ignores per-request thinking budgets.
  • Add --vision for image input. It needs extra VRAM, so reduce the context if you use it.

Measured on a single 24 GB SM120 GPU

Settings: NVFP4 KV cache, MTP with 3 draft tokens, 4 lanes.

Test Result
Largest KV pool that starts (4 lanes) 180,224 tokens (196,608 fails)
Needle retrieval, 173,801-token prompt Found; peak VRAM 23.5 GB
Decode, code: short / 100k context 86 / 76 tok/s (MTP acceptance about 79%)
Decode, prose: short / 100k context 58 / 50 tok/s (MTP acceptance about 41%)
Prefill at 100k context about 2,240 tok/s

Hard coding / agent benchmark

This is a private benchmark: hidden pytest suites run in an offline Docker sandbox. Settings: temperature 0.6, top_p 0.95, server-side thinking budgets of 16,384 tokens (coding) and 6,144 tokens per agent turn. The baseline is a Qwen3.8-27B NVFP4 NInfer build with DFlash2, run with the same harness.

Task DaisyRose (this repo) Qwen3.8-27B NInfer baseline
SQL engine (joins, GROUP BY, 3-valued NULL) 18/30 4/30
Cron engine (L, #n, DOM/DOW OR rule) 41/43 30/43
Segmented WAL + crash recovery 24/24 24/24
HTTP/1.1 parser, anti-smuggling 37/56 55/56
npm semver + backtracking resolver 85/89 87/89
MVCC store (SI + serializable, GC) 24/24 23/24
Coding total 229/266 223/266
Agentic: payments repo (5 bugs + feature) 46/47 47/47
Agentic: SQLite repo (optimistic locking) 25/25 25/25

These are single runs on one private suite, so treat them as indicative, not definitive.

Notes

  • Thinking: DaisyRose thinks by default and can produce very long reasoning. Set --default-thinking-budget to suit your use.
  • Refusals: this is a format conversion only. The model's alignment and refusal behaviour are unchanged.
  • Not official: this is an unofficial community conversion, not affiliated with RadixArk, the Qwen team or the NInfer authors. Report issues with this conversion here, not upstream.
Downloads last month
2
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for artisticalien17/DaisyRose-27B-NInfer-NVFP4

Base model

Qwen/Qwen3.8-27B
Quantized
(27)
this model