Instructions to use artisticalien17/DaisyRose-27B-NInfer-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NInfer
How to use artisticalien17/DaisyRose-27B-NInfer-NVFP4 with NInfer:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
DaisyRose-27B β NInfer NVFP4
An unofficial conversion of RadixArk/DaisyRose-27B-NVFP4 into a single NInfer v3 artifact. It runs DaisyRose-27B on one 24 GB Blackwell (SM120) GPU with a long context and MTP speculative decoding.
All credit for the model goes to RadixArk (DaisyRose-27B, an RL-trained agentic fine-tune) and the Qwen team (Qwen3.8-27B). This repository only changes the file format. See the original model card for training details, intended use and benchmarks.
Files
| File | Purpose |
|---|---|
daisyrose_27b_nvfp4.ninfer |
The NInfer v3 artifact: text, vision and MTP |
daisyrose_27b_nvfp4.ninfer.conversion.json |
Per-tensor conversion report |
daisyrose_recipe.py |
Conversion recipe |
converter.patch |
Two small converter fixes needed to read this ModelOpt checkpoint |
SHA256SUMS |
Checksums |
LICENSE |
Apache-2.0, unchanged from the original |
Precision
| Part | Format |
|---|---|
| MLP, all 64 layers | NVFP4, imported as shipped |
| Full-attention and Gated DeltaNet projections | FP8 E4M3, imported as shipped (codes bit-exact) |
| lm_head | FP8 row-wise, quantized from the original BF16 weights |
| Token embedding | FP8 row-wise, from the shipped BF16 weights |
| Vision encoder, MTP head | Q8/Q6/Q5/Q4 group formats, NInfer defaults |
| Norms, GDN conv/a/b, dt_bias, A_log | BF16/FP32, unchanged |
Weights occupy about 18.5 GiB on the GPU.
What differs from the source checkpoint
- FP8 scales. The source stores one FP32 scale per FP8 tensor, while NInfer expects one BF16 scale per row. The single scale is repeated on every row. The FP8 codes are unchanged. The only difference is the BF16 rounding of the scale, at most about 0.2% per tensor.
- lm_head. NInfer has no NVFP4 kernel for the vocabulary-width output head, so the head was
re-quantized to FP8 from the original BF16
lm_head.weightofRadixArk/DaisyRose-27B. This matches the precision of the official Qwen3.8-27B NInfer NVFP4 recipe.
The runtime is stock NInfer. The patches are only needed if you want to re-run the conversion.
Requirements
- NVIDIA Blackwell GPU (SM120: RTX 5090 / RTX PRO 6000 / DGX Spark class), 24 GB or more
- NInfer at commit
4de9e42599b3f1fb893cac64a063e6f7d2581515
git clone https://github.com/Neroued/ninfer.git && cd ninfer
git checkout 4de9e42599b3f1fb893cac64a063e6f7d2581515
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120a
cmake --build build -j --target ninfer-serve
Download and serve
hf download artisticalien17/DaisyRose-27B-NInfer-NVFP4 --local-dir ./daisyrose-ninfer
cd daisyrose-ninfer && sha256sum -c SHA256SUMS && cd ..
./build/apps/ninfer-serve ./daisyrose-ninfer/daisyrose_27b_nvfp4.ninfer \
--host 127.0.0.1 --port 8081 --model-id daisyrose \
--max-context 180224 --kv-capacity 180224 --kv-dtype nvfp4 \
--max-concurrency 4 --pending-timeout-ms 600000 --device-state-slots 0 \
--prefill-chunk 1024 --spec mtp --draft-tokens 3 \
--preserve-thinking --default-max-tokens 32768 --default-thinking-budget 16384
- This serves an OpenAI-compatible API at
http://127.0.0.1:8081/v1, withmodelset todaisyrose. - The KV pool (180,224 tokens) is shared by the 4 lanes. One chat on its own can use all of it.
- The thinking budget is set server-wide. NInfer ignores per-request thinking budgets.
- Add
--visionfor image input. It needs extra VRAM, so reduce the context if you use it.
Measured on a single 24 GB SM120 GPU
Settings: NVFP4 KV cache, MTP with 3 draft tokens, 4 lanes.
| Test | Result |
|---|---|
| Largest KV pool that starts (4 lanes) | 180,224 tokens (196,608 fails) |
| Needle retrieval, 173,801-token prompt | Found; peak VRAM 23.5 GB |
| Decode, code: short / 100k context | 86 / 76 tok/s (MTP acceptance about 79%) |
| Decode, prose: short / 100k context | 58 / 50 tok/s (MTP acceptance about 41%) |
| Prefill at 100k context | about 2,240 tok/s |
Hard coding / agent benchmark
This is a private benchmark: hidden pytest suites run in an offline Docker sandbox. Settings: temperature 0.6, top_p 0.95, server-side thinking budgets of 16,384 tokens (coding) and 6,144 tokens per agent turn. The baseline is a Qwen3.8-27B NVFP4 NInfer build with DFlash2, run with the same harness.
| Task | DaisyRose (this repo) | Qwen3.8-27B NInfer baseline |
|---|---|---|
| SQL engine (joins, GROUP BY, 3-valued NULL) | 18/30 | 4/30 |
| Cron engine (L, #n, DOM/DOW OR rule) | 41/43 | 30/43 |
| Segmented WAL + crash recovery | 24/24 | 24/24 |
| HTTP/1.1 parser, anti-smuggling | 37/56 | 55/56 |
| npm semver + backtracking resolver | 85/89 | 87/89 |
| MVCC store (SI + serializable, GC) | 24/24 | 23/24 |
| Coding total | 229/266 | 223/266 |
| Agentic: payments repo (5 bugs + feature) | 46/47 | 47/47 |
| Agentic: SQLite repo (optimistic locking) | 25/25 | 25/25 |
These are single runs on one private suite, so treat them as indicative, not definitive.
Notes
- Thinking: DaisyRose thinks by default and can produce very long reasoning. Set
--default-thinking-budgetto suit your use. - Refusals: this is a format conversion only. The model's alignment and refusal behaviour are unchanged.
- Not official: this is an unofficial community conversion, not affiliated with RadixArk, the Qwen team or the NInfer authors. Report issues with this conversion here, not upstream.
- Downloads last month
- 2