mrfakename/novel-humanize-4b-ONNX

novel-humanize-4b ONNX

ONNX weights of gugujiji/novel-humanize-4b for Transformers.js and onnxruntime-web (WebGPU).

• onnx/model_q4f16.onnx: 4-bit weights (block size 32), fp16 activations and KV cache, about 2.3 GB, split into two external data files for the browser.
• Built with the onnxruntime-genai model builder (-p int4 -e webgpu, unfused QKV, shared int4 embedding / LM head). Rotary embedding is fused into GroupQueryAttention, so the graph takes input_ids, attention_mask and past_key_values.* (no position_ids).

Use the same system prompt, user wrapper and sampling as the original model card (temperature 1.0, top_p 0.95, top_k 20, presence penalty 0.3, thinking off).

Browser demo: https://huggingface.co/spaces/mrfakename/novel-humanize-4b-webgpu

License: CC BY-NC 4.0, same as the original weights. Non-commercial use only.