qwen-image-2.1-p150

Qwen-Image-2.1 port on one Tenstorrent Blackhole p150a. Weights: Qwen/Qwen-Image-2.1 · Port: code/models/experimental/qwen_image_2_1 in this repo

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart (Python)

Prerequisite: a Python environment with a built tt-metal / ttnn at tt-metal 8b98410e730 with patches/tt-metal-eth-dispatch.patch applied.

hf download changh95/qwen-image-2.1-p150 --exclude "image/*" --local-dir qwen-image-2.1-p150 && cd qwen-image-2.1-p150
pip install -e code/                  # inside an environment that has tt-metal (ttnn); the Python API only
pip install -e "code/[server,test]"   # also the HTTP server (fastapi, uvicorn, pydantic) and the tests (pytest)

from qwen_image_p150 import QwenImage

with QwenImage.from_pretrained(device_id=0) as model:
    out = model("White furry llama with black sunglasses, smiling and happy, jumping", seed=42, num_steps=40)
    out.image.save("llama.png")   # PIL.Image, 1024x1024 RGB
    print(out.timings)            # seconds per stage

from_pretrained() warms up the model. The first call is as fast as the next calls (about 15 s for 40 steps). To prepare other step counts, use warmup_variants=[{"num_steps": [40, 25]}] or model.warmup(num_steps=25). Keep prompts at 256 tokens or fewer.

• Relative paths (for example media/edit_ref_llama.jpg in code/PYTHON.md) are correct when the current directory is the model repo root (qwen-image-2.1-p150). From a different directory, use the full path.
• from_pretrained downloads the pinned weights Qwen/Qwen-Image-2.1 (revision 790c926, approximately 33 GB) to the HF cache.
• It opens the chip with ETH dispatch, 1 command queue and the 12×10 grid. Then it loads the text encoder, the DiT and the VAE decoder, captures the metal traces and makes one warm-up image.
• The first load on a new chip compiles the kernels and takes approximately 10 minutes. A load with a warm kernel cache takes approximately 2 minutes.
• Without the ETH-dispatch patch, the model uses Tensix dispatch and shows a RuntimeWarning. The images are the same, but the calls are slower.
• After the load, one image takes approximately 15 s.
• The with block releases the traces and the chip at the end. You can also call model.close().
• The output is identical to the HTTP /predict image for the same prompt, seed and steps (tested bit for bit).

	name	meaning
input	prompt	str, or a list of str (one image for each prompt). Prompts up to 256 tokens use the pre-compiled path.
input	images	editing only: 1–4 condition images as PIL.Image, file path, PNG/JPEG bytes, numpy array or torch tensor. Needs from_pretrained(editing=True).
option	seed	seed of the initial noise (default 42). For a list of prompts: one int or one seed for each prompt.
option	num_steps	Euler steps, 1–100 (default 40). num_inference_steps is an alias.
option	return_latents	also return the final latents (default False).
option	from_pretrained(weights=None, device_id=0, device=None, dispatch="auto", editing=False, warmup=True, download=True)	weights (HF repo id or local snapshot), chip, an open device to share, "eth" / "tensix" dispatch, editing encoders, warm-up image, offline mode
output	image	PIL.Image RGB, composited over white (the server image). 1024×1024 for text-to-image.
output	image_rgba, numpy(rgba=False)	PIL.Image RGBA (the 4-channel VAE output); uint8 array [H, W, 3] or [H, W, 4]
output	timings	seconds: encode_s, prefix_s (0 for a resident prompt), denoise_s (40 steps and VAE decode in one trace), decode_s, total_s, call_s
output	images, save(path), size, latents	[image] (diffusers convention), write to a file, (width, height), latents when requested

Throughput helpers:

• model(["prompt 1", "prompt 2"], seed=[1, 2]) and model.generate_many(prompts) make one image for each prompt, in order. Each result is identical to a single call.
• The 4 most recent prompts stay on the device. A repeated prompt skips the text encoder and the text-prefix pass.
• from_pretrained(device=...) uses a ttnn device that you opened. Then close() does not close that device.
• Calls are serialised with a lock, so you can share one model object between threads.

code/PYTHON.md gives the full API reference and an editing example. code/examples/quickstart.py runs the snippet and writes llama.png and llama.json (the timings).

Serving (HTTP)

tt-model pull  changh95/qwen-image-2.1-p150 --with-weights
tt-model serve changh95/qwen-image-2.1-p150      # or, with tt-cli: tt serve changh95/qwen-image-2.1-p150
curl -s localhost:20000/predict -H 'Content-Type: application/json' \
  -d '{"prompt": "White furry llama with black sunglasses, smiling and happy, jumping", "seed": 42, "num_steps": 40}' \
  | python3 -c 'import sys, json, base64; d = json.load(sys.stdin); open("llama.png", "wb").write(base64.b64decode(d["image"])); print(d["timing_ms"])'
tt model stop changh95/qwen-image-2.1-p150       # tt-cli

• Weights Qwen/Qwen-Image-2.1 go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port). The first start compiles kernels and captures the metal traces (several minutes). The server is ready when the log says Application startup complete.
• POST /predict: prompt (string), optional seed (42), num_steps (40, 1..100), return_rgba (false), and for editing images (1-4 base64 PNG/JPEG condition images).
• The response has image (base64 PNG, RGB composited over white), image_rgba when requested, and timing_ms. Text-to-image gives 1024x1024. Editing gives approximately 1 MP with the aspect ratio of the last condition image.
• GET /health, GET /info (device, worker grid, precision policy), GET /v1/models (stub).

Demo

prompt: "White furry llama with black sunglasses, smiling and happy, jumping" (seed 42, 40 steps, 1024x1024)
p150a output (media/sample.png)

prompt: 'Character 1 - "White furry llama with sunglasses", Character 2 - "Cat with glasses wearing a hoodie". These two characters are jumping, hooraying together. Happy' (seed 42, 40 steps; condition images media/edit_ref_llama.jpg, media/edit_ref_gongnyang.jpg)
condition images:
p150a output (media/edit_demo_p150.png)

prompt: 'Character 1 - "White furry llama with sunglasses", Character 2 - "Cat with glasses wearing a hoodie". A dark medieval journey scene painted in one consistent illustration style: the two characters travel along a muddy dirt road through a stormy medieval countryside, rolling hills, a stone castle on a hill and a small village in the distance, heavy rain clouds and lightning. Character 1 walks on all four feet like a pack animal, overloaded with heavy sacks, suitcases and baggage strapped to its back, crying with tears, exhausted and desperate. Character 2 keeps exactly the same face as in its reference image: the same soft round grey tabby cat face, small pink nose, and the same big round black-rimmed glasses, clearly recognizable, no helmet, the green hood of the hoodie resting over the armor collar. Only the expression changes: angry and commanding, brows frowning, eyes narrowed, mouth open shouting an order at Character 1, head turned toward Character 1. Character 2 is an Electric Knight in full plate armor: electric-blue lightning arcs crackle over the armor, the sword blade is wreathed in blue lightning, glowing blue runes pulse on the shield, and the blue glow lights the wet ground, while the face stays naturally lit and recognizable. Character 2 walks behind Character 1, drives it like a slave, points forward with the lightning sword in one hand, holds the shield in the other, and a rope leash runs from the knight's belt to the harness of Character 1. Dramatic low light, no white background.' (seed 42, 40 steps; same condition images)
p150a output (media/edit_demo2_p150.png)

Demo & Performances

Text-to-image, demo prompt, 1024x1024, 40 steps, seed 42, batch 1, warm, traced (2026-10-05, code commit 67afcdc). Device configuration for all numbers in this section: dispatch on ETH cores, 1 command queue, 12×10 compute grid (120 Tensix cores). This is the configuration of one p150 card. The runs used two chips of a Blackhole Galaxy, with the grid capped at 12×10. Chip 3 is the independent verification run. Chip 5 is the developer run. The sustained clock is different on each chip.

Metric	Chip 3 (verification)	Chip 5
End-to-end generate(), resident prompt (bench_baseline --repeat 3, warm runs)	14.92–15.04 s	15.31–15.61 s
End-to-end generate(), new prompt	14.71–15.11 s	15.42–15.58 s
DiT denoising step (one step trace, replayed 40 times)	352.8 ms/step (AICLK 1131–1137 MHz)	371.6–373.0 ms/step (AICLK 1056–1062 MHz)
40 denoising steps + VAE decode + uint8 post-processing (one trace)	14.51 s	15.21–15.26 s
Text encoder · text-prefix K/V (new prompt only)	64.5 ms · 79.1 ms	64.5 ms · 77.6 ms
Python model() call (qwen_image_p150 API, demo prompt)	14.67 s median · 14.52 s min (6 calls, the first with a new prompt)	15.32–15.38 s (3 calls)
HTTP POST /predict wall time, resident prompt · new prompt	14.92 s median · 14.93–14.99 s	15.45–15.73 s · 15.49–15.71 s
First call after from_pretrained() · steady median (bench_first_call --variant t2i)	14.710 s · 14.710 s	15.516 s · 15.496 s

• On each chip, the Python call and the HTTP request take the same time as the bench run. The API and the server add no time.
• The bench image, the Python image and the HTTP image are bit-identical (SHA-256 10459e67…). They are also identical to the reference image of the accuracy gate. The accuracy gate results did not change.
• The VAE decode (1024x1024 RGBA, traced) takes 381 ms, and the H2D · D2H transfers per image are 512 KB noise up · 4 MB uint8 RGBA down (1.6–1.9 ms). These values are from the 2026-10-03 run in the same configuration. The code for these stages did not change.
• The 2026-10-03 run in the same configuration gave 15.16 s median end to end and 365.9–369.2 ms/step. The differences follow the sustained clock of each chip.

RTX 5090 reference measurements (2026-09-22) are unchanged. They use diffusers bf16 in eager PyTorch 2.11 on the same host, with model CPU offload, because the DiT and the text encoder do not fit together in 32 GB. Full table: GPU_COMPARISON.md.

Stage	RTX 5090 (bf16, offload)	This build (chips 3 and 5)	Result
End to end per image, warm	28.8–33.0 s	14.92–15.61 s	Blackhole 1.84–2.21× faster
One denoising step, sustained	377–387 ms	352.8–373.0 ms	Blackhole 1.01–1.10× faster
Text encoding per new prompt	2.4–4.2 s	0.142–0.144 s (TE + text-prefix K/V)	Blackhole 16.7–29.6× faster

Most of the end-to-end lead comes from the CPU offload on the GPU. The p150a keeps all weights resident on the chip. On the denoising step alone, the two are almost equal.

Caveats

• Does not scale to multiple p150a in a mesh configuration. The measured text-to-image build uses a 12x10 compute grid. It gets this grid because dispatch moves from Tensix cores to ETH cores. Thus this build assumes that the user does not need chip-to-chip ethernet communication. (SCALING.md is about batch size on one card, not about multiple chips.)
• Default device configuration of the Python API, the server code and demo/demo.py: ETH dispatch, 1 command queue, 12×10 = 120 cores, text-to-image. Editing (editing=True or QWEN_IMAGE_EDITING=1) is an opt-in. It uses Tensix dispatch, because ETH dispatch hangs when the editing encoders are loaded. On a p150, Tensix dispatch gives an 11×10 grid. The editing path was not measured again for this build. The container image (2026-09-22) serves with editing on and Tensix dispatch.
• Text-to-image and image editing with 1-2 condition images (up to 4 accepted, ~2.3 GB of prefix K/V per 1 MP image). Batch 1, 40 steps by default, no classifier-free guidance (the model is meant to be sampled without it).
• Precision: text encoder weights bfp8_b; DiT weights bf16. Text-to-image steps use LoFi matmuls. Editing steps use HiFi2 matmuls (QWEN_EDIT_MM_FID), because LoFi lost the identity of a character in a two-image edit. Attention uses HiFi2. Layer norms use HiFi4 with fp32 accumulation. OPT_REPORT.md gives the changes of the optimized build.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model readiness probe does not 404.

Licensing

• Weights: Qwen/Qwen-Image-2.1, Qwen Research License (research use; see the weights repo's LICENSE). Fetched from upstream, not redistributed here.
• Port and serving code (code/): Apache-2.0. patches/tt-metal-eth-dispatch.patch is a patch to tt-metal (Apache-2.0).

Provenance

These are the exact sources the container image was built from. code/ has since been updated (2026-10-03 optimized build, see OPT_REPORT.md; 2026-10-04 Python package qwen_image_p150; 2026-10-05 p150 device configuration as the default) and is newer than the image. tt-model serve runs the image's code until the image is rebuilt. tt-model.yaml still describes the image:

component	built from
tt-metal	a local checkout — commit not published
code/ digest (image)	f8615edc0fc5c021 (sha256, first 16 hex digits; the current code/ differs)
built	2026-09-22T11:38:56+00:00 by tt-model 0.1.0

The optimized build was measured with tt-metal 8b98410e730 and patches/tt-metal-eth-dispatch.patch. VERIFICATION_2026-10-03.md gives the independent check of the numbers above (section "p150 ETH-dispatch compliance (2026-10-05)").