iamkaikai

/

MiniMax-H3-Single-Frame-VAE-500K

Like 54

Hi, and thanks for releasing this — the single-frame decoder solves a real problem. Decoding one
temporal slice with the official H3 VAE falls apart (I measured 20.58 dB with a heavy orange cast
against 34.49 dB with yours, same latent), so this fills a gap for anyone pulling stills out of H3.

I wanted to use it in ComfyUI and ran into the fact that the stock VAELoader can't load the file.
That's not a defect on your side — it's a contract mismatch. Your load_decoder.py loads a complete
VAE and replaces two submodules, which is the right design. ComfyUI instead builds a VAE from the
file, and its detection requires an encoder key:

elif "decoder.transformer_blocks.0.scale1" in sd and "encoder.down.5.block.0.conv1.weight" in sd:

A decoder-only file misses that, so ComfyUI logs No VAE weights detected, leaves
first_stage_model = None, and falls back to Stable Diffusion geometry (4 latent channels, 8x upscale
instead of 24 and 32x). The output is blocks, which looks like a broken checkpoint but isn't.

So I did two things:

1. Converted weights to ComfyUI's key layout, grafting encoder.*, quant_conv,
   latents_mean/std and mask_token from the official H3 VAE — the same graft your script performs,
   and sound because you froze the encoder during training.
2. Loader nodes for ComfyUI, including one that reads your original file unmodified and does
   the translation in memory, so nobody has to trust my converted copy.

Two transforms turned out not to be derivable from the key names, and getting either wrong gives
wrong output with no load error. I settled both by correlating against the official VAE rather than
guessing:

• to_qkv is interleaved per head — (32 heads, 3, 64 dim_head), not stacked [q;k;v]. It matches
  qkv.view(B, S, -1, 3 * dim_head) in comfy/ldm/minimax/vae.py. Correlation 0.998.
• ff.w1's halves are swapped relative to diffusers, because ComfyUI reads gate, x = w1(x).chunk(2).
  Correlation: straight −0.013, swapped +2.00.

Validation: same latent, raw decoder(post_quant_conv(z)) on both sides — your diffusers path with
the original file against the converted file in ComfyUI — gives 72.92 dB, i.e. fp16 noise, with
0 missing and 0 unexpected keys.

One finding that may interest you regardless of ComfyUI. ComfyUI tiles internally with a default
tile_size of 256, and that value is bad for your decoder. Measured on a 1056x640 image, with "seam"
meaning edge energy at tile boundaries relative to the image average (1.0 = invisible):

512 lines up with your curriculum — 475k of the 500k images were at ≤512px — so 512-wide tiles keep
each piece inside the regime with training mass. Larger tiles push past it; smaller ones multiply
seams. Disabling tiling is worse above ~768px: a 32px block pattern appears (block ratio 2.94 at 1024,
3.45 at 1536, against ~1.5 with tiling). This may be worth a line in your card even for diffusers users
who decode large images in one pass.

The offer: everything is yours if you want it. Copy the converted files into this repo, take the
node code and publish an official one — no attribution needed, no strings. If you do, I'll delete my
copy, since it only exists because there was nowhere else to put the ComfyUI-side files. A single
official source is better for everyone.

Repo (a duplicate of yours, clearly marked as such): https://huggingface.co/Alissonerdx/MiniMax-H3-Single-Frame-VAE-500K-Comfy
Nodes: https://github.com/alisson-anjos/ComfyUI-BFSNodes (v1.24.0)

Happy to open a PR instead, or to re-run any of these measurements if you want them checked