dima0000/Alpamayo2-Super-NVFP4-language-smoke

Alpamayo 2 Super NVFP4 experiment

Controlled evaluation status — 2026-09-23

BF16 reference complete; the requested quantized comparison remains incomplete. The saved reference contains 20 full-model rows on an NVIDIA A100-SXM4-80GB: five clips, seeds 42 and 43, and free/fixed-token modes. Every observed diffusion-noise hash matches the requested noise. Fixed-token replay reproduces the paired BF16 trajectory exactly (maximum delta zero).

BF16 reference, free generation (10 rows)	Measurement
Mean ADE	3.519291 m
Mean FDE	8.645417 m
Median rollout latency, A100	3.268258 s
Maximum allocated memory	68.930434 GiB
Maximum reserved memory	69.052734 GiB

The runner implements current native NVFP4, original BF16 K/V restoration, and BF16 K/V plus FP8 expert MLPs. It checks paired inputs/noise, exact BF16-token replay, module execution, native kernels, trajectory error, latency, and memory. Both candidate attempts produced zero measured rows; no precision winner or fresh RTX 5090 verification is available. A100 timing must not be interpreted as a same-GPU speed comparison with RTX 5090.

The first candidate attempt failed because ninja was missing from the virtual environment's PATH. The launcher now fixes that path and checks CUDA, nvcc, and ninja before model downloads. The replacement host failed that preflight because PyTorch could not initialize CUDA despite nvidia-smi detecting its RTX 5090. The final capacity check found no RTX 5090 stock in the probed CUDA 13.0/13.1 pools. All four pods created for this work are stopped. Conservative elapsed-time cost is approximately $0.92, below the authorized $1.35 cap; provider billing is not settled.

All 21 CPU tests pass. The BF16 collection, result-return, local Hub-upload, and failure-recovery paths were exercised. Successful quantized completion remains unvalidated. The packed checkpoint stays at 6300bd8fde2bf6d29a09a1ff56af11d57889219b; model weights are unchanged and the repository remains private.

See the roadmap, status and hashes, and current scripts. The preparation archive is unchanged: overlay the current loose scripts after extraction and apply shared_noise.patch to the pinned source. The Windows staging launcher expects the existing local layout ROOT/work/stage_controlled_evaluation.py with payload files in ROOT/outputs; its HF account authentication remains local. The archived BF16 manifest has a legacy default policy: current label; its phase, all result rows, and execution record identify BF16. The updated runner records the actual policies list.

Historical RTX 5090 experiment — 2026-09-21

The following sections describe the earlier native NVFP4 experiment and its historical figures; they are separate from the incomplete 2026-09-23 controlled evaluation.

Read the full test reflection and limitations before interpreting the scores. Native NVFP4 and memory fit were demonstrated on one RTX 5090. General driving-quality preservation or improvement was not established. The five-clip means are sensitive to one large improvement, two clips regress versus BF16, and the shared text/diffusion random-number stream means equal starting seeds did not guarantee identical initial diffusion noise. The cause of the 0103cc09 improvement remains unresolved.

Publication status: this repository contains licensed driving-data tensors and dataset-related benchmark artifacts, including in history. NVIDIA's dataset agreement separately restricts their redistribution and disclosure. It remains private while publication permission or a clean model-and-code release is resolved. The model-weight license does not grant rights to the separate driving dataset.

Target: one RTX 5090 with 32 GB VRAM. Total RunPod experiment budget: US$5.

The packed checkpoint is saved in the private repository dima0000/Alpamayo2-Super-NVFP4-language-smoke. Its successful checkpoint commit is 6300bd8fde2bf6d29a09a1ff56af11d57889219b. The full packed model now runs with native NVFP4 acceleration on an actual RTX 5090. All 448 packed layers use FlashInfer/CUTLASS, and the full-model CUDA trace confirms an SM120 block-scaled FP4 kernel. The paired five-clip test ran on one GPU:

Measurement	Packed fallback	Native NVFP4
Median model rollout	19.075 s	4.921 s
Peak PyTorch allocated memory	28.705 GiB	29.002 GiB
Peak PyTorch reserved memory	28.955 GiB	29.113 GiB
Mean ADE	1.963 m	1.588 m
Mean FDE	5.021 m	4.271 m

This is 3.876x faster than the packed dequantization fallback on the same RTX 5090. The tested model and workload fit the card without CPU offload. These measurements cover one trajectory, ten diffusion steps, and 4,580 input tokens; they do not establish fit at larger batches or contexts. A roughly five-second rollout still needs substantial speed work for a frequent closed-loop planning cycle.

Raw RTX 5090 reports are pinned to commit 12453f0bc3f77555c188a098b1e82327ff05e22c. rtx5090_results.json combines their raw rows, kernel trace, summaries and BF16 reference. The model weights are unchanged from the successful quantization checkpoint.

The RunPod GPU is stopped. Its console reports compute and storage not running, $0.00/hour and $2.74 credit remaining. The observed balance decrease across all attempts is approximately $2.28; our conservative time-based total is $3.43, below the $5 budget.

What was quantized

ModelOpt 0.43.0 quantized 448 language transformer projections, containing 31,205,621,760 weights, to NVFP4. Vision, the action expert, embeddings, the output head, and KV cache retain BF16. The seven safetensors shards total 26,770,262,024 bytes. This is an experimental adaptation of the released Alpamayo 2 wrapper; changing the model name in an Alpamayo 1.5 recipe is insufficient because their APIs differ.

NVFP4 represents values with four-bit floating-point numbers, FP8 scales for blocks of 16 values, and a global scale. Calibration estimates appropriate scales using real multimodal driving inputs. Packing reduces weight storage. Native execution additionally requires a kernel that consumes those packed values directly.

The saved weights do not need to be quantized again for the native adapter. The adapter reorders the existing FP8 scales once, quantizes incoming activations, and calls FlashInfer's CUTLASS FP4 GEMM. Outputs return in BF16. The native scale cache uses extra VRAM, which is included in the full-model memory measurement.

Sources: NVIDIA NVFP4 explanation, FlashInfer FP4 GEMM API, FlashInfer installation requirements.

Quality and compression check

The completed run used an RTX PRO 6000 with 96 GB VRAM. Five calibration clips and five different evaluation clips used six cameras, four frames per camera, and 4,580 input tokens. Each rollout used seed 42, one trajectory sample and ten diffusion steps, predicting 64 future positions over 6.4 seconds. Ground-truth future positions were kept separate from model inputs.

Execution phase	Mean ADE (m)	Mean FDE (m)	Median rollout (s)	Peak allocated VRAM (GiB)
Original BF16	2.05784	5.91994	2.31011	68.93043
NVFP4 quantize/dequantize simulation	1.92369	4.85703	3.63346	69.00075
Packed checkpoint, fallback	1.96314	5.02098	18.37306	28.70541
Packed checkpoint, fresh process	1.96314	5.02098	18.37296	28.70541

The fresh process reproduced the packed checkpoint's trajectories exactly. None of the 448 packed layers used native FP4 GEMM in that run. The fallback latency therefore does not measure native NVFP4 performance, and the later RTX 5090 run above establishes fit for the tested workload.

ADE is average position error across the predicted horizon. FDE is error at its endpoint. The packed model's mean trajectory deviation from BF16 was 1.16961 m. Although average ADE was 0.09470 m lower, four of five clips had worse ADE; one large improvement lowered the mean. These five clips are a functional smoke test, not evidence that quantization improves driving quality or establishes driving safety. The clips are calibration/evaluation disjoint; no claim is made that they were absent from pretraining.

The native run's mean ADE is 1.58759 m versus the original BF16 reference's 2.05784 m. However, two of five native clips have worse ADE and FDE than BF16. For clip 00ee8960, FDE rises from 2.23022 m to 5.07541 m. The mean trajectory deviation from BF16 is 1.32095 m. Native versus fallback trajectory deviation averages 0.41770 m, and generated reasoning text changes on two clips. Lower errors on this small selection do not establish a quality improvement or preservation across the driving distribution.

Native execution check

nvfp4_runtime.py implements a focused adapter for the pinned ModelOpt checkpoint and FlashInfer 0.6.18 on SM120. It preserves the checkpoint format, caches swizzled block scales, and attaches the native function to ModelOpt's packed linear modules.

Before downloading the full checkpoint, its GPU gate checks four representative matrix shapes derived from the model configuration at 1, 32, 128 and 4,580 input rows. It compares native outputs with FP32 multiplication of the same packed FP4 values and scale factors. It also requires a CUDA trace containing a CUTLASS FP4 GEMM kernel. This avoids treating a backend label alone as proof of native execution.

Full-model validation compared fallback and native trajectories, latency and memory on the same RTX 5090 using identical inputs and sampling settings. Timed rollouts follow a warmup; a separate profile records native CUDA kernels. The script fails if packed layers retain the fallback. Native activation quantization can round differently from ModelOpt's fallback, so trajectory changes are measured rather than assumed to be zero.

All 16 native matrix checks passed, with a maximum relative RMS error of 0.00300734 against the packed-value FP32 reference. The CUDA profile contains a CUTLASS SM120 block-scaled FP4 kernel. See native_gate.json. This gate checks kernel correctness; it does not by itself establish whole-model quality or speed. The subsequent full-model run additionally exercised every packed layer, including projection shapes outside this small gate, and captured a native FP4 kernel during a complete trajectory rollout.

Timing includes sample_trajectories_from_data, after CPU inputs have been copied to the GPU and before output conversion back to CPU. It excludes downloads, model load, kernel compilation and input preprocessing. Each phase has a warmup and one timed rollout per clip. Sampling resets seed 42, top-p 0.98 and temperature 0.6; native arithmetic changes some generated outputs, so the measured speedup is an application comparison, not an identical-output GEMM microbenchmark. The CUDA profile runs separately from timed clips. Memory counters are PyTorch allocator peaks, excluding driver/context and other processes.

Run on a Linux RTX 5090

Use Python 3.12, a CUDA 12.9 development toolkit and a compatible NVIDIA driver. The tested RunPod image is runpod/pytorch:1.3.1-cu1290-torch280-ubuntu2404. The Linux lock pins PyTorch 2.8.0+cu129, Transformers 4.57.1, ModelOpt 0.43.0, FlashInfer 0.6.18, and their dependencies. FlashInfer compiles the required kernels on first use. A system C++ compiler, CUDA compiler and sufficient host RAM/disk are needed. The completed pod profile reported 125 GB host RAM and 180 GB temporary disk. Ordinary HTTPS transfers with four workers are used because the Xet downloader repeatedly timed out on the tested host. These are test resources, not a measured minimum. The tested driver was 580.126.09. Native Windows execution has not been tested; reproduce the validated setup in Linux on the RTX 5090.

Download the preparation archive from the private repository while signed in with the account that has model/dataset access. Extract it into bundle/. It includes the prepared input tensors and scripts; it contains no credentials. The artifact checksum is recorded in upload_receipt.json. Do not download all repository history to obtain the model.

From an activated Python 3.12 virtual environment, install the lock and pinned upstream source. Use a terminal with the Hugging Face account already authenticated:

python -m pip install uv==0.12.5
uv pip sync bundle/requirements-linux.txt
git clone --no-checkout https://github.com/NVlabs/alpamayo2.git alpamayo2
git -C alpamayo2 checkout --detach 5e7975f4a2100ee8ac1a62b79239bbabefddcbef
export PYTHONPATH="$PWD/alpamayo2/src"
export HF_HUB_ENABLE_HF_TRANSFER=0
export HF_HUB_DISABLE_XET=1
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
export MAX_JOBS=8
python bundle/nvfp4_runtime.py --output native-gate

Download only the root checkpoint files, pinning the already verified checkpoint commit:

from huggingface_hub import HfApi, snapshot_download

repo = "dima0000/Alpamayo2-Super-NVFP4-language-smoke"
revision = "6300bd8fde2bf6d29a09a1ff56af11d57889219b"
info = HfApi().model_info(repo, revision=revision)
root_files = [entry.rfilename for entry in info.siblings if "/" not in entry.rfilename]
snapshot_download(repo, revision=revision, local_dir="checkpoint", allow_patterns=root_files, max_workers=4)

Run the paired benchmark with a new output directory:

python bundle/alpamayo2_nvfp4.py \
  --eval-only --native-nvfp4 \
  --checkpoint "$PWD/checkpoint" \
  --inputs bundle/prepared_inputs.pt \
  --evaluation-manifest alpamayo2/examples/public_golden_validation_samples.json \
  --output benchmark-5090

For integration into an application, enable checkpoint restoration before loading the model, then enable the native adapter before any trajectory sampling:

import torch
from alpamayo2_nvfp4 import configure_modelopt
from nvfp4_runtime import enable_native_nvfp4
from alpamayo2_super.models.alpamayo2_super import Alpamayo2Super

configure_modelopt()
model = Alpamayo2Super.from_pretrained(
    "./checkpoint", dtype=torch.bfloat16, device_map="cuda:0",
    attn_implementation="sdpa",
).eval()
print(enable_native_nvfp4(model))
# Prepare inputs and call sample_trajectories_from_data using the released helpers.

The application must add bundle/ to its Python import path for these two adapter imports. The checkpoint requires Alpamayo's wrapper and ModelOpt; it is not a generic AutoModel text-generation checkpoint. Loading the checkpoint alone does not enable the custom native backend. Use remains subject to upstream model and dataset license terms.

Reproducibility and spending

run_config.json is the source of truth for upstream revisions, samples and run settings. requirements-linux.txt pins the environment; bundle_manifest.json records artifact sizes and SHA-256 values. preflight.json records the successful CUDA packing/reload gate from the original quantization run. inputs_report.json records the prepared driving inputs. preparation_status.json records the completed experiment status and spending.

runpod_run.py validates the current pod identity, price, cumulative spending and rental start. It arms a separate stop process, reserves 15 minutes for upload and 2 minutes for shutdown, uploads diagnostics even after failure, and requests pod stop in its cleanup. work/stage_pod.py additionally arms a bootstrap watchdog before dependency installation. For this reports-only native validation, a recovery controller reused these functions, kept the original 90-minute hard stop, and reduced the upload reservation to five minutes while preserving the two-minute stop margin. This allowed a slow checkpoint download to finish on the same pod. The default full-quantization launcher retains its 15-minute reservation because checkpoint uploads are much larger. Rental prices, pod identity and prior spending must be refreshed for any later paid run; the bundled values are a preparation snapshot, not current account state.

The 12 local preparation tests mock RunPod APIs and verify failure cleanup; they are not a replacement for observing the real pod stop. The protection is a guest watchdog plus external control, not a provider-enforced dollar cap.

For meaningful quality assessment beyond this smoke test, evaluate a larger disjoint set covering turns, stopping, intersections, road users, weather and lighting. Report paired ADE/FDE changes and tails, trajectory acceleration/jerk and boundary violations, and inspect regressions clip by clip. Any closed-loop driving claim needs simulator rollouts measuring collisions, route completion, traffic violations and comfort. Keep sampling settings and runtime versions fixed when comparing precision choices.

Model-weight license: OpenMDW-1.1, retained from the pinned NVIDIA model release. Upstream model origin: nvidia/Alpamayo2-Super. The upstream Alpamayo source is separately licensed under Apache-2.0. This repository adds experimental language-projection quantization, a custom native adapter, and evaluation reports; it is not an NVIDIA-certified quantized release. The dataset-related artifacts retain their separate terms.