Qwen3VL-PI_v3-Bridge-RT-1

A Vision-Language-Action (VLA) model from the StarVLA project, combining a Qwen3-VL-4B-Instruct backbone with a layer-wise cross-attention flow-matching action head (QwenPI_v3). The model is co-trained on the Bridge V2 and RT-1 / Fractal slices of the Open X-Embodiment (OXE) collection, and is evaluated on the SimplerEnv WidowX benchmark.

QwenPI_v3 is StarVLA's open-weight realisation of the π₀.₅ recipe:

Layer-wise cross-DiT flow-matching action head — every VLM layer's hidden state participates in cross-attention with the action DiT, instead of consuming only the last-layer feature.
Compressed Action DiT — per-layer LayerNorm + Linear projectors compress the 2560-d Qwen3-VL hidden states down to a 1024-d DiT latent, shrinking the action-head footprint by ~6× while preserving the layer-wise interaction.
Discretised-state language injection — proprioceptive state is quantised into 256 bins and appended to the instruction as plain tokens ([STATE] <bins> [ACTION]), so the VLM can attend to robot state with no additional encoder.

Model Summary


Architecture	`QwenPI_v3` (Qwen3-VL + layer-wise cross-DiT flow-matching head)
VLM backbone	`Qwen3-VL-4B-Instruct`
Action head	Layer-wise Flow-Matching DiT (36 layers, 1024 hidden, 16 heads)
Action chunk	16 steps
Action / state dim	7 / 7 (delta end-effector)
Image resolution	224 × 224, single 3rd-person view
Inference timesteps	4 (flow matching)
Total parameters	≈ 5.07 B
License metadata	MIT in this repository's existing Hub front matter; no separate `LICENSE` file is packaged
Codebase	starVLA/starVLA

Parameter breakdown

Module	Parameters	Share
`qwen_vl_interface` (Qwen3-VL-4B)	4,437,815,808	87.5 %
`action_model` (layer-wise FM DiT, hidden 1024)	538,678,305	10.6 %
`project_layers` (per-layer 2560 → 1024 projectors)	94,593,024	1.9 %
Total	5,071,087,137	100 %

Training Data

Co-training mixture bridge_rt_1 (1 : 1 sampling):

Dataset	Embodiment	Source
`bridge_orig_1.0.0_lerobot`	WidowX	IPEC-COMMUNITY/bridge_orig_lerobot
`fractal20220817_data_0.1.0_lerobot` (RT-1)	Google Robot	IPEC-COMMUNITY/fractal20220817_data_lerobot

Action representation: delta end-effector (7-d, gripper included)
Image observation: single primary RGB view, resized to 224 × 224
Per-dataset normalisation statistics are stored in dataset_statistics.json.

Training Recipe


Total steps	100,000 configured; only the 50k weight is released
Warm-up steps	5,000
Per-device batch size	24
Hardware	GPU type/count is not recorded by the portable config
Precision	bf16, mixed-precision + gradient checkpointing
Optimizer	AdamW (β₁ = 0.9, β₂ = 0.95, ε = 1e-8, wd = 1e-8)
LR (base / VLM)	1e-5
LR (action head)	1e-4
LR scheduler	`cosine_with_min_lr` (min lr 5e-7)
Gradient clipping	1.0
Flow-matching noise	β-distribution (α=1.5, β=1.0), s = 0.999
Repeated diffusion steps	8
Frozen modules	Archived launcher overrides to empty; saved config records `true`, so confirm the resolved run value
Attention impl.	FlashAttention-2

The saved recipe is preserved in config.yaml and config.full.yaml. These files contain environment-local paths that must be mapped before use. The archived launcher is also environment-specific and must be security-reviewed rather than executed unchanged.

Evaluation — SimplerEnv WidowX

Following the standard SimplerEnv WidowX protocol on four pick-and-place tasks (24 episodes per task per run). Numbers are success rates (↑).

Step	PutCarrotOnPlate	PutEggplantInBasket	PutSpoonOnTableCloth	StackGreenCubeOnYellowCube	Average
40k	0.688	0.917	0.750	0.333	0.672
50k	0.625	1.000	0.792	0.375	0.698
60k	0.667	1.000	0.750	0.167	0.646

Best average: 69.8 % at the 50k checkpoint (steps_50000_pytorch_model.pt), which we ship as the recommended checkpoint.

For comparison with other StarVLA frameworks on the same bridge_rt_1 mixture and protocol see the StarVLA Model Zoo.

Repository Layout

.
├── README.md                 # this model card
├── config.yaml               # minimal training config
├── config.full.yaml          # fully resolved training config
├── run_oxe_train.sh          # archived environment-specific launcher; audit before use
├── dataset_statistics.json   # per-dataset action/state normalisation stats
├── summary.jsonl             # training step summary
├── success_summary/          # SimplerEnv evaluation logs and plots
│   ├── success_summary.csv
│   ├── raw_success.txt
│   └── success_plot.png
└── checkpoints/
    ├── steps_50000_pytorch_model.pt   # ← recommended checkpoint
    └── ...                            # per-step evaluation logs

How to Use

This checkpoint is consumed by the StarVLA framework; it is not a standalone Transformers from_pretrained() directory. Clone StarVLA, download the whole Hub repository, and retain its directory layout:

git clone https://github.com/starVLA/starVLA.git
cd starVLA
# Follow installation instructions in the StarVLA README.
huggingface-cli download StarVLA/Qwen3VL-PI_v3-Bridge-RT_1 \
  --local-dir Qwen3VL-PI_v3-Bridge-RT_1

CKPT="$PWD/Qwen3VL-PI_v3-Bridge-RT_1/checkpoints/steps_50000_pytorch_model.pt"
python deployment/model_server/server_policy.py \
  --ckpt_path "$CKPT" --port 6678 --use_bf16 \
  --config_override framework.qwenvl.base_vlm=Qwen/Qwen3-VL-4B-Instruct

For end-to-end SimplerEnv evaluation see examples/simBenchmarks/SimplerEnv.

Intended Use & Limitations

Intended use. Research on vision-language-action models, manipulation policy learning, and as a baseline for π-style flow-matching action heads on top of open-weight VLMs.

Out-of-scope / limitations.

Trained only on Bridge (WidowX) + RT-1 (Google Robot) with a 7-d delta-EE action space — generalisation to other embodiments / action spaces is not guaranteed.
Single 224 × 224 third-person view; no wrist camera, no depth.
Evaluated only on SimplerEnv WidowX simulation; behaviour on real robots has not been validated by the released checkpoint.
Inherits any biases / failure modes of the underlying Qwen3-VL-4B model.
Not safety-tuned. Do not deploy on physical robots without an external safety layer.

Evidence and evaluation boundary

Architecture and training values were checked against the packaged configs; the released artifact list contains only checkpoints/steps_50000_pytorch_model.pt.
The repository includes success_summary/ and per-step evaluation logs for 40k, 50k, and 60k, so the reported table is auditable. Evaluation logs at a step do not imply that the corresponding weight is downloadable.
The public config records gradient_accumulation_steps: 1, but does not establish GPU type/count. No global batch size is claimed here.
The config/launcher disagreement for freeze_modules prevents a definitive frozen-module claim without the original resolved launch log.
The MIT tag is retained from this target repository's already-published Hub metadata, not inferred from the StarVLA code license. No separate model LICENSE file is packaged, and users must also comply with the Qwen3-VL base model terms.
Reproducing the result requires the saved normalization statistics and the same SimplerEnv task, camera, action and execution-horizon protocol.

Citation

If you use this checkpoint, please cite StarVLA:

@article{starvla2026,
  title   = {StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing},
  author  = {StarVLA Community},
  journal = {arXiv preprint arXiv:2604.05014},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.05014}
}

And the underlying VLM backbone:

@misc{qwen3vl,
  title  = {Qwen3-VL},
  author = {Qwen Team},
  year   = {2025},
  url    = {https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct}
}

Acknowledgements

Qwen Team for the Qwen3-VL backbone.
Physical Intelligence for the π₀ / π₀.₅ flow-matching action-head recipe that inspired QwenPI_v3.
Open X-Embodiment and IPEC-COMMUNITY for the LeRobot conversions of Bridge V2 and RT-1.
SimplerEnv for the evaluation protocol.