Real-world Q-RECAP policy sweep (fail*_ps)
OpenPI ฯ0.5 policies trained with Q-RECAP (CFG + 4-bit LM QAT) on real-world multitask data.
Each subfolder is a different failure penalty with primitive_success shaping on the last step
(failure_reward - 500 * (1 - p)).
This repo contains policy checkpoints only (stage4 actor). Value / advantage artifacts are not included.
Layout
fail300_ps/actor/model_state_dict/full_weights.pt
fail500_ps/actor/model_state_dict/full_weights.pt
...
norm_stats.json # shared; also copied under each fail*_ps/
manifest.csv
Included runs
| Folder | failure_reward | training step |
|---|---|---|
fail300_ps |
-300 | global_step_30000 |
fail400_ps |
-400 | global_step_30000 |
fail500_ps |
-500 | global_step_30000 |
fail600_ps |
-600 | global_step_30000 |
fail750_ps |
-750 | global_step_30000 |
fail1000_ps |
-1000 | global_step_30000 |
Load in RLinf
export Q_RECAP_POLICY_PATH="/path/to/this-repo/fail500_ps/actor"
# eval / deploy with:
# examples/recap/cfg/config/realworld_cfg_openpi_q_recap_qat_lm4bit.yaml
# data.advantage_tag=realworld_q_recap_fail500_ps_N8_ckpt18000_q30
RLinf resolves actor/model_state_dict/full_weights.pt under model_path.
Base model
Fine-tuned from the real-world multitask ฯ0.5 checkpoint used in this project
(see Q_RECAP_RAW_POLICY_PATH / Q_RECAP_POLICY_PATH in env.sh).
Confirm license terms for the base OpenPI / aiha weights before redistribution.