pi05_franka_relative_10hz_unified
pi0.5 policy for the Franka FR3 with relative end-effector pose actions. It is trained on data from both side cameras (RealSense D435if and D455), with every side image rendered in one unified view, so it can be served with either camera.
The repo root holds the most recent checkpoint, and each uploaded checkpoint is tagged step-<N> (every 10k steps and the final one). model.safetensors contains the EMA weights (decay 0.99).
Unified side view
The unified view is the D435if viewpoint with only the top 426 of its 480 rows (848×426); the D455 cannot see the bottom rows. D435if images are cropped to it. D455 images are re-rendered into it with a perspective warp computed from the recorded calibrations: a homography induced by the plane z = 5 cm, plus the D455 lens distortion. The warp is off by less than 2 px (at 224 px) for points up to 15 cm above the table. See openpi.policies.franka_cameras.
At inference the side camera defaults to the D455. A client using the D435if sends "observation/side_camera": "d435if" with the observation. The side image must be the full camera frame; resizing it is fine, but cropping or padding is not.
Training
- Data: D435if dataset (515 episodes, 139,823 frames, 14 tasks) and D455 dataset (255 episodes, 57,708 frames, 6 tasks), concatenated.
- Validation: 25 D455 episodes (10% per task, seed 0; 5,516 frames) are held out: 13, 15, 24, 29, 39, 74, 80, 87, 93, 99, 112, 123, 136, 140, 153, 156, 180, 190, 191, 197, 208, 219, 226, 228, 229. Every 1k steps the run logs
val/loss(fixed noise),val/grasp_point_err_cm,val/pos_err_1s_cmandval/gripper_close_within_0.3s(sampled chunks on frames before a grasp) to wandb, using the raw (non-EMA) weights. - Initialization:
lerobot/pi05_base. - Schedule: 75,000 steps, global batch 64 (4× A100-40GB, 4 per GPU × 4 accumulation steps), cosine LR with 1k warmup, peak 2.5e-5, decaying to 2.5e-6. EMA 0.99, bf16.
- State / actions: same as
pi05_franka_relative_10hz. Norm stats are computed over the training data of both datasets and stored underassets/franka_d435if_d455_unified. - openpi config:
pi05_franka_relative_10hz_unified.
Evaluation
None yet.