Robotics
Safetensors
openpi
pi05
franka

pi05_franka_relative_10hz_unified

pi0.5 policy for the Franka FR3 with relative end-effector pose actions. It is trained on data from both side cameras (RealSense D435if and D455), with every side image rendered in one unified view, so it can be served with either camera.

The repo root holds the most recent checkpoint, and each uploaded checkpoint is tagged step-<N> (every 10k steps and the final one). model.safetensors contains the EMA weights (decay 0.99).

Unified side view

The unified view is the D435if viewpoint with only the top 426 of its 480 rows (848×426); the D455 cannot see the bottom rows. D435if images are cropped to it. D455 images are re-rendered into it with a perspective warp computed from the recorded calibrations: a homography induced by the plane z = 5 cm, plus the D455 lens distortion. The warp is off by less than 2 px (at 224 px) for points up to 15 cm above the table. See openpi.policies.franka_cameras.

At inference the side camera defaults to the D455. A client using the D435if sends "observation/side_camera": "d435if" with the observation. The side image must be the full camera frame; resizing it is fine, but cropping or padding is not.

Training

  • Data: D435if dataset (515 episodes, 139,823 frames, 14 tasks) and D455 dataset (255 episodes, 57,708 frames, 6 tasks), concatenated.
  • Validation: 25 D455 episodes (10% per task, seed 0; 5,516 frames) are held out: 13, 15, 24, 29, 39, 74, 80, 87, 93, 99, 112, 123, 136, 140, 153, 156, 180, 190, 191, 197, 208, 219, 226, 228, 229. Every 1k steps the run logs val/loss (fixed noise), val/grasp_point_err_cm, val/pos_err_1s_cm and val/gripper_close_within_0.3s (sampled chunks on frames before a grasp) to wandb, using the raw (non-EMA) weights.
  • Initialization: lerobot/pi05_base.
  • Schedule: 75,000 steps, global batch 64 (4× A100-40GB, 4 per GPU × 4 accumulation steps), cosine LR with 1k warmup, peak 2.5e-5, decaying to 2.5e-6. EMA 0.99, bf16.
  • State / actions: same as pi05_franka_relative_10hz. Norm stats are computed over the training data of both datasets and stored under assets/franka_d435if_d455_unified.
  • openpi config: pi05_franka_relative_10hz_unified.

Evaluation

None yet.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
4B params
Tensor type
F32
·
BF16
·
Video Preview
loading

Model tree for muacha/pi05_franka_relative_10hz_unified

Finetuned
(909)
this model

Datasets used to train muacha/pi05_franka_relative_10hz_unified