strands-isaaclab-open-drawer-franka-policy

rsl_rl PPO policy for a Franka Panda opening a cabinet drawer (Isaac-Open-Drawer-Franka), trained with strands-robots' isaaclab train_policy provider (PR #4227). Rollouts recorded through strands: cagataydev/strands-isaaclab-open-drawer-franka.

Results

4096 envs × 400 PPO iterations (157 M env steps, 21 min 6 s, Newton/MJWarp, 1× L40S): mean reward 1.63 → 97.8, drawer-open success 1.00. Recorded 12 rollouts: mean return 60.0.

PPO iteration	mean reward	drawer-open success	env-steps/s
0	1.63	0.002	88,003
25	36.14	0.940	95,673
50	50.87	1.000	95,401
100	71.89	1.000	95,659
200	90.95	1.000	153,266
300	96.91	1.000	152,153
399	97.82	1.000	125,944

Files

file	what
model_399.pt	rsl_rl checkpoint (actor-critic + optimizer), final iteration
exported/policy.pt	TorchScript actor incl. observation normalizer; input (N, 31) → (N, 8) (7 arm joints + gripper)
exported/policy.onnx (+ .data)	ONNX export of the same actor
params/agent.yaml, params/env.yaml	exact Isaac Lab agent / env configs of this run
train_curve.json	per-iteration training log parsed by the strands provider
record.json, verify.json	recording + verification reports
playback.gif, playback.mp4, frame.png	rendered rollouts
examples/	recording script used to make the dataset

How it was made with strands-robots

# Isaac Lab in its OWN venv (its pins clash with strands; strands never imports it)
uv venv --python 3.12 ~/il && uv pip install --python ~/il/bin/python --prerelease=allow \
  --index https://pypi.nvidia.com --index-strategy unsafe-best-match "isaaclab[rsl-rl,isaacsim]==3.0.0rc1"
export ISAACLAB_PYTHON=~/il/bin/python
export OMNI_KIT_ACCEPT_EULA=YES          # you accept the NVIDIA Omniverse / Isaac Sim EULA yourself
pip install "git+https://github.com/cagataycali/robots@feat/isaaclab-trainer"   # strands-robots with PR #4227

As an agent tool call (the train_policy tool is a Strands @tool):

from strands import Agent
from strands_robots.tools.train_policy import train_policy

agent = Agent(tools=[train_policy])
agent("Train the Franka to open the cabinet drawer with the isaaclab provider: task Isaac-Open-Drawer-Franka, 400 iterations, seed 1.")
# -> train_policy(action="train", provider="isaaclab", steps=400, seed=1, output_dir="runs/c6_open_drawer",
#                 extra={"task": "Isaac-Open-Drawer-Franka", "timeout_s": 7200})

As plain Python (exactly what produced this run):

from strands_robots.tools.train_policy import train_policy

job = train_policy(action="train", provider="isaaclab", steps=400, seed=1, output_dir="runs/c6_open_drawer",
                   extra={"task": "Isaac-Open-Drawer-Franka", "timeout_s": 7200})   # task default: 4096 envs, Newton/MJWarp physics
train_policy(action="status", provider="isaaclab", job_id="<job_id from the result>")

Under the hood: python -m isaaclab train --rl_library rsl_rl --task Isaac-Open-Drawer-Franka --max_iterations 400 --seed 1 in $ISAACLAB_PYTHON. Job id of this run: isaaclab-20260929-074641-de56a64da9e9. Docs: docs/learn/training/isaaclab.md · PR: strands-labs/robots#4227.

Use it

Play it in Isaac Lab:

huggingface-cli download cagataydev/strands-isaaclab-open-drawer-franka-policy --local-dir drawer_policy
OMNI_KIT_ACCEPT_EULA=YES $ISAACLAB_PYTHON -m isaaclab play --rl_library rsl_rl --task Isaac-Open-Drawer-Franka \
  --num_envs 16 --checkpoint drawer_policy/model_399.pt     # add --video --video_length 300

Or use the exported actor anywhere (TorchScript):

import torch
pi = torch.jit.load("drawer_policy/exported/policy.pt").eval()
actions = pi(obs)          # obs: (N, 31) Isaac Lab 'policy' observation group -> (N, 8)

As a strands Policy (what the recording used):

import json, numpy as np, torch
from strands_robots.policies.base import Policy

class RslRlJitPolicy(Policy):
    def __init__(self, jit_path, action_names, device="cpu"):
        self.net = torch.jit.load(jit_path, map_location=device).eval()
        self.device, self.action_names, self.robot_state_keys = device, list(action_names), []
    @property
    def provider_name(self): return "isaaclab_rsl_rl_jit"
    def set_robot_state_keys(self, keys): self.robot_state_keys = list(keys)
    def reset(self, seed=None): self.net.reset()
    async def get_actions(self, observation_dict, instruction, **kw):
        x = torch.as_tensor(np.asarray(observation_dict["policy_obs"], np.float32), device=self.device)
        with torch.inference_mode():
            y = self.net(x.unsqueeze(0))[0].cpu().numpy()
        return [dict(zip(self.action_names, map(float, y)))]

names = json.load(open("drawer_policy/record.json"))["action_names"]
policy = RslRlJitPolicy("drawer_policy/exported/policy.pt", names)
chunk = policy.get_actions_sync({"policy_obs": obs_31}, "open the drawer")   # [{"arm_action.panda_joint1": ..., "gripper_action.0": ...}]

Record your own dataset with strands:

The final checkpoint was rolled out and recorded with examples/record_trained_policy.py (included): rebuild the task env in play mode with an RTX camera → load model_399.pt with rsl_rl and export TorchScript/ONNX → wrap the actor as a strands Policy (RslRlJitPolicy, max |Δa| vs rsl_rl = 3.6e-07) → step with policy.get_actions_sync(...) → write every frame through strands DatasetRecorder (LeRobot v3) → verify with strands verify_dataset.

OMNI_KIT_ACCEPT_EULA=YES PYTHONPATH=/path/to/strands-robots $ISAACLAB_PYTHON examples/record_trained_policy.py \
  --task Isaac-Open-Drawer-Franka --checkpoint model_399.pt --episodes 12 --frames 300 \
  --cam fixed --cam_name front --eye 1.5,-1.5,1.4 --target 0.5,0,0.4 \
  --task_str "grasp the drawer handle and pull the drawer open" --robot_type franka_panda --root out/ds --repo_id cagataydev/strands-isaaclab-open-drawer-franka
$ISAACLAB_PYTHON examples/record_trained_policy.py --verify out/ds --repo_id cagataydev/strands-isaaclab-open-drawer-franka

Provenance

• strands-robots: feat/isaaclab-trainer @ fa66fc68 — strands-labs/robots#4227 (isaaclab train_policy provider; DatasetRecorder; verify_dataset)
• Isaac Lab 3.0.0rc1 · Isaac Sim 6.1.0.0 · Newton / MJWarp (task default physics) · rsl-rl-lib 5.4.1 (PPO) · lerobot 0.6.1
• GPU: 1× NVIDIA L40S (46 GB), shared with the AnymalD rough-terrain training
• Seeds: training seed 1 (params/agent.yaml, params/env.yaml); recording seed 7
• Training job: isaaclab-20260929-074641-de56a64da9e9, 2026-09-29

Limitations

• Simulation only; no real Franka or cabinet was used; no sim-to-real claims.
• Release candidates: Isaac Lab 3.0.0rc1 / Isaac Sim 6.1.0.0. Trained on the task's default Newton/MJWarp physics; the checkpoint does not remember the preset (IL-X-011) — replay on the same physics.
• Metrics/success_rate is Isaac Lab's drawer-open criterion; it saturates early (≥ 0.99 from iteration 37) while the reward keeps rising (the drawer gets opened further/faster), so compare runs by reward too.
• Recording: 12 parallel envs, 300 frames (5 s @ 60 fps) each from one fixed camera; the episodes are fixed-length windows, not cut at success. observation.state = 9 joint pos + 31-D policy observation + root pos/quat (47-D), not a standard LeRobot robot state.
• create_policy("rl") in strands cannot load rsl_rl checkpoints yet (IL-X-006); use the exported TorchScript + the wrapper above.

License

Card choice: license: other — our generated data / weights under CC-BY-4.0, plus NVIDIA notices. Why:

• Trajectories, rendered camera video, playback clips and trained weights are user-generated content made with NVIDIA Isaac Sim / Isaac Lab; the NVIDIA Omniverse License Agreement (governs Isaac Sim 6.1, isaacsim/LICENSE.txt) §2.1 allows distributing "user generated content that you develop using Omniverse, such as video, audio, stills, models, 3D assets and screen captures". We release it under CC-BY-4.0.
• No NVIDIA Content is redistributed: the Franka Panda USD (IsaacLab/Robots/FrankaEmika/Legacy/panda_instanceable.usd) and the Sektion cabinet USD (Props/Sektion_Cabinet/sektion_cabinet_instanceable.usd) come from the Isaac Lab / Isaac Sim asset packs and are not in this repo (params/env.yaml only references their paths). "Franka" is a trademark of Franka Robotics; no endorsement by Franka Robotics or NVIDIA is implied.
• params/*.yaml are Isaac Lab configurations (BSD-3-Clause); examples/* are Apache-2.0 like strands-robots. Running Isaac Sim requires your own acceptance of the NVIDIA Isaac Sim / Omniverse EULA. Full text: LICENSE.md.