Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 145 additions & 0 deletions examples/isaaclab_bimanual_robot/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
<!--

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please rename to README.md so that it's consistent with the rest of the code base. thanks.

SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Bimanual Robot Example using IsaacLab

This directory contains minimal usage examples for teleoperating bimanual robot arms in IsaacLab.

##UR bimanual
![ur bimanual](./images/bimanual_ur.png)


## Prerequisites


### 1.Isaac Teleop

Follow [Install Isaac Teleop](https://isaac-sim.github.io/IsaacLab/main/source/how-to/cloudxr_teleoperation.html#cloudxr-teleoperation) to install Isaac Teleop.

After installation, start the ClourXR server:

```bash
python -m isaacteleop.cloudxr
```

### 2.Isaac Lab

Follow [Install Isaac Lab](https://isaac-sim.github.io/IsaacLab/main/source/setup/installation/index.html) to install Isaac Lab with virtual environment `env_isaaclab`.

After installation, activate the virtual environment:

```bash
source <your_isaaclab_path>/env_isaaclab/bin/activate
```

### 3. Install this repository to `env_isaaclab`

```bash
cd examples/isaaclab_bimanual_robot
python -m pip install -e .

## If you use uv:
# uv pip install -e .
```

## How to run

1. Source cloudxr env

```bash
source ~/.cloudxr/run/cloudxr.env
```

2. Run the task

List the two environments:

```bash
python scripts/list_envs.py
```

And you should see:

```
+--------+------------------------------+---------------------------------+-----------------------------------------------------------------------------------------------+
| S. No. | Task Name | Entry Point | Config |
+--------+------------------------------+---------------------------------+-----------------------------------------------------------------------------------------------+
| 1 | Template-FlexivRizon-Play-v0 | isaaclab.envs:ManagerBasedRLEnv | bimanual.tasks.manager_based.flexiv_rizon.flexiv_rizon_bimanual_env:FlexivRizonBimanualEnvCfg |
| 2 | Template-UR10-Play-v0 | isaaclab.envs:ManagerBasedRLEnv | bimanual.tasks.manager_based.ur10.ur10_bimanual_env:UR10BimanualEnvCfg |
+--------+------------------------------+---------------------------------+-----------------------------------------------------------------------------------------------+
```

To examine the task environment only (without XR or Teleop):

```bash
python scripts/zero_agent.py --task Template-UR10-Play-v0 --num_envs=1

python scripts/zero_agent.py --task Template-FlexivRizon-Play-v0 --num_envs=1
```

Teleoperation with XR controllers:

```bash
python scripts/teleop_se3_agent_bimanual_xr.py \
--task Template-UR10-Play-v0 \
--teleop_device motion_controllers \
--num_envs 1 \
--xr \
--reverse_rotation_yz

python scripts/teleop_se3_agent_bimanual_xr.py \
--task Template-FlexivRizon-Play-v0 \
--teleop_device motion_controllers \
--num_envs 1 \
--xr \
--sensitivity 0.5 \
--enable_gripper
```

Connect you XR devices before running the task using the [Isaac Teleop Web client](https://nvidia.github.io/IsaacTeleop/client/).


In IsaacLab UI, click the `Start AR` button in the bottom right panel.

<img width="300" src="./images/start_xr.png" >

Key bindings:
- Press any button on the left controller to start teleoperation
- Move the left controller to control the left arm
- Move the right controller to control the right arm
- Press x or y buttons on the left controller to reset


Record data:

```bash
python scripts/record_se3_agent_bimanual_xr.py \
--task Template-UR10-Play-v0 \
--teleop_device motion_controllers \
--num_envs 1 \
--xr \
--dataset_file ./datasets/dataset.hdf5

# or with flexiv rizon
python scripts/record_se3_agent_bimanual_xr.py \
--task Template-FlexivRizon-Play-v0 \
--teleop_device motion_controllers \
--num_envs 1 \
--xr \
--sensitivity 0.5 \
--enable_gripper \
--dataset_file ./datasets/dataset_flexiv.hdf5
```

![flexiv rizon teleoperation](./images/bimanual_flexiv.png)

Key bindings:
- Press any button on the left controller to start teleoperation
- Move the left controller to control the left arm
- Move the right controller to control the right arm
- Press x or y buttons on the left controller to reset
- Press a or b buttons on the right controller to save the trajectory and reset env

The dataset will be saved to `./datasets/` folder.
4 changes: 4 additions & 0 deletions examples/isaaclab_bimanual_robot/datasets/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Ignore everything in this directory
*
# Except this .gitignore file
!.gitignore
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
158 changes: 158 additions & 0 deletions examples/isaaclab_bimanual_robot/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

[project]
name = "bimanual"
version = "0.1.0"

[project.optional-dependencies]
data = [
"lerobot",
]

[tool.setuptools.packages.find]
where = ["source/"]



[tool.ruff]
line-length = 120
target-version = "py310"

# Exclude directories
extend-exclude = [
"logs",
"_isaac_sim",
".vscode",
"_*",
".git",
"assets",
"datasets",
]

[tool.ruff.lint]
# Enable flake8 rules and other useful ones
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # pyflakes
"I", # isort
"UP", # pyupgrade
"C90", # mccabe complexity
# "D", # pydocstyle
"SIM", # flake8-simplify
"RET", # flake8-return
]

# Ignore specific rules (matching your flake8 config)
ignore = [
"E402", # Module level import not at top of file
"D401", # First line should be in imperative mood
"RET504", # Unnecessary variable assignment before return statement
"RET505", # Unnecessary elif after return statement
"SIM102", # Use a single if-statement instead of nested if-statements
"SIM103", # Return the negated condition directly
"SIM108", # Use ternary operator instead of if-else statement
"SIM117", # Merge with statements for context managers
"SIM118", # Use {key} in {dict} instead of {key} in {dict}.keys()
"UP006", # Use 'dict' instead of 'Dict' type annotation
"UP018", # Unnecessary `float` call (rewrite as a literal)
]

[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"] # Allow unused imports in __init__.py files

[tool.ruff.lint.mccabe]
max-complexity = 30

[tool.ruff.lint.pydocstyle]
convention = "google"

[tool.ruff.lint.isort]

# Custom import sections with separate sections for each Isaac Lab extension
section-order = [
"future",
"standard-library",
"third-party",
# Group omniverse extensions separately since they are run-time dependencies
# which are pulled in by Isaac Lab extensions
"omniverse-extensions",
# Group Isaac Lab extensions together since they are all part of the Isaac Lab project
"isaaclab",
"isaaclab-contrib",
"isaaclab-rl",
"isaaclab-mimic",
"isaaclab-tasks",
"isaaclab-assets",
# First-party is reserved for project templates
"first-party",
"local-folder",
]

[tool.ruff.lint.isort.sections]
# Define what belongs in each custom section

"omniverse-extensions" = [
"isaacsim",
"omni",
"pxr",
"carb",
"usdrt",
"Semantics",
"curobo",
]

"isaaclab" = ["isaaclab"]
"isaaclab-assets" = ["isaaclab_assets"]
"isaaclab-contrib" = ["isaaclab_contrib"]
"isaaclab-rl" = ["isaaclab_rl"]
"isaaclab-mimic" = ["isaaclab_mimic"]
"isaaclab-tasks" = ["isaaclab_tasks"]

[tool.ruff.format]

docstring-code-format = true

[tool.pyright]

include = ["source", "scripts"]
exclude = [
"**/__pycache__",
"**/_isaac_sim",
"**/docs",
"**/logs",
".git",
".vscode",
]

typeCheckingMode = "basic"
pythonVersion = "3.11"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yizhouzhao-nvidia IIRC, this this still using Isaac Lab 2.3? So this example is actually using the legacy CloudXR path in Isaac Lab rather than the new Isaac Teleop path.

For the simple XR controller path this is probably find and we can actually land this as-is and then make a follow up PR to upgrade this to Isaac Lab 3.0. That PR can be used as a follow up example as well.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rwiltz wdyt?

pythonPlatform = "Linux"
enableTypeIgnoreComments = true

# This is required as the CI pre-commit does not download the module (i.e. numpy, torch, prettytable)
# Therefore, we have to ignore missing imports
reportMissingImports = "none"
# This is required to ignore for type checks of modules with stubs missing.
reportMissingModuleSource = "none" # -> most common: prettytable in mdp managers

reportGeneralTypeIssues = "none" # -> raises 218 errors (usage of literal MISSING in dataclasses)
reportOptionalMemberAccess = "warning" # -> raises 8 errors
reportPrivateUsage = "warning"


[tool.codespell]
skip = '*.usd,*.usda,*.usdz,*.svg,*.png,_isaac_sim*,*.bib,*.css,*/_build'
quiet-level = 0
# the world list should always have words in lower case
ignore-words-list = "haa,slq,collapsable,buss,reacher,thirdparty"


[tool.pytest.ini_options]

markers = [
"isaacsim_ci: mark test to run in isaacsim ci",
]
72 changes: 72 additions & 0 deletions examples/isaaclab_bimanual_robot/scripts/list_envs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

"""
Script to print all the available environments in Isaac Lab.

The script iterates over all registered environments and stores the details in a table.
It prints the name of the environment, the entry point and the config file.

All the environments are registered in the `Bimanual` extension. They start
with `Template-` in their name.
"""

"""Launch Isaac Sim Simulator first."""

import argparse

from isaaclab.app import AppLauncher

# add argparse arguments
parser = argparse.ArgumentParser(description="List Isaac Lab environments.")
parser.add_argument("--keyword", type=str, default=None, help="Keyword to filter environments.")
# parse the arguments
args_cli = parser.parse_args()

# launch omniverse app
app_launcher = AppLauncher(headless=True)
simulation_app = app_launcher.app


"""Rest everything follows."""

import gymnasium as gym
from prettytable import PrettyTable

import bimanual.tasks # noqa: F401


def main():
"""Print all environments registered in `Bimanual` extension."""
# print all the available environments
table = PrettyTable(["S. No.", "Task Name", "Entry Point", "Config"])
table.title = "Available Environments in Isaac Lab"
# set alignment of table columns
table.align["Task Name"] = "l"
table.align["Entry Point"] = "l"
table.align["Config"] = "l"

# count of environments
index = 0
# acquire all Isaac environments names
for task_spec in gym.registry.values():
if "Template-" in task_spec.id and (args_cli.keyword is None or args_cli.keyword in task_spec.id):
# add details to table
table.add_row([index + 1, task_spec.id, task_spec.entry_point, task_spec.kwargs["env_cfg_entry_point"]])
# increment count
index += 1

print(table)


if __name__ == "__main__":
try:
# run the main function
main()
except Exception as e:
raise e
finally:
# close the app
simulation_app.close()
Loading