From 08d1202429bd02881b479b790c824ae4e6f3e6fa Mon Sep 17 00:00:00 2001 From: yuecideng Date: Tue, 12 May 2026 12:44:59 +0800 Subject: [PATCH] docs: improve navigation, add cross-references and new guides - Add project overview paragraph to root index page - Add numbered learning path with prerequisites to tutorial index - Fix broken dangling sentence in modular_env tutorial - Clean up resources/task/index toctree - Resolve add_robot duplication: replace guide with checklist + link to tutorial - Add "See Also" / "Next Steps" sections across RL tutorial, agents, online data, create_scene, basic_env, RL overview, gym/env overview - Add custom functors guide (function-style and class-style patterns) - Add configuration guide (@configclass, FunctorCfg, JSON config) - Fix guides/index.rst title underline Co-Authored-By: Claude Opus 4.6 --- docs/source/features/agents.md | 9 + docs/source/features/online_data.md | 8 + docs/source/guides/add_robot.rst | 599 ++------------------------ docs/source/guides/configuration.md | 293 +++++++++++++ docs/source/guides/custom_functors.md | 390 +++++++++++++++++ docs/source/guides/index.rst | 6 +- docs/source/index.rst | 5 +- docs/source/overview/gym/env.md | 2 + docs/source/overview/rl/index.rst | 8 + docs/source/resources/task/index.rst | 1 - docs/source/tutorial/basic_env.rst | 8 + docs/source/tutorial/create_scene.rst | 9 + docs/source/tutorial/index.rst | 31 +- docs/source/tutorial/modular_env.rst | 2 +- docs/source/tutorial/rl.rst | 8 + 15 files changed, 815 insertions(+), 564 deletions(-) create mode 100644 docs/source/guides/configuration.md create mode 100644 docs/source/guides/custom_functors.md diff --git a/docs/source/features/agents.md b/docs/source/features/agents.md index 7cb2356d..89602c93 100644 --- a/docs/source/features/agents.md +++ b/docs/source/features/agents.md @@ -164,3 +164,12 @@ embodichain/agents/ │ └── prompt/ # Prompt templates (LangChain) └── prompts/ # Agent prompt templates ``` + +--- + +## See Also + +- [Online Data Streaming](online_data.md) — Streaming live simulation data for training +- [RL Architecture](../overview/rl/index.rst) — RL training pipeline and algorithms +- [Atomic Actions Tutorial](../tutorial/atomic_actions.rst) — Action primitives used by the CodeAgent +- [Supported Tasks](../resources/task/index.rst) — Available task environments diff --git a/docs/source/features/online_data.md b/docs/source/features/online_data.md index c186aef6..dccd38d1 100644 --- a/docs/source/features/online_data.md +++ b/docs/source/features/online_data.md @@ -143,3 +143,11 @@ It shows item mode, batch mode, and dynamic chunk sizes. Run it with: ```bash python examples/agents/datasets/online_dataset_demo.py ``` + +--- + +## See Also + +- [EmbodiAgent](agents.md) — Hierarchical agent that uses online data for training +- [RL Architecture](../overview/rl/index.rst) — RL training pipeline +- [Data Generation Tutorial](../tutorial/data_generation.rst) — Generating offline datasets diff --git a/docs/source/guides/add_robot.rst b/docs/source/guides/add_robot.rst index 5110fcc0..f437fd0b 100644 --- a/docs/source/guides/add_robot.rst +++ b/docs/source/guides/add_robot.rst @@ -1,571 +1,54 @@ -.. _tutorial_add_robot: +.. _guide_add_robot: -Adding a New Robot -================== +Adding a New Robot — Quick Reference +===================================== -.. currentmodule:: embodichain.lab.sim.robots +This guide provides a checklist and key reference for adding a new robot to EmbodiChain. For the full step-by-step walkthrough with code examples, see :doc:`/tutorial/add_robot`. -This tutorial guides you through adding a new robot to EmbodiChain. You'll learn the file structure, key components, and patterns used for robot definitions. +Checklist +--------- -EmbodiChain supports two approaches for defining robots: +1. **Prepare the URDF** — Place your URDF file (and associated meshes) in the robot assets directory. +2. **Create the config class** — Inherit from ``RobotCfg``, implement ``from_dict`` and ``_build_default_cfgs``. +3. **Define control parts** — Group joints into logical sets (e.g., ``arm``, ``gripper``). +4. **Configure IK solver** — Choose ``OPWSolverCfg``, ``SRSSolverCfg``, or a generic ``SolverCfg``. +5. **Set drive properties** — Configure stiffness, damping, and max effort per joint group. +6. **Implement** ``build_pk_serial_chain`` — Required for PyTorch-Kinematics IK support. +7. **Register in** ``embodichain/lab/sim/robots/__init__.py``. +8. **Add documentation** — Create ``docs/source/resources/robot/my_robot.md`` and update ``resources/robot/index.rst``. +9. **Test** — Add a ``__main__`` block or use the ``preview-asset`` CLI to verify. -1. **Single-file approach**: For simpler robots (like ``CobotMagic``) -2. **Package approach**: For complex robots with multiple variants (like ``DexforceW1``) +Approaches +---------- -Choose the approach based on your robot's complexity. +- **Single-file** (simple robots): One ``my_robot.py`` with everything. +- **Package** (complex robots): Directory with ``types.py``, ``params.py``, ``utils.py``, ``cfg.py``, ``__init__.py``. ---- - -Prerequisites -~~~~~~~~~~~~~~ - -Before adding a new robot, ensure you have: - -- URDF file(s) for your robot -- Robot's kinematic parameters (DH parameters or joint limits) -- Understanding of your robot's joint structure and control parts - ---- - -Approach 1: Single-File Robot (Simple Robots) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Use this approach for robots with a single variant and straightforward configuration. - -File: ``embodichain/lab/sim/robots/my_robot.py`` - -.. dropdown:: Complete Example: CobotMagic-style Robot - :icon: code - - .. literalinclude:: ../../../embodichain/lab/sim/robots/cobotmagic.py - :language: python - :linenos: - -Step-by-Step Guide ------------------- - -1. **Create the configuration class** inheriting from ``RobotCfg``: - - .. code-block:: python - - from __future__ import annotations - - from typing import Dict, List, Any - import numpy as np - - from embodichain.lab.sim.cfg import ( - RobotCfg, - URDFCfg, - JointDrivePropertiesCfg, - RigidBodyAttributesCfg, - ) - from embodichain.lab.sim.solvers import SolverCfg, OPWSolverCfg - from embodichain.lab.sim.utility.cfg_utils import merge_robot_cfg - from embodichain.data import get_data_path - from embodichain.utils import configclass - - @configclass - class MyRobotCfg(RobotCfg): - urdf_cfg: URDFCfg = None - control_parts: Dict[str, List[str]] | None = None - solver_cfg: Dict[str, "SolverCfg"] | None = None - -2. **Implement the ``from_dict`` class method** for flexible initialization: - - .. code-block:: python - - @classmethod - def from_dict(cls, init_dict: Dict[str, Any]) -> "MyRobotCfg": - cfg = cls() - default_cfgs = cls()._build_default_cfgs() - for key, value in default_cfgs.items(): - setattr(cfg, key, value) - cfg = merge_robot_cfg(cfg, init_dict) - return cfg - -3. **Define ``_build_default_cfgs``** with your robot's defaults: - - .. code-block:: python - - @staticmethod - def _build_default_cfgs() -> Dict[str, Any]: - # URDF path - urdf_path = get_data_path("MyRobot/my_robot.urdf") - - # URDF configuration (for multi-component robots) - urdf_cfg = URDFCfg( - components=[ - { - "component_type": "arm", - "urdf_path": urdf_path, - "transform": np.eye(4), # 4x4 transform matrix - }, - ] - ) - - # Control parts - group joints for control - control_parts = { - "arm": [ - "JOINT1", "JOINT2", "JOINT3", - "JOINT4", "JOINT5", "JOINT6", - ], - "gripper": ["JOINT7", "JOINT8"], - } - - # Solver configuration for IK - solver_cfg = { - "arm": OPWSolverCfg( - end_link_name="link6", - root_link_name="base_link", - tcp=np.array([...]), # Tool center point transform - ), - } - - # Drive properties - joint physics parameters - drive_pros = JointDrivePropertiesCfg( - stiffness={ - "JOINT[1-6]": 7e4, # Regex pattern for joints 1-6 - "JOINT[7-8]": 3e2, - }, - damping={ - "JOINT[1-6]": 1e3, - "JOINT[7-8]": 3e1, - }, - max_effort={ - "JOINT[1-6]": 3e6, - "JOINT[7-8]": 3e3, - }, - ) - - return { - "uid": "MyRobot", - "urdf_cfg": urdf_cfg, - "control_parts": control_parts, - "solver_cfg": solver_cfg, - "drive_pros": drive_pros, - "attrs": RigidBodyAttributesCfg( - mass=0.1, - static_friction=0.95, - dynamic_friction=0.9, - linear_damping=0.7, - angular_damping=0.7, - ), - } - -4. **Implement ``build_pk_serial_chain``** for PyTorch-Kinematics: - - .. code-block:: python - - def build_pk_serial_chain( - self, device: torch.device = torch.device("cpu"), **kwargs - ) -> Dict[str, "pk.SerialChain"]: - from embodichain.lab.sim.utility.solver_utils import ( - create_pk_chain, - create_pk_serial_chain, - ) - - urdf_path = get_data_path("MyRobot/my_robot.urdf") - chain = create_pk_chain(urdf_path, device) - - arm_chain = create_pk_serial_chain( - chain=chain, - end_link_name="link6", - root_link_name="base_link" - ).to(device=device) - - return {"arm": arm_chain} - -5. **Register in** ``embodichain/lab/sim/robots/__init__.py``: - - .. code-block:: python - - from .my_robot import MyRobotCfg - ---- - -Approach 2: Package-Based Robot (Complex Robots) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Use this approach for robots with multiple variants (e.g., different arm types, versions, or configurations). - -File Structure +Key Parameters -------------- -For complex robots, create a package directory: - -.. code-block:: - - robots/ - └── my_robot/ - ├── __init__.py # Exports the main config class - ├── types.py # Enums for robot variants - ├── params.py # Kinematics parameters - ├── utils.py # Manager classes and builders - └── cfg.py # Main configuration class - -Step-by-Step Guide ------------------ - -1. **types.py** - Define enums for robot variants: - - .. code-block:: python - - from enum import Enum - - class MyRobotVersion(Enum): - V010 = "v010" - V020 = "v020" - - class MyRobotArmKind(Enum): - STANDARD = "standard" - EXTENDED = "extended" - - class MyRobotSide(Enum): - LEFT = "left" - RIGHT = "right" - -2. **params.py** - Define kinematics parameters: - - .. code-block:: python - - from dataclasses import dataclass - import numpy as np - from typing import Optional - - @dataclass - class MyRobotArmKineParams: - arm_side: MyRobotSide - arm_kind: MyRobotArmKind - version: MyRobotVersion - - dh_params: np.ndarray = None # DH parameters (N x 4) - qpos_limits: np.ndarray = None # Joint limits (N x 2) - link_lengths: np.ndarray = None # Link lengths - T_b_ob: np.ndarray = None # Base to origin transform - T_e_oe: np.ndarray = None # End-effector transform - -3. **utils.py** - Manager classes and builder functions: - - .. code-block:: python - - class ArmManager: - """Manages arm URDF and configuration.""" - pass - - def build_my_robot_assembly_urdf_cfg(...): - """Build URDF assembly from components.""" - pass - - def build_my_robot_cfg(...): - """Build complete robot configuration.""" - pass - -4. **cfg.py** - Main configuration class: - - .. code-block:: python - - @configclass - class MyRobotCfg(RobotCfg): - version: MyRobotVersion = MyRobotVersion.V010 - arm_kind: MyRobotArmKind = MyRobotArmKind.STANDARD - - @classmethod - def from_dict(cls, init_dict: Dict) -> "MyRobotCfg": - # Implementation similar to single-file approach - pass - -5. **__init__.py** - Export the config: - - .. code-block:: python - - from .cfg import MyRobotCfg - -6. **Register in** ``robots/__init__.py``: - - .. code-block:: python - - from .my_robot import * - ---- - -Key Configuration Parameters -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Regardless of the approach, your robot config needs these core parameters: - -+---------------------+------------------------+----------------------------------+ -| Parameter | Type | Description | -+=====================+========================+==================================+ -| ``uid`` | str | Unique robot identifier | -+---------------------+------------------------+----------------------------------+ -| ``urdf_cfg`` | URDFCfg | URDF file and components | -+---------------------+------------------------+----------------------------------+ -| ``control_parts`` | Dict[str, List[str]] | Joint groups for control | -+---------------------+------------------------+----------------------------------+ -| ``solver_cfg`` | Dict[str, SolverCfg] | IK solver configurations | -+---------------------+------------------------+----------------------------------+ -| ``drive_pros`` | JointDrivePropertiesCfg | Joint stiffness, damping, force | -+---------------------+------------------------+----------------------------------+ -| ``attrs`` | RigidBodyAttributesCfg | Mass, friction, damping | -+---------------------+------------------------+----------------------------------+ - -URDF Configuration ------------------ ++---------------------+----------------------------+----------------------------------+ +| Parameter | Type | Description | ++=====================+============================+==================================+ +| ``uid`` | str | Unique robot identifier | ++---------------------+----------------------------+----------------------------------+ +| ``urdf_cfg`` | URDFCfg | URDF file and components | ++---------------------+----------------------------+----------------------------------+ +| ``control_parts`` | Dict[str, List[str]] | Joint groups for control | ++---------------------+----------------------------+----------------------------------+ +| ``solver_cfg`` | Dict[str, SolverCfg] | IK solver configurations | ++---------------------+----------------------------+----------------------------------+ +| ``drive_pros`` | JointDrivePropertiesCfg | Joint stiffness, damping, force | ++---------------------+----------------------------+----------------------------------+ -The ``URDFCfg`` allows composing robots from multiple URDF files: - -.. code-block:: python - - urdf_cfg = URDFCfg( - components=[ - { - "component_type": "arm", - "urdf_path": arm_urdf, - "transform": np.eye(4), - }, - { - "component_type": "gripper", - "urdf_path": gripper_urdf, - "transform": gripper_transform, - }, - ] - ) - -Control Parts -------------- - -Group joints logically for different control modes: - -.. code-block:: python - - control_parts = { - "arm": ["JOINT1", "JOINT2", "JOINT3", "JOINT4", "JOINT5", "JOINT6"], - "gripper": ["JOINT7", "JOINT8"], - } - -Use regex patterns for flexible matching: -- ``"JOINT[1-6]"`` matches JOINT1 through JOINT6 -- ``"(LEFT|RIGHT)_ARM.*"`` matches all arm joints - -Drive Properties ----------------- - -Configure joint physics behavior: - -.. code-block:: python - - drive_pros = JointDrivePropertiesCfg( - stiffness={ - "ARM_JOINTS": 1e4, # High stiffness for arm joints - "GRIPPER_JOINTS": 3e2, # Lower stiffness for gripper - }, - damping={ - "ARM_JOINTS": 1e3, - "GRIPPER_JOINTS": 3e1, - }, - max_effort={ - "ARM_JOINTS": 1e5, - "GRIPPER_JOINTS": 1e3, - }, - ) - -IK Solver Configuration ------------------------ - -Choose the appropriate solver for your robot: - -- **OPWSolverCfg**: For 6-axis industrial arms (like CobotMagic) -- **SRSSolverCfg**: For robots with specific kinematics (like DexforceW1) -- **SolverCfg**: Generic solver configuration - -.. code-block:: python - - solver_cfg = { - "arm": OPWSolverCfg( - end_link_name="link6", - root_link_name="base_link", - tcp=np.array([...]), # Tool center point - ), - } - ---- - -Using Your Robot -~~~~~~~~~~~~~~~~ - -After adding the robot, use it in your code: - -.. code-block:: python - - from embodichain.lab.sim import SimulationManager, SimulationManagerCfg - from embodichain.lab.sim.robots import MyRobotCfg - - # Create simulation - sim_cfg = SimulationManagerCfg(headless=False, num_envs=2) - sim = SimulationManager(sim_cfg) - - # Create robot config - robot_cfg = MyRobotCfg.from_dict({ - "uid": "my_robot", - }) - - # Add robot to simulation - robot = sim.add_robot(cfg=robot_cfg) - ---- - -Testing Your Robot -~~~~~~~~~~~~~~~~~~ - -Add a test block at the bottom of your robot config file: - -.. code-block:: python - - if __name__ == "__main__": - from embodichain.lab.sim import SimulationManager, SimulationManagerCfg - - sim_cfg = SimulationManagerCfg(headless=True, num_envs=2) - sim = SimulationManager(sim_cfg) - - robot_cfg = MyRobotCfg.from_dict({"uid": "my_robot"}) - robot = sim.add_robot(cfg=robot_cfg) - - print("Robot added successfully!") - ---- - -Best Practices -~~~~~~~~~~~~~~ - -1. **Use the** ``@configclass`` **decorator** for all config classes -2. **Provide** ``from_dict`` **method** for flexible initialization -3. **Use regex patterns** for joint names in drive properties -4. **Keep kinematics parameters** separate in ``params.py`` for complex robots -5. **Include** ``build_pk_serial_chain`` **method** for IK support -6. **Add** ``to_dict`` **and** ``save_to_file`` **methods** for serialization -7. **Test with** ``__main__`` **block** before integrating -8. **Add robot documentation** in ``docs/source/resources/robot/`` for user reference - ---- - -Adding Robot Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When adding a new robot, create documentation in ``docs/source/resources/robot/`` to help users understand and use your robot. - -File Location -------------- - -Create a markdown file: ``docs/source/resources/robot/my_robot.md`` - -Recommended Structure ---------------------- - -.. code-block:: markdown - - # MyRobot - - Brief description of the robot and its manufacturer. - -
- MyRobot -

MyRobot

-
- - ## Key Features - - - Feature 1 - - Feature 2 - - Feature 3 - - --- - - ## Robot Parameters - - | Parameter | Description | - |-----------|-------------| - | Joints | Number of joints | - | DOF | Degrees of freedom | - | ... | ... | - - --- - - ## Quick Initialization Example - - ```python - from embodichain.lab.sim import SimulationManager, SimulationManagerCfg - from embodichain.lab.sim.robots import MyRobotCfg - - config = SimulationManagerCfg(headless=False, sim_device="cpu", num_envs=2) - sim = SimulationManager(config) - - robot = sim.add_robot(cfg=MyRobotCfg.from_dict({})) - ``` - - --- - - ## Configuration Parameters - - ### Main Configuration Items - - - **uid**: Unique identifier - - **urdf_cfg**: URDF configuration - - **control_parts**: Control groups - - **solver_cfg**: IK solver configuration - - **drive_pros**: Joint drive properties - - **attrs**: Physical attributes - - ### Custom Usage Example - - ```python - custom_cfg = { - "uid": "my_robot", - # Add parameters - } - cfg = MyRobotCfg.from_dict(custom_cfg) - robot = sim.add_robot(cfg=cfg) - ``` - - --- - - ## References - - - Manufacturer product page - - URDF file paths - - Related documentation - -Register the Robot in Index ---------------------------- - -After creating the robot documentation, add it to the index file at ``docs/source/resources/robot/index.rst``: - -.. code-block:: rst - - .. toctree:: - :maxdepth: 1 - - Dexforce W1 - CobotMagic - MyRobot # Add your robot here - ---- - -Next Steps -~~~~~~~~~~ - -After adding your robot: +.. tip:: -- Add robot documentation in ``docs/source/resources/robot/`` -- Update ``docs/source/resources/robot/index.rst`` to include the new robot -- Add task environments that use your robot -- Configure sensors (cameras, force sensors) -- Implement custom IK solvers if needed -- Add motion planning support + See the :doc:`full tutorial ` for complete code examples of both approaches. -.. tip:: - **Using an AI coding agent?** These skills can help when extending your robot: +See Also +-------- - - **/add-task-env** — Scaffold a task environment that uses your new robot. - - **/add-functor** — Add observation, reward, or randomization functors for robot-specific tasks. - - **/add-test** — Write tests for your robot config or task environment. - - **/pre-commit-check** — Verify all code passes CI checks before committing. +- :doc:`/tutorial/add_robot` — Full step-by-step tutorial +- :doc:`/tutorial/robot` — Using robots in simulation +- :doc:`/overview/sim/solvers/index` — IK solver reference +- :doc:`/resources/robot/index` — Existing robot documentation diff --git a/docs/source/guides/configuration.md b/docs/source/guides/configuration.md new file mode 100644 index 00000000..c031b891 --- /dev/null +++ b/docs/source/guides/configuration.md @@ -0,0 +1,293 @@ +# Configuration Guide + +EmbodiChain uses a declarative configuration system built on Python dataclasses. This guide explains the key patterns: `@configclass`, `FunctorCfg`, and JSON configuration files. + +--- + +## The `@configclass` Decorator + +All configuration objects use the `@configclass` decorator, which is similar to Python's `@dataclass` with additional validation and serialization support. + +```python +from embodichain.utils import configclass +from dataclasses import MISSING + + +@configclass +class MyManagerCfg: + param_a: float = 1.0 + param_b: str = MISSING # Required — must be set by caller + param_c: int = 10 +``` + +- **Optional parameters** have default values. +- **Required parameters** use `MISSING` as the default — callers must provide them. +- All parameters are typed for IDE auto-completion and static analysis. + +--- + +## Configuration Hierarchy + +EmbodiChain configs form a nested hierarchy: + +``` +EmbodiedEnvCfg +├── sim_cfg: SimulationManagerCfg +│ ├── render_cfg: RenderCfg +│ ├── physics_config: PhysicsCfg +│ └── gpu_memory_config: GPUMemoryCfg +├── robot: RobotCfg +│ ├── urdf_cfg: URDFCfg +│ ├── drive_pros: JointDrivePropertiesCfg +│ └── solver_cfg: Dict[str, SolverCfg] +├── sensor: List[SensorCfg] +├── events: EventCfg +├── observations: ObservationCfg +├── rewards: RewardCfg +├── actions: ActionTermCfg +├── dataset: DatasetFunctorCfg +└── extensions: Dict[str, Any] +``` + +Each sub-config can be set independently, allowing fine-grained control over the environment. + +--- + +## Functor Configuration + +Functors are configured through specialized config classes that inherit from `FunctorCfg`. The base class has three fields: + +```python +@configclass +class FunctorCfg: + func: Callable | Functor = MISSING # The function or class to call + params: dict[str, Any] = dict() # Keyword arguments + extra: dict[str, Any] = dict() # Optional metadata +``` + +### Specialized Config Classes + +| Config Class | Extra Fields | Used By | +|---|---|---| +| `ObservationCfg` | `mode`, `name` | ObservationManager | +| `EventCfg` | `mode`, `interval_step`, `is_global` | EventManager | +| `RewardCfg` | `weight`, `mode` | RewardManager | +| `ActionTermCfg` | `mode` | ActionManager | +| `DatasetFunctorCfg` | `mode` | DatasetManager | + +### Python Config Example + +```python +from embodichain.utils import configclass +from embodichain.lab.gym.envs.managers.cfg import ( + ObservationCfg, + RewardCfg, + EventCfg, + SceneEntityCfg, +) +from embodichain.lab.gym.envs.managers.observations import get_object_pose + + +@configclass +class MyObsCfg: + object_pose: ObservationCfg = ObservationCfg( + func=get_object_pose, + mode="add", + name="object/pose", + params={"entity_cfg": SceneEntityCfg(uid="my_cube")}, + ) + + +@configclass +class MyRewardCfg: + distance: RewardCfg = RewardCfg( + func="distance_between_objects", + weight=0.5, + params={ + "source_entity_cfg": SceneEntityCfg(uid="cube"), + "target_entity_cfg": SceneEntityCfg(uid="target"), + }, + ) + + +@configclass +class MyEventCfg: + randomize_light: EventCfg = EventCfg( + func="randomize_light", + mode="interval", + interval_step=5, + params={"light_uid": "main_light"}, + ) +``` + +--- + +## JSON Configuration + +For RL training and data generation, EmbodiChain uses JSON config files. The JSON config mirrors the Python config structure but uses string names instead of direct function references. + +### Environment Config (`gym_config.json`) + +```json +{ + "max_episodes": 100, + "max_episode_steps": 600, + "env": { + "num_envs": 4, + "sim_cfg": { + "sim_device": "cuda:0", + "headless": true + }, + "robot": { + "uid": "robot", + "urdf_cfg": {"fpath": "robots/my_robot/my_robot.urdf"} + }, + "control_parts": ["arm"], + "sensor": [ + { + "uid": "cam_high", + "type": "StereoCamera", + "height": 540, + "width": 960 + } + ], + "actions": { + "delta_qpos": { + "func": "DeltaQposTerm", + "params": {"scale": 0.1} + } + }, + "events": { + "randomize_table": { + "func": "randomize_visual_material", + "mode": "interval", + "interval_step": 10, + "params": {"uid": "table"} + } + }, + "observations": { + "obj_pose": { + "func": "get_object_pose", + "mode": "add", + "name": "object/pose", + "params": {"entity_cfg": {"uid": "cube"}} + } + }, + "rewards": { + "distance": { + "func": "distance_between_objects", + "weight": 0.5, + "params": { + "source_entity_cfg": {"uid": "cube"}, + "target_entity_cfg": {"uid": "target"} + } + } + }, + "dataset": { + "lerobot": { + "func": "LeRobotRecorder", + "mode": "save", + "params": { + "save_path": "/path/to/output", + "robot_meta": {"robot_type": "DexforceW1"}, + "use_videos": true + } + } + }, + "extensions": { + "success_threshold": 0.1 + } + } +} +``` + +### RL Training Config (`train_config.json`) + +```json +{ + "trainer": { + "exp_name": "push_cube", + "seed": 42, + "device": "cuda:0", + "iterations": 500, + "buffer_size": 1024 + }, + "env": { + "id": "PushCubeRL", + "cfg": { + "num_envs": 4, + "actions": { + "delta_qpos": { + "func": "DeltaQposTerm", + "params": {"scale": 0.1} + } + } + } + }, + "policy": { + "name": "actor_critic", + "actor": { + "type": "mlp", + "network_cfg": {"hidden_sizes": [256, 256], "activation": "relu"} + }, + "critic": { + "type": "mlp", + "network_cfg": {"hidden_sizes": [256, 256], "activation": "relu"} + } + }, + "algorithm": { + "name": "ppo", + "cfg": { + "learning_rate": 0.0001, + "n_epochs": 10, + "batch_size": 64, + "gamma": 0.99, + "gae_lambda": 0.95, + "clip_coef": 0.2 + } + } +} +``` + +--- + +## String-Based Function Resolution + +In JSON configs, functor functions are specified by name (string). EmbodiChain resolves these strings at runtime by searching registered modules. For example: + +- `"distance_between_objects"` resolves to `embodichain.lab.gym.envs.managers.rewards.distance_between_objects` +- `"DeltaQposTerm"` resolves to `embodichain.lab.gym.envs.managers.actions.DeltaQposTerm` +- `"get_object_pose"` resolves to `embodichain.lab.gym.envs.managers.observations.get_object_pose` + +When writing custom functors, make sure they are imported in the module's `__init__.py` so the resolver can find them. + +--- + +## `SceneEntityCfg` in JSON + +When referencing scene entities in JSON, use a dictionary with a `uid` key: + +```json +{"uid": "my_cube"} +``` + +This is automatically converted to a `SceneEntityCfg` object at runtime. + +--- + +## Tips + +1. **Start from an existing config.** Copy a config file from `configs/gym/` and modify it for your task. +2. **Use Python configs for development.** They provide IDE auto-completion and type checking. +3. **Use JSON configs for experiments.** They are easier to version, diff, and share. +4. **Validate configs early.** Run your environment with a short episode count to catch config errors before long training runs. +5. **Keep config pairs together.** For action-bank tasks, version `gym_config.json` and `action_config.json` together. + +--- + +## See Also + +- [Custom Functors Guide](custom_functors.md) — How to write observation, reward, event, and action functors +- [Embodied Environments](../overview/gym/env.md) — Full environment configuration reference +- [Tutorial: Modular Environment](../tutorial/modular_env.rst) — Complete example using config-driven setup +- [Tutorial: RL Training](../tutorial/rl.rst) — RL training configuration walkthrough diff --git a/docs/source/guides/custom_functors.md b/docs/source/guides/custom_functors.md new file mode 100644 index 00000000..383754f1 --- /dev/null +++ b/docs/source/guides/custom_functors.md @@ -0,0 +1,390 @@ +# Writing Custom Functors + +Functors are the building blocks of EmbodiChain's manager system. They define how observations are computed, rewards are calculated, events are triggered, actions are preprocessed, and datasets are recorded. + +This guide explains the two functor styles (function and class), how to register them in manager configs, and provides examples for each functor type. + +--- + +## Functor Basics + +Every functor is configured through a `FunctorCfg` object with three fields: + +| Field | Type | Description | +|-------|------|-------------| +| `func` | `Callable \| Functor` | The function or class to call. **Required.** | +| `params` | `dict` | Keyword arguments passed to the function. | +| `extra` | `dict` | Optional metadata (e.g., observation shapes). | + +The `func` field can be: +- A **function** (callable) — receives the environment as the first argument, plus any `params` as keyword arguments. +- A **class** inheriting from `Functor` — instantiated with `(cfg, env)`, then called via `__call__`. + +--- + +## Function-Style Functors + +Function-style functors are plain Python functions. They are stateless and easy to write. Use them when your functor is a simple computation that doesn't need to maintain state between calls. + +### General Pattern + +```python +def my_functor(env, obs, **kwargs) -> torch.Tensor: + """Compute something from the environment state. + + Args: + env: The environment instance. + obs: The current observation dictionary. + **kwargs: Additional parameters from FunctorCfg.params. + + Returns: + A tensor of shape (num_envs, ...). + """ + # Access environment state + value = compute_value(env) + + return value +``` + +The exact signature depends on the functor type (see below). + +### Example: Observation Functor + +Observation functors receive `(env, obs)` plus any params. They must return a tensor. + +```python +from __future__ import annotations +import torch +from embodichain.lab.gym.envs import EmbodiedEnv +from embodichain.lab.gym.envs.managers.observations import EnvObs +from embodichain.lab.sim.cfg import SceneEntityCfg + + +def get_object_height( + env: EmbodiedEnv, + obs: EnvObs, + entity_cfg: SceneEntityCfg, +) -> torch.Tensor: + """Get the Z-coordinate (height) of an object. + + Args: + env: The environment instance. + obs: The current observation dictionary. + entity_cfg: Scene entity configuration with the object UID. + + Returns: + Tensor of shape (num_envs, 1) with the object height. + """ + obj = env.sim.get_rigid_object(entity_cfg.uid) + pose = obj.get_local_pose(to_matrix=True) # (num_envs, 4, 4) + height = pose[:, 2, 3:4] # Extract Z from translation + return height +``` + +Register it in your environment config: + +```python +from embodichain.lab.gym.envs.managers.cfg import ObservationCfg, SceneEntityCfg +from embodichain.utils import configclass + + +@configclass +class MyObsCfg: + obj_height: ObservationCfg = ObservationCfg( + func=get_object_height, + mode="add", + name="object/height", + params={"entity_cfg": SceneEntityCfg(uid="my_cube")}, + ) +``` + +Or in JSON: + +```json +"observations": { + "obj_height": { + "func": "get_object_height", + "mode": "add", + "name": "object/height", + "params": {"entity_cfg": {"uid": "my_cube"}} + } +} +``` + +### Example: Reward Functor + +Reward functors receive `(env, obs, action, info)` plus any params. They return a tensor of shape `(num_envs,)`. + +```python +import torch +from embodichain.lab.gym.envs import EmbodiedEnv +from embodichain.lab.sim.cfg import SceneEntityCfg + + +def target_height_reward( + env: EmbodiedEnv, + obs: dict, + action, + info: dict, + entity_cfg: SceneEntityCfg = None, + target_height: float = 0.5, +) -> torch.Tensor: + """Reward for lifting an object to a target height. + + Returns: + Negative distance to the target height. Shape (num_envs,). + """ + obj = env.sim.get_rigid_object(entity_cfg.uid) + pose = obj.get_local_pose(to_matrix=True) + current_height = pose[:, 2, 3] + return -torch.abs(current_height - target_height) +``` + +Register it: + +```python +from embodichain.lab.gym.envs.managers.cfg import RewardCfg +from embodichain.utils import configclass + + +@configclass +class MyRewardCfg: + lift_reward: RewardCfg = RewardCfg( + func=target_height_reward, + weight=1.0, + params={ + "entity_cfg": SceneEntityCfg(uid="my_cube"), + "target_height": 0.5, + }, + ) +``` + +--- + +## Class-Style Functors + +Class-style functors inherit from `Functor` and implement `__init__(cfg, env)` and `__call__(...)`. Use them when you need to: + +- Maintain state across calls (e.g., caching, counters) +- Perform expensive initialization once +- Implement a `reset()` method for per-episode cleanup + +### General Pattern + +```python +from embodichain.lab.gym.envs.managers import Functor +from embodichain.lab.gym.envs.managers.cfg import FunctorCfg + + +class MyFunctor(Functor): + """A stateful functor.""" + + def __init__(self, cfg: FunctorCfg, env): + super().__init__(cfg, env) + # Initialize state, buffers, etc. + self._counter = 0 + + def reset(self, env_ids=None): + """Called on environment reset.""" + self._counter = 0 + + def __call__(self, env, obs, **kwargs): + """Called every step.""" + self._counter += 1 + # Compute and return result +``` + +### Example: Observation Functor with Caching + +```python +from __future__ import annotations +import torch +from embodichain.lab.gym.envs import EmbodiedEnv +from embodichain.lab.gym.envs.managers import Functor +from embodichain.lab.gym.envs.managers.cfg import FunctorCfg, ObservationCfg +from embodichain.lab.sim.cfg import SceneEntityCfg + + +class get_object_mass(Functor): + """Get the mass of a rigid object, with caching. + + Caches the result to avoid repeated queries to the physics engine. + Cache is cleared on environment reset. + """ + + def __init__(self, cfg: FunctorCfg, env: EmbodiedEnv): + super().__init__(cfg, env) + self._cache = {} + + def reset(self, env_ids=None): + self._cache.clear() + + def __call__( + self, + env: EmbodiedEnv, + obs, + entity_cfg: SceneEntityCfg, + ) -> torch.Tensor: + uid = entity_cfg.uid + if uid in self._cache: + return self._cache[uid].clone() + + obj = env.sim.get_rigid_object(uid) + mass = obj.get_mass() # (num_envs, 1) + + self._cache[uid] = mass.clone() + return mass +``` + +### Example: Action Functor + +Action functors inherit from `ActionTerm` and implement `process_action`. They transform raw policy actions into robot control commands. + +```python +from __future__ import annotations +import torch +from embodichain.lab.gym.envs.managers.actions import ActionTerm +from embodichain.lab.gym.envs.managers.cfg import ActionTermCfg + + +class DeltaQposTerm(ActionTerm): + """Delta joint position: current_qpos + scale * action -> target qpos. + + The policy outputs a position offset, which is added to the current + joint positions to get the target. + """ + + def __init__(self, cfg: ActionTermCfg, env): + super().__init__(cfg, env) + self._scale = cfg.params.get("scale", 1.0) + + @property + def input_key(self) -> str: + return "qpos" + + @property + def action_dim(self) -> int: + return len(self._env.active_joint_ids) + + def process_action(self, action: torch.Tensor) -> torch.Tensor: + return action * self._scale + self._env.robot.get_qpos() +``` + +Register it in JSON config: + +```json +"actions": { + "delta_qpos": { + "func": "DeltaQposTerm", + "params": {"scale": 0.1} + } +} +``` + +--- + +## Functor Signature Reference + +Each functor type has a specific call signature: + +### Observation Functors + +```python +def my_obs_functor(env, obs, **params) -> torch.Tensor +``` + +- `env`: The environment instance. +- `obs`: The current observation dictionary. +- Additional params from `ObservationCfg.params`. +- Returns: tensor of shape `(num_envs, ...)`. + +Config class: `ObservationCfg` with `mode` (`"add"` or `"modify"`) and `name`. + +### Reward Functors + +```python +def my_reward_functor(env, obs, action, info, **params) -> torch.Tensor +``` + +- `env`: The environment instance. +- `obs`: The current observation dictionary. +- `action`: The action taken this step. +- `info`: The info dictionary. +- Additional params from `RewardCfg.params`. +- Returns: tensor of shape `(num_envs,)`. + +Config class: `RewardCfg` with `weight` and `mode` (`"add"` or `"replace"`). + +### Event Functors + +```python +def my_event_functor(env, env_ids, **params) -> None +``` + +- `env`: The environment instance. +- `env_ids`: The environment IDs affected by this event. +- Additional params from `EventCfg.params`. +- Returns: `None` (events modify the environment in-place). + +Config class: `EventCfg` with `mode` (`"startup"`, `"reset"`, or `"interval"`) and `interval_step`. + +### Action Functors + +```python +class MyActionTerm(ActionTerm): + def process_action(self, action: torch.Tensor) -> torch.Tensor +``` + +- `action`: Raw action from the policy, shape `(num_envs, action_dim)`. +- Returns: transformed action tensor. + +Config class: `ActionTermCfg` with `mode` (`"pre"` or `"post"`). + +### Dataset Functors + +Dataset functors handle recording and saving. In most cases you should use the built-in `LeRobotRecorder` rather than writing a custom one. + +Config class: `DatasetFunctorCfg` with `mode` (`"save"`). + +--- + +## Using `SceneEntityCfg` in Params + +Many functors need to reference scene objects (robots, rigid objects, sensors). Instead of passing string UIDs directly, use `SceneEntityCfg`: + +```python +from embodichain.lab.sim.cfg import SceneEntityCfg + +params = { + "entity_cfg": SceneEntityCfg(uid="my_cube"), +} +``` + +The manager automatically resolves `SceneEntityCfg` objects to the actual simulation entities at runtime. + +--- + +## File Placement + +| Functor Type | Recommended Location | +|---|---| +| Observation | `embodichain/lab/gym/envs/managers/observations.py` | +| Reward | `embodichain/lab/gym/envs/managers/rewards.py` | +| Event | `embodichain/lab/gym/envs/managers/events.py` or `embodichain/lab/gym/envs/managers/randomization/` | +| Action | `embodichain/lab/gym/envs/managers/actions.py` | +| Dataset | `embodichain/lab/gym/envs/managers/datasets.py` | + +For task-specific functors, place them in the task module file (e.g., alongside the task environment class). + +Remember to: +- Add the functor to `__all__` in the module. +- Add the Apache 2.0 license header. +- Use type annotations with `from __future__ import annotations`. + +--- + +## See Also + +- [Configuration Guide](configuration.md) — How to set up `@configclass` configs and JSON files +- [Embodied Environments](../overview/gym/env.md) — Full environment architecture +- [Tutorial: Modular Environment](../tutorial/modular_env.rst) — Using functors in a complete environment diff --git a/docs/source/guides/index.rst b/docs/source/guides/index.rst index e5c0f2de..f44ad5a0 100644 --- a/docs/source/guides/index.rst +++ b/docs/source/guides/index.rst @@ -1,10 +1,14 @@ How-to Guides -========= +============= + +Practical guides for common tasks in EmbodiChain. .. toctree:: :maxdepth: 1 :hidden: + custom_functors + configuration add_robot cli diff --git a/docs/source/index.rst b/docs/source/index.rst index c3a47f2f..4bae98ae 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,7 +1,9 @@ EmbodiChain Documentation ========================= -Welcome to the EmbodiChain! +EmbodiChain is a GPU-accelerated robotics simulation framework for embodied AI research. It provides tools for building generating and processing simulation assets and scenes, creating robot learning environments, generating expert demonstration data, training policies with imitation learning and reinforcement learning, and deploying models into real world. + +The framework is built on top of `DexSim `_, a high-performance physics and rendering engine, designed for Embodied AI research and production use. Table of Contents ================= @@ -59,4 +61,3 @@ Table of Contents :titlesonly: api_reference/index - diff --git a/docs/source/overview/gym/env.md b/docs/source/overview/gym/env.md index cb545b5c..88f44fb9 100644 --- a/docs/source/overview/gym/env.md +++ b/docs/source/overview/gym/env.md @@ -305,6 +305,8 @@ For a complete example of a modular environment setup, please refer to the {ref} - {ref}`tutorial_modular_env` - Advanced modular environment setup - {ref}`tutorial_rl` - Reinforcement learning training guide - {doc}`/api_reference/embodichain/embodichain.lab.gym.envs` - Complete API reference for EmbodiedEnv and configurations +- {doc}`/guides/custom_functors` - How to write custom functors +- {doc}`/guides/configuration` - Configuration system guide ```{toctree} :maxdepth: 1 diff --git a/docs/source/overview/rl/index.rst b/docs/source/overview/rl/index.rst index cac282f4..df2fd29e 100644 --- a/docs/source/overview/rl/index.rst +++ b/docs/source/overview/rl/index.rst @@ -79,3 +79,11 @@ See also config.md train_script.md multi_gpu.md + +See Also +-------- + +- :doc:`/tutorial/rl` — Step-by-step RL training tutorial +- :doc:`/overview/gym/env` — EmbodiedEnv configuration and Action Manager +- :doc:`/features/online_data` — Online data streaming pipeline +- :doc:`/resources/task/index` — Available RL task environments diff --git a/docs/source/resources/task/index.rst b/docs/source/resources/task/index.rst index 998f6614..1c65e7e1 100644 --- a/docs/source/resources/task/index.rst +++ b/docs/source/resources/task/index.rst @@ -6,6 +6,5 @@ Supported Tasks .. toctree:: :maxdepth: 1 - Push Cube Pour Water diff --git a/docs/source/tutorial/basic_env.rst b/docs/source/tutorial/basic_env.rst index 257cd47b..443fbe97 100644 --- a/docs/source/tutorial/basic_env.rst +++ b/docs/source/tutorial/basic_env.rst @@ -185,3 +185,11 @@ This tutorial showcases several important features of EmbodiChain environments: .. tip:: **Using an AI coding agent?** Once you're ready to create your own task environment, use the **/add-task-env** skill to scaffold the file with the correct structure, ``@register_env`` decorator, base class methods, and test stub. Use **/add-test** to write tests and **/pre-commit-check** to verify everything passes CI before committing. + +Next Steps +~~~~~~~~~~ + +- :doc:`modular_env` — Build advanced config-driven environments with ``EmbodiedEnv`` +- :doc:`rl` — Train RL agents with PPO or GRPO +- :doc:`/overview/gym/env` — Full environment architecture and manager reference +- :doc:`/guides/custom_functors` — Write custom observation, reward, and event functors diff --git a/docs/source/tutorial/create_scene.rst b/docs/source/tutorial/create_scene.rst index 244bd932..da13d5ec 100644 --- a/docs/source/tutorial/create_scene.rst +++ b/docs/source/tutorial/create_scene.rst @@ -89,3 +89,12 @@ You can also pass arguments to customize the simulation. For example, to run in python scripts/tutorials/sim/create_scene.py --headless --num_envs --device Now that we have a basic understanding of how to create a scene, let's move on to more advanced topics. + +Next Steps +~~~~~~~~~~ + +- :doc:`create_softbody` — Add deformable bodies to your scene +- :doc:`robot` — Load and control a robot +- :doc:`sensor` — Add cameras and capture sensor data +- :doc:`basic_env` — Create your first Gymnasium environment +- :doc:`/overview/sim/sim_manager` — Full SimulationManager API reference diff --git a/docs/source/tutorial/index.rst b/docs/source/tutorial/index.rst index 6e6ae292..33b95a8b 100644 --- a/docs/source/tutorial/index.rst +++ b/docs/source/tutorial/index.rst @@ -1,6 +1,36 @@ Tutorials ========= +These tutorials walk you through EmbodiChain step by step, from creating your first simulation scene to training RL agents. Each tutorial includes a complete runnable script and a line-by-line explanation. + +Suggested Learning Path +~~~~~~~~~~~~~~~~~~~~~~~ + +Follow the tutorials in this order for the best learning experience: + +**Phase 1: Simulation Basics** + +1. :doc:`create_scene` — Set up a simulation, add objects, and run the render loop. **Start here.** +2. :doc:`create_softbody` and :doc:`create_cloth` — Add deformable bodies to your scenes. +3. :doc:`rigid_object_group` — Manage collections of rigid objects efficiently. +4. :doc:`robot` — Load and control a robot in simulation. +5. :doc:`sensor` — Add cameras and capture RGB/depth/segmentation data. +6. :doc:`solver` — Configure IK solvers for end-effector control. +7. :doc:`motion_gen` — Generate smooth trajectories with motion planners. +8. :doc:`atomic_actions` — Use built-in action primitives (pick, place, move). +9. :doc:`gizmo` — Interactively control robots with on-screen gizmos. + +**Phase 2: Environments** + +10. :doc:`basic_env` — Create a simple Gymnasium environment with ``BaseEnv``. Prerequisite: Phase 1 basics. +11. :doc:`modular_env` — Build a config-driven environment with ``EmbodiedEnv``, managers, and randomization. Prerequisite: :doc:`basic_env`. +12. :doc:`data_generation` — Generate expert demonstration datasets for imitation learning. Prerequisite: :doc:`modular_env`. +13. :doc:`rl` — Train RL agents with PPO or GRPO. Prerequisite: :doc:`basic_env`. + +**Phase 3: Extending the Framework** + +14. :doc:`add_robot` — Add a new robot model to EmbodiChain. + .. toctree:: :maxdepth: 1 :hidden: @@ -20,4 +50,3 @@ Tutorials modular_env data_generation rl - diff --git a/docs/source/tutorial/modular_env.rst b/docs/source/tutorial/modular_env.rst index d155dab2..eef801c3 100644 --- a/docs/source/tutorial/modular_env.rst +++ b/docs/source/tutorial/modular_env.rst @@ -64,7 +64,7 @@ The ``randomize_table_mat`` event varies visual appearance: - **Mode**: ``"interval"`` - triggers every 10 steps - **Features**: Random textures from COCO dataset and base color variations -for more randomization events, please refer +For more randomization events, please refer to :doc:`/overview/gym/event_functors`. Observation Configuration ------------------------- diff --git a/docs/source/tutorial/rl.rst b/docs/source/tutorial/rl.rst index 28054648..db1c7ab1 100644 --- a/docs/source/tutorial/rl.rst +++ b/docs/source/tutorial/rl.rst @@ -420,3 +420,11 @@ Best Practices - **Checkpoints**: Regular checkpoints are saved to ``outputs//checkpoints/``. Use these to resume training or evaluate policies. +See Also +-------- + +- :doc:`/overview/rl/index` — RL module architecture and component reference +- :doc:`/overview/gym/env` — EmbodiedEnv configuration and Action Manager +- :doc:`basic_env` — Creating basic Gymnasium environments +- :doc:`modular_env` — Advanced modular environments with managers +- :doc:`/resources/task/index` — List of available RL task environments