Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ uv run train --algo sac --task g1_wbt_obs --sim mujoco training.use_amp=true

The `g1_wbt_obs` owner is the deploy-aligned off-policy observation profile: a
pelvis IMU state (`pelvis_local_linvel` / `pelvis_gyro` / `pelvis_upvector`) plus
per-term observation history (`noise_config.obs_history_length: 5`), byte-aligned
with the deploy-time `ObservationManager`. Deploy tooling lives under
`scripts/deploy/`, and the observation alignment is cross-checked by
`tests/scripts/test_obs_alignment_g1_wbt.py`. When a Motrix sim2sim replay needs a
checkpoint from another log root, pass the absolute path through `uv run eval`:
per-term observation history (`noise_config.obs_history_length: 5`), flattened
oldest-first per term so a hardware runtime assembling per-term history reads the
same vector. That ordering is guarded by
`tests/scripts/test_obs_alignment_g1_wbt.py`; the hardware-side contract is
documented in the sim-to-real deployment guide. When a Motrix sim2sim replay
needs a checkpoint from another log root, pass the absolute path through
`uv run eval`:

```bash
uv run eval --algo sac --task g1_motion_tracking --sim motrix \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ page in this section drills into one stage.
## What "sim-to-real" means in UniLab

A deployable UniLab policy is the exported policy plus the exact observation
and action contracts used by the selected task owner. The G1 WBT helper path
materializes this as `policy.onnx`, `deploy_config.yaml`, and a motion binary;
other robots need an equivalent hardware-side runtime that:
and action contracts used by the selected task owner. UniLab ships the training
side and the ONNX export; every robot needs a hardware-side runtime that:

1. Reads sensors → assembles the **same observation vector** the policy saw
in simulation.
Expand Down
124 changes: 67 additions & 57 deletions docs/sphinx/source/en/3-deployment/1-sim_to_real/2-g1_whole_body.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,21 @@

::::{admonition} Hardware target
:class: note
Unitree G1 humanoid (29-DoF variant). Joints are assumed in the order exported
by `scripts/deploy/export_deploy_config.py` from
`src/unilab/assets/robots/g1/scene_flat.xml`; verify that order before
hardware bring-up.
Unitree G1 humanoid (29-DoF variant). Joint order comes from the task owner's
scene (`src/unilab/assets/robots/g1/scene_flat.xml`, actuator order); verify
that order against your SDK motor indices before hardware bring-up.
::::

This guide walks the **last mile** between a converged G1 motion-tracking
policy and a closed-loop run on the robot.
This guide covers the **observation and action contract** a G1 motion-tracking
policy expects on hardware. The repository does not ship a G1 deploy runtime —
you supply the hardware-side loop, and this page tells you what it must
reproduce.

## 0. Verify your sim-side checkpoint

```bash
# Replay the policy headlessly and produce a video.
uv run eval --algo ppo --task g1_motion_tracking --sim motrix --load-run -1 \
uv run eval --algo sac --task g1_wbt_obs --sim mujoco --load-run -1 \
--render-mode record
```

Expand All @@ -25,39 +26,51 @@ What to look for in the video:
- Joint velocities and actions remain finite and within the expected range.
- Contact timing looks consistent with the reference motion.

If any of those is off, fix the sim-side checkpoint or deploy contract before
hardware bring-up.
If any of those is off, fix the sim-side checkpoint before hardware bring-up.

## 1. Export
## 1. Pick the owner, then read its contract off the YAML

Use the training playback path to export `policy.onnx`, then export the G1 WBT
deploy config and motion binary with the committed deployment helpers:
Every field your hardware loop needs is declared in the task owner YAML. The
deploy-oriented G1 owners are:

```bash
uv run eval --algo ppo --task g1_motion_tracking --sim motrix --load-run -1

uv run scripts/deploy/export_deploy_config.py \
--output logs/deploy/deploy_config.yaml

uv run scripts/deploy/export_motion_bin.py \
--output logs/deploy/dance1.bin
```{list-table}
:header-rows: 1
:widths: 34 22 44

* - Owner
- Actor obs width
- Notes
* - `conf/offpolicy/task/sac/g1_wbt_obs/mujoco.yaml`
- 514 (H=5)
- Proprio history, no state estimation: drops `base_lin_vel` and
`motion_anchor_pos_b`, pelvis IMU.
* - `conf/ppo/task/g1_motion_tracking_deploy/mujoco.yaml`
- 154 (H=1)
- Single-step mimic actor layout, per-joint `action_scale` list.
```

The deployment-side prototype consumes:
::::{admonition} Read the width off the env, not off this table
:class: warning
Actor obs width is a function of the owner's `noise_config` flags — see
`_actor_obs_dim` in `src/unilab/envs/motion_tracking/g1/tracking_obs.py` for
`g1_wbt_obs`, and `mimic_actor_obs_dim` in
`src/unilab/envs/motion_tracking/common/observations.py` for the deploy owner.
If the ONNX input width disagrees with what your hardware loop assembles, that
is a contract bug, not a hardware tuning problem.
::::

Export `policy.onnx` through the training playback path:

```
runs/<run>/
└── policy.onnx
logs/deploy/
├── deploy_config.yaml
└── dance1.bin
```bash
uv run eval --algo sac --task g1_wbt_obs --sim mujoco --load-run -1
```

## 2. Observation contract

For the committed G1 WBT deploy helper, the observation layout is exported into
`deploy_config.yaml` as `obs_layout`. `scripts/deploy/export_deploy_config.py`
is the source of truth for the segment order:
For `g1_wbt_obs`, the actor obs is assembled in this order (see
`_build_actor_obs` in `src/unilab/envs/motion_tracking/g1/tracking_obs.py`).
Single-step reference terms come first, then each proprio term's full history
flattened **oldest-first**:

```{list-table}
:header-rows: 1
Expand All @@ -77,32 +90,39 @@ is the source of truth for the segment order:
- anchor orientation term from the reference and robot torso frames
* - `gyro`
- 3 per history step
- IMU gyro term
- IMU gyro term (`env.sensor.gyro`, `pelvis_gyro` for this owner)
* - `joint_pos_rel`
- 29 per history step
- measured joint position minus `default_angles`
- measured joint position minus the `stand` keyframe joint angles
* - `dof_vel`
- 29 per history step
- joint velocity term
* - `last_actions`
- 29
- 29 per history step
- previous raw actor output
```

The export script also records each segment's `history_length` and verifies the
total `obs_dim`. `scripts/deploy/sim_prototype.py` refuses to run when the ONNX
input width and `deploy_config.yaml` `obs_dim` disagree.
History depth `H` is `env.noise_config.obs_history_length` (5 for this owner).
Per-term oldest-first ordering is guarded by
`tests/scripts/test_obs_alignment_g1_wbt.py`; mirror that ordering on hardware
or the policy reads a permuted vector.

## 3. Actuator interface

The G1 deploy prototype maps actor output exactly as:
`action * action_scale + default_angles`, then clips to `joint_lower` /
`joint_upper` and applies EMA smoothing from `ema_alpha`.
Map actor output as `action * action_scale + default_angles`, then clamp to the
scene's joint range before the target reaches the motor driver.

- `action_scale` is `env.control_config.action_scale` in the owner YAML. It may
be a **scalar** (2.0 for `g1_wbt_obs`) or a **per-joint list** (29 entries for
`g1_motion_tracking_deploy`). Reproduce the owner's form exactly — do not
average a list, take its first entry, or broadcast a scalar over a list owner.
- `default_angles` is the `stand` keyframe joint block of the owner's scene.
- Joint limits and gains come from the same scene XML (`jnt_range`, position
actuator `gainprm` / `biasprm`).

- Action = target joint position, **scaled** by the `action_scale` entry in
`deploy_config.yaml`.
- Clamp the target to the generated joint range before it reaches the motor
driver.
Training applies the target directly with no smoothing. If hardware jitter
forces you to add smoothing, verify the sim2sim impact first — every step of lag
pushes observations out of the training distribution.

## 4. Reference motion sync

Expand All @@ -125,8 +145,7 @@ specifics:

- Reject non-finite actions and shape mismatches before applying
`action_scale`.
- Clamp generated targets with `joint_lower` / `joint_upper` from
`deploy_config.yaml`.
- Clamp generated targets with the joint range from the owner's scene XML.
- Keep watchdog, pose monitor, and operator-stop thresholds in the deploy
controller and test them independently of the policy.

Expand All @@ -145,18 +164,9 @@ and `last_actions` wiring mistakes are easiest to catch.
## 7. What to log

Log the **full observation vector**, **full action vector**, and **wall
clock** for every step. Before hardware bring-up, validate the same ONNX,
deploy config, and motion binary through the MuJoCo deployment prototype:

```bash
uv run scripts/deploy/sim_prototype.py \
--onnx runs/<run>/policy.onnx \
--config logs/deploy/deploy_config.yaml \
--motion logs/deploy/dance1.bin
```

A mismatch between the ONNX input width and `deploy_config.yaml` `obs_dim` is a
deployment contract bug, not a hardware tuning problem.
clock** for every step. Compare the first hardware observation window against a
sim episode built from the same owner YAML — that diff localizes unit, frame,
and ordering mistakes faster than any reward inspection.

## See also

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,38 +26,21 @@ uv run eval --algo sac --task g1_walk_flat --sim mujoco --load-run -1

`uv run eval` sets playback mode and maps `--load-run` to the checkpoint
selector used by the routed training script. The exported file is written into
the selected run directory. For deployment
prototypes, keep the exported `policy.onnx` together with the deploy-side
configuration and motion assets used by the runtime.
the selected run directory. For deployment, keep the exported `policy.onnx`
together with the task owner YAML it was trained from — that YAML is the
authority on the observation and action contract the runtime must reproduce.

## G1 Deployment Prototype
## Verifying the Exported Graph

The committed G1 WBT deployment helpers use these artifacts:
The playback path validates the exported graph against PyTorch before writing
it, so a successful export already establishes numerical parity. What it does
**not** establish is that your hardware-side loop assembles the same input
vector. Before hardware bring-up:

| Artifact | Producer |
| --- | --- |
| `policy.onnx` | Training playback export above. |
| `deploy_config.yaml` | `scripts/deploy/export_deploy_config.py`. |
| `dance1.bin` or another motion binary | `scripts/deploy/export_motion_bin.py`. |

Example validation run:

```bash
uv run scripts/deploy/export_deploy_config.py \
--output logs/deploy/deploy_config.yaml

uv run scripts/deploy/export_motion_bin.py \
--output logs/deploy/dance1.bin

uv run scripts/deploy/sim_prototype.py \
--onnx runs/<run>/policy.onnx \
--config logs/deploy/deploy_config.yaml \
--motion logs/deploy/dance1.bin
```

`scripts/deploy/sim_prototype.py` checks that the ONNX input width matches the
`obs_dim` in `deploy_config.yaml` and then drives the policy in MuJoCo with the
same observation layout the deployment side expects.
- Read the actor obs width off the env (not off a doc table) and confirm it
matches the ONNX input width.
- Confirm your term order and per-term history ordering against the owner's
`_build_actor_obs`. For G1 whole-body tracking, see {doc}`2-g1_whole_body`.

## See Also

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,37 +43,34 @@ flowchart LR
```

Keep the hard real-time safety checks in the deploy controller, not in the
training script. The repository's G1 helper path exports deploy config and runs
a MuJoCo prototype; it does not implement a production motor-driver safety
loop.
training script. The repository does not implement a production motor-driver
safety loop — that boundary is yours to build and test.

## What the policy assumes you've configured

The G1 deployment helper exports these fields into `deploy_config.yaml`:
The policy expects the action mapping and limits its training owner declared.
For the G1 WBT owner (`conf/offpolicy/task/sac/g1_wbt_obs/mujoco.yaml`):

```yaml
action_scale: 2.0
ema_alpha: 1.0
default_angles: [...]
joint_lower: [...]
joint_upper: [...]
kp: [...]
kd: [...]
```
| Quantity | Authority |
| --- | --- |
| `action_scale` | `env.control_config.action_scale` (scalar `2.0` for this owner; other owners declare a per-joint list) |
| `default_angles` | `stand` keyframe joint block of the owner's scene XML |
| joint limits | `jnt_range` in the scene XML |
| `kp` / `kd` | position actuator `gainprm` / `biasprm` in the scene XML |

`scripts/deploy/sim_prototype.py` consumes the same fields and applies
`action * action_scale + default_angles`, joint clipping, and EMA smoothing.
Hardware controllers should consume generated config rather than hand-copying
joint ranges or gains.
Derive these from the owner YAML and its scene, and reproduce the owner's
`action_scale` **form** exactly — a scalar owner and a per-joint-list owner are
not interchangeable. Do not hand-copy joint ranges or gains into a second place
that can silently drift from the asset.

## Hand-off testing

Before integrating policy → safety → motor, test the safety layer in
isolation:

1. Inject a NaN action and verify the command is rejected.
2. Inject an out-of-range joint target and verify clamping uses
`joint_lower` / `joint_upper` from `deploy_config.yaml`.
2. Inject an out-of-range joint target and verify clamping uses the joint range
from the owner's scene XML.
3. Cut the policy feed mid-run and verify the controller enters its configured
safe state.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ budgets as robot-specific measurements, not UniLab defaults.
| Surface | Repo evidence | What it covers |
| --- | --- | --- |
| One-step action delay | `control_config.simulate_action_latency` in locomotion and G1 motion-tracking envs | Executes the previous action instead of the current action. |
| G1 WBT observation history | `noise_config.obs_history_length` and `scripts/deploy/export_deploy_config.py` | Exports per-term `obs_layout` history for `gyro`, `joint_pos_rel`, `dof_vel`, and `last_actions`. |
| G1 WBT observation history | `noise_config.obs_history_length` in `conf/offpolicy/task/sac/g1_wbt_obs/mujoco.yaml` | Per-term history for `gyro`, `joint_pos_rel`, `dof_vel`, and `last_actions`. |
| Sharpa tactile contact latency | `domain_rand.contact_latency` in Sharpa in-hand configs | Keeps previous tactile contact values for sampled contact channels. |
| Deploy-side ONNX contract check | `scripts/deploy/sim_prototype.py` | Validates `obs_layout`, `obs_dim`, ONNX input width, clipping, and EMA action smoothing for the G1 WBT path. |
| Obs history ordering guard | `tests/scripts/test_obs_alignment_g1_wbt.py` | Asserts per-term oldest-first flatten for the G1 WBT actor obs. |

## Action Latency

Expand All @@ -30,23 +30,13 @@ The checked-in G1 WBT owner enables this flag in

## Observation Lag And History

The G1 WBT deployment helpers do not guess observation width. They export a
schema with `obs_layout`, per-term `history_length`, and `obs_dim`; the
prototype then assembles the same layout and refuses mismatches.
Observation width is a function of the owner's `noise_config`, not something a
hardware runtime may guess. For the G1 WBT owner, `obs_history_length: 5` gives
each proprioceptive term a 5-step history flattened oldest-first, while
reference terms stay single-step. See {doc}`2-g1_whole_body` for the full term
order.

```bash
uv run scripts/deploy/export_deploy_config.py \
--output logs/deploy/deploy_config.yaml

uv run scripts/deploy/sim_prototype.py \
--onnx runs/<run>/policy.onnx \
--config logs/deploy/deploy_config.yaml \
--motion logs/deploy/dance1.bin
```

Do not lag command/reference terms unless the training owner did so. In the G1
WBT schema, reference terms stay single-step while proprioceptive terms carry
history.
Do not lag command/reference terms unless the training owner did so.

## Deploy-Side Measurements

Expand All @@ -58,10 +48,10 @@ Record these per policy tick in the hardware runtime:
4. actuator command send timestamp
5. the action vector before and after clamp / smoothing

Compare the observation vector against a sim rollout built from the same
`deploy_config.yaml`. If the measured pipeline needs filtering or buffering,
encode the matching behavior in the task owner and re-export the deployment
artifacts.
Compare the observation vector against a sim rollout built from the same task
owner YAML. If the measured pipeline needs filtering or buffering, encode the
matching behavior in the task owner and retrain, rather than adding it only on
the deploy side.

## Symptoms Of Mismatch

Expand Down
Loading
Loading