Feat/a2arm pos force - #1050
Closed
lemon-star608 wants to merge 194 commits into
Closed
Conversation
…umba perf(managers): continue reducing update_state cost (#1316)
Multi-rank off-policy collectors pin their MuJoCo BatchEnvPool workers to a per-rank CPU block via EnvCfg.cpu_ids, but the collector's host-side compute did not follow: Numba's parallel kernels sized their pool from the host CPU count and drifted across rank boundaries, and the OpenBLAS pool spawned at import kept the host-wide mask. NpEnv.__init__ now applies apply_env_cpu_runtime(cfg.cpu_ids) on the cold path: the process is confined to the block (existing threads pinned individually via /proc/self/task, later threads — including Numba's lazily-launched pool — inherit the mask) and Numba's pool is sized to len(cpu_ids) unless NUMBA_NUM_THREADS is set explicitly. cpu_ids=None keeps the single-rank path bit-identical. Backend-agnostic: any env declaring cpu_ids (e.g. motrix once it grows affinity support) gets the same confinement.
…lock perf(env): confine DP collector host compute to the per-rank CPU block
perf(mjwarp): reduce g1 motion tracking reset latency
… probes (#1328) Diagnostic probes for the SAC/MuJoCo single-GPU collector CPU under-utilization report: pool thread-count scaling on the G1 scene, and per-phase wall/CPU attribution of a full task env step. New files only; no behavior change.
…hread-sizing benchmark(env): add MuJoCo pool thread-scaling and env-step phase-CPU probes (#1328)
os.sched_setaffinity/sched_getaffinity are Linux-only, so mypy on darwin rejected the direct attribute access (attr-defined) and the unit tests' monkeypatch.setattr/delattr failed because the attributes do not exist. Resolve the affinity symbols via getattr at call time (identical runtime semantics, still monkeypatchable) and pass raising=False to the test monkeypatch seams so they work whether or not the host exposes them.
fix(base): cpu_runtime 在非 Linux 主机上的类型检查与测试修复
Reward displays (tensorboard reward/mean and the terminal logger) lagged badly on off-policy and APPO runs: - collectors sent metrics only every num_envs * 10 env steps, so the reported reward changed just once per ~10 learner iterations; - runners then averaged the last 100 (off-policy) or 50 (APPO) reports, each already a rolling 100-episode mean, delaying the visible curve by ~1000 iterations. Report metrics every collector cycle, keep the runner-side window at the last 10 reports, and bound the per-worker episode reward/length buffers with deque(maxlen=100) instead of lists that grew for the whole run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ManagerBasedRlEnv.reset() replaced state.info["log"] with the reset-only extras (Episode_Reward/*), wiping the fresh per-step reward/* entries that _update_state_in_read_phase() had just computed for the current transition. On any step where at least one env resets — with thousands of envs, nearly every step — collectors therefore saw no reward/* keys at all, so the per-term reward components in tensorboard and the terminal logger stayed frozen at one stale value for thousands of iterations (observed as long flat staircases on reward/motion_* etc.). Merge instead of replace on the autoreset path: the pre-reset per-step entries stay, reset extras layer on top. Standalone (non-autoreset) resets are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(logging): make collector reward reporting timely
…rsubscription (#1328) Single-GPU (cpu_ids=None) BatchEnvPool sizing now uses the CPUs actually usable by this process (len(os.sched_getaffinity(0)), falling back to os.cpu_count() where unavailable) instead of 2*cpu_count(). Measured on this machine (sac + g1_motion_tracking + mujoco, num_envs=4096, 2000 iters, steady-state second-half median): 86,162 -> 96,262 steps/s (+11.7%), collector env_step 47.6ms -> 41.9ms. Explicit cpu_ids still fixes nthread = len(cpu_ids).
lemon-star608
force-pushed
the
feat/a2arm-pos-force
branch
from
August 27, 2026 08:13
1714b6b to
d44e8a6
Compare
…hread-sizing perf(mujoco): 单卡默认 pool 线程数收敛到有效 CPU 数，消除 2× 超订 (#1328)
lemon-star608
force-pushed
the
feat/a2arm-pos-force
branch
from
August 27, 2026 08:23
d44e8a6 to
8758945
Compare
Collaborator
|
后续移动到unilab-gallery或者unitree-rl-unilab或者单独仓库 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
功能概述
主要改动
A2Arm 位置-力控制环境
A2ArmPosForce环境及对应配置。CSE-PPO 训练流程
scripts/train_cse_ppo.py训练入口。conf/ppo_cse/下的算法和任务配置。MuJoCo 交互控制
scripts/play_a2arm_pos_force_interactive.py。验证
make test-all