From fd5c69a53a38f0ef6c1564b01df726e0bd678129 Mon Sep 17 00:00:00 2001 From: pareenaverma Date: Thu, 27 Aug 2026 14:27:07 -0400 Subject: [PATCH] Tech review of mappo LP --- .../1-prepare-environment.md | 54 +++++++++++++- .../2-configure-training.md | 70 ++++++++++++++++--- .../3-validate-checkpoint.md | 46 +++++++++++- .../4-deploy-gui.md | 26 ++++++- .../5-export-actor.md | 13 +++- .../_index.md | 12 +++- 6 files changed, 204 insertions(+), 17 deletions(-) diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/1-prepare-environment.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/1-prepare-environment.md index 0bed838189..eb1526d3e4 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/1-prepare-environment.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/1-prepare-environment.md @@ -7,6 +7,32 @@ weight: 2 layout: learningpathall --- +## Understand the reference environment + +This Learning Path trains a multi-agent proximal policy optimization (MAPPO) policy entirely on Arm CPUs. It does not need a GPU. + +BenchMARL defines and runs the experiment, TorchRL provides the reinforcement-learning components, and VMAS simulates many navigation worlds in a vectorized PyTorch batch. MAPPO trains an actor that selects each agent's actions and a centralized critic used only during training. The reference configuration shares one actor across all three agents, which makes the later actor-only export possible. + +The reference experiment was tested on this AWS configuration: + +| Component | Tested configuration | +| --- | --- | +| Instance | `m9g.48xlarge` | +| Processor | AWS Graviton5 | +| Architecture | `aarch64` | +| vCPUs | 192 | +| Memory | 768 GiB | +| Operating system | Ubuntu 24.04 | +| Storage | 512 GB EBS volume | + +The M9g instance is EBS-only, so the storage volume is provisioned separately from the instance. The workflow does not depend on an AWS-specific API and can run on other Arm-based cloud instances. A smaller instance uses fewer vectorized environments and takes longer to process the same training-frame budget. + +AWS Graviton5 provides one hardware thread per core on this instance. The one-environment-per-workload-CPU rule is still a starting point rather than a fixed mapping, because VMAS processes the environments as tensor batches. + +{{% notice Note %}} +The tested configuration is a reproducibility reference, not a measured minimum requirement. Package installation, training logs, and checkpoints need persistent storage, but this Learning Path does not establish 512 GB as the minimum capacity. +{{% /notice %}} + ## Inspect the Arm cloud instance Confirm that the instance uses the Arm64 architecture: @@ -33,12 +59,13 @@ You can also inspect the processor topology: lscpu ``` -Save the CPU count and reserve one CPU for operating-system and runtime activity: +Save the CPU count and leave one CPU out of the workload calculation: ```bash export CORE_COUNT=$(nproc) export RESERVED_CPUS=1 export WORKLOAD_CPUS=$((CORE_COUNT - RESERVED_CPUS)) +test "$WORKLOAD_CPUS" -ge 1 || { echo "This sizing policy needs at least 2 CPUs" >&2; exit 1; } ``` Verify the values: @@ -47,6 +74,8 @@ Verify the values: echo "TotalCPUs=$CORE_COUNT ReservedCPUs=$RESERVED_CPUS WorkloadCPUs=$WORKLOAD_CPUS" ``` +This calculation reduces the size of the PyTorch thread pools and the VMAS batch. It does not pin the workload to specific CPUs or prevent the operating system from scheduling work on them. + ## Understand agents and vectorized environments The agent count and vectorized environment count control different parts of the workload: @@ -121,6 +150,16 @@ Verify the installation: ```bash python -c 'import platform, torch; print("Architecture:", platform.machine()); print("PyTorch:", torch.__version__); print("CUDA available:", torch.cuda.is_available()); print("CUDA device:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "N/A")' ``` +On the tested system, the output was: + +```output +Architecture: aarch64 +PyTorch: 2.13.0+cu130 +CUDA available: False +CUDA device: N/A +``` + +The PyTorch version can differ when you run an unpinned `pip` installation. `CUDA available: False` is expected on the CPU-only M9g instance, even if the wheel version contains a CUDA suffix. ## Install BenchMARL and VMAS @@ -145,6 +184,15 @@ Verify the software stack: python -c 'import torch, torchrl, benchmarl, vmas; print("PyTorch:", torch.__version__); print("TorchRL: OK"); print("BenchMARL: OK"); print("VMAS: OK")' ``` +On the tested system, the output was: + +```output +PyTorch: 2.13.0+cu130 +TorchRL: OK +BenchMARL: OK +VMAS: OK +``` + Record the BenchMARL revision used for the experiment: ```bash @@ -152,3 +200,7 @@ git rev-parse HEAD ``` Keep this revision with your experiment notes so you can reproduce the software environment later. + +## What you've accomplished + +You have verified the `aarch64` environment, sized the initial VMAS workload, and installed the training stack. Next, you will keep the training-frame budget consistent while adapting the vectorized environment count to the available CPUs. diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/2-configure-training.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/2-configure-training.md index 2398548a4d..2584a65170 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/2-configure-training.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/2-configure-training.md @@ -9,6 +9,8 @@ layout: learningpathall ## Choose the agent count and devices +MAPPO uses the centralized critic to learn from joint training information while each agent acts from its own observation. With `share_policy_params=true`, the three agents use the same actor parameters but receive different observations. + Move to the BenchMARL repository: ```bash @@ -39,6 +41,7 @@ Recreate the CPU sizing variables so the configuration works after a new SSH log export CORE_COUNT=$(nproc) export RESERVED_CPUS=1 export WORKLOAD_CPUS=$((CORE_COUNT - RESERVED_CPUS)) +test "$WORKLOAD_CPUS" -ge 1 || { echo "This sizing policy needs at least 2 CPUs" >&2; exit 1; } export N_ENVS=$WORKLOAD_CPUS ``` @@ -49,13 +52,16 @@ export FRAMES_PER_ENV_PER_BATCH=100 export FRAMES_PER_BATCH=$((N_ENVS * FRAMES_PER_ENV_PER_BATCH)) ``` -Run 100 training batches: +Use 1,910,000 frames as the target training budget. Calculate enough complete batches to meet or slightly exceed that target: ```bash -export TRAINING_BATCHES=100 +export TARGET_MAX_FRAMES=1910000 +export TRAINING_BATCHES=$(((TARGET_MAX_FRAMES + FRAMES_PER_BATCH - 1) / FRAMES_PER_BATCH)) export MAX_FRAMES=$((FRAMES_PER_BATCH * TRAINING_BATCHES)) ``` +BenchMARL collects complete batches. `MAX_FRAMES` therefore equals the target on the 192-vCPU reference system and can be up to one batch larger on another instance. Keeping the target fixed prevents CPU count from reducing the amount of training. + Evaluate every 20 batches using ten evaluation episodes: ```bash @@ -66,10 +72,10 @@ export EVAL_EPISODES=10 The reference sizing gives: -| Total CPUs | Workload CPUs / environments | Frames per batch | Total frames | Evaluation interval | -| ---: | ---: | ---: | ---: | ---: | -| 64 | 63 | 6,300 | 630,000 | 126,000 | -| 192 | 191 | 19,100 | 1,910,000 | 382,000 | +| Total CPUs | Environments | Frames per batch | Training batches | Target frames | Actual frames | Evaluation interval | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 64 | 63 | 6,300 | 304 | 1,910,000 | 1,915,200 | 126,000 | +| 192 | 191 | 19,100 | 100 | 1,910,000 | 1,910,000 | 382,000 | {{% notice Note %}} When VMAS sampling runs on CUDA, tune `N_ENVS` for the GPU instead of deriving it from the CPU count. @@ -90,8 +96,11 @@ Name the run so that the agent count, environment count, sampling device, and tr export RUN_NAME="agents_${AGENTS}__envs_${N_ENVS}__sampling_${SAMPLING_DEVICE}__train_${TRAIN_DEVICE}" export RUN_DIR="$OUTPUT_ROOT/$RUN_NAME" mkdir -p "$RUN_DIR" +ln -sfn "$RUN_DIR" "$OUTPUT_ROOT/latest" ``` +The `latest` symbolic link gives later sections a stable way to recover the run directory after a new SSH login. + ## Limit CPU thread parallelism Cap the main CPU thread pools at the number of workload CPUs: @@ -105,18 +114,35 @@ export NUMEXPR_MAX_THREADS=$WORKLOAD_CPUS This avoids library thread pools using more CPU threads than the workload allocation. +Record the software revision, installed packages, and shell configuration with the run: + +```bash +export BENCHMARL_REVISION=$(git rev-parse HEAD) +python -m pip freeze > "$RUN_DIR/software-versions.txt" +declare -px \ + AGENTS SAMPLING_DEVICE TRAIN_DEVICE \ + CORE_COUNT RESERVED_CPUS WORKLOAD_CPUS N_ENVS \ + FRAMES_PER_ENV_PER_BATCH FRAMES_PER_BATCH TARGET_MAX_FRAMES \ + TRAINING_BATCHES MAX_FRAMES EVAL_EVERY_BATCHES EVAL_INTERVAL EVAL_EPISODES \ + OUTPUT_ROOT RUN_NAME RUN_DIR \ + OMP_NUM_THREADS MKL_NUM_THREADS OPENBLAS_NUM_THREADS NUMEXPR_MAX_THREADS \ + BENCHMARL_REVISION > "$RUN_DIR/run.env" +``` + +`run.env` contains only the variables listed in the command. It does not copy credentials or the rest of your shell environment. + ## Validate the configuration Print the complete configuration before training: ```bash -echo "Agents=$AGENTS TotalCPUs=$CORE_COUNT ReservedCPUs=$RESERVED_CPUS WorkloadCPUs=$WORKLOAD_CPUS Environments=$N_ENVS FramesPerBatch=$FRAMES_PER_BATCH MaxFrames=$MAX_FRAMES EvalInterval=$EVAL_INTERVAL EvalEpisodes=$EVAL_EPISODES Sampling=$SAMPLING_DEVICE Training=$TRAIN_DEVICE" +echo "Agents=$AGENTS TotalCPUs=$CORE_COUNT ReservedCPUs=$RESERVED_CPUS WorkloadCPUs=$WORKLOAD_CPUS Environments=$N_ENVS FramesPerBatch=$FRAMES_PER_BATCH TrainingBatches=$TRAINING_BATCHES MaxFrames=$MAX_FRAMES EvalInterval=$EVAL_INTERVAL EvalEpisodes=$EVAL_EPISODES Sampling=$SAMPLING_DEVICE Training=$TRAIN_DEVICE" ``` For a 192-CPU system, the reference values are: ```output -Agents=3 TotalCPUs=192 ReservedCPUs=1 WorkloadCPUs=191 Environments=191 FramesPerBatch=19100 MaxFrames=1910000 EvalInterval=382000 EvalEpisodes=10 Sampling=cpu Training=cpu +Agents=3 TotalCPUs=192 ReservedCPUs=1 WorkloadCPUs=191 Environments=191 FramesPerBatch=19100 TrainingBatches=100 MaxFrames=1910000 EvalInterval=382000 EvalEpisodes=10 Sampling=cpu Training=cpu ``` Do not start training if a required field is blank. @@ -126,7 +152,31 @@ Do not start training if a required field is blank. Start MAPPO training: ```bash -python benchmarl/run.py algorithm=mappo task=vmas/navigation task.n_agents="$AGENTS" experiment.sampling_device="$SAMPLING_DEVICE" experiment.train_device="$TRAIN_DEVICE" experiment.on_policy_n_envs_per_worker="$N_ENVS" experiment.on_policy_collected_frames_per_batch="$FRAMES_PER_BATCH" 'experiment.loggers=[csv]' experiment.render=false experiment.evaluation=true experiment.max_n_frames="$MAX_FRAMES" experiment.checkpoint_at_end=true experiment.prefer_continuous_actions=true experiment.evaluation_interval="$EVAL_INTERVAL" experiment.evaluation_episodes="$EVAL_EPISODES" experiment.save_folder="$RUN_DIR" +python benchmarl/run.py \ + algorithm=mappo \ + task=vmas/navigation \ + task.n_agents="$AGENTS" \ + experiment.sampling_device="$SAMPLING_DEVICE" \ + experiment.train_device="$TRAIN_DEVICE" \ + experiment.on_policy_n_envs_per_worker="$N_ENVS" \ + experiment.on_policy_collected_frames_per_batch="$FRAMES_PER_BATCH" \ + 'experiment.loggers=[csv]' \ + experiment.render=false \ + experiment.evaluation=true \ + experiment.max_n_frames="$MAX_FRAMES" \ + experiment.checkpoint_at_end=true \ + experiment.prefer_continuous_actions=true \ + experiment.evaluation_interval="$EVAL_INTERVAL" \ + experiment.evaluation_episodes="$EVAL_EPISODES" \ + experiment.save_folder="$RUN_DIR" ``` -BenchMARL performs training and periodic evaluation and saves a checkpoint when the run completes. +This is the argument set used for the reference experiment. The tested BenchMARL and VMAS defaults define the 18-value observation and `256, 256` actor architecture checked by the exporter. BenchMARL saves a checkpoint when training completes. + +{{% notice Important %}} +Training takes several hours on the reference system. Run the command in a persistent terminal session, such as `tmux`, so an SSH disconnection does not terminate the process. This tested command saves its checkpoint at the end of training. +{{% /notice %}} + +## What you've accomplished + +You have configured a CPU-sized VMAS batch without changing the target training budget. BenchMARL now evaluates the policy periodically and saves the final checkpoint with the package versions, BenchMARL revision, and workload variables recorded alongside the run. diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/3-validate-checkpoint.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/3-validate-checkpoint.md index 5c754a065a..39f9090176 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/3-validate-checkpoint.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/3-validate-checkpoint.md @@ -7,6 +7,19 @@ weight: 4 layout: learningpathall --- +## Restore the run configuration + +Activate the training environment and recover the saved variables. The `latest` link was created when you configured the run: + +```bash +source $HOME/venvs/mappo/bin/activate +export RUN_DIR=$HOME/mappo_navigation_runs/latest +source "$RUN_DIR/run.env" +cd $HOME/BenchMARL +``` + +If you want to validate an older run, set `RUN_DIR` to that run directory before you source `run.env`. + ## Find the checkpoint BenchMARL creates an experiment directory below `RUN_DIR` with a structure similar to: @@ -36,9 +49,12 @@ Display and verify the path: ```bash echo "$CHECKPOINT" -test -f "$CHECKPOINT" && echo "Checkpoint found: $CHECKPOINT" +test -n "$CHECKPOINT" && test -f "$CHECKPOINT" || { echo "No checkpoint found under $RUN_DIR" >&2; exit 1; } +echo "Checkpoint found: $CHECKPOINT" ``` +The command stops the shell if no checkpoint exists, which prevents later commands from deriving paths from an empty value. + ## Keep `config.pkl` with the checkpoint Determine the BenchMARL experiment directory: @@ -86,6 +102,34 @@ python -c "import pickle; f=open('$SOURCE_EXPERIMENT_DIR/config.pkl','rb'); task This value comes from the trained experiment and prevents a checkpoint from being registered with a stale shell value for the agent count. +Save the checkpoint variables so later sections can restore them after a new SSH login: + +```bash +declare -px CHECKPOINT SOURCE_EXPERIMENT_DIR CHECKPOINT_AGENTS >> "$RUN_DIR/run.env" +``` + +## Evaluate the reloaded policy + +Run BenchMARL's evaluation entry point against the selected checkpoint: + +```bash +python benchmarl/evaluate.py "$CHECKPOINT" +``` + +This command reconstructs the experiment from `config.pkl`, loads the trained state, and runs the saved evaluation configuration. A successful run proves that BenchMARL can execute the policy, while the earlier file checks prove only that the artifacts exist. + +List the CSV logs produced by the experiment: + +```bash +find "$SOURCE_EXPERIMENT_DIR" -type f -name '*.csv' -print +``` + +Compare the first and final evaluation returns in these logs. Returns vary with the random seed and package versions, so this Learning Path does not use an unverified numeric threshold. A flat or falling return means that checkpoint creation succeeded but the policy did not demonstrate learning. + {{% notice Security %}} Only load `config.pkl` from a training run you trust. Python pickle files can execute code when loaded. {{% /notice %}} + +## What you've accomplished + +You have selected a valid checkpoint, confirmed that its matching configuration is present, recovered deployment metadata, and reloaded the policy for evaluation. The checkpoint and `config.pkl` now form the portable BenchMARL experiment used by the optional GUI stage. diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/4-deploy-gui.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/4-deploy-gui.md index 7639402ac1..bd59351563 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/4-deploy-gui.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/4-deploy-gui.md @@ -11,6 +11,8 @@ layout: learningpathall The MARL GUI used in this stage is a separate application from BenchMARL and VMAS. It is not created automatically by the training workflow. +The GUI source and helper scripts are not included in the Arm Learning Paths repository. Treat this stage as optional unless you have the companion checkout described in the prerequisites. Checkpoint validation and actor export do not depend on the GUI. + If you already have the companion GUI code, use that checkout and continue with the steps below. If you do not have the GUI code, you can either skip this stage and continue to the actor-export section, or create an equivalent visualization application. A compatible GUI should be able to: @@ -48,8 +50,12 @@ The GUI reloads the full BenchMARL experiment, so use the same Python environmen ```bash source $HOME/venvs/mappo/bin/activate +export RUN_DIR=$HOME/mappo_navigation_runs/latest +source "$RUN_DIR/run.env" ``` +Sourcing `run.env` restores `CHECKPOINT`, `SOURCE_EXPERIMENT_DIR`, the agent count, and the device labels saved in the previous section. + Set the GUI root to your local checkout: ```bash @@ -197,16 +203,28 @@ Confirm that the output again contains: ## Start the GUI -Start the application: +Start the application on the cloud instance. Bind it to the loopback interface so it is not exposed directly through the instance network: ```bash cd "$GUI_ROOT" ``` ```bash -./run_demo.sh --host 0.0.0.0 --port 8045 +./run_demo.sh --host 127.0.0.1 --port 8045 +``` + +From your local workstation, open an SSH tunnel to the instance: + +```bash +ssh -L 8045:127.0.0.1:8045 ubuntu@INSTANCE_PUBLIC_IP ``` +Keep the SSH connection open and visit `http://127.0.0.1:8045` in your local browser. Replace `ubuntu` and `INSTANCE_PUBLIC_IP` with the SSH user and address for your instance. + +{{% notice Security %}} +The SSH tunnel avoids opening TCP port 8045 to the internet. If you intentionally bind the GUI to `0.0.0.0`, restrict the cloud firewall or security-group rule to a trusted source address and confirm that the GUI's authentication is suitable for your environment. +{{% /notice %}} + Refresh the browser after registration. Select the model in this order: ```text @@ -237,3 +255,7 @@ The GUI uses a single VMAS environment for interactive playback. This is indepen | GUI registry | `$GUI_ROOT/configs/models.yaml` | Drives model selection | The full BenchMARL checkpoint is the correct artifact for the GUI. For a downstream inference runtime, you can export only the actor parameters, as shown in the next section. + +## What you've accomplished + +You have preserved the BenchMARL directory layout, registered the checkpoint with the companion GUI, reloaded the deployed model, and accessed interactive playback through a secure tunnel. Next, you will extract the shared actor for a runtime that does not include BenchMARL. diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/5-export-actor.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/5-export-actor.md index cf6a83f466..0b8bff5a85 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/5-export-actor.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/5-export-actor.md @@ -47,7 +47,7 @@ lidar_range - measured_range ``` {{% notice Note %}} -The exporter in this section is intentionally specific to the validated shared MAPPO actor. It verifies the task configuration, parameter sharing, MLP layer sizes, activation type, absence of normalization layers, LiDAR-ray count, and actor tensor shapes before it creates an artifact. If any of these assumptions change, the exporter stops rather than silently creating an incompatible model. +The exporter in this section is intentionally specific to the validated shared MAPPO actor. It verifies the task configuration, parameter sharing, MLP layer sizes, activation type, absence of normalization layers, and actor tensor shapes before it creates an artifact. The tested BenchMARL task configuration does not store the VMAS LiDAR-ray default, so the exporter uses 12 when that field is absent and verifies the resulting 18-input actor shape. {{% /notice %}} ## Create the actor exporter @@ -55,9 +55,14 @@ The exporter in this section is intentionally specific to the validated shared M Move to the BenchMARL directory: ```bash +source $HOME/venvs/mappo/bin/activate +export RUN_DIR=$HOME/mappo_navigation_runs/latest +source "$RUN_DIR/run.env" cd $HOME/BenchMARL ``` +This restores the original checkpoint path and activates the packages used to create it. If you copied the experiment to another system, update `CHECKPOINT` before you run the exporter. + Create `export_mappo_actor.py` with the following code: ```python @@ -577,6 +582,8 @@ Finite: True Within [-1,1]: True ``` +This is an interface smoke test. It checks tensor shapes, finite arithmetic, and action bounds, but it does not show that the policy navigates successfully. Use the BenchMARL evaluation and GUI playback checks for behavioral validation. + You now have two forms of the trained policy: ```text @@ -592,3 +599,7 @@ Downstream inference runtime ``` The downstream runtime must reproduce the observation ordering and action interpretation recorded in `metadata_json`. The actor-only artifact does not contain the VMAS environment, critic, or robot-specific sensor and control integration. + +## What you've accomplished + +You have extracted the validated shared MAPPO actor into a portable NumPy artifact and confirmed that its deterministic output matches the trained TorchRL policy. You also recorded the observation layout and action semantics needed to integrate the actor with a downstream runtime. diff --git a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/_index.md b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/_index.md index 28f64d7100..01ddd543bf 100644 --- a/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/train-mappo-navigation-arm-cloud/_index.md @@ -13,11 +13,11 @@ who_is_this_for: This Learning Path is for cloud and machine learning developers learning_objectives: - Configure a vectorized BenchMARL and VMAS workload for an Arm cloud instance. - Train and evaluate a multi-agent navigation policy with MAPPO. - - Package and validate the trained BenchMARL checkpoint for a cloud visualization GUI. + - Package and validate the trained BenchMARL checkpoint when the companion cloud visualization GUI is available. - Extract and validate the shared actor as a smaller inference-only artifact. prerequisites: - - An Arm64 Ubuntu cloud instance with SSH access, `sudo` privileges, and internet access. + - An `aarch64` cloud instance running Ubuntu 24.04 with SSH access, `sudo` privileges, and internet access. The reference experiment was tested on an AWS Graviton5 `m9g.48xlarge` instance with 192 vCPUs, 768 GiB of memory, and 512 GB of EBS storage. You can use another Arm-based cloud instance, but training time and the effective environment count will differ. - Familiarity with Linux, Python, PyTorch, and reinforcement learning concepts such as observations, actions, rewards, and policies. - A local checkout of the companion MARL GUI containing `tools/deploy_checkpoint.py` and `tools/inspect_checkpoint.py` if you want to complete the GUI deployment section. @@ -60,6 +60,14 @@ further_reading: title: The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games link: https://arxiv.org/abs/2103.01955 type: website + - resource: + title: Amazon EC2 M9g instances + link: https://aws.amazon.com/ec2/instance-types/m9g/ + type: documentation + - resource: + title: PyTorch installation guidance + link: https://pytorch.org/get-started/locally/ + type: documentation ### FIXED, DO NOT MODIFY # ================================================================================