Repository navigation
Description
Summary
A downstream SDK consumer of aci_edge_sandboxes needs usable ICMP inside the guest and enforced ICMP policy rules (allow and deny). With the runtime bundle below, neither works. We'd like to know what NVX/OpenVMM support at this pin and what guest configuration is intended, so that the ACI layer can be aligned.
Related: #277 (ICMP-only egress rules, closed 2026-10-09, before this pin).
Versions
| Component | Version |
|---|---|
| Rust crates (crates.io) | aci_edge_sandboxes 0.1.1, aci_edge_sandboxes_model 0.1.1, nvx =0.1.0-alpha.6 |
| Runtime bundle | Azure/aci-edge-sandbox v0.1.0-build188642429, asset aci-edge-sandboxes-0.1.0-build188642429.zip, SHA-256 cad4db7c574a73aaa1c5a07e7a740cd2b01d5b83c62c81b2196d63bad9ecc126 (the same bundle as crate 0.1.0) |
| NVX in the bundle | 6b485c82c9fdf4f592e1221efc7d6d41ab9bc66c |
| OpenVMM in the bundle | c32c57d129a36fc8c82308d6659f8d7ab5955474 (OpenVMM log: version="0.3.0+gc32c57d12") |
| Host | Windows 11 x64, Windows Hypervisor Platform |
| Guest image | mcr.microsoft.com/azurelinux/base/python:3 |
The bundle's agent library (aci_edge_agent.dll) is ACI's sandbox agent. It translates the crate's network policy into NVX/OpenVMM options in an ACI layer on top of nvx, so some behavior below comes from that ACI layer rather than from NVX. Each section says which.
Observed
1. ICMP policy rules are rejected (ACI layer, not NVX)
An egress rule with "protocol": "icmp" fails at provision:
Error { code: PolicyValidation, message: "OpenVMM cannot select ICMP by itself" }
This message comes from ACI's policy translation, not from NVX. As far as we can tell from source at this pin, NVX's lowering (nvx-rs/src/host/network/lowering.rs) emits CIDR:icmp rules, and OpenVMM's egress rule parser (vm/devices/net/net_backend_resources/src/egress.rs) accepts them. ACI will change its layer; we have not verified end-to-end enforcement by NVX/OpenVMM.
2. ICMP sockets fail in the guest even with egress allowed
With {"egress": {"default": "allow"}, "ingress": {"default": "deny"}}, the workload runs as uid 0 without CAP_NET_RAW, and unprivileged ping sockets are disabled:
uid 0
CapEff: 00000000200004fb
CapBnd: 00000000200004fb
NoNewPrivs: 1
ping_group_range 1 0
dgram errno 13 Permission denied # socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)
raw errno 1 Operation not permitted # socket(AF_INET, SOCK_RAW, IPPROTO_ICMP)
We have not determined whether this capability set and ping_group_range come from NVX's guest init or from ACI's guest configuration. Because no ICMP socket could be opened, we couldn't test whether OpenVMM relays ICMP.
Reproducer
A minimal consumer of the published crate. With the agent-bundled feature, the build downloads and verifies the runtime bundle above.
Cargo.toml:
[package]
name = "nvx-net-repro"
version = "0.1.0"
edition = "2024"
[dependencies]
aci_edge_sandboxes = { version = "=0.1.1", default-features = false, features = ["agent-bundled"] }
serde_json = "1"src/main.rs:
//! Usage: nvx-net-repro <state-root> <policy.json> (--python <script.py> | <guest command...>)
//! Set REPRO_SNAPSHOTS=1 to enable platform snapshots (templates and clones).
use std::sync::Arc;
use aci_edge_sandboxes::agent::{AgentBackend, BundledAgent, PlatformSnapshots, SandboxDefaults};
use aci_edge_sandboxes::{
AciEdgeSandbox, ExecRequest, ImageSource, NetworkPolicy, ProvisionRequest, SandboxSpec,
};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut args = std::env::args().skip(1);
let state = args.next().expect("state root");
let policy: NetworkPolicy =
serde_json::from_str(&std::fs::read_to_string(args.next().expect("policy.json"))?)?;
let rest: Vec<String> = args.collect();
let bundle = BundledAgent::discover()?;
let mut setup = bundle.setup(state).with_defaults(SandboxDefaults::default().with_image(
ImageSource::Reference("mcr.microsoft.com/azurelinux/base/python:3".into()),
));
if std::env::var_os("REPRO_SNAPSHOTS").is_some() {
setup = setup.with_platform_snapshots(PlatformSnapshots::new(2, 512));
}
let backend = Arc::new(AgentBackend::new(bundle.config(setup))?);
let client = AciEdgeSandbox::from_shared(backend.clone());
let request = ProvisionRequest::new().with_network(policy);
let id = client.provision_with(&request, &SandboxSpec::new())?.sandbox_id;
let started = client.start(&id)?;
eprintln!("bootMode: {:?}", started.boot_mode());
let exec = match rest.as_slice() {
[flag, script] if flag == "--python" => {
ExecRequest::argv(["python3".to_owned(), "-c".to_owned(), std::fs::read_to_string(script)?])
}
_ => ExecRequest::command_line(rest.join(" ")),
};
let output = client.exec(&id, &exec)?.wait_with_output()?;
println!("outcome: {}", output.outcome);
print!("{}{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr));
client.stop(&id)?;
client.deprovision(&id)?;
Ok(())
}Run it with cargo run --release -- <state-root> <policy.json> --python <probe.py>, using a short state root such as C:\nvxr.
Policies and probe:
// icmp-rule.json: expect a rejection today
{ "egress": { "default": "deny", "allow": [ { "to": [ { "cidr": "1.1.1.1/32" } ], "ports": [ { "protocol": "icmp" } ] } ] }, "ingress": { "default": "deny" } }
// egress-allow.json
{ "egress": { "default": "allow" }, "ingress": { "default": "deny" } }# icmp_probe.py
import socket, os
print("uid", os.getuid())
for line in open("/proc/self/status"):
if line.startswith(("CapEff", "CapBnd", "NoNewPrivs")): print(line.strip())
print("ping_group_range", open("/proc/sys/net/ipv4/ping_group_range").read().strip())
for name, kind in (("dgram", socket.SOCK_DGRAM), ("raw", socket.SOCK_RAW)):
try: socket.socket(socket.AF_INET, kind, socket.IPPROTO_ICMP).close(); print(name, "ok")
except OSError as e: print(name, "errno", e.errno, e.strerror)Expected (downstream requirement)
- ICMP egress rules, ICMP-only as in Support additional egress protocol combinations #277, are enforced: allowed destinations are reachable by ICMP, and others are blocked, including by deny rules.
- A workload can use ICMP, for example echo, without the consumer depending on a particular ping implementation. We don't assume this requires unrestricted raw sockets.
Questions
- At NVX
6b485c82/ OpenVMMc32c57d1on Windows WHP, is ICMP (and ICMPv6) policy enforcement supported end to end, including relaying echo to allowed destinations? Any limits, such as echo only? - What guest configuration does NVX intend for usable ICMP: for example,
net.ipv4.ping_group_rangecovering the workload's group forSOCK_DGRAMICMP, or an opt-in capability? Is that controlled by NVX's guest init, or left to the integrator? - Any interaction with published listeners or other policies that would restrict this?
Activity
guest: open ICMP echo sockets to every group
doc: specify the ICMP echo support
openvmm: pin the ICMP echo relay
doc: specify the ICMP echo support
openvmm: pin the ICMP echo relay