Skip to content

test-microvm: cap a busy loop with cpu.max - #1300

Closed
Pedro Henrique Penna (ppenna) wants to merge 1 commit into
agents/issue-1193-cpu-controllerfrom
agents/issue-1193-cpu-bandwidth-scenario
Closed

Pedro Henrique Penna (ppenna) wants to merge 1 commit into
agents/issue-1193-cpu-controllerfrom
agents/issue-1193-cpu-bandwidth-scenario

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Refs #1193. Stacked on #1299, which builds the CPU controller's bandwidth limit into the guest kernel.

#1299 lets a guest agent cap an execution's CPU time with its cgroup's cpu.max, and pins the kernel settings to the build's checks, but no scenario shows that the limit holds in a guest. This adds one.

What changed

  • The cpu-bandwidth scenario. Its guest script, cpu-bandwidth.sh.in, mounts cgroup v2 unless it is mounted, enables the CPU controller in the root cgroup, limits a cgroup to a quarter of a CPU, 25000 100000 in its cpu.max, and runs a busy loop in it for a second; the loop moves itself into the cgroup by writing 0 to its cgroup.procs. The guest then reports the limit that cpu.max reads back, the wall time from /proc/uptime, and the cgroup's cpu.stat. A kernel without the controller fails the script at once, with NVX-CPU-BANDWIDTH-FAIL the kernel has no CPU controller.
  • The check. check_cpu_bandwidth reads the one report through the module's _single_marker_value and parse_fields, and requires the limit that the guest wrote, at least one throttled period, and at most half of the wall time of CPU, twice the limit's share. The kernel enforces the quota at its 10 ms ticks, so the loop can overrun a period's 25 ms by about a tick, and the period in which the loop starts gives it a full quota; the loop used 26.8% to 28.1% of the wall time in every run below. A loop without the limit uses all of a CPU.
  • The suite. The scenario is a default scenario, dispatched after the others. plan-microvm-shards --seconds cpu-bandwidth=2 put it in the shorter shard of each backend and moved no other scenario; a regeneration from CI's reports will replace the estimate.
  • The documents. doc/ci.md describes the scenario, and the inventory names it among the CPU controller's validation.

Changes to existing tests (tests-modified)

check_test_changes.py --base origin/dev reports two changes, both additions to the registries of the end-to-end harness; no existing scenario or test changes.

Change Why
MICROVM_TEST_SCENARIOS gains cpu-bandwidth. The tuple lists the scenarios that a default run and the shards run.
_run_scenarios gains the dispatch of cpu-bandwidth, after every other scenario. Each scenario of the tuple runs through its own block of the runner.

microvm_shards.json, the shards' CI wiring, adds the scenario and its estimated duration to one shard of each backend, whose total grows by 2 s. The new unit test, test_cpu_bandwidth_requires_cpu_max_to_throttle_a_busy_loop, checks the limit that the rendered script writes and its marker, accepts a report at exactly half a CPU, and refuses one a microsecond over it, one without a throttled period, one whose cpu.max holds another limit or none, and a guest output with no report or two.

Validation

Locally, on Windows: the commands of validate-nvx (1,816 tests, 73 skipped, among them all 344 of test_microvm_tests.py; test_adversarial.py; test_hosts.py), Ruff, Pyright for Linux and Windows, the Markdown link check, the test layout check, and check-kernel-config passed. On nvx-kvm-test (Linux), at 5e4893313, with NVX_RS naming a release build of nvx-rs and without the Azure setup packages that validate-nvx installs: its tests (1,816, 35 skipped), the command help, and check-kernel-config passed.

On the test VMs of the host inventory, all virtual-machine hosts, with the release artifacts of 4cfe8330e (v0.1.0-dev.4cfe8330e719) and either their kernel or #1299's, built by the Docker build's kernel-artifacts target. 5e4893313 is this branch's head before its commit message last changed, with the same tree; 8a5847698 was its first revision, whose check read the report by hand.

Profile Backend test-microvm --scenario cpu-bandwidth at 5e4893313, release kernel With #1299's kernel Default suite at 8a5847698, #1299's kernel
nvx-kvm-test KVM failed: the kernel has no CPU controller passed: 277,064 µs of CPU in 1.01 s, 10 of 10 periods throttled passed, 58 scenario runs in 559 s
nvx-mshv-test MSHV failed: the kernel has no CPU controller passed: 279,546 µs in 1.01 s, 10 of 10; with the Ubuntu guest 271,019 µs, 10 of 10; with the Azure Linux guest 274,928 µs in 1.00 s, 9 of 10 passed, 58 runs in 503 s
nvx-whp-test WHP failed: the kernel has no CPU controller passed: 278,406 µs in 1.01 s, 10 of 10 passed, 58 runs in 626 s

At 8a5847698, the scenario also failed with the release kernel and passed with #1299's on all three, alone and in the default suites, in 1.5 to 1.8 s.

Left to CI: the microVM shards that run the scenario on the shared KVM, MSHV, and WHP runners, which a draft pull request skips.

The guest kernel builds the CPU controller's bandwidth limit, so that a
guest agent can cap an execution's CPU time with its cgroup's cpu.max
(#1193), but no scenario showed that the limit holds. The cpu-bandwidth
scenario enables the controller in the root cgroup, limits a cgroup to
a quarter of a CPU, 25 ms of every 100 ms period, and runs a busy loop
in it for a second. The guest reports the limit that cpu.max reads
back, the wall time, and the cgroup's cpu.stat, and the harness
requires the limit, a throttled period, and at most half a CPU, twice
the limit's share, which a loop without the limit exceeds.

The scenario runs in the default suite, in the shorter shard of each
backend, which plan-microvm-shards chose with an estimate of 2 s; it
took 1.5 to 1.8 s. doc/ci.md describes it, and the kernel feature
inventory names it among the CPU controller's validation. A unit test
checks the limit that the scenario's script writes, its marker, and the
check's bound and failures.

On nvx-kvm-test (Linux/KVM), nvx-mshv-test (Linux/MSHV), and
nvx-whp-test (Windows/WHP), all virtual machines, with the release
artifacts of 4cfe833, the scenario failed with the release kernel,
which has no CPU controller, and passed with the kernel of the CPU
controller's change. In each of 11 runs, among them one each of the
Ubuntu and the Azure Linux guest on nvx-mshv-test, its limit held the
loop to 26.8% to 28.1% of the wall time and throttled it in 9 or 10 of
10 periods. The default suite passed with that kernel on all three.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

This pull request changes existing tests

It modifies or removes 2 existing test asset(s), listed below. Review each with care: a change to a test can hide a regression that the test caught. See doc/contribute.md#changes-to-existing-tests.

End-to-end tests

Change What Before After
modified helper assignment MICROVM_TEST_SCENARIOS in scripts/nvx_tools/microvm_tests.py scripts/nvx_tools/microvm_tests.py:99 scripts/nvx_tools/microvm_tests.py:99
modified helper def _run_scenarios in scripts/nvx_tools/microvm_tests.py scripts/nvx_tools/microvm_tests.py:10890 scripts/nvx_tools/microvm_tests.py:10957

Relocated unchanged: 0; added: 8. Compared the merge base 478cd69f6aac with 5dae66276a16.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the added CPU bandwidth test coverage.

No open issues 🎉

What changed in this PR

Adds end-to-end coverage for the guest CPU bandwidth controller enabled by #1299, validating that cpu.max throttles a busy loop as intended for #1193.

Changes:

  • Adds a cpu-bandwidth scenario that applies a quarter-CPU quota and checks the reported limit, throttling, and CPU usage.
  • Registers the scenario in the default suite and all backend shard plans, with unit tests for report validation.
  • Documents the scenario and links it to the kernel feature inventory.
File Description
scripts/​test_microvm_tests.py Tests script rendering and report acceptance and rejection cases.
scripts/​nvx_tools/​microvm_tests.py Registers, runs, and validates the CPU bandwidth scenario.
scripts/​nvx_tools/​microvm_test_scripts/​cpu-bandwidth.sh.in Applies the quota, runs a busy loop, and reports cgroup statistics.
scripts/​nvx_tools/​microvm_shards.json Adds the scenario and estimated duration to each backend’s shard plan.
scripts/​nvx_tools/​kernel_features.py Records the scenario as CPU controller validation.
doc/​ci.md Describes the scenario and its pass criteria.

Review effort: Balanced

@ppenna

Copy link
Copy Markdown
Contributor Author

Closing without merging: this will not be implemented, because it goes away with the nvx refactor (feature/nvx-refactor). #1193 is closed as not planned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests-modified Changes existing tests, test data, or test wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants