Skip to content

feat(runtime): add process-local per-model RPM admission limits - #1116

Open
LortuArte wants to merge 2 commits into
Twigpine:mainfrom
LortuArte:lortuarte/rpm-390
Open

LortuArte wants to merge 2 commits into
Twigpine:mainfrom
LortuArte:lortuarte/rpm-390

Conversation

@LortuArte

@LortuArte LortuArte commented Oct 3, 2026 •

Copy link
Copy Markdown

Refs #390 — approved RPM-only slice. This does not implement the broader cost/TPM request.

Summary

Zero currently has no local per-model RPM admission gate at the turn-session boundary.

This change adds a process-local sliding 60-second request window keyed by resolved model ID and rejects request N+1 before the wrapped TurnSession.Stream method is entered.

The limiter is wired into:

  • zero exec
  • interactive sessions
  • model escalation

while preserving the existing provider/transport selection.

FileConfig.modelRPM configures the cap. Unset or zero means unlimited.

Project configuration may tighten an existing user limit, but may not relax it.

Admitted requests consume their slot even when the provider subsequently fails.

Scope / non-goals

This PR is intentionally limited to the scope discussed in #390:

  • RPM only
  • per process
  • sliding 60-second window
  • pre-send admission
  • per resolved model ID
  • native Go
  • clean zero exec error

It does not add:

  • TPM limits
  • monetary/cost budgets
  • persistence across restarts
  • cross-process coordination
  • account-wide limits
  • hidden sleeps or retries
  • Python or AEGIS integration
  • any new runtime dependency

Separate processes have independent windows and restarting Zero clears the limiter state.

The limiter counts TurnSession.Stream admissions. It does not claim to count setup/prewarm traffic, compaction traffic, provider-internal retries, discovery calls, or requests made outside this boundary.

See docs/MODEL_RPM.md.

Regression proof

The regression tests verify that request N+1 is rejected before entering the wrapped transport.

As a negative control, I temporarily bypassed the admission wrapper.

With the gate removed:

  • the provider regression test failed with N+1 was not denied: <nil>
  • the exec regression test observed two provider calls instead of one

The wrapper was then restored before final validation.

This verifies that the new tests fail when the behavior they are intended to protect is removed.

Concurrency result

A 100-request concurrent test with a configured RPM limit of 7 produced:

  • 100 total attempts
  • 7 admissions
  • 7 wrapped transport calls
  • 93 local rejections

No additional admitted request reached the wrapped transport.

Validation environment

Original validation run: 2026-09-27.

Revalidated for delivery on 2026-10-03.

Before opening this draft, I rechecked the upstream repository and confirmed that main still points to the exact validated base:

99721c762f37cd43ac511007a5f51d1846df959e

The submitted branch was compared against the preserved validation files before publication.

Tested on:

  • Linux amd64
  • Go 1.26.6
  • base commit 99721c762f37cd43ac511007a5f51d1846df959e

Passing checks

  • focused RPM/config/provider/CLI/errhint tests: PASS
  • affected RPM/escalation tests under -race: PASS
  • make fmt-check: PASS
  • go vet ./...: PASS
  • go run ./cmd/zero-release build: PASS
  • go run ./cmd/zero-release smoke: PASS
  • make vulncheck: PASS (No vulnerabilities found)
  • git diff HEAD --check: PASS
  • patch application against the clean base with git apply --check: PASS

Full-suite limitation

go test ./... does not pass completely in this validation environment.

It reports 14 failing tests.

I repeated the same suite on an unmodified clean checkout of main at:

99721c762f37cd43ac511007a5f51d1846df959e

under the same environment and network restrictions.

The same 14 test names fail on clean main.

No additional failed-test names were introduced by this patch.

The reproduced failures are:

  1. TestAltScreenTranscriptScrollKeepsFooterFixed
  2. TestPlatformTransportRoundTrip
  3. TestServeCleansStatusThroughBoundDirectoryAfterSwap
  4. TestServePreservesPreviousStatusWhenReplacementFails
  5. TestServeRemovesSocketBoundAfterFinalRuntimeSwap
  6. TestServeSupportsReadOnlyCustomRuntimeDirectory
  7. TestServerEndToEnd
  8. TestServerPublishesDefaultStatusAfterCrashReportCreatesRuntimeDirectory
  9. TestServerRejectsUnknownCommand
  10. TestServerSecondInstanceFails
  11. TestTerminateAndReapDaemonProcess
  12. TestTerminateCommandKillsChildAfterLeaderExits
  13. TestTerminateCommandStopsUnconfiguredCommand
  14. TestUnixTransportFallbackAndSocketMode

The failures include Unix-socket restrictions (operation not permitted), daemon/process lifecycle tests and one transcript-title assertion.

This comparison is evidence that the same failed-test names reproduce on the clean base in this runner.

It is not being presented as a full-suite PASS and it does not establish macOS or Windows validation.

No unrelated test was disabled or modified to hide these failures.

Static lint

make lint-static is also not completely clean in this environment.

It reports four findings:

  • internal/installtest/workflow_permissions_test.go:20:19 — QF1001
  • internal/proxydial/proxydial.go:67:15 — QF1008
  • internal/proxydial/proxydial.go:72:33 — QF1008
  • internal/tools/web_fetch.go:315:21 — QF1008

The same four findings reproduce on clean main, and these files are not modified by this PR.

I left the unrelated code unchanged.

Network isolation note

The new CLI fixtures explicitly inject a simulated/no-op MCP runtime.

For the broader validation run, external HTTP was rejected using a local deny-only HTTP proxy so tests did not make live MCP requests.

That proxy is not an OS-level network sandbox.

No credentials, live provider accounts, or real funds were used by the new fixtures.

The lint/vulnerability tooling was allowed to obtain its normal public tooling/data.

Windows revalidation — 2026-10-04

After the initial draft review, I fixed a recovery-path issue where ModelRPM could be lost when Resolve returned ErrNoActiveProvider.

Regression evidence:

  • before the fix, the new regression test failed with ModelRPM[gpt-test] = 0
  • after the fix, ./internal/config passes
  • the interactive provider-fallback test confirms the RPM limiter is preserved downstream
  • go test ./internal/config ./internal/cli -count=1: PASS
  • go test -race ./internal/config ./internal/cli -count=1: PASS
  • go vet ./...: PASS
  • formatting check: PASS
  • go run ./cmd/zero-release build: PASS
  • go run ./cmd/zero-release smoke: PASS
  • govulncheck: PASS (No vulnerabilities found)
  • git diff HEAD --check: PASS

Windows environment:

  • Windows amd64
  • Go 1.26.6
  • CGO enabled for race validation
  • MSYS2 UCRT64 GCC 16.2.0

The repository-wide go test ./... is still not all-green in this Windows environment.

On the PR branch, one full-suite run failed in internal/tools at TestExecCommandForegroundServerReturnsSessionAndServesHTTP; that same test then passed in isolation.

I then created a detached clean worktree at the exact base commit:

99721c762f37cd43ac511007a5f51d1846df959e

On that clean base:

  • TestExecCommandForegroundServerReturnsSessionAndServesHTTP passed 5/5 in isolation
  • a full go test ./... run still failed, but in a different unrelated baseline test: TestACPEndToEndPrompt with context deadline exceeded

I have not modified unrelated tests or runtime code to hide these environment/baseline failures.

Latest fix commit:

9de88d736ed1d1202e1093d958bcb44d7c1f2836

Maintainer review status

The RPM implementation, recovery-path regression fix, focused tests, race validation, vet, build, smoke, formatting, vulnerability scan and diff hygiene all pass locally.

The repository-wide Windows suite is not all-green, but the clean base also produces unrelated full-suite failures under the same environment.

The PR is ready for project CI and maintainer review. I have intentionally left unrelated baseline/environment failures unchanged.

Disclosure

AI-assisted implementation and local verification.

This PR does not claim maintainer endorsement, Zero adoption of AEGIS, or production certification.

Summary by CodeRabbit

  • New Features
    • Configure per-model completion limits with modelRPM. Limits apply to interactive turns and zero exec; requests over the limit are rejected locally with guidance on when to retry.
    • Project settings can tighten, but not relax, limits set in user configuration.
    • Limits are shared across sessions and model switches within a process; separate processes have independent limits.
  • Documentation
    • Added a guide to per-model limits, including configuration rules, timing, and which requests count. The user documentation now links to it.

@coderabbitai

coderabbitai Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: Twigpine/zero/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b1c1cd9b-9c03-4aad-b334-c1436ce3ba0d
📥 Commits

Reviewing files that changed from the base of the PR and between c9c30a5 and 9de88d7.

📒 Files selected for processing (3)
  • internal/cli/app_test.go
  • internal/config/resolver.go
  • internal/config/resolver_test.go

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.


Walkthrough

Adds configurable per-model completion limits with a process-local sliding-window limiter. The limiter applies to exec and interactive turn sessions, including provider escalation. Configuration validation, error hints, tests, and user documentation are also added.

Changes

Per-model RPM limits

Layer / File(s) Summary
Configure and resolve model limits
internal/config/types.go, internal/config/model_rpm.go, internal/config/resolver.go, internal/config/model_rpm_test.go, internal/config/resolver_test.go
Configuration accepts modelRPM values, normalizes model names, rejects invalid or conflicting limits, and applies project and provider-command limits without relaxing the user cap. Resolved configuration carries the merged limits.
Enforce the sliding-window limit
internal/zeroruntime/rpm.go, internal/zeroruntime/rpm_test.go
A process-local limiter tracks per-model stream admissions over 60 seconds. It rejects over-limit calls before the underlying stream, reports retry duration, and counts admitted calls even when the underlying stream fails.
Apply limits to provider sessions
internal/providers/factory.go, internal/providers/turn_session.go, internal/providers/escalation.go, internal/providers/model_rpm_test.go
Configured sessions wrap streams with the limiter. Escalation passes the limiter to switched-provider sessions. Tests cover optimized and unoptimized session paths.
Wire limits into CLI and interactive turns
internal/cli/app.go, internal/cli/exec.go, internal/cli/model_rpm_test.go, internal/cli/app_test.go, internal/tui/options.go, internal/tui/model.go, internal/errhint/errhint.go, internal/errhint/errhint_test.go, docs/MODEL_RPM.md, docs/README.md
Exec and TUI setup pass the limiter into provider sessions. Local RPM errors receive CLI and TUI guidance. Tests cover both entry points, and the documentation describes configuration and limiter behavior.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant ConfiguredTurnSessions
  participant ModelRPMLimiter
  participant TurnSessionProvider
  CLI->>ConfiguredTurnSessions: provide profile and limiter options
  ConfiguredTurnSessions->>ModelRPMLimiter: wrap the selected model session
  CLI->>ModelRPMLimiter: call Stream
  ModelRPMLimiter->>TurnSessionProvider: forward admitted Stream call
Loading

Suggested reviewers: anandh8x, gnanam1990, pierrunoyt

Merge Risk: ⚪ Minimal · up to 9de88

The change adds process-local per-model admission limits before provider streams, and the inspected configuration flow preserves the documented cap behavior. No actionable merge-blocking risk is established; it appears ready for normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 9de88

The request cap can be silently lost when startup enters provider setup to repair an incomplete configuration. Later ordinary turns in that process can then run without the configured cap. This is a bounded admission-control defect; no broader privilege expansion was established.

Retained concerns

  • Medium · reliability · inferred: When provider resolution requires setup and no usable saved-provider fallback applies, interactive startup discards valid ModelRPM settings. Completing setup activates a provider without rebuilding the limiter, so subsequent ordinary turns for a configured capped model are ungated until a later successful startup. The recovery branch predates this PR, but its interaction with the newly introduced admission policy creates a control-lifecycle defect.
Security review details

Security Blast Radius

  • inferred — The identified defect affects ordinary turns in an interactive process that entered the clearing recovery branch and then completed setup. It removes local request-rate containment for configured models; it does not establish expanded tenant access or credential privileges.

Security Findings and Attack Paths

  • inferred — The supported failure path is valid cap configuration, recoverable provider-resolution failure, cleared configuration, operator-completed setup, and ordinary turns using a captured nil limiter. Attacker-controlled completion of setup was not demonstrated. Excluding setup traffic from accounting does not exclude ordinary turns after setup.

Trust Boundaries and Controls

  • observed — Project and provider-command inputs can add or tighten a cap but cannot relax an existing positive cap. Provider-command ModelRPM is cleared before the ordinary merge, preventing a second overwrite. The usable saved-provider fallback retains the resolved cap instead of entering the clearing recovery branch.

Resilience and Maintainability Implications

  • observed — The wrapper delegates session cleanup without refunding process-owned admissions. The agent loop defers closing the current session, keeps the old session when opening an escalation target fails, and closes the old session after successful replacement.

Hardening Proposals

  • proposed — Separate validated admission policy from recoverable provider-selection state. Provider repair should preserve that policy and attach newly activated providers to the same process-owned limiter without erasing active admission history.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 35 functions across 19 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: process-local per-model RPM admission limits.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Oct 3, 2026
@LortuArte
LortuArte marked this pull request as ready for review October 4, 2026 07:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant