diff --git a/CHANGELOG.md b/CHANGELOG.md index ced046d..85fb838 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,70 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) --- +## [0.14.9] - 2026-08-07 + +v3.38 wire-drift close — three real contract bugs that diverged from backend source code. Verified against `backend/src/proxy/http/protocol.rs`, `backend/src/proxy/middleware/auth.rs`, and CLAUDE.md §5 / §13 — not against comments or documentation. No SDK_MIN_VERSION bump. No on-wire change (backend already shipped the matching wire shape; this SDK release closes the consumer side). + +### Fixed + +- **Capabilities probe route** — `nullrun.capabilities.CAPABILITIES_PATH` was `"/health"` (a generic liveness endpoint) instead of the canonical `"/api/v1/capabilities"`. Pre-fix, every `init()` probe returned `None` and `is_v3_ready()` was always `False`, so every v3 capability flag (`server_minted_execution_id` / `per_execution_reservations` / `enforcement_modes_soft` / `heartbeat_time_based`) was a runtime no-op — even when the backend was v3-ready. The new probe URL matches `backend/src/proxy/http/protocol.rs::capabilities_handler` (canonical wire contract since 2025-04). +- **API_KEY_* error code granularity (v3.38 backend split)** — backend v3.38 split the `API_KEY_REVOKED` bucket into five distinct wire codes: `API_KEY_EXPIRED` / `API_KEY_DISABLED` / `API_KEY_INVALID` / `API_KEY_MISSING` / `API_KEY_MALFORMED` (mirrors CLAUDE.md §13 vocabulary). Pre-fix, only `API_KEY_REVOKED` was mapped in `_V3_ERROR_CODE_MAP`; the other five silently fell through to the generic HTTP-status fallback at `transport.py:~2616` and never surfaced as `NullRunAuthError`, losing both the exception class and the diagnostic `wire_code`. The map now covers all six wire codes. The envelope parser filters unknown `details` keys to a known kwargs set (`{error_code, user_action, retryable, docs_url, cause}`) and parks extras on `self.details` — the pre-fix behaviour was to forward every detail as a kwarg and raise `TypeError` on the first unknown key (the regression appeared once v3.38 EXPIRED responses started emitting `expires_at` in details). +- **`NullRunAuthError.wire_code`** — the exception class gains a `wire_code: str | None = None` constructor kwarg that defaults to `"API_KEY_REVOKED"` for backwards compat. Mirrors the existing `NullRunChainError.backend_code` pattern at `breaker/exceptions.py:448`. Handlers can now branch on the granular lifecycle signal instead of inferring from message strings. + +### Added + +- **`decision == "soft_pass"` handler in `check_workflow_budget`** — the runtime's `/gate` decision dispatcher gains a `soft_pass` branch (currently the only branch missing from the source). Pre-fix the branch was absent, so soft-mode calls that proceeded via the chain's overdraft cap fell through the default allow path with no log line and no `soft_overdraft_used` counter increment — silent budget drift. The new branch: + - calls `metrics.inc_runtime("soft_overdraft_used")` so the dashboard can graph soft-cap pressure + - logs at WARNING with `overdraft_used_cents` / `max_overdraft_cents` / `remaining_overdraft_cents` from the backend response so operators can see which chains are burning overdraft + - returns normally (the `allow` semantic is correct — the gate already authorised the call via the chain's overdraft cap) + +### Tests + +- `tests/test_v3_38_drift_fixes.py` — 14 new regression tests across three classes: + - `CAPABILITIES_PATH` is `"/api/v1/capabilities"` (constant pin); probe against canonical route with v3 payload yields `is_v3_ready() == True` (negative pin against `/health` mocks). + - `_V3_ERROR_CODE_MAP` covers all six wire codes (6-case parametrise); `NullRunAuthError.wire_code` surfaces the granular backend code (default to `API_KEY_REVOKED`); envelope parser filters unknown details without raising `TypeError`. + - Static-source scan pins the `soft_pass` branch structure (counter increment, WARNING log, `overdraft_used_cents` reference) — mirroring the `migration_drift_tests` pattern used elsewhere in the SDK and backend. A future refactor that drops the branch fails the test in CI rather than at first production `/check`. +- `tests/conftest.py` / `tests/test_capabilities.py` / `tests/test_init_contract.py` updated to mock `/api/v1/capabilities` (was `/health`). + +### Compatibility + +- **No SDK_MIN_VERSION bump.** All three fixes are consumer-side; the backend already shipped the matching wire shape. +- **No public API change.** `CAPABILITIES_PATH` / `_V3_ERROR_CODE_MAP` / `NullRunAuthError` are internal implementation details; the public surface (`nullrun.init(...)`, `@protect`, `decision`-keyed `GateResponse` parsing) is unchanged. +- **Test suite: 1457 passed, 7 skipped** (no regressions from the wire-drift close; pre-fix the affected tests were passing on the wrong-shape mock responses). + +--- + +## [0.14.8] - 2026-08-06 + +Execution Graph v0 — additive sub-agent lineage. The backend landed `parent_execution_id` as an optional wire field on `/api/v1/gate` (backend commit `87fae759`, not pushed yet) so an SDK spawning a sub-agent can name the parent's `execution_id`. Backend validates ownership against the parent's `execution:{id}` Redis binding (mirrors the `/cancel` ownership check) and rejects cross-org / cross-key / not-found with `403 PARENT_EXECUTION_*`. This release ships the SDK-side forward path, the matching capability flag, and the three-way error-code mapping. Wire change is strictly additive (omitted when `None`); no SDK_MIN_VERSION bump. + +### Added + +- **`parent_execution_id` on `/check` (gate)** — `Transport.check(check_request=...)` forwards the optional `parent_execution_id` field from `check_request` onto the wire when the caller passes a non-None string. Omitted entirely when absent or explicitly `None`, so legacy / single-shot callers keep the previous payload shape. Mirrors the additive forward pattern used by `chain_id` / `tool_arguments` / `idempotency_key` at `src/nullrun/transport.py:1607-1626`. Sub-agent SDKs stamp the field manually from a caller-supplied UUID; auto-injection from a "current execution_id" contextvar is deferred (v0 is intentionally caller-owned). +- **`execution_graph` capability flag** — `parse_capabilities` reads the new `execution_graph: bool` from `/api/v1/capabilities` (nested under `capabilities:` with top-level fallback for pre-1.0.0 backends). `ServerCapabilities.execution_graph` exposes the flag so SDKs can probe whether the deployment supports sub-agent lineage before sending the field. Pre-Graph backends silently ignore unknown fields, but the probe lets SDKs surface a clean diagnostic at `init()` rather than a 400 on the first call. +- **`NullRunChainError.parent_execution_id`** — the chain error class gains an optional `parent_execution_id: str | None = None` constructor kwarg (mirroring the existing `chain_id` kwarg at `breaker/exceptions.py:425`). When the backend rejects a sub-agent call with `PARENT_EXECUTION_*`, the offending parent id is preserved on the exception so cookbook code can log / surface it without re-parsing the message string. + +### Changed + +- **Three new error codes mapped to `NullRunChainError`** — `PARENT_EXECUTION_NOT_FOUND`, `PARENT_EXECUTION_ORG_MISMATCH`, `PARENT_EXECUTION_KEY_MISMATCH` (all 403) are added to `_V3_ERROR_CODE_MAP` at `src/nullrun/transport.py:2675-2685`. Mapped to `NullRunChainError` (not a new class) because the diagnostic profile is identical to `CHAIN_CROSS_ORG` / `CHAIN_ORG_MISMATCH` — 403-class security errors with `(org_id, api_key_id)` ownership semantics. Diagnostic clarity wins over a new exception class per CLAUDE.md §13 philosophy. + +### Tests + +- `tests/test_transport.py::TestParentExecutionIdForwarding` — 3 new tests: `test_check_forwards_parent_execution_id_when_present` (round-trips from `check_request` → wire JSON), `test_check_omits_parent_execution_id_when_absent` (legacy / single-shot callers keep the old payload shape), `test_check_omits_parent_execution_id_when_none_explicit` (explicit `None` is treated as "no parent" / single-shot). + +### Compatibility + +- **Backward-compatible additive wire change.** Pre-Execution-Graph SDKs that never set `parent_execution_id` continue to work unchanged — the field is omitted entirely from the wire. +- **Backward-compatible capability flag.** Pre-Graph backends return `execution_graph: false` (or omit the field entirely); the SDK treats both as "don't send the parent field". `is_v3_ready()` is unchanged — the flag is informational, not a hard gate. +- **Backward-compatible exception class.** `NullRunChainError` gains a kwarg with a default; the existing 4-arg call sites (CHAIN_MAX_DURATION_EXCEEDED, CHAIN_CROSS_ORG, CHAIN_ORG_MISMATCH, CHAIN_NOT_FOUND/EXPIRED) continue to work unchanged. +- No on-wire change for legacy callers. No SDK_MIN_VERSION bump. The `parent_execution_id` field is omitted on the wire whenever the caller does not pass it explicitly. + +### Refs + +- Backend commit `87fae759` (not pushed; awaiting local review + push authorisation). Additive wire contract at `backend/src/proxy/http/gate/schemas.rs:62-73`; ownership validation at `backend/src/proxy/http/gate/internal.rs` (lifts `parent_execution_id` parsing before the validation block + persistence call site); migration 266 adds `execution_records.parent_execution_id` + partial index for graph queries (Tasks #11-14, not in v0). + +--- + ## [0.14.7] - 2026-08-04 Init contract hardening — strip leading and trailing whitespace from `api_key` (and the `NULLRUN_API_KEY` env fallback) BEFORE the truthiness check in `nullrun.init()` and `NullRunRuntime.__init__`. Pre-fix, whitespace-only strings (`" "`, `"\t"`, `"\n"`) are TRUTHY in Python and silently slipped past the empty-key guard; they were stored on the runtime and reached the gateway as a malformed `Authorization: Bearer ***` header, surfacing as a backend 401 only on the first `/gate` call rather than at startup. diff --git a/README.md b/README.md index cbba340..11c8b54 100644 --- a/README.md +++ b/README.md @@ -1,116 +1,302 @@ -

- PyPI version - Python versions - License - Downloads -

- -

- CI - Coverage - Stars - Documentation -

- -# nullrun - -**Enforcement gateway for AI agents.** - -Stop runaway agents before they burn the budget. NullRun sits between your -code and your LLM calls, tracking cost and tool usage so a single agent can't -take down your account. - -> ⚠️ **Status: alpha.** The public API may shift between minor versions. -> Pin your dependency and read the [CHANGELOG](./CHANGELOG.md) on every -> upgrade. +
+ + +NullRun — Runtime decision layer for AI agents + +# NullRun + +**Ship AI agents with real-time budget, policy, and human-approval gates.** + +Zero-refactor cost control, tool policy enforcement, and audit trail for any +LLM-powered agent — works with OpenAI, Anthropic, LangGraph, CrewAI, AutoGen, +LlamaIndex, and your own stack. + +[Quickstart](#-quickstart) · [Docs](https://docs.nullrun.io) · [Examples](https://github.com/nullrunio/nullrun-examples) + + +
+ PyPI version + Python versions + License + Downloads +
+ + +
+ CI + Coverage + Stars + Last commit +
+ + +
+ protocol v3.31 + Zero-code instrumentation + Server-authoritative cost +
+ +
--- -## Install +> ⚠️ **Status: alpha (v0.14.7, protocol v3.31.6).** The public API may shift between minor versions. Pin your dependency and read the [CHANGELOG](https://github.com/nullrunio/nullrun-sdk-python/blob/master/CHANGELOG.md) before upgrading. + +--- + +## Why NullRun? + +AI agents can overspend, call dangerous tools, and act without audit trails. +Existing observability tools tell you **after** the fact. NullRun enforces **before** the action. + +| Without NullRun | With NullRun | +|---|---| +| Agent calls `gpt-4o` 10,000 times → surprise $5,000 invoice | Hard budget cap → SDK blocks at 402 before invocation | +| Agent runs `bash rm -rf /` | Tool policy → SDK blocks at 403 before execution | +| Sensitive action with no human in the loop | Approval flow → SDK pauses and waits for WS `approval_resolved` push | +| Cost & calls scattered across 4 libraries | Single source of truth: per-org, per-workflow, per-execution | +| Runaway SDK loop calling `/gate` without `/track` | Per-reservation rate cap → 402 budget error (see `docs/errors/NR-R001.md`) | + +--- + +## Features + +| | | +|---|---| +| **Hard & soft budget gates** — atomic Redis-enforced, no client-trust model | **Tool policy enforcement** — block dangerous tools before execution | +| **Human-in-the-loop approvals** — pause agent and await `approval_resolved` via WS push | **Immutable audit trail** — every decision, every tool call, every cent | +| **Zero-code instrumentation** — `nullrun.init()` patches `httpx` once for any vendor | **LangGraph, CrewAI, AutoGen, LlamaIndex** — first-class integrations | +| **Memory-safe streaming** — 16 MiB response body cap (anti-OOM); full body for usage extraction | **Lightweight** — no LLM-key storage, no proxy required | +| **Server-authoritative cost** — wire protocol v3.31, server-minted execution IDs | **MCP support** — expose tools to agents via Model Context Protocol | + +--- + +## Architecture + +```mermaid +%%{init: { +'flowchart': { + 'curve': 'basis', + 'htmlLabels': true, + 'nodeSpacing': 80, + 'rankSpacing': 90 +} +}}%% + +flowchart LR +%% ========================= +%% AI RUNTIME +%% ========================= +subgraph USER ["👤 AI Runtime"] +direction TB +A["🤖 Agent"] +end + +%% ========================= +%% NULLRUN LAYER +%% ========================= + +subgraph LIB ["📦 NullRun Enforcement Layer"] +direction TB +B["NullRun SDK
Interceptor"] +C["🚦 Runtime Gate"] +P["📜 Policy Engine"] +H["👤 Human Approval"] + +end + +%% ========================= +%% PRODUCTION +%% ========================= + +subgraph PROD ["⚙️ Production Actions"] +direction TB + +T["🛠 Tools"] +API["🌐 External APIs"] +DB["🗄 Databases"] +end + +STATE["🗂 Audit + Runtime State"] + +%% ========================= +%% FLOW +%% ========================= + +A -->|"protected action"| B +B -->|"authorize"| C +C --> P +P -->|"allow"| T +P -->|"allow"| API +P -->|"allow"| DB +C -->|"require approval"| H +H -->|"approved"| T +C --> STATE + +%% ========================= +%% COLORS +%% ========================= +classDef user fill:#dbeafe,stroke:#2563eb,color:#0f172a +classDef sdk fill:#dcfce7,stroke:#16a34a,color:#0f172a +classDef srv fill:#fed7aa,stroke:#ea580c,color:#0f172a +classDef store fill:#f5d0fe,stroke:#a21caf,color:#0f172a +classDef ok fill:#bbf7d0,stroke:#16a34a,color:#0f172a +classDef wait fill:#fef08a,stroke:#ca8a04,color:#0f172a + +class A user +class B sdk +class C,P,H srv +class STATE store +class T,API,DB ok +class H wait + +style USER fill:#f8fafc,stroke:#64748b,stroke-width:1px +style LIB fill:#f8fafc,stroke:#64748b,stroke-width:1px +style PROD fill:#f8fafc,stroke:#64748b,stroke-width:1px +``` + +The gate is **server-authoritative** — the SDK never trusts client-supplied +cost. Redis is the source of truth for budget and tool-policy state; Postgres +holds the immutable audit log. + +--- + +```mermaid +sequenceDiagram + +participant Agent +participant SDK +participant Gate +participant Policy +participant Human +participant Tool + + +Agent->>SDK: execute(tool) +SDK->>Gate: authorize(action) +Gate->>Policy: evaluate rules + +alt Allowed +Policy-->>Gate: allow +Gate-->>SDK: continue +SDK->>Tool: execute +else Approval required +Policy-->>Gate: approval_required +Gate-->>SDK: wait +Gate->>Human: request approval +Human-->>Gate: approved +Gate-->>SDK: resume +SDK->>Tool: execute +else Blocked +Policy-->>Gate: deny +Gate-->>SDK: exception +end +``` + +## Quickstart + +Install: ```bash pip install nullrun +export NULLRUN_API_KEY="nr_..." # get one at https://nullrun.io/control-center/api-keys ``` -## Quick start - -Wrap any function that calls an LLM with `@protect` and you're done — cost -and tool calls are tracked automatically. +### Option — decorator (3 lines) ```python from nullrun import protect @protect def my_agent(prompt: str) -> str: - return call_my_llm(prompt) + return call_llm(prompt) + ``` +--- -Or drop in zero-code auto-instrumentation for the LLM libraries you already -use. Pass your API key once at startup; supported vendors are detected -automatically. +## How NullRun compares -```python -import nullrun -import openai +| | **NullRun** | LangChain callbacks | Helicone | Portkey | OpenLLMetry | +|---|---|---|---|---|---| +| **Enforce before execution** | ✅ | ❌ observe-only | ⚠️ async | ⚠️ async | ❌ | +| **Server-authoritative budget** | ✅ | ❌ | ❌ | ❌ | ❌ | +| **Tool-call policy** | ✅ | ❌ | ❌ | ⚠️ limited | ❌ | +| **Human-in-the-loop approvals** | ✅ | ❌ | ❌ | ❌ | ❌ | +| **Zero-code instrumentation** | ✅ | ✅ | ✅ | ✅ | ✅ | +| **Immutable audit trail** | ✅ | ⚠️ | ✅ | ✅ | ✅ | +| **Streaming memory cap (anti-OOM)** | ✅ | ❌ | ⚠️ | ⚠️ | ❌ | +| **MCP support** | ✅ | ⚠️ | ❌ | ❌ | ⚠️ | -nullrun.init(api_key="nr_...") +> NullRun is the only option that **blocks** expensive or dangerous calls *before* they happen, not just observes them. -client = openai.OpenAI() -client.chat.completions.create(...) # tracked, no other changes needed -``` -## Configuration +--- + +## Examples + +Runnable, copy-pastable examples live in a separate repo so you can adapt without cloning the SDK source: + +- **LangGraph** — multi-node agent with budget + approval [→](https://github.com/nullrunio/nullrun-examples/tree/main/langgraph) +- **CrewAI** — multi-agent crew with shared budget [→](https://github.com/nullrunio/nullrun-examples/tree/main/crewai) +- **AutoGen** — group-chat agent with policy gating [→](https://github.com/nullrunio/nullrun-examples/tree/main/autogen) +- **LlamaIndex** — RAG pipeline with cost-per-query enforcement [→](https://github.com/nullrunio/nullrun-examples/tree/main/llama-index) +- **Custom tools** — register your own tools for policy [→](https://github.com/nullrunio/nullrun-examples/tree/main/custom-tools) +- **Multi-agent** — shared budget across sub-agents [→](https://github.com/nullrunio/nullrun-examples/tree/main/multi-agent) + +--- -Two environment variables cover almost every setup: +## Roadmap -| Variable | Default | Purpose | +| Version | Status | Highlights | |---|---|---| -| `NULLRUN_API_KEY` | — | Your NullRun API key. **Required.** | -| `NULLRUN_API_URL` | `https://api.nullrun.io` | Backend base URL (override for self-hosted). | +| **v0.14.x** (current) | ✅ alpha | Wire protocol v3.31, server-minted execution IDs, MCP, anti-OOM streaming cap | +| **v0.15** | 🚧 in progress | OpenTelemetry exporter, Redis-backed offline queue, hardened init contract | +| **v0.16** | 📋 planned | Cost prediction from prompt, semantic tool policy (regex → AST) | +| **v1.0** | 🎯 beta target | Stable wire contract, full async support, type-safe decisions | -Everything else — batching, transport tuning, mTLS, vendor-specific options -— lives in the docs: +[Full roadmap & RFCs →](https://docs.nullrun.io/roadmap) -- 📘 **Full configuration reference**: +--- -## Examples +## Development setup -A growing set of runnable examples (LangGraph, OpenAI Agents, raw OpenAI, -Anthropic, multi-agent) is maintained in a separate repo so you can copy -and adapt without cloning the SDK source: +```bash +git clone https://github.com/nullrunio/nullrun-sdk-python +cd nullrun-sdk-python +python -m venv .venv && source .venv/bin/activate +pip install -e ".[dev]" +pytest -q +``` -- 🧪 **Examples repo**: +We follow [Conventional Commits](https://www.conventionalcommits.org/), +require tests for new public API, and run `ruff` + `mypy` in CI. -## Documentation +--- -Concept guides, integration recipes, and the full Python API reference: +## Security -- 📖 +NullRun does **not** store or proxy your LLM provider keys — it sits beside your existing clients and observes the calls. The gate is **server-authoritative** for cost: even a malicious SDK cannot inflate spend by sending a fake `cost_cents` to `/track`. -## Project & organisation +See the security policy at for the threat model and disclosure policy. + +To report a vulnerability: **support@nullrun.io**. + +--- + +## Community & support + +- **GitHub Issues**: +- **GitHub Discussions**: +- **Enterprise support**: support@nullrun.io + +--- + +--- -This SDK is one part of the NullRun platform. +
-- 🏢 **Organisation**: -- 🐛 **Issues**: -- 📝 **Changelog**: +Made with care by [NullRun](https://nullrun.io) and contributors. -## License +[⭐ Star us on GitHub](https://github.com/nullrunio/nullrun-sdk-python) · [📖 Read the docs](https://docs.nullrun.io) -Apache-2.0 +
diff --git a/docs/assets/banner.svg b/docs/assets/banner.svg new file mode 100644 index 0000000..c014664 --- /dev/null +++ b/docs/assets/banner.svg @@ -0,0 +1,230 @@ + + + NullRun — Runtime Authorization for AI Agents + Hero banner for NullRun: runtime authorization for AI agents. Shows the brand mark, wordmark, tagline, feature chips, the website nullrun.io, and a live decision log mockup with allow / flag / block states. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + NULLRUN + + + + NULLRUN.IO + + + + + + + + + + + + + + + + + + NullRun + + + Runtime Authorization for AI Agents + + + + + + + + + + + Server-authoritative + + + + + Zero-code + + + + + + + + Live + + + + + nullrun.io + + + + + + + + + + + + + + + + + + + + DECISION LOG · LIVE + + + + + + + + + + + + + + + 00:01:23 + claude-sonnet-4-6 · tools/bash + + + + ALLOW + + + + + + + + 00:01:24 + claude-sonnet-4-6 · execute_code + + + + FLAG + + + + + + + + 00:01:25 + claude-sonnet-4-6 · rm -rf /tmp + + + + BLOCK + + + + + + + + 00:01:26 + gpt-4o · chat.completions + + + + ALLOW + + + + + diff --git a/pyproject.toml b/pyproject.toml index 5b08112..a9e9e32 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -152,11 +152,40 @@ name = "nullrun" # startup. The strip normalises the value before storage so the # HMAC signing path and the Authorization header see the same # canonical form on both sides of the wire. -version = "0.14.7" +# 0.14.9 (2026-08-07): v3.38 wire-drift close — three real +# contract bugs that diverged from backend source. (1) +# ``nullrun.capabilities.CAPABILITIES_PATH`` was ``/health`` (a +# generic liveness endpoint) instead of the canonical +# ``/api/v1/capabilities``; pre-fix every ``init()`` probe +# returned None and ``is_v3_ready()`` was always False, leaving +# the v3 capability flags as runtime no-ops. (2) Backend v3.38 +# split the ``API_KEY_REVOKED`` bucket into five distinct wire +# codes (``API_KEY_EXPIRED`` / ``API_KEY_DISABLED`` / +# ``API_KEY_INVALID`` / ``API_KEY_MISSING`` / +# ``API_KEY_MALFORMED``) — pre-fix only ``API_KEY_REVOKED`` was +# mapped in ``_V3_ERROR_CODE_MAP``, so the other five silently +# fell through to the generic HTTP-status fallback and never +# surfaced as ``NullRunAuthError``, losing both the exception +# class and the diagnostic ``wire_code``. (3) Backend returns +# ``decision == "soft_pass"`` for soft-mode calls that proceed +# via the chain's overdraft cap (CLAUDE.md §5); pre-fix +# ``check_workflow_budget`` had no branch for ``soft_pass`` and +# it fell through the default allow path with no log line and +# no ``soft_overdraft_used`` counter increment — silent budget +# drift. The new soft_pass branch increments the counter via +# ``metrics.inc_runtime("soft_overdraft_used")`` and logs at +# WARNING with ``overdraft_used_cents`` so operators have +# visibility into which chains are burning overdraft. Three +# real bugs closed; no SDK_MIN_VERSION bump; no on-wire change. +version = "0.14.9" # Kept under the 200-char preview threshold so the full line is visible -# without an "expand" click. Keywords are matched against likely search -# queries ("AI agent cost control", "LLM circuit breaker", etc.). -description = "NullRun Python SDK — enforcement gateway for AI agents. Circuit-breaker, policy enforcement and observability for OpenAI, Anthropic, LangGraph, LlamaIndex, CrewAI, AutoGen." +# without an "expand" click. The headline is the canonical §1 statement +# from positioning.md — "runtime decision layer for tool-using AI agents" +# (not "enforcement gateway", which undersells Phase 1 typed action +# predicates and 3 MCP-aware enforcement). Vendor list kept for +# searchability; "BusinessImpact" + "MCP-aware" surface the two newest +# differentiators. Keywords below match the likely search queries. +description = "NullRun Python SDK — runtime decision layer for tool-using AI agents." readme = "README.md" license = { text = "Apache-2.0" } requires-python = ">=3.10" diff --git a/src/nullrun/__init__.py b/src/nullrun/__init__.py index b56797c..f65b2be 100644 --- a/src/nullrun/__init__.py +++ b/src/nullrun/__init__.py @@ -27,16 +27,16 @@ def my_agent(query): from nullrun.__version__ import __version__ # Module-level lock that serialises the three singleton-slot writes -# inside `init `. See plan item B3. +# inside `init `. _init_lock = _threading.Lock() # --------------------------------------------------------------------------- -# Curated public surface (Phase 3.4) +# Curated public surface # --------------------------------------------------------------------------- # These six names are imported eagerly so they show up in `dir(nullrun)` and -# in tab-completion — that's the "track AI cost in 5 minutes" surface. All -# other names (legacy Breaker exports, instrumentation, exceptions, …) live -# in `_LAZY_EXPORTS` below and are loaded on first access via __getattr__. +# in tab-completion. All other names (legacy Breaker exports, +# instrumentation, exceptions, …) live in `_LAZY_EXPORTS` below and are +# loaded on first access via __getattr__. from nullrun.decorators import protect # the gate decorator from nullrun.runtime import track_event, track_llm, track_tool @@ -327,9 +327,9 @@ def my_agent: except Exception as e: # noqa: BLE001 — best-effort logger.warning("previous runtime shutdown raised during init(): %s", e) - # Phase 3 (2026-07-05): install the runtime in the registry - # so every consumer (decorators, @protect, track_*) sees the - # same instance regardless of which init path we use. + # Install the runtime in the registry so every consumer + # (decorators, @protect, track_*) sees the same instance + # regardless of which init path we use. from nullrun._registry import get_registry registry = get_registry() @@ -349,11 +349,11 @@ def my_agent: NullRunRuntime._instance = runtime # v3.12 / 0.12.0 — server-minted execution_id default ON. Probe - # the backend's /health endpoint and log any version mismatch - # so the operator sees the gap at startup rather than on the - # first failed /check. We do NOT fail init — the gate still - # rejects with 400 PROTOCOL_TOO_OLD, and the SDK's role is - # advisory here. + # the backend's /api/v1/capabilities endpoint and log any + # version mismatch so the operator sees the gap at startup + # rather than on the first failed /check. We do NOT fail init — + # the gate still rejects with 400 PROTOCOL_TOO_OLD, and the + # SDK's role is advisory here. try: from nullrun.__version__ import __version__ from nullrun.capabilities import ( @@ -367,20 +367,20 @@ def my_agent: for w in warnings: logger.warning("nullrun.init: %s", w) else: - # /health unreachable — most likely the operator - # hasn't pointed the SDK at the right host. We don't - # fail init (the user might intentionally init + # /api/v1/capabilities unreachable — most likely the + # operator hasn't pointed the SDK at the right host. + # We don't fail init (the user might intentionally init # before network is ready) but we log at INFO so the # operator sees it. logger.info( - "nullrun.init: could not probe %s/health — " + "nullrun.init: could not probe %s/api/v1/capabilities — " "v3 capability negotiation skipped", runtime.api_url, ) except Exception as e: # noqa: BLE001 — best-effort probe logger.debug("nullrun.init: capability probe raised %s", e) - # Phase D6: wire auto-instrumentation AFTER the runtime is fully + # Wire auto-instrumentation AFTER the runtime is fully # constructed. In 0.3.0 api_key is required, so this branch is # unconditional — we always have a remote LLM traffic source if # auto-instrumentation libraries are installed. @@ -418,8 +418,8 @@ def my_agent: "get_trace_id": ("nullrun.context", "get_trace_id"), "get_span_id": ("nullrun.context", "get_span_id"), "get_agent_id": ("nullrun.context", "get_agent_id"), - # T4 (2026-06-27): per-call context for /gate pre-flight. Users - # call `set_call_context(model=..., tools=[...])` inside + # Per-call context for /gate pre-flight. Users call + # `set_call_context(model=..., tools=[...])` inside # `with workflow(...)` so the backend's budget + tool_block # enforcement sees real values instead of the previous fake # `"budget-precheck"` sentinel and empty tool list. @@ -436,27 +436,26 @@ def my_agent: "set_chain_op": ("nullrun.context", "set_chain_op"), # Instrumentation "NullRunCallback": ("nullrun.instrumentation", "NullRunCallback"), - # NOTE (Sprint 1.2 / B11-B12): `patch_openai` and `unpatch_openai` - # were removed from `_LAZY_EXPORTS` because they pointed at - # non-existent attributes on `nullrun.instrumentation` (the actual - # function is `patch_openai_agents`, with different semantics — - # it patches `agents.Runner`, not the `openai` SDK). The pre-fix - # lazy entries caused `AttributeError` on first access, which is - # a worse failure mode than a clean `ImportError` from + # NOTE: `patch_openai` and `unpatch_openai` were removed from + # `_LAZY_EXPORTS` because they pointed at non-existent + # attributes on `nullrun.instrumentation` (the actual function + # is `patch_openai_agents`, with different semantics — it patches + # `agents.Runner`, not the `openai` SDK). The pre-fix lazy + # entries caused `AttributeError` on first access, which is a + # worse failure mode than a clean `ImportError` from # `from nullrun import patch_openai` failing because the symbol # is no longer in the lazy table. - # Toolbox — framework-specific wrappers (Phase 1 Commit 6). - # The previous `instrument ` helper lived at - # `nullrun.instrumentation.langgraph.instrument`; it is now - # `nullrun.toolbox.langgraph.wrapper`. Reachable as + # Toolbox — framework-specific wrappers. The previous `instrument ` + # helper lived at `nullrun.instrumentation.langgraph.instrument`; + # it is now `nullrun.toolbox.langgraph.wrapper`. Reachable as # `from nullrun import wrapper` for one-line import. "wrapper": ("nullrun.toolbox.langgraph", "wrapper"), - # Span / trace context (Phase 2 Commit 3). - # `tracing.py` is the structured replacement for the loose `_trace_id` - # / `_span_id` contextvars in `nullrun.context`. `SpanContext` is a - # single value (parent + children derive from it); `set_span` / - # `reset_span` are the token-based API the runtime and `@protect` - # use to push/pop the active span. + # Span / trace context. `tracing.py` is the structured replacement + # for the loose `_trace_id` / `_span_id` contextvars in + # `nullrun.context`. `SpanContext` is a single value (parent + + # children derive from it); `set_span` / `reset_span` are the + # token-based API the runtime and `@protect` use to push/pop the + # active span. "SpanContext": ("nullrun.tracing", "SpanContext"), "get_current_span": ("nullrun.tracing", "get_current_span"), "create_root_span": ("nullrun.tracing", "create_root_span"), @@ -465,7 +464,7 @@ def my_agent: "reset_span": ("nullrun.tracing", "reset_span"), # Decorators "sensitive": ("nullrun.decorators", "sensitive"), - # Actions (Phase 3) + # Actions "ActionHandler": ("nullrun.actions", "ActionHandler"), "ActionType": ("nullrun.actions", "ActionType"), "ActionEvent": ("nullrun.actions", "ActionEvent"), @@ -473,7 +472,7 @@ def my_agent: "handle_action": ("nullrun.actions", "handle_action"), "register_action_handler": ("nullrun.actions", "register_action_handler"), "get_action_handler": ("nullrun.actions", "get_action_handler"), - # Exceptions (Phase 3 + Layer 1) + # Exceptions (Layer 1) "NullRunError": ("nullrun.breaker.exceptions", "NullRunError"), "NullRunBlockedException": ("nullrun.breaker.exceptions", "NullRunBlockedException"), "NullRunAuthenticationError": ("nullrun.breaker.exceptions", "NullRunAuthenticationError"), @@ -488,8 +487,8 @@ def my_agent: "NullRunStatus": ("nullrun.observability.status", "NullRunStatus"), "RecentError": ("nullrun.observability.status", "RecentError"), "WorkflowState": ("nullrun.observability.status", "WorkflowState"), - # Sprint 2.2: zombie exception classes removed. See the - # NOTE block in breaker/exceptions.py for the list. + # Zombie exception classes removed. See the NOTE block in + # breaker/exceptions.py for the list. "WorkflowPausedException": ("nullrun.breaker.exceptions", "WorkflowPausedException"), "WorkflowKilledException": ("nullrun.breaker.exceptions", "WorkflowKilledException"), "WorkflowKilledInterrupt": ("nullrun.breaker.exceptions", "WorkflowKilledInterrupt"), @@ -550,10 +549,10 @@ def __dir__() -> list[str]: __all__ = [ # Version (single value, always public) "__version__", - # Phase 3.4: the curated public surface — six symbols. - # Everything else stays importable as `from nullrun import X` for - # backward compatibility, but does NOT appear in `dir(nullrun)` - # until the user actually accesses it. + # The curated public surface — six symbols. Everything else + # stays importable as `from nullrun import X` for backward + # compatibility, but does NOT appear in `dir(nullrun)` until the + # user actually accesses it. "init", "protect", # gate decorator "track_llm", @@ -612,11 +611,11 @@ def __dir__() -> list[str]: "init_or_die", ] -# Sprint 2.1: the SDK-side ``decision_history`` module was deleted. -# Decision history is a backend + dashboard surface only — the SDK -# does not (and cannot) replay LLM calls because NULLRUN does not -# store request/response payloads or hold client LLM keys. The -# orphan ``start_recording`` / ``stop_recording`` methods on +# The SDK-side ``decision_history`` module was deleted. Decision +# history is a backend + dashboard surface only — the SDK does not +# (and cannot) replay LLM calls because NULLRUN does not store +# request/response payloads or hold client LLM keys. The orphan +# ``start_recording`` / ``stop_recording`` methods on # ``NullRunRuntime`` are kept as no-op stubs for one minor version # for backward compatibility; they will be removed in 0.5.0. # Do NOT re-export ReplayManager / ReplaySession / ReplayEvent / diff --git a/src/nullrun/__version__.py b/src/nullrun/__version__.py index b610149..ffeab1c 100644 --- a/src/nullrun/__version__.py +++ b/src/nullrun/__version__.py @@ -1,5 +1,33 @@ """NullRun Platform SDK. +v3.38 / 0.14.9 (2026-08-07) — wire-drift close: three real +contract bugs that diverged from backend source code. +(1) ``nullrun.capabilities.CAPABILITIES_PATH`` was ``/health`` +(legacy liveness endpoint) instead of the canonical +``/api/v1/capabilities``. Pre-fix every ``init()`` probe +returned None and ``is_v3_ready()`` was always False, leaving +the v3 capability flags as runtime no-ops. +(2) Backend v3.38 split the ``API_KEY_REVOKED`` bucket into +five distinct wire codes (``API_KEY_EXPIRED`` / +``API_KEY_DISABLED`` / ``API_KEY_INVALID`` / +``API_KEY_MISSING`` / ``API_KEY_MALFORMED``) — pre-fix only +``API_KEY_REVOKED`` was mapped in ``_V3_ERROR_CODE_MAP``, so +the other five silently fell through to the generic +HTTP-status fallback and never surfaced as +``NullRunAuthError``, losing both the exception class and the +diagnostic ``wire_code``. +(3) Backend returns ``decision == "soft_pass"`` for soft-mode +calls that proceed via the chain's overdraft cap (CLAUDE.md +§5); pre-fix ``check_workflow_budget`` had no branch for +``soft_pass`` and it fell through the default allow path with +no log line and no ``soft_overdraft_used`` counter increment +— silent budget drift. The new soft_pass branch increments +the counter via ``metrics.inc_runtime("soft_overdraft_used")`` +and logs at WARNING with ``overdraft_used_cents`` so +operators have visibility into which chains are burning +overdraft. +Recommended upgrade path: 0.14.8 -> 0.14.9 (or 0.14.7 -> 0.14.9). + v3.31.6 / 0.14.7 (2026-08-04) — init contract hardening: strip whitespace from ``api_key`` before the truthiness check. @@ -79,8 +107,8 @@ window turned the run red even when the ``test`` (3.10/3.11/3.12) matrix was fully green. The marker itself (``reruns=2``, ``release_after_ms=200``) was already in place - from the Sprint 0 audit — the missing piece was the plugin on - the coverage leg. This release matches the install on + from the audit — the missing piece was the plugin on the + coverage leg. This release matches the install on ``ci.yml:41-45``. 2. ``tests/test_actions.py::TestPauseAction::test_is_paused_respects_cooldown`` @@ -143,7 +171,7 @@ --- v3.30 / 0.14.4 (2026-07-27) — ToolParameters Approval Rules -wire contract (Tier 2 / Разрыв 2 follow-up). +wire contract. Pre-fix 0.14.0, a ``track_tool`` event payload containing a ``Decimal`` (e.g. ``refund_amount`` from a @@ -200,7 +228,7 @@ v3.28 / 0.14.0 (2026-07-23) — hardening pass on the money contract. -Closes the four review gaps from the Phase 1.1 / UX follow-up: +Closes the four review gaps from the UX follow-up: 1. **Dedicated error types** -- ``InvalidMoneyPrecisionError`` and ``InvalidMoneyAmountError`` (both subclass @@ -247,7 +275,7 @@ * Server's ``approval_timeout`` is clamped to ``[1, 3600]s`` on the SDK side as defence against a malformed / overshooting backend that returns ``0`` or ``2147483647`` - in the Разрыв 1c field. + in the server approval-timeout field. Public API change (additive only, backward-compatible): @@ -303,16 +331,16 @@ --- -v3.27 / 0.13.13 (2026-07-21) — Разрыв 1c SDK sync. +v3.27 / 0.13.13 (2026-07-21) — approval-timeout wire sync. -Backend commit ``0ad03b9`` (Разрыв 1c, gate hot-path trigger) -added ``approval_timeout_seconds: Option`` and -``approval_expires_at: Option`` to the GateResponse +Backend commit ``0ad03b9`` (gate hot-path trigger that prompted +this SDK sync) added ``approval_timeout_seconds: Option`` +and ``approval_expires_at: Option`` to the GateResponse wire format. Before this SDK fix, the approval wait path used ``NULLRUN_APPROVAL_TIMEOUT_SECONDS`` env default (default 300s) as the ONLY source of wait duration — which is exactly -the Разрыв 3 class of bug that the backend sweeper was written -to prevent on the backend side. +the silent-desync class of bug that the backend sweeper was +written to prevent on the backend side. Concretely: a backend approval rule configured with ``expires_in_seconds=20`` (short-approval use case) would @@ -328,11 +356,11 @@ kwarg ``timeout_seconds: float | None = None``. When set to a positive number, used as the event.wait() timeout (server-authoritative, takes precedence over the env - default). When ``None`` (legacy backend without Разрыв 1c - field, or malformed response), falls back to - ``self._approval_timeout_seconds`` (env default) — - pre-Разрыв 1c behaviour preserved. When set to a - non-positive number (0 or negative), also falls back to + default). When ``None`` (legacy backend without the + server-side approval-timeout field, or malformed response), + falls back to ``self._approval_timeout_seconds`` (env + default) — pre-server-side behaviour preserved. When set + to a non-positive number (0 or negative), also falls back to env default; we explicitly reject these because ``event.wait(timeout=0)`` deadlocks on the very first call. @@ -643,7 +671,7 @@ /track single-event path. Chain-mode loops that re-use the *same* chain_id across many gate calls still rely on the cache collapsing to one roundtrip, which is the - intentional design (CLAUDE.md §18 BUG #5 — gate_cache + intentional design (BUG #5 — gate_cache debounce). Operators who need a fresh ``/gate`` call on every ``@protect`` invocation can opt out via ``NULLRUN_GATE_CACHE_DISABLE=1`` (env var, no code @@ -1160,5 +1188,5 @@ """ -__version__ = "0.14.7" +__version__ = "0.14.9" __platform_version__ = "1.0.0" diff --git a/src/nullrun/_registry.py b/src/nullrun/_registry.py index e96c857..76c3096 100644 --- a/src/nullrun/_registry.py +++ b/src/nullrun/_registry.py @@ -16,7 +16,7 @@ of the three pointing at a dead runtime, dropping ``span_start`` / ``span_end`` events on the floor (see audit 2026-07-05 H2). -Phase 3 unifies the three writers behind a single +The three writers are unified behind a single :class:`RuntimeRegistry` so every consumer reads from one place. The class-level ``NullRunRuntime._instance`` is preserved as a proxy for backward compatibility (test fixtures, third-party @@ -27,7 +27,7 @@ ------------- The registry uses an ``RLock`` because the same thread can re-enter during a ``get_instance`` -> ``shutdown`` -> ``get_instance`` sequence -(Phase 5 #5.3 documented the original deadlock from a plain Lock). +(B5 #5.3 documented the original deadlock from a plain Lock). Readers (the hot path on every ``@protect`` call) take a snapshot of the instance pointer once and release the lock immediately; they do NOT hold the lock across downstream calls (e.g. ``runtime diff --git a/src/nullrun/_singleton.py b/src/nullrun/_singleton.py index 978663a..515802e 100644 --- a/src/nullrun/_singleton.py +++ b/src/nullrun/_singleton.py @@ -1,6 +1,6 @@ # Backwards-compat proxy descriptor for ``NullRunRuntime._instance``. -# Phase 3 (2026-07-05) refactored the singleton slot into the +# The singleton slot was refactored into the # ``nullrun._registry.RuntimeRegistry`` so there is exactly one # source of truth. External code (test fixtures, third-party # extensions, dashboard scripts) still introspects @@ -71,9 +71,9 @@ def install_module_proxy(module, attribute_name: str = "_runtime") -> None: Backwards-compat for code that imports nullrun.runtime._runtime or nullrun.decorators._runtime directly — historically these - were plain module attributes holding the active runtime. After - Phase 3 the registry is the source of truth, so the module - attribute is now a property-style proxy. + were plain module attributes holding the active runtime. The + registry is the source of truth now, so the module attribute + is a property-style proxy. Args: module: The module object to patch. diff --git a/src/nullrun/actions.py b/src/nullrun/actions.py index b782a28..f4d117c 100644 --- a/src/nullrun/actions.py +++ b/src/nullrun/actions.py @@ -186,13 +186,13 @@ def handle( try: action_type = ActionType(action.lower()) except ValueError: - # Sprint 1.5 (B14): pre-fix this degraded silently to - # ``ActionType.BLOCK`` and triggered ``_default_block`` - # which raises ``NullRunBlockedException``. That made - # the SDK into a DoS amplifier: a single malformed - # ``action`` from the server (or a MITM, or a server - # schema regression) would block every subsequent tool - # call in the workflow with no actionable error. + # Pre-fix this degraded silently to ``ActionType.BLOCK`` + # (B14) and triggered ``_default_block`` which raises + # ``NullRunBlockedException``. That made the SDK into a + # DoS amplifier: a single malformed ``action`` from the + # server (or a MITM, or a server schema regression) + # would block every subsequent tool call in the workflow + # with no actionable error. # # Post-fix: log at ERROR, record the event for forensic # visibility, and DO NOT invoke any handler. The diff --git a/src/nullrun/breaker/__init__.py b/src/nullrun/breaker/__init__.py index 4502213..c2068e8 100644 --- a/src/nullrun/breaker/__init__.py +++ b/src/nullrun/breaker/__init__.py @@ -7,10 +7,10 @@ remain so that `runtime.py`, `transport.py`, `actions.py`, and the test suite can share a single error vocabulary. -Sprint 2.2: zombie exception classes (CostLimitExceeded -ApprovalRequired, BreakerTimeout) were removed because they had -zero in-tree callers. See the NOTE block in -``nullrun.breaker.exceptions`` for the full list. +Removed zombie exception classes (CostLimitExceeded, ApprovalRequired, +BreakerTimeout) are not re-exported because they had zero in-tree +callers. See the NOTE block in ``nullrun.breaker.exceptions`` for +the full list. """ from nullrun.breaker.circuit_breaker import CBState, CircuitBreaker diff --git a/src/nullrun/breaker/circuit_breaker.py b/src/nullrun/breaker/circuit_breaker.py index 134b44c..7f11345 100644 --- a/src/nullrun/breaker/circuit_breaker.py +++ b/src/nullrun/breaker/circuit_breaker.py @@ -199,9 +199,8 @@ def _on_state_change(self, old_state: CBState, new_state: CBState) -> None: """Record state transition metrics.""" if new_state == CBState.OPEN: metrics.inc_transport("circuit_open_count") - # Sprint 3 follow-up (B24): also bump the - # ``circuit_breaker_opens`` global counter on - # ``TransportMetrics`` (was 0-call). This is the + # Also bump the global ``circuit_breaker_opens`` counter + # on ``TransportMetrics`` (was 0-call). This is the # cross-CB-instance counter — the operator alerts # on its rate, not on the per-CB ``circuit_open_count``. metrics.inc_transport("circuit_breaker_opens") @@ -227,11 +226,11 @@ def _on_closed(self) -> None: @property def state(self) -> CBState: - # Phase 0.3.1: hold the lock for the whole transition so - # concurrent threads do not race into HALF_OPEN. The - # previous version only held the lock for the dict read - # which let two workers independently decide they should - # both probe in HALF_OPEN at the same wall-clock moment. + # Hold the lock for the whole transition so concurrent + # threads do not race into HALF_OPEN. The previous + # version only held the lock for the dict read which + # let two workers independently decide they should both + # probe in HALF_OPEN at the same wall-clock moment. # The fix also publishes HALF_OPEN to Redis (was defined # but never called) so other workers see the state via # ``_check_global_state`` instead of falling back to @@ -301,8 +300,8 @@ def _maybe_apply_open_jitter_sync(self) -> None: if self._state == CBState.OPEN and self._opened_at is not None: time_in_open = time.monotonic() - self._opened_at if time_in_open >= self._recovery_timeout: - # Phase 8: cap at 5s (was 30s). 5s is plenty to - # spread reconnects across workers. + # Cap at 5s. 5s is plenty to spread reconnects + # across workers. jitter = random.uniform(0, 5.0) time.sleep(jitter) diff --git a/src/nullrun/breaker/exceptions.py b/src/nullrun/breaker/exceptions.py index 2e1135c..b3b5683 100644 --- a/src/nullrun/breaker/exceptions.py +++ b/src/nullrun/breaker/exceptions.py @@ -320,7 +320,7 @@ class RateLimitError(NullRunTransportError): """Raised when the gateway returns HTTP 429 with a ``Retry-After`` header (or JSON body field). - Phase 4: subclass of ``NullRunTransportError`` so + Subclass of ``NullRunTransportError`` so ``except NullRunTransportError`` keeps catching it. Surfaces ``retry_after`` (seconds) and ``upgrade_url`` so callers can schedule a retry or surface a billing upgrade prompt. @@ -397,17 +397,25 @@ class NullRunProtocolError(NullRunInfrastructureError): class NullRunChainError(NullRunDecision): """Chain-related failure. - Covers four backend codes: ``CHAIN_MAX_DURATION_EXCEEDED`` (402) - ``CHAIN_CROSS_ORG`` (403), ``CHAIN_ORG_MISMATCH`` (403), and - ``CHAIN_NOT_FOUND`` / ``CHAIN_EXPIRED`` (404). Splitting the - chain codes into their own class (rather than reusing + Covers backend codes: ``CHAIN_MAX_DURATION_EXCEEDED`` (402), + ``CHAIN_CROSS_ORG`` (403), ``CHAIN_ORG_MISMATCH`` (403), + ``CHAIN_NOT_FOUND`` / ``CHAIN_EXPIRED`` (404), and the + Execution Graph v0 (2026-08-06) trio: + ``PARENT_EXECUTION_NOT_FOUND`` / ``PARENT_EXECUTION_ORG_MISMATCH`` + / ``PARENT_EXECUTION_KEY_MISMATCH`` (all 403). Splitting the + chain-and-lineage codes into their own class (rather than reusing NullRunBlockedException) gives cookbook code a clean way to distinguish "you forgot to start a chain" from "your tool is - blocked" without string-matching the message. + blocked" from "your sub-agent references an execution you do not + own" without string-matching the message. Attributes: chain_id: Chain that triggered the error (may be None on a cross-org collision). + parent_execution_id: Execution Graph v0 (2026-08-06) — the + parent execution_id from the rejected sub-agent call. + Distinct from chain_id (lifecycle of one SDK run) — the + Execution Graph tracks spawn topology across runs. """ error_code = "NR-CH001" @@ -424,12 +432,19 @@ def __init__( message: str, *, chain_id: str | None = None, + parent_execution_id: str | None = None, backend_code: str | None = None, details: dict[str, Any] | None = None, status_code: int | None = None, **kwargs: Any, ) -> None: self.chain_id = chain_id + # Execution Graph v0 (2026-08-06): when the backend rejects + # a sub-agent call with PARENT_EXECUTION_*, the offending + # parent_execution_id is preserved on the exception so + # cookbook code can log / surface it without re-parsing the + # message string. ``None`` for non-lineage chain errors. + self.parent_execution_id = parent_execution_id self.backend_code = backend_code or self.error_code self.details = details or {} # 2026-07-04: preserve the wire HTTP @@ -466,7 +481,7 @@ class NullRunConsumeOverbudgetError(NullRunDecision): user_action = ( "The actual cost exceeded the reservation by more than the " "epsilon_cents tolerance. The reservation was NOT silently " - "re-reserved (CLAUDE.md §25). Either reduce the call's " + "re-reserved. Either reduce the call's " "expected cost before /check (model downgrade, fewer tokens) " "or increase the per-policy ``epsilon_cents`` after manual " "review — never bypass the invariant by retrying." @@ -498,13 +513,12 @@ def __init__( class NullRunWorkflowInactiveError(NullRunDecision): - """Workflow soft-deleted; gate blocks per-key traffic ( - — Sprint 6 v1 12.2 hot-path wiring). + """Workflow soft-deleted; gate blocks per-key traffic. Raised when the workflow's ``is_active`` flag is false (soft delete + ``killed_at`` not null) AND an active API key still - tries to drive traffic against it. Per the fail-CLOSED contract - in, the SDK must not let the agent body run in + tries to drive traffic against it. Per the fail-CLOSED contract, + the SDK must not let the agent body run in this state — a soft-deleted workflow implies the operator intentionally revoked it. """ @@ -550,10 +564,10 @@ class NullRunRateLimitRedisError(NullRunInfrastructureError): error_code = "NR-R002" user_action = ( "The NullRun backend cannot reach Redis for the aggregate " - "rate limit. The request was rejected (fail-CLOSED per " - "CLAUDE.md §4) because the rate limit is the authoritative " - "gate, not a soft advisory. Retry after the operator " - "confirms Redis is healthy — check status.nullrun.io." + "rate limit. The request was rejected (fail-CLOSED) because " + "the rate limit is the authoritative gate, not a soft " + "advisory. Retry after the operator confirms Redis is " + "healthy — check status.nullrun.io." ) retryable = True @@ -656,6 +670,14 @@ class NullRunAuthError(NullRunAuthenticationError): Subclass of:class:`NullRunAuthenticationError` so existing ``except NullRunAuthenticationError`` clauses keep matching. + + The wire error code (one of ``API_KEY_REVOKED`` / + ``API_KEY_EXPIRED`` / ``API_KEY_DISABLED`` / ``API_KEY_INVALID`` + / ``API_KEY_MISSING`` / ``API_KEY_MALFORMED`` per v3.38) is + stored on ``self.wire_code`` so callers can branch on the + granular lifecycle state without clobbering the SDK-side + ``error_code`` taxonomy (``NR-A003``). Pattern mirrors + :class:`NullRunChainError.backend_code`. """ error_code = "NR-A003" @@ -666,6 +688,19 @@ class NullRunAuthError(NullRunAuthenticationError): ) retryable = False + def __init__( + self, + message: str, + *, + wire_code: str | None = None, + **kwargs: Any, + ) -> None: + self.wire_code = wire_code or "API_KEY_REVOKED" + # Preserve the historical ``self.message`` attribute — some + # user code reads ``exc.message`` instead of ``str(exc)``. + self.message = message + super().__init__(message, **kwargs) + # --------------------------------------------------------------------------- # Block decisions (budget, loop, rate, tool-block) @@ -712,9 +747,7 @@ class NullRunBlockedException(NullRunDecision): storm) where there was no wire response. Lets FastAPI / Starlette exception handlers map to the correct HTTP status without re-deriving it from - ``type(exc).__name__``. Drift fix 2026-07-04 - (P1-1: SDK_README's NR-B004 → 429 claim was - wrong; the real wire status is 402). + ``type(exc).__name__``. """ error_code = "NR-X001" # generic block; subclasses override @@ -798,12 +831,12 @@ class NullRunToolBlockedError(NullRunBlockedException): retryable = False -# NOTE (Sprint 2.2): the following six exception classes were removed -# in 0.4.0 because they had no callers in the SDK or in any -# test. They were zombie public surface — defined but never raised. -# If a real use case emerges in the future, they should be re-added -# with at least one in-tree caller and a regression test that -# exercises the raise path: +# NOTE: the following six exception classes were removed in 0.4.0 +# because they had no callers in the SDK or in any test. They were +# zombie public surface — defined but never raised. If a real use +# case emerges in the future, they should be re-added with at least +# one in-tree caller and a regression test that exercises the raise +# path: # - CostLimitExceeded # - ApprovalRequired # - BreakerTimeout diff --git a/src/nullrun/business_impact.py b/src/nullrun/business_impact.py index e96625c..5ca6ca9 100644 --- a/src/nullrun/business_impact.py +++ b/src/nullrun/business_impact.py @@ -1,13 +1,12 @@ """BusinessImpact + action_digest (SDK mirror of backend). -Phase 1 / MVP 1.0 (Разрыв 1c follow-up). The Python SDK -must produce the *exact* same SHA-256 hex digest the Rust +The SDK must produce the *exact* same SHA-256 hex digest the Rust backend computes, so the digest re-check on /execute re-check matches byte-for-byte. Drift between SDK and backend would be caught at the first mismatch attack on a real customer. Wire format mirrors `backend::proxy::gate::business_impact`: -- discriminated union with a single MVP variant `kind="money"` +- discriminated union with a single variant `kind="money"` - `MoneyImpact(direction, amount_minor, currency, ...)` - `Condition(MoneyAmount(direction, operator, threshold_minor, currency))` lives on the **rule side** in the backend; the @@ -49,10 +48,9 @@ EQ = "eq" -# MVP 1.0: `money` kind for per-call flat amounts. -# MVP 1.1 (ToolParameters / Phase 1 / Tier 2): `tool_call` kind -# for free-form tool-call argument bags matched against -# ToolParameters Approval Rules on the backend. +# `money` kind for per-call flat amounts. +# `tool_call` kind for free-form tool-call argument bags matched +# against ToolParameters Approval Rules on the backend. KIND_MONEY = "money" KIND_TOOL_CALL = "tool_call" @@ -67,15 +65,15 @@ @dataclass class MoneyImpact: - """Flat per-call money amount, USD-centric in MVP 1.0. + """Flat per-call money amount. Attributes: direction: "outflow" (refund/payout) or "inflow" (charge/invoice). - MVP approval rules only fire on outflow. + Approval rules only fire on outflow. amount_minor: integer cents for USD, MUST be non-negative. Negatives are rejected at validate() time. Sign convention is `direction`, not `+/- amount` — do not switch. - currency: ISO-4217 (3 uppercase letters). MVP is "USD". The + currency: ISO-4217 (3 uppercase letters). Default is "USD". The backend treats any other currency as a no-match against a USD-only rule (separate per-currency rule needed by author). extractor_id: self-reported SDK extractor id (e.g. "nullrun.money.path"). @@ -131,7 +129,7 @@ def to_wire_dict(self) -> dict[str, Any]: @dataclass class ToolCallParams: - """Free-form tool-call argument bag (Phase 1 / Tier 2 wire shape). + """Free-form tool-call argument bag. Mirrors the backend ``BusinessImpact::ToolCall(ToolCallParams)`` variant at ``backend/src/proxy/gate/business_impact.rs:62-307``. @@ -260,15 +258,16 @@ def business_impact_to_dict(impact: BusinessImpact) -> dict[str, Any]: # Dataclasses that mirror the Rust backend's discriminated union via # `kind` discriminator. In Python we represent the union as a # tagged dict at the wire layer and a small class hierarchy at the -# in-process layer. MVP 1.0 only materializes MoneyImpact. +# in-process layer. The SDK validates the variant at construction +# time so the backend never sees malformed output. @dataclass class BusinessImpact: """Top-level BusinessImpact union. - MVP 1.0: `Money` only. - MVP 1.1 (Phase 1 / Tier 2): adds `ToolCall` for free-form - tool-call argument bags matched against ToolParameters - Approval Rules on the backend. + Variants: + `Money`: flat per-call money amount (cents, USD-centric). + `ToolCall`: free-form tool-call argument bag matched + against ToolParameters Approval Rules on the backend. The SDK validates the variant at construction time so the backend never sees malformed output. diff --git a/src/nullrun/capabilities.py b/src/nullrun/capabilities.py index 90150bd..d04700e 100644 --- a/src/nullrun/capabilities.py +++ b/src/nullrun/capabilities.py @@ -40,7 +40,7 @@ This module is intentionally lazy: the probe only fires once at `init `, not on every transport call. -## Drift history +## Capability history * 2026-07-06 — fixed P0 (audit §1 capabilities): - probe URL was ``/health`` (legacy v1/v2); backend exposes the @@ -74,20 +74,20 @@ SDK_MIN_VERSION_FOR_V3 = "0.12.0" -# Wire path for the canonical capabilities endpoint. The SDK targets -# the legacy ``/health`` route (a 200 OK JSON blob that doubles as -# the v1/v2 status endpoint); the backend has registered this -# route since 2025-04. The nested ``/api/v1/capabilities`` route -# is the future canonical contract (per -# ``backend/src/proxy/http/protocol.rs:189``) but is opt-in for -# backends < 1.0.0 — we probe the older URL so the SDK works -# against any 1.0.0-rc.0+ backend without coordination. -CAPABILITIES_PATH = "/health" +# Wire path for the canonical capabilities endpoint. The backend +# exposes this at ``/api/v1/capabilities`` (per +# ``backend/src/proxy/http/protocol.rs:189``) since 2025-04. The +# legacy ``/health`` route returns a generic liveness payload — +# it does NOT carry the v3-gating fields, so probing there always +# returned None and ``is_v3_ready()`` was always False, leaving +# every capability flag a no-op at runtime. See capability +# history note in module docstring (2026-07-06 fix). +CAPABILITIES_PATH = "/api/v1/capabilities" @dataclass(frozen=True) class RateLimitFailScope: - """Per CLAUDE.md §9 — fail-OPEN/CLOSED matrix for rate limiting. + """Fail-OPEN/CLOSED matrix for rate limiting. ``aggregate`` controls the per-org aggregate bucket; ``per_key`` controls the per-API-key bucket. Each is either ``"open"`` (fail-OPEN: @@ -133,6 +133,15 @@ class ServerCapabilities: decision_log: bool = False outbox_async_drain: bool = False idempotency_keys: bool = False + # Execution Graph v0 (2026-08-06, backend): additive + # `parent_execution_id` wire field on /gate. SDKs probe this + # flag before sending the field; pre-Graph backends silently + # ignore unknown fields, but the probe lets SDKs surface a + # clean diagnostic at `init()` ("sub-agent mode requires + # server v0.5+") instead of a 400 on the first call. NOT + # included in `is_v3_ready()` -- it's informational, not a + # hard gate. + execution_graph: bool = False rate_limit_fail_scope: RateLimitFailScope = field( default_factory=lambda: RateLimitFailScope() ) @@ -173,6 +182,7 @@ def as_dict(self) -> dict[str, Any]: "decision_log": self.decision_log, "outbox_async_drain": self.outbox_async_drain, "idempotency_keys": self.idempotency_keys, + "execution_graph": self.execution_graph, "rate_limit_fail_scope": { "aggregate": self.rate_limit_fail_scope.aggregate, "per_key": self.rate_limit_fail_scope.per_key, @@ -252,6 +262,11 @@ def _v3_flag(name: str) -> bool: decision_log=_v3_flag("decision_log"), outbox_async_drain=_v3_flag("outbox_async_drain"), idempotency_keys=_v3_flag("idempotency_keys"), + # Execution Graph v0 (2026-08-06, backend): additive flag + # -- defaults to False so pre-Graph backends (which omit + # the field entirely) yield a fail-closed view where the + # SDK does NOT send `parent_execution_id`. + execution_graph=_v3_flag("execution_graph"), rate_limit_fail_scope=_parse_rate_limit_scope(caps.get("rate_limit_fail_scope")), ) diff --git a/src/nullrun/context.py b/src/nullrun/context.py index 050f018..fd4c1e8 100644 --- a/src/nullrun/context.py +++ b/src/nullrun/context.py @@ -3,9 +3,9 @@ Provides workflow and trace context for automatic event correlation. -Sprint 2.7 (B27): the previously-defined ``_organization_id_var`` / -``_api_key_id_var`` contextvars and the ``get_organization_id`` / -``get_api_key_id`` getters were removed because: +The previously-defined ``_organization_id_var`` / ``_api_key_id_var`` +contextvars and the ``get_organization_id`` / ``get_api_key_id`` +getters were removed (B27) because: 1. No code path ever wrote to them — both getters always returned ``None``. 2. ``observability.TenantFilter`` (the only consumer) was @@ -31,25 +31,24 @@ _agent_id_var: ContextVar[str | None] = ContextVar("agent_id", default=None) _attempt_index_var: ContextVar[int] = ContextVar("attempt_index", default=0) -# T4 (2026-06-27): per-call context that flows into the /gate pre-flight -# request so the backend can compute projected_cost and tool_block -# decisions from real data instead of the previous fake "budget-precheck" +# Per-call context that flows into the /gate pre-flight request so +# the backend can compute projected_cost and tool_block decisions +# from real data instead of the previous fake "budget-precheck" # sentinel. Both default to None/empty; users opt in by calling # ``set_call_context(model=..., tools=[...])`` inside a ``with workflow(...)`` # block. When unset, the backend falls back to its default pricing and -# skips tool-block enforcement on /gate (per-key tool_block is enforced -# on /track only — see gate/internal.rs T3). +# skips tool-block enforcement on /gate (per-key tool_block is +# enforced on /track only). _call_model_var: ContextVar[str | None] = ContextVar("call_model", default=None) _call_tools_var: ContextVar[tuple[str, ...]] = ContextVar("call_tools", default=()) -# Разрыв 3 / 2026-07-28: per-call MCP tool class + annotations. -# Set via the new ``set_mcp_tool_context`` helper when the SDK -# recognises an MCP server. The gate honors `tool_class` over -# its own `classify_tool(tool_name)` parse, and uses -# `mcp_annotations` to evaluate `mcp_destructive_policy` / -# `mcp_readonly_policy`. ``None`` means "I don't know" — the -# gate treats absent values as unknown (NOT as false), so a -# server that forgets to set annotations cannot accidentally get -# a read-only bypass. +# Per-call MCP tool class + annotations. Set via the +# ``set_mcp_tool_context`` helper when the SDK recognises an MCP +# server. The gate honors `tool_class` over its own +# `classify_tool(tool_name)` parse, and uses `mcp_annotations` to +# evaluate `mcp_destructive_policy` / `mcp_readonly_policy`. +# ``None`` means "I don't know" — the gate treats absent values +# as unknown (NOT as false), so a server that forgets to set +# annotations cannot accidentally get a read-only bypass. _call_mcp_class_var: ContextVar[str | None] = ContextVar( "call_mcp_class", default=None ) @@ -438,8 +437,8 @@ def set_call_context( """Set per-call context (model name, tool list) for the next /gate pre-flight check. - T4 (2026-06-27): replaces the previous fake ``model="budget-precheck"`` - and ``estimated_tokens=1`` always-default / always-empty pre-flight. + Replaces the previous fake ``model="budget-precheck"`` and + ``estimated_tokens=1`` always-default / always-empty pre-flight. Call inside a ``with workflow(...)`` block before ``@protect`` to give the backend real data. @@ -450,9 +449,9 @@ def set_call_context( /track will compute from real token counts. tools: List of tool names the call intends to use. Backend matches each against the workflow's effective - ``blocked_tools`` aggregate (T3 in backend) and returns - block on any match. Pass ``None`` to leave whatever was - previously set, ``[]`` to clear. + ``blocked_tools`` aggregate and returns block on any + match. Pass ``None`` to leave whatever was previously + set, ``[]`` to clear. """ if model is not None: _call_model_var.set(model) @@ -464,8 +463,8 @@ def set_mcp_tool_context( tool_class: str | None = None, annotations: dict[str, bool | None] | None = None, ) -> None: - """Разрыв 3 / 2026-07-28: forward the cached MCP tool class + - annotations to the next ``/check`` call. + """Forward the cached MCP tool class + annotations to the next + ``/check`` call. Use after fetching ``tools/list`` from an MCP server — the SDK caches the response and on each subsequent ``/check`` should @@ -526,7 +525,7 @@ def workflow(name: str | None = None) -> Generator[str, None, None]: Yields: The workflow_id string """ - # Phase 5 #5.6: emit a real UUID4 with dashes (matching + # Emit a real UUID4 with dashes (matching # ``generate_trace_id``). The previous ``wf-{hex32}`` format # was inconsistent with the rest of the SDK's id generation. workflow_id = name or str(uuid.uuid4()) @@ -600,7 +599,7 @@ def agent(name: str | None = None) -> Generator[str, None, None]: Yields: The agent_id string """ - # P2-4 / S-8: emit a real UUID4 with dashes (matching + # Emit a real UUID4 with dashes (matching # ``generate_trace_id`` / ``generate_span_id``). The previous # ``f"agent-{uuid.uuid4.hex}"`` format was 32 hex chars # without dashes; backend UUID-typed columns (cost_events. diff --git a/src/nullrun/decorators.py b/src/nullrun/decorators.py index 7330dca..923cffe 100644 --- a/src/nullrun/decorators.py +++ b/src/nullrun/decorators.py @@ -1,11 +1,11 @@ """ Decorators for the NullRun SDK. -Public surface (Phase 2 Commit 4): `protect` is the only gate decorator. -It takes NO parameters — span hierarchy is built automatically from the -caller's context via contextvars, and the workflow is derived from the -API key on the backend (the dashboard surfaces the agent's name from -the key's `name` field). +Public surface: `protect` is the only gate decorator. It takes NO +parameters — span hierarchy is built automatically from the caller's +context via contextvars, and the workflow is derived from the API key +on the backend (the dashboard surfaces the agent's name from the +key's `name` field). Usage: # Basic — auto-init from env, auto-build span tree @@ -68,8 +68,8 @@ def researcher(q): F = TypeVar("F", bound=Callable[..., Any]) -# Phase 3: expanded sensitive-arg keys. The original 7-key set -# missed obvious PII tokens and credential names; ``@sensitive`` and +# Expanded sensitive-arg keys. The original 7-key set missed +# obvious PII tokens and credential names; ``@sensitive`` and # ``_safe_kwargs`` would have shipped them in the audit log. # Matching is case-insensitive (see ``_safe_kwargs`` which calls # ``.lower `` on the key). @@ -135,14 +135,14 @@ def _safe_repr(value: object, max_len: int = 50) -> str: convention only. ``_safe_repr`` is now the single source of truth. """ r = repr(value) - # Phase 1: redact ``details={...}`` substrings on the FULL repr. + # Redact ``details={...}`` substrings on the FULL repr. # Cheap (single linear scan over the string), and ensures the # ``details=`` substring is replaced before we potentially # truncate it away. r = _strip_details_balanced(r) - # Phase 2: truncate to ``max_len`` so a giant repr doesn't bloat - # span events. We append ``...`` so consumers can - # see the cut happened. + # Truncate to ``max_len`` so a giant repr doesn't bloat span + # events. We append ``...`` so consumers can see the + # cut happened. if len(r) > max_len: return r[:max_len] + "..." return r @@ -199,11 +199,10 @@ def _safe_args(fn: Callable[..., Any], args: tuple[Any, ...]) -> list[Any]: return masked -# SEC-29: strip the `details={...}` payload from an exception's -# string form before it lands in the span_end audit event. -# Phase 3 replaced the previous one-level regex with a -# balanced-brace walker that handles nested dicts and dict values -# that contain `{` / `}` in their string content. +# Strip the `details={...}` payload from an exception's string form +# before it lands in the span_end audit event. The current walker +# handles nested dicts and dict values that contain `{` / `}` in +# their string content. _DETAILS_REDACTED = "" # the payload only — caller prepends "details=" @@ -269,17 +268,15 @@ def _strip_details_balanced(text: str) -> str: def _safe_error_str(error: BaseException | None) -> str | None: - """Return a log-safe string for ``error`` (SEC-29, Phase 3).""" + """Return a log-safe string for ``error``.""" if error is None: return None raw = str(error) return _strip_details_balanced(raw) -# Module-level cache for the runtime instance — the @protect decorator needs -# The legacy module-level slot was removed in -# Phase 3 (2026-07-05). Reads/writes now route through the -# registry (see nullrun._singleton._RuntimeProxyModule). +# The legacy module-level slot was removed. Reads/writes now route +# through the registry (see nullrun._singleton._RuntimeProxyModule). def _get_or_create_runtime() -> NullRunRuntime: @@ -526,11 +523,10 @@ def sync_wrapper(*args: Any, **kwargs: Any) -> Any: return result except BaseException as exc: # noqa: BLE001 error = exc - # Round 3 (Phase 0.4.0): unify the "blocked" signal at - # the @protect boundary so callers can catch a single - # NullRunBlockedException for both policy blocks and - # sensitive-tool blocks. Direct calls to - # check_workflow_budget still raise the original + # Unify the "blocked" signal at the @protect boundary so + # callers can catch a single NullRunBlockedException for + # both policy blocks and sensitive-tool blocks. Direct + # calls to check_workflow_budget still raise the original # exception type so callers that distinguish hard vs # soft blocks keep that signal. if isinstance(exc, (WorkflowKilledInterrupt, WorkflowPausedException)): @@ -651,14 +647,14 @@ def _enforce_sensitive_tool( # the /execute payload that lands in the audit log. masked_args = _safe_args(fn, args) - # Phase 1 / MVP 1.0: if the wrapped function carries an - # ``_nullrun_extractor`` attribute (set by the @sensitive - # decorator's ``impact=money_outflow(...)`` argument), extract - # the typed action impact from the live args before sending - # /execute. The extractor returns a fully-validated - # BusinessImpact; we then compute its action_digest and pass - # both onto the wire so the backend can stamp the approval row - # AND verify the digest on the post-approval re-check. + # If the wrapped function carries an ``_nullrun_extractor`` + # attribute (set by the @sensitive decorator's + # ``impact=money_outflow(...)`` argument), extract the typed + # action impact from the live args before sending /execute. + # The extractor returns a fully-validated BusinessImpact; we + # then compute its action_digest and pass both onto the wire + # so the backend can stamp the approval row AND verify the + # digest on the post-approval re-check. # # If the extractor raises (bad arg name, wrong type, negative # amount, etc.), we fail-CLOSED per ADR-008: a sensitive tool @@ -681,12 +677,11 @@ def _enforce_sensitive_tool( business_impact_dict = impact.to_wire_dict() action_digest_hex = compute_action_digest(impact) elif isinstance(extractor, ToolParamsExtractor): - # Phase 1 / MVP 1.1 (Tier 2): free-form tool-call - # argument bag, matched against ToolParameters - # Approval Rules on the backend. Same wire envelope - # (BusinessImpact) and same digest contract as the - # Money variant -- only the discriminator and the - # ``params`` field differ. + # Free-form tool-call argument bag, matched against + # ToolParameters Approval Rules on the backend. + # Same wire envelope (BusinessImpact) and same + # digest contract as the Money variant -- only the + # discriminator and the ``params`` field differ. impact = extractor.impact_for(fn, args, kwargs) business_impact_dict = impact.to_wire_dict() action_digest_hex = compute_action_digest(impact) @@ -767,18 +762,17 @@ def _enforce_sensitive_tool( workflow_id = get_workflow_id() or UNKNOWN_WORKFLOW_ID try: - # Round 3 (Phase 0.4.0): pass on_transport_error="raise" so - # the transport raises NullRunTransportError on network / 5xx - # failure instead of returning a synthetic dict. The arm - # below converts the typed error into NullRunBlockedException - # so the caller's `except NullRunBlockedException` catches it - # uniformly. + # Pass on_transport_error="raise" so the transport raises + # NullRunTransportError on network / 5xx failure instead of + # returning a synthetic dict. The arm below converts the + # typed error into NullRunBlockedException so the caller's + # `except NullRunBlockedException` catches it uniformly. # - # Phase 1 / MVP 1.0: thread the typed impact + digest - # through. When the decorator did NOT see an extractor, both - # are None and the runtime.execute() drops them from the - # payload; the backend then uses the approval_id-only - # grant consume (Phase 0 fallback). + # Thread the typed impact + digest through. When the + # decorator did NOT see an extractor, both are None and the + # runtime.execute() drops them from the payload; the + # backend then uses the approval_id-only grant consume + # (the legacy approval_id-only fallback). result = runtime.execute( fn.__name__, {"args": masked_args, "kwargs": masked}, @@ -957,10 +951,10 @@ def sensitive( def charge_card(amount: int) -> str: ... - Phase 1 / MVP 1.0: ``@sensitive(impact=money_outflow(...))`` - attaches a typed ``MoneyImpactExtractor`` to the function via - the ``_nullrun_extractor`` attribute. The wrapper reads it - inside ``_enforce_sensitive_tool`` to extract a typed + ``@sensitive(impact=money_outflow(...))`` attaches a typed + ``MoneyImpactExtractor`` to the function via the + ``_nullrun_extractor`` attribute. The wrapper reads it inside + ``_enforce_sensitive_tool`` to extract a typed ``BusinessImpact`` + ``action_digest`` from the live call arguments and forward them to /execute, so the backend can stamp the approval row with the digest and refuse tampered @@ -974,7 +968,7 @@ def refund_customer(amount_cents: int, customer_id: str): Args: fn: the function to decorate. May be None when used with keyword arguments (the ``@sensitive(impact=...)`` form). - impact: Phase 1 typed action extractor. Currently only + impact: typed action extractor. Currently only ``MoneyImpactExtractor`` (returned by ``money_outflow(argument=...)``) is supported. @@ -1090,10 +1084,9 @@ def _find_extractor_in_chain(fn: Any) -> Any: def _do_sensitive_register(fn: F) -> F: - # Phase 1 / MVP 1.1 (Tier 2 -- ToolParameters): if @sensitive - # was applied bare (no impact=...), auto-attach a default - # ``ToolParamsExtractor(include_all=True)`` so the tool is - # immediately eligible for ToolParameters Approval Rules + # If @sensitive was applied bare (no impact=...), auto-attach a + # default ``ToolParamsExtractor(include_all=True)`` so the tool + # is immediately eligible for ToolParameters Approval Rules # without requiring every user to write # ``@sensitive(impact=tool_params())`` explicitly. # @@ -1127,10 +1120,10 @@ def _do_sensitive_register(fn: F) -> F: # The extractor module is loaded above us on every path # we care about; this ImportError guard is defensive in # case the SDK is shrunk (e.g. for a hypothetical - # tool-only build). Falling back to legacy Phase 0 path - # is the safe default -- the wire payload drops the - # business_impact field and the backend uses approval_id- - # only grant consume. + # tool-only build). Falling back to the legacy + # approval_id-only grant consume is the safe default -- + # the wire payload drops the business_impact field and the + # backend uses approval_id-only grant consume. pass try: @@ -1207,8 +1200,8 @@ def get_protected_runtime() -> NullRunRuntime | None: return None -# Phase 3 (2026-07-05): install the registry-backed proxy on the -# module class (see nullrun._singleton for the rationale). +# Install the registry-backed proxy on the module class +# (see nullrun._singleton for the rationale). from nullrun._singleton import install_runtime_proxy install_runtime_proxy(__name__) diff --git a/src/nullrun/extractor.py b/src/nullrun/extractor.py index e8ad150..2a20b86 100644 --- a/src/nullrun/extractor.py +++ b/src/nullrun/extractor.py @@ -1,4 +1,4 @@ -"""BusinessImpact extraction for @sensitive tools (Phase 1 / MVP 1.0). +"""BusinessImpact extraction for @sensitive tools. This module is the SDK-side counterpart of the backend's ``BusinessImpact`` discriminated union. It exposes a single @@ -41,7 +41,7 @@ the function signature is refactored. Concretely: @nullrun.sensitive(impact=nullrun.money_outflow(argument="amount")) - def refund(amount: int) -> ... # Phase 0 path: 50 = 50 cents + def refund(amount: int) -> ... # 50 = 50 cents (minor units) def refund(amount: Decimal) -> ... # 50 = $50.00 (5000 cents) If ``units`` were implicit-from-type, renaming ``amount``'s @@ -175,7 +175,7 @@ def refund(amount: Decimal) -> ... # 50 = $50.00 (5000 cents) # # ``minor`` = the value is already in minor units (cents, pence, # satoshi-style). The SDK stores the value verbatim on the wire. -# This is the Phase 0 / pre-Decimal path: a function declared +# This is the pre-Decimal path: a function declared # ``def refund(amount_cents: int)`` already works in minor units # so the operator just has to add the decorator and the wire # shape does not change. @@ -605,7 +605,7 @@ def _to_minor_units( class MoneyImpactExtractor: """Declarative money-impact extractor. - ``units`` discriminator semantics (Phase 1.1 / UX follow-up): + ``units`` discriminator semantics: - ``units="minor"`` (default): the bound argument is already in minor units. ``int`` is the canonical type; @@ -740,9 +740,8 @@ def money_outflow( ``InvalidCurrencyError`` at decorator-application time. ``units`` defaults to ``"minor"`` for backward compatibility - with the Phase 0 / pre-Decimal path. New code that - passes Decimal amounts in major units should pass - ``units="major"`` explicitly. + with the pre-Decimal path. New code that passes Decimal + amounts in major units should pass ``units="major"`` explicitly. ``enforce_business_cap`` defaults to ``True`` so any debit above the per-currency cap goes through the explicit @@ -761,7 +760,7 @@ def money_outflow( # ============================================================================ -# Phase 1 / MVP 1.1 -- ToolParameters (Разрыв 2 / Tier 2) +# ToolParameters extractor # ============================================================================ # # The ToolParamsExtractor is the SDK-side mirror of the backend's @@ -804,7 +803,7 @@ def money_outflow( class ToolParamsExtractor: - """Phase 1 / MVP 1.1 tool-params impact extractor. + """ToolParameters impact extractor. Captures the live call's kwargs into a free-form JSON object that the backend matches against ToolParameters Approval @@ -992,11 +991,11 @@ def tool_params( ) -> ToolParamsExtractor: """Shorthand constructor used by ``@sensitive(impact=tool_params(...))``. - Phase 1 / MVP 1.1 (Tier 2): every bare ``@sensitive`` tool - auto-attaches a ``ToolParamsExtractor(include_all=True)`` - (see ``_do_sensitive_register`` in decorators.py), so most - users never need to call this function explicitly. The - factory below is for two opt-in cases: + Every bare ``@sensitive`` tool auto-attaches a + ``ToolParamsExtractor(include_all=True)`` (see + ``_do_sensitive_register`` in decorators.py), so most users + never need to call this function explicitly. The factory + below is for two opt-in cases: 1. Explicit ``{rule_param: arg_name}`` mapping when the rule name diverges from the function arg name:: diff --git a/src/nullrun/instrumentation/__init__.py b/src/nullrun/instrumentation/__init__.py index 4f32aca..681dd2e 100644 --- a/src/nullrun/instrumentation/__init__.py +++ b/src/nullrun/instrumentation/__init__.py @@ -4,8 +4,7 @@ Provides low-level instrumentation primitives for various AI frameworks. The user-facing "wrap my compiled app" helpers live in `nullrun.toolbox` (e.g. `nullrun.toolbox.langgraph.wrapper` -which replaced `nullrun.instrumentation.langgraph.instrument` -in Phase 1 Commit 6). +which replaced `nullrun.instrumentation.langgraph.instrument`). The v0.x ``openai.ChatCompletion.create`` patcher was removed in 0.4.0 — ``openai>=1.0`` does not expose that attribute. All diff --git a/src/nullrun/instrumentation/_safe_patch.py b/src/nullrun/instrumentation/_safe_patch.py index 5535f85..55b5948 100644 --- a/src/nullrun/instrumentation/_safe_patch.py +++ b/src/nullrun/instrumentation/_safe_patch.py @@ -1,15 +1,15 @@ """ Centralised error handling for auto-instrumentation patchers. -Sprint 2.9 (B47): pre-fix, the auto-instrumentation modules had -25+ instances of ``try/except Exception: pass # pragma: no cover`` -scattered across ``auto.py``, ``auto_requests.py``, ``autogen.py`` -``crewai.py``, ``llama_index.py``. If a patch failed in production -(typically because the vendored SDK changed a method signature) -the SDK would silently degrade and the user would have no idea -why their costs were no longer being tracked. +The pre-fix auto-instrumentation modules had 25+ instances of +``try/except Exception: pass # pragma: no cover`` scattered across +``auto.py``, ``auto_requests.py``, ``autogen.py``, ``crewai.py`` +``llama_index.py``. If a patch failed in production (typically +because the vendored SDK changed a method signature) the SDK would +silently degrade and the user would have no idea why their costs +were no longer being tracked. -The fix: every patch call goes through ``safe_patch `` which: +The fix: every patch call goes through ``safe_patch`` (B47) which: - Returns ``True``/``False`` based on patch outcome. - Logs at WARNING with the patch name + the actual exception (so a SRE can grep for ``Auto-instrumentation patch X failed`` diff --git a/src/nullrun/instrumentation/auto.py b/src/nullrun/instrumentation/auto.py index adb53c8..fed2b56 100644 --- a/src/nullrun/instrumentation/auto.py +++ b/src/nullrun/instrumentation/auto.py @@ -1,9 +1,9 @@ """ Vendor-independent auto-instrumentation for NullRun SDK. -Phase D of the hardening plan: a single `nullrun.init(api_key=...)` call should -track every LLM call regardless of vendor. The user does not need to remember -to call `patch_openai ` or wire callbacks. +A single `nullrun.init(api_key=...)` call should track every LLM call +regardless of vendor. The user does not need to remember to call +`patch_openai` or wire callbacks. Three observation paths feed a single sink (`runtime.track`): @@ -181,7 +181,7 @@ def _openai_extractor(body: bytes, status: int) -> ExtractedUsage | None: # body bytes) never collides with the callback's scheme and # the dedup LRU cannot collapse duplicates. "id": payload.get("id"), - # Phase 4.1: explicit cache / reasoning / finish / tool fields. + # Explicit cache / reasoning / finish / tool fields. # Previously these were reachable only via raw_usage (now # stripped at the wire boundary). Backend gate/budget/loop # detection now sees them as first-class columns. @@ -402,7 +402,7 @@ def _cohere_extractor(body: bytes, status: int) -> ExtractedUsage | None: Note: Cohere streaming has no usage in stream — only non-streaming responses carry it. Documented in the plan. - 2026-07-13 (drift fix #N): v2 has THREE schema changes the SDK + 2026-07-13: v2 has THREE schema changes the SDK silently missed: 1. ``tool_calls`` live under ``message.tool_calls`` (not at @@ -734,7 +734,7 @@ def _check_kill_before_send(runtime: Any, request: httpx.Request) -> None: - no workflow can be resolved (no active context, no API key binding) - the cached state is anything other than Killed / Paused - Note: prior to T3-S2 (0.3.0) this also short-circuited in + Note: prior to 0.3.0 this also short-circuited in `local_mode` (no api_key). The local_mode branch is gone because api_key is now required at runtime construction — every runtime has a remote control plane to consult. @@ -750,11 +750,11 @@ def _check_kill_before_send(runtime: Any, request: httpx.Request) -> None: # rather than crashing the user's transport hook. if not hasattr(runtime, "_resolve_workflow_id"): return - # Phase 5 #5.8: the kill check is independent of which LLM host - # the user is talking to. Previously the check was gated on the - # extractor table, so a custom LLM endpoint silently bypassed the - # dashboard KILL switch. The kill state lives in `_remote_states` - # which is keyed by workflow, not by host. + # The kill check is independent of which LLM host the user is + # talking to. Previously the check was gated on the extractor + # table, so a custom LLM endpoint silently bypassed the dashboard + # KILL switch. The kill state lives in `_remote_states` which is + # keyed by workflow, not by host. workflow_id = runtime._resolve_workflow_id(None) if not workflow_id: return @@ -784,14 +784,14 @@ def _check_kill_before_send(runtime: Any, request: httpx.Request) -> None: # once, the extractor runs, and a fresh Response is returned with the same # body bytes — callers see no behavioural change. -# NOTE (Sprint 2.3): the ``_STREAMING_CONTENT_TYPES`` constant was -# defined here but only consumed in ``auto_requests.py`` (same -# constant is re-defined there). The streaming branch in the -# httpx transport wrapper does not actually consult this table -# it just reads the body and lets the extractors return ``None`` -# for non-usage bodies. The constant is deleted to avoid the -# false impression that this module has streaming-specific -# behaviour. See auto.py module docstring §"Streaming". +# NOTE: the ``_STREAMING_CONTENT_TYPES`` constant was defined here +# but only consumed in ``auto_requests.py`` (same constant is +# re-defined there). The streaming branch in the httpx transport +# wrapper does not actually consult this table — it just reads the +# body and lets the extractors return ``None`` for non-usage bodies. +# The constant is deleted to avoid the false impression that this +# module has streaming-specific behaviour. See auto.py module +# docstring §"Streaming". class NullRunSyncTransport(httpx.BaseTransport): """Synchronous httpx transport that emits a `llm_call` event for known @@ -866,9 +866,9 @@ def _rebuild( # against the post-decompression byte count. req = getattr(response, "_request", None) or request headers = response.headers.copy() - # Phase 6 #6.2: also strip Transfer-Encoding so downstream - # HTTP clients (and httpx itself) don't try to chunk-decode - # an already-buffered body. + # Also strip Transfer-Encoding so downstream HTTP clients + # (and httpx itself) don't try to chunk-decode an + # already-buffered body. for enc in ( "content-encoding", "Content-Encoding", "transfer-encoding", "Transfer-Encoding", @@ -929,8 +929,8 @@ def _emit( # a known provider. See plan at # `~/.claude/plans/async-swinging-hanrahan.md`. try: - # Phase 4.1: lift cache / reasoning / finish / tool names - # out of raw_usage onto the event itself. The backend's + # Lift cache / reasoning / finish / tool names out of + # raw_usage onto the event itself. The backend's # gate/budget/loop detection needs them as first-class # columns; raw_usage is no longer on the wire (stripped # at the track boundary — see _WIRE_STRIP_FIELDS in @@ -1050,9 +1050,9 @@ def _rebuild( # zlib.error. req = getattr(response, "_request", None) or request headers = response.headers.copy() - # Phase 6 #6.2: also strip Transfer-Encoding so downstream - # HTTP clients (and httpx itself) don't try to chunk-decode - # an already-buffered body. + # Also strip Transfer-Encoding so downstream HTTP clients + # (and httpx itself) don't try to chunk-decode an + # already-buffered body. for enc in ( "content-encoding", "Content-Encoding", "transfer-encoding", "Transfer-Encoding", @@ -1091,9 +1091,9 @@ def _emit( # `_extract_model_from_request_body` is sync-only); leave # model as the response-body value or None. try: - # Phase 4.1: see sync _emit for rationale. Async path - # uses identical event shape so the dedup key space - # stays unified across sync + async transports. + # See sync _emit for rationale. Async path uses + # identical event shape so the dedup key space stays + # unified across sync + async transports. # # Audit 2026-06-29 (unified fingerprint): see sync # _emit for the rationale — async transport must use the @@ -1170,12 +1170,11 @@ def _fingerprint_for(host: str, body: bytes, status: int) -> str: def _fingerprint_for_event_dict(event: dict[str, Any]) -> str: """Stable fingerprint for a generic event dict. - Phase 3 of the production-readiness plan: ``runtime.track_event`` - was the only emit path that did NOT set ``_fingerprint``, so two - observers firing for the same LLM call (the user's manual - ``track_event`` plus the httpx transport hook) produced two - ``/track`` POSTs. This helper gives the dedup LRU a stable key - derived from the event's content. + ``runtime.track_event`` was the only emit path that did NOT set + ``_fingerprint``, so two observers firing for the same LLM call + (the user's manual ``track_event`` plus the httpx transport hook) + produced two ``/track`` POSTs. This helper gives the dedup LRU a + stable key derived from the event's content. """ try: payload = json.dumps(event, sort_keys=True, default=str).encode("utf-8") @@ -1652,8 +1651,8 @@ def _emit_from_agents_result(runtime: Any, result: Any) -> None: if prompt == 0 and completion == 0 and total == 0: continue try: - # Phase 4.1: lift cache / reasoning / finish / tool fields - # from raw_usage onto the event itself, mirroring the + # Lift cache / reasoning / finish / tool fields from + # raw_usage onto the event itself, mirroring the # sync/async httpx transport shape. The Agents SDK emits # the OpenAI usage shape so the field names line up. prompt_details = usage.get("prompt_tokens_details") or {} @@ -1847,13 +1846,13 @@ def auto_instrument(runtime: Any) -> bool: at least one path was installed (so the caller can log a useful 'instrumented N paths' message). - Sprint 2.9 (B47): every patch call is wrapped in ``safe_patch`` - which logs at WARNING if the patch raised a non-ImportError - exception. Pre-fix the 25+ scattered ``try/except Exception: - pass # pragma: no cover`` blocks meant a vendor SDK breaking - change (e.g. a renamed method) would silently disable cost - tracking with no log line. The operator would only find out - when the bill arrived. + Every patch call is wrapped in ``safe_patch`` (B47) which logs + at WARNING if the patch raised a non-ImportError exception. The + pre-fix ``try/except Exception: pass # pragma: no cover`` blocks + meant a vendor SDK breaking change (e.g. a renamed method) + would silently disable cost tracking with no log line. The + operator would only find out when the bill arrived. + """ global _auto_installed with _auto_lock: @@ -2015,7 +2014,7 @@ def reset_for_tests() -> None: # events. This is exposed here so tests can introspect / clear the LRU # without poking into the runtime module. -DEDUP_LRU_MAX = 4096 # Phase 6 #6.7: 4096 entries give a 410ms dedup window at 10K events/sec +DEDUP_LRU_MAX = 4096 # 4096 entries give a 410ms dedup window at 10K events/sec # P0-3: streaming-OOM cap. Pre-fix, the sync transport # called ``response.read `` and the async transport called diff --git a/src/nullrun/instrumentation/auto_requests.py b/src/nullrun/instrumentation/auto_requests.py index d0f3ccd..1810914 100644 --- a/src/nullrun/instrumentation/auto_requests.py +++ b/src/nullrun/instrumentation/auto_requests.py @@ -1,6 +1,5 @@ """ -Auto-instrumentation for the `requests` library — Phase P2 of the audit -fix plan. +Auto-instrumentation for the `requests` library. Mirrors `auto.py` (the httpx transport hook) for the `requests` HTTP client. The motivation: 30-50% of real codebases use `requests` directly @@ -27,7 +26,8 @@ - Double-emission guard: `request._nullrun_tracked = True` is set on the PreparedRequest after a successful track, so a future `urllib3` patch (which `requests` uses under the hood) can skip - already-tracked requests. See plan section P2 / "requests ↔ urllib3". + already-tracked requests. See the "requests ↔ urllib3" section of + the audit notes. 0.9.0: counter-bump helpers (`_safe_bump_coverage` `_bump_streaming_skipped`) are gone — coverage is now derived from @@ -35,8 +35,8 @@ and `metadata.streaming_skipped: bool` so the backend can compute coverage_pct from `spans.metadata` directly. -`aiohttp` is deliberately out of scope for this phase — see -`docs/known-limitations.md` and the plan's open questions. +`aiohttp` is deliberately out of scope — see +`docs/known-limitations.md` for the rationale. """ from __future__ import annotations diff --git a/src/nullrun/instrumentation/langgraph.py b/src/nullrun/instrumentation/langgraph.py index dc54352..87d34c4 100644 --- a/src/nullrun/instrumentation/langgraph.py +++ b/src/nullrun/instrumentation/langgraph.py @@ -157,7 +157,7 @@ def extract_usage_from_response(response: Any, provider: str, model: str) -> dic Returns raw usage dict - backend will normalize and compute cost. SDK does NOT compute cost - this is intentional (backend is source of truth). - Phase 4.1: also extracts cache_read_tokens, cache_write_tokens + Also extracts cache_read_tokens, cache_write_tokens, reasoning_tokens, finish_reason, and tool_names so the backend's gate/budget/loop detection can see them as first-class columns. Fields are best-effort — different LangChain providers expose @@ -302,8 +302,8 @@ def extract_usage_from_response(response: Any, provider: str, model: str) -> dic # Final response should have usage_metadata pass - # Phase 4.1: extract the second-tier fields the backend gate/budget - # loop detection now needs. We pull from the same response object + # Extract the second-tier fields the backend gate/budget loop + # detection now needs. We pull from the same response object # LangChain already loaded — no extra HTTP, no schema surprise. # All five fields are best-effort: any provider that doesn't expose # them simply leaves the default value (0 / None / []). @@ -645,8 +645,8 @@ def on_llm_end(self, response: Any, **kwargs: Any) -> None: pass # Build event with RAW usage data (no cost computation in SDK!) - # Phase 4.1: lift cache / reasoning / finish / tool names out - # of raw_usage onto the event itself, mirroring the httpx + # Lift cache / reasoning / finish / tool names out of + # raw_usage onto the event itself, mirroring the httpx # transport shape so the dedup key space stays unified. # 0.9.0: tag metadata.tracked based on whether the model # extraction produced a real value (not the literal diff --git a/src/nullrun/observability/__init__.py b/src/nullrun/observability/__init__.py index 840d1fd..3219acb 100644 --- a/src/nullrun/observability/__init__.py +++ b/src/nullrun/observability/__init__.py @@ -67,8 +67,8 @@ class TransportMetrics: circuit_half_open_count: int = 0 circuit_closed_count: int = 0 fallback_mode_activations: int = 0 - # Sprint 1.5 (B13): HMAC verification failures on the control - # plane WebSocket. Pre-fix, a signature mismatch on a signed + # HMAC verification failures on the control plane WebSocket + # (B13). Pre-fix, a signature mismatch on a signed # ``state_change`` / ``key_rotated`` / ``policy_invalidated`` # message was logged at WARNING and the message was silently # dropped — meaning a forged or mis-rotated kill command could diff --git a/src/nullrun/observability/error_hooks.py b/src/nullrun/observability/error_hooks.py index 66c412e..622be59 100644 --- a/src/nullrun/observability/error_hooks.py +++ b/src/nullrun/observability/error_hooks.py @@ -142,13 +142,13 @@ def __post_init__(self) -> None: # from one thread and fired from another (e.g. register at app # startup, fire from a transport background thread). # -# Phase 4 (2026-07-05): the hot path is has_hooks(), which -# previously took an RLock.acquire on every call (100+ raises/min -# in a busy agent is enough to show up in profiles). We now keep -# the hook list under the same RLock but expose has_hooks() -# as a lock-free len() check. The list itself is private; -# callers always go through the public functions (which take -# the lock for the read snapshot during dispatch). +# The hot path is has_hooks(), which previously took an +# RLock.acquire on every call (100+ raises/min in a busy agent +# is enough to show up in profiles). We now keep the hook list +# under the same RLock but expose has_hooks() as a lock-free +# len() check. The list itself is private; callers always go +# through the public functions (which take the lock for the read +# snapshot during dispatch). _lock = threading.RLock() _hooks: list[ErrorHook] = [] diff --git a/src/nullrun/runtime.py b/src/nullrun/runtime.py index e0339cf..1c5d932 100644 --- a/src/nullrun/runtime.py +++ b/src/nullrun/runtime.py @@ -24,7 +24,7 @@ | `_emit_span_start` / `_emit_span_end` | n/a -- never blocks | n/a | n/a | | `/track` batch path (legacy) | OPEN-on-network-error (event dropped, no retry) | n/a -- circuit breaker backoff applies | none | -**Drift fix 2026-07-04:** the SDK_README.md claim +**Readme correction (2026-07-04):** the SDK_README.md claim "Fail-OPEN на инфраструктурных сбоях. Если backend недоступен, бюджет не блокирует агента" is **partially wrong** — it conflates SDK-side transport failure with backend-side budget-enforcement failure. The @@ -103,11 +103,11 @@ logger = logging.getLogger(__name__) -# Phase 0.3.1: sentinel used when a gate fires outside a -# ``with workflow(...)`` context. The double-underscore prefix -# namespacing avoids collision with a user workflow that happens -# to be named ```` (the previous literal was a -# collision hazard). Wire compat: still a string. +# Sentinel used when a gate fires outside a ``with workflow(...)`` +# context. The double-underscore prefix namespacing avoids +# collision with a user workflow that happens to be named +# ```` (the previous literal was a collision hazard). +# Wire compat: still a string. UNKNOWN_WORKFLOW_ID: str = "__nullrun_unknown__" # 2026-07-04 (BUG #5): in-process gate cache for chain-mode @@ -180,13 +180,13 @@ def is_strict_mode_forced(tool_name: str) -> bool: # worth the simplicity of a hard-coded threshold). SERVER_MINTED_RESERVATION_MAX_AGE_SECONDS: float = 295.0 -# Phase 0 review (2026-07-23): hard cap on server-supplied -# approval_timeout_seconds. The backend is authoritative for the -# approval window, but a misconfigured backend (or a malicious -# proxy in front of one) could advertise an absurdly long -# timeout (e.g. 1e9 seconds) and lock the calling thread -# indefinitely. We clamp the server value to this ceiling as -# the maximum time we'll ever wait for an operator click. +# Hard cap on server-supplied approval_timeout_seconds. The +# backend is authoritative for the approval window, but a +# misconfigured backend (or a malicious proxy in front of one) +# could advertise an absurdly long timeout (e.g. 1e9 seconds) and +# lock the calling thread indefinitely. We clamp the server +# value to this ceiling as the maximum time we'll ever wait for +# an operator click. # The env default `NULLRUN_APPROVAL_TIMEOUT_SECONDS` is also # clamped — see check_workflow_budget / runtime.execute for the # exact clamp call. @@ -202,10 +202,10 @@ def is_strict_mode_forced(tool_name: str) -> bool: def _validate_approval_timeout(value: object, log_prefix: str) -> float | None: """Validate and clamp a server-supplied approval_timeout_seconds. - Phase 0 review (2026-07-23): the server is authoritative for the - approval window, but it could advertise 0 (deadlock), 1e9 (lock the - thread for years), a non-numeric string, or `None`. We refuse to - forward anything outside `[MIN_APPROVAL_TIMEOUT_SECONDS, + The server is authoritative for the approval window, but it + could advertise 0 (deadlock), 1e9 (lock the thread for + years), a non-numeric string, or `None`. We refuse to forward + anything outside `[MIN_APPROVAL_TIMEOUT_SECONDS, MAX_APPROVAL_TIMEOUT_SECONDS]` and return `None` so the caller falls back to `_approval_timeout_seconds` (the env default). @@ -243,10 +243,10 @@ def _validate_approval_timeout(value: object, log_prefix: str) -> float | None: return candidate -# Phase 4.1: privacy boundary. Fields that MUST NOT leave the SDK on -# the wire. The transport layer (POST /api/v1/track/batch) reads -# whatever is in the event dict, so anything not allowlisted ends up -# in the user's audit log on the backend side. We strip: +# Privacy boundary: fields that MUST NOT leave the SDK on the +# wire. The transport layer (POST /api/v1/track/batch) reads +# whatever is in the event dict, so anything not allowlisted ends +# up in the user's audit log on the backend side. We strip: # # * ``cost_cents`` -- the SDK does not estimate cost; the backend # recomputes it from tokens + the org's pricing policy. Sending @@ -258,7 +258,7 @@ def _validate_approval_timeout(value: object, log_prefix: str) -> float | None: # which prompts went through dedup, defeating the purpose. # * ``raw_usage`` -- the vendor's full usage dict (OpenAI # ``prompt_tokens_details``, Anthropic ``cache_*_input_tokens`` -# etc.) — Phase 4.1 moved every field we care about out of +# etc.) -- every field we care about has been lifted out of # raw_usage onto the event itself, so the original dict is now # just an opaque blob of provider-specific data. Carrying it on # the wire is a privacy regression: provider response payloads @@ -271,15 +271,15 @@ def _validate_approval_timeout(value: object, log_prefix: str) -> float | None: _WIRE_STRIP_FIELDS: frozenset[str] = frozenset({"cost_cents", "_fingerprint", "raw_usage"}) -# Phase 3 (2026-07-05): metaclass is what routes the legacy -# NullRunRuntime._instance class-attribute access through the -# registry (see :class:`nullrun._singleton._NullRunRuntimeMeta`). -# The descriptor protocol only fires on class-level access if the -# descriptor lives on the metaclass — defining _instance on -# the class body would route reads through type.__getattribute__ -# and never call our __get__. Keeping the metaclass minimal -# (it only owns _instance) means every other attribute behaves -# exactly as before. +# The metaclass routes the legacy NullRunRuntime._instance +# class-attribute access through the registry (see +# :class:`nullrun._singleton._NullRunRuntimeMeta`). The descriptor +# protocol only fires on class-level access if the descriptor +# lives on the metaclass -- defining _instance on the class body +# would route reads through type.__getattribute__ and never call +# our __get__. Keeping the metaclass minimal (it only owns +# _instance) means every other attribute behaves exactly as +# before. from nullrun._singleton import _NullRunRuntimeMeta @@ -392,11 +392,12 @@ def __init__( ) # organization_id is set by _authenticate; stays None until then. self.organization_id: str | None = None - # Phase 139+: workflow_id is set by _authenticate from the API - # key's binding (organization_api_keys.workflow_id). Used as a - # fallback for /check, /status, and span events when the user - # hasn't entered a `with workflow(...)` context. None on legacy - # keys (pre-139 or never used) -- call sites must NOT invent one. + # workflow_id is set by _authenticate from the API key's + # binding (organization_api_keys.workflow_id). Used as a + # fallback for /check, /status, and span events when the + # user hasn't entered a `with workflow(...)` context. None + # on legacy keys (pre-139 or never used) -- call sites + # must NOT invent one. self.workflow_id: str | None = None self._test_mode = _test_mode @@ -417,12 +418,11 @@ def __init__( self._transport: Transport | None = None # Local enforcement state - # Phase 0.3.1: the BoundedDict-based per-workflow cost / - # loop / retry counters have been removed alongside - # ``_check_local_limits``. As of 0.7.0 ALL local - # enforcement (LoopTracker / RateTracker / _local_check / - # hardcoded thresholds) has been removed -- the SDK is a - # thin client, the backend is authoritative. + # The BoundedDict-based per-workflow cost / loop / retry + # counters have been removed alongside ``_check_local_limits``. + # As of 0.7.0 ALL local enforcement (LoopTracker / RateTracker + # / _local_check / hardcoded thresholds) has been removed -- + # the SDK is a thin client, the backend is authoritative. self._workflow_start_time: float = time.time() # Layer 3: ring buffer for the ``nullrun.status `` recent @@ -440,11 +440,11 @@ def __init__( self._last_backend_attempt_at: float | None = None self._last_backend_attempt_ok: bool | None = None - # Phase D: dedup LRU. Multiple observation paths (httpx transport - # LangChain callback, OpenAI Agents tracer) can fire for the same - # LLM call. We collapse them to a single track per fingerprint. - # The fingerprint is computed at the observation point and passed - # via the `_fingerprint` event field. + # Dedup LRU. Multiple observation paths (httpx transport, + # LangChain callback, OpenAI Agents tracer) can fire for + # the same LLM call. We collapse them to a single track per + # fingerprint. The fingerprint is computed at the observation + # point and passed via the `_fingerprint` event field. from nullrun.instrumentation.auto import make_dedup_state self._seen_track_fingerprints = make_dedup_state() @@ -465,15 +465,15 @@ def __init__( # Remote control plane state (per-workflow, pushed from server via WS). # Unified model: effective_state = max(local_state, remote_state). # All writes and reads go through the `_remote_state_for` / - # `_set_remote_state` helpers (Phase 5 #5.1) so the WS callback - # the HTTP poll, and the gate check can run concurrently - # without a TOCTOU race. RLock because the same thread can - # re-enter via the gate's get-then-set sequence. + # `_set_remote_state` helpers so the WS callback, the HTTP + # poll, and the gate check can run concurrently without a + # TOCTOU race. RLock because the same thread can re-enter + # via the gate's get-then-set sequence. self._remote_states: dict[str, dict[str, Any]] = {} self._states_lock = threading.RLock() - # Drift section 7 (2026-07-06): human-approval pending registry. - # When a /gate response carries decision="require_approval", + # Human-approval pending registry. When a /gate response + # carries decision="require_approval", # the SDK stores the (approval_id, workflow_id, execution_id) # tuple here and blocks until either: # - the WS push arrives with outcome="approved" (release @@ -501,11 +501,12 @@ def __init__( _t = 300.0 self._approval_timeout_seconds: float = _t - # Phase B: control plane transport. The SDK connects to the server's - # WS endpoint and receives state push events (killed/paused) within - # ~100ms of the operator action -- vs the previous 1s HTTP poll. - # The HTTP poll path is preserved as a fallback when - # `NULLRUN_TRANSPORT=http` is set (env var defaults to `ws`). + # Control plane transport. The SDK connects to the server's + # WS endpoint and receives state push events (killed/paused) + # within ~100ms of the operator action -- vs the previous 1s + # HTTP poll. The HTTP poll path is preserved as a fallback + # when `NULLRUN_TRANSPORT=http` is set (env var defaults to + # `ws`). self._transport_mode: str = os.getenv("NULLRUN_TRANSPORT", "ws").lower() self._ws_thread: threading.Thread | None = None self._ws_stop_event = threading.Event() @@ -570,12 +571,12 @@ def __init__( # Initialize action handler self._action_handler = ActionHandler() - # Phase 1.4: Sensitive tools that require strict mode (pre-execution enforcement) - # These tools MUST go through /execute endpoint, NOT direct execution - # Phase 4 (2026-07-05): is_sensitive_tool is the hot - # path on every @protect call against a sensitive tool. - # We keep a pre-lowercased mirror so the read does not - # have to build a set comprehension on every call. The + # Sensitive tools that require strict mode (pre-execution + # enforcement). These tools MUST go through /execute + # endpoint, NOT direct execution. ``is_sensitive_tool`` is + # the hot path on every @protect call against a sensitive + # tool. We keep a pre-lowercased mirror so the read does + # not have to build a set comprehension on every call. The # cache is mutated alongside _sensitive_tools under # _tools_lock (see add/remove_sensitive_tool below) and # every value is lowercased at insertion time. @@ -616,12 +617,11 @@ def __init__( # comprehension per call). Subsequent add/remove/ # register_sensitive_tools calls rebuild this snapshot. self._sensitive_tools_lower = frozenset(t.lower() for t in self._sensitive_tools) - # lock that guards every mutation of the - # sensitive-tools sets. The pre-fix code did - # ``self._strict_mode_tools.add(tool_name)`` from - # ``add_sensitive_tool`` without holding any lock; the - # reader in ``is_sensitive_tool`` (line 1270-ish) did - # ``tool_name in self._strict_mode_tools`` without a lock. + # Lock that guards every mutation of the sensitive-tools + # sets. Reads and writes to these sets are guarded so a + # concurrent reader cannot observe a mid-mutation snapshot + # on a free-threaded build. The lock is uncontended on the + # read path so the cost is one acquire per call. # Under CPython's GIL the set mutation is atomic at the # bytecode level, but the snapshot you read can still be # stale mid-mutation (a single-threaded read can see the @@ -637,11 +637,10 @@ def __init__( def get_instance(cls) -> "NullRunRuntime": """Get the singleton runtime instance. - Thread-safe: the singleton lock is held for the full read-compare- - rebuild sequence (Phase 5 #5.3). The previous version dropped the - lock between shutdown and the recursive get_instance, creating a - window where a concurrent caller could observe a half-shutdown - runtime. + Thread-safe: the singleton lock is held for the full + read-compare-rebuild sequence. The lock prevents a + concurrent caller from observing a half-shutdown runtime + between an inner shutdown and the recursive rebuild. """ with cls._lock: # Re-read env vars at every call site so credential rotation @@ -949,20 +948,21 @@ def _authenticate(self) -> None: raise err self.organization_id = org_id - # Phase 139+: pick up the workflow this key is bound to. - # `None` on legacy keys (pre-139 or never-used) -- call - # sites that NEED a workflow (check_workflow_budget - # check_control_plane, span events) will fall through to - # the contextvar when self.workflow_id is None, exactly - # like before. New keys always have this set. + # Pick up the workflow this key is bound to. + # `None` on legacy keys (pre-139 or never-used) -- + # call sites that NEED a workflow + # (check_workflow_budget, check_control_plane, span + # events) will fall through to the contextvar when + # self.workflow_id is None, exactly like before. + # New keys always have this set. self.workflow_id = data.get("workflow_id") - # Phase 0.3.1: pre-Phase-139 API keys do not return - # workflow_id, so the SDK cannot honour the - # dashboard's KILL/PAUSE for that workflow. Emit a - # one-time WARNING so the operator knows to rotate - # the key. Without this, the kill switch silently - # no-ops (a real safety hole for legacy users). + # Legacy API keys do not return workflow_id, so the + # SDK cannot honour the dashboard's KILL/PAUSE for + # that workflow. Emit a one-time WARNING so the + # operator knows to rotate the key. Without this, + # the kill switch silently no-ops (a real safety + # hole for legacy users). if self.workflow_id is None: masked_key = ( (self.api_key[:8] + "***") @@ -972,7 +972,7 @@ def _authenticate(self) -> None: logger.warning( f"API key {masked_key!s} is a legacy key with no " f"workflow binding; remote kill/pause will not be " - f"honoured. Rotate to a Phase 139+ key in the " + f"honoured. Rotate to a workflow-bound key in the " f"dashboard to enable control plane enforcement." ) @@ -1037,10 +1037,11 @@ def _start_transport(self) -> None: def _start_remote_polling(self) -> None: """Start the control-plane background listener. - Phase B: defaults to WebSocket push for sub-second kill/pause - propagation. Set `NULLRUN_TRANSPORT=http` to fall back to the - legacy 1-second HTTP poll (kept for environments where the WS - endpoint is blocked or for parity with old SDK behavior). + Defaults to WebSocket push for sub-second kill/pause + propagation. Set `NULLRUN_TRANSPORT=http` to fall back to + the legacy 1-second HTTP poll (kept for environments where + the WS endpoint is blocked or for parity with old SDK + behavior). """ if self._transport_mode == "http": self._start_http_poller() @@ -1057,12 +1058,13 @@ def _start_http_poller(self) -> None: logger.info("Started remote state poller (HTTP)") def _start_ws_listener(self) -> None: - """Phase B: connect the WebSocket push channel in a background thread. + """Connect the WebSocket push channel in a background thread. - The thread runs its own asyncio loop so the WS receive task can - drive `_remote_states` from server pushes without contending with - the user's main loop. Reconnects with exponential backoff on - disconnect (handled inside `WebSocketConnection`). + The thread runs its own asyncio loop so the WS receive task + can drive `_remote_states` from server pushes without + contending with the user's main loop. Reconnects with + exponential backoff on disconnect (handled inside + `WebSocketConnection`). """ if not self.organization_id: logger.warning( @@ -1207,24 +1209,23 @@ def _resolve_workflow_id(self, explicit: str | None = None) -> str | None: 1. `explicit` -- passed by the call site (e.g. contextvar in track_event or the user-supplied arg in check_control_plane) - 2. `self.workflow_id` -- bound to the API key by the server - (Phase 139+). Set during _authenticate. None on legacy - keys. + 2. `self.workflow_id` -- bound to the API key by the server. + Set during _authenticate. None on legacy keys. 3. None -- caller is in cloud mode but has no workflow scope. /check falls through to org-level policy; /status is skipped; span events are emitted without workflow_id (orphan, as before). - The SDK does NOT auto-generate a workflow_id. The Phase 139 - invariant -- workflow is derived server-side from the key, never - invented by the SDK -- is preserved. + The SDK does NOT auto-generate a workflow_id. The + invariant -- workflow is derived server-side from the key, + never invented by the SDK -- is preserved. """ if explicit: return explicit return self.workflow_id def _remote_state_for(self, workflow_id: str) -> dict[str, Any]: - """Return the cached remote state for `workflow_id` (Phase 5 #5.1). + """Return the cached remote state for `workflow_id`. Thread-safe via `_states_lock`. If no state has been pushed yet, returns an empty dict (so callers can do @@ -1253,9 +1254,9 @@ def _fetch_remote_state(self, workflow_id: str) -> None: backend/src/proxy/handlers.rs:9758, accepts X-API-Key OR Authorization: Bearer). Pre-swap the HTTP-poll path silently 401'd on every poll, so the legacy HTTP-poll fallback never - observed a remote kill/pause. WS push (the default mode since - Phase 5) does NOT go through this code path, so the WS control - plane is unaffected. + observed a remote kill/pause. WS push (the default mode) + does NOT go through this code path, so the WS control plane + is unaffected. Backend ``StatusResponse`` (handlers.rs:9747-9756) returns ``workflow_id, state, version, reason?, updated_at @@ -1292,11 +1293,10 @@ def _fetch_remote_state(self, workflow_id: str) -> None: logger.debug(f"Failed to fetch remote state for {workflow_id}: {e}") def _handle_approval_resolved(self, payload: dict[str, Any]) -> None: - """Drift section 7 (2026-07-06): WS push handler for an - approval resolution. Releases the matching gate - reservation (approved) or raises WorkflowKilledInterrupt - (denied) so the agent can resume from the same - execution_id. + """WS push handler for an approval resolution. Releases + the matching gate reservation (approved) or raises + WorkflowKilledInterrupt (denied) so the agent can resume + from the same execution_id. Args: payload: The WsMessage::ApprovalResolved dict from the @@ -1340,8 +1340,7 @@ def _wait_for_approval_resolution( execution_id: str, timeout_seconds: float | None = None, ) -> dict[str, Any]: - """Drift section 7 (2026-07-06) + Разрыв 1c (2026-07-21): - block the calling thread until the WS approval push + """Block the calling thread until the WS approval push arrives (or the per-approval timeout elapses). The WS handler (``_handle_approval_resolved`` above) sets the threading Event when the push lands; this method waits @@ -1353,16 +1352,14 @@ def _wait_for_approval_resolution( execution_id: Execution the approval gates. timeout_seconds: Server-authoritative wait duration from the /gate response field - ``approval_timeout_seconds`` (added in Разрыв 1c, - 2026-07-21). When set, this overrides - ``self._approval_timeout_seconds`` (the + ``approval_timeout_seconds``. When set, this + overrides ``self._approval_timeout_seconds`` (the ``NULLRUN_APPROVAL_TIMEOUT_SECONDS`` env default) so the SDK can never silently desync from the backend row's actual expiry. When - ``None`` (legacy backend without Разрыв 1c - field, or malformed response), falls back to the - env-derived default — same behavior as pre-Разрыв - 1c SDKs. + ``None`` (legacy backend without that field, or + malformed response), falls back to the + env-derived default. Returns: The entry dict, with ``outcome`` populated (either @@ -1370,13 +1367,10 @@ def _wait_for_approval_resolution( a sentinel ``{"outcome": "timeout", "timed_out": True}``. **The caller is expected to fail-CLOSED on timeout** — - raise ``WorkflowKilledInterrupt``. The Разрыв 1c - contract deliberately rejected a `/status` poll - fallback here (per `references/razriv1c-approval-flow.md`, - Phase 4 H4): a silent timeout must not silently - approve a privileged action. The legacy docstring - "fall back to legacy /status poll path" was incorrect - and is removed as part of the 2026-07-23 review. + raise ``WorkflowKilledInterrupt``. The contract + deliberately rejects a `/status` poll fallback here: + a silent timeout must not silently approve a + privileged action. Raises: Nothing. Approval timeouts are returned, not raised, @@ -1384,19 +1378,19 @@ def _wait_for_approval_resolution( (raise WorkflowKilledInterrupt on denied OR on timeout, resume on approved). """ - # Per-approval timeout resolution (Разрыв 1c, 2026-07-21): - # prefer the server-authoritative value from the /gate - # response so the SDK never times out before the - # backend's expiry sweeper (Разрыв 3 class of bug). - # Fall back to the env default only on missing or - # out-of-range value — both signal "backend didn't send a - # sane value" and we preserve the pre-Разрыв 1c behaviour. + # Per-approval timeout resolution: prefer the + # server-authoritative value from the /gate response so + # the SDK never times out before the backend's expiry + # sweeper. Fall back to the env default only on missing + # or out-of-range value -- both signal "backend didn't + # send a sane value" and we preserve the legacy + # behaviour. # - # Phase 0 review (2026-07-23): we clamp the server value - # to `[MIN, MAX]`. A misconfigured backend advertising - # 0 (deadlock), 1e9 (lock the thread for years), or any - # other garbage value will not stall the agent loop — - # we fall back to the env default instead. + # Clamp the server value to `[MIN, MAX]`. A + # misconfigured backend advertising 0 (deadlock), 1e9 + # (lock the thread for years), or any other garbage + # value will not stall the agent loop -- we fall back + # to the env default instead. if timeout_seconds is not None: try: candidate = float(timeout_seconds) @@ -1487,20 +1481,19 @@ def check_control_plane(self, workflow_id: str) -> None: WorkflowPausedException: If workflow is paused on server WorkflowKilledInterrupt: If workflow is killed on server """ - # Phase 139+: prefer the explicit arg (contextvar-supplied), fall - # back to the API key's bound workflow. None on legacy keys -- + # Prefer the explicit arg (contextvar-supplied), fall back + # to the API key's bound workflow. None on legacy keys -- # in that case there's no workflow to check, so we no-op - # (preserves pre-139 behavior for keys that have never been - # workflow-bound). + # (preserves the legacy behavior for keys that have never + # been workflow-bound). resolved = self._resolve_workflow_id(workflow_id or None) if not resolved: return workflow_id = resolved - # Ensure we have the latest remote state - # Phase 5 #5.1: use the lock-protected getter so a concurrent - # WS push can't drop the state between the membership check - # and the read. + # Ensure we have the latest remote state. Use the + # lock-protected getter so a concurrent WS push can't drop + # the state between the membership check and the read. remote_state = self._remote_state_for(workflow_id) if not remote_state: # Fetch synchronously if not in cache yet @@ -1536,9 +1529,6 @@ def check_workflow_budget(self) -> None: before the wrapped function runs, so a workflow with no remaining budget never gets to spend tokens. - Sprint 3.1: bumps the ``check_calls`` metric so the dashboard - can show the rate of pre-flight budget checks. - Decision → exception mapping: "block" → WorkflowKilledInterrupt (hard policy / reservation error) "throttle"→ WorkflowPausedException (insufficient budget, can resume) @@ -1566,10 +1556,10 @@ def check_workflow_budget(self) -> None: logger.debug("check_workflow_budget: skipped via NULLRUN_SKIP_BUDGET_CHECK=1") return - # Sprint 3.1 (B23): bump the ``check_calls`` counter so the - # dashboard can show the rate of pre-flight budget checks - # and the operator can verify the pre-flight is actually - # running (not silently always-skipped). + # Bump the ``check_calls`` counter so the dashboard can show + # the rate of pre-flight budget checks and the operator can + # verify the pre-flight is actually running (not silently + # always-skipped). metrics.inc_runtime("check_calls") from nullrun.context import ( @@ -1582,28 +1572,27 @@ def check_workflow_budget(self) -> None: get_workflow_id, ) - # Phase 139+: prefer the user-set contextvar (explicit `with - # workflow(...)` block), fall back to the API key's bound - # workflow. Returns None only on legacy keys that have never - # been workflow-bound -- in that case the check is silently - # skipped, exactly as before this change. + # Prefer the user-set contextvar (explicit `with workflow(...)` + # block), fall back to the API key's bound workflow. Returns + # None only on legacy keys that have never been + # workflow-bound -- in that case the check is silently + # skipped. workflow_id = self._resolve_workflow_id(get_workflow_id()) if not workflow_id: return - # T4 (2026-06-27): use the real model name from the call - # context if the user set it via `set_call_context(model=...)` - # (or via a future `with workflow(..., model=...)` block). - # Pre-T4 this always sent the literal string "budget-precheck" - # — a fake sentinel that: - # 1. forced backend pricing lookup to fall through to the - # default 3.0 rate, so projected_cost was always computed - # against the wrong per-model rate - # 2. blocked any future per-model budget tier (model-specific - # caps) from being enforced correctly. - # Sending `None` is fine — backend `calculate_projected_cost` - # defaults to claude-sonnet-4 when model is unset, and tool_block - # enforcement on /gate is best-effort when no tools are sent. + # Use the real model name from the call context if the user + # set it via `set_call_context(model=...)` (or via a future + # `with workflow(..., model=...)` block). Earlier SDK + # versions always sent the literal string "budget-precheck" + # -- a fake sentinel that forced backend pricing lookup to + # fall through to the default rate, so projected_cost was + # always computed against the wrong per-model rate and + # blocked any future per-model budget tier (model-specific + # caps) from being enforced correctly. Sending `None` is + # fine -- backend `calculate_projected_cost` defaults when + # model is unset, and tool_block enforcement on /gate is + # best-effort when no tools are sent. call_model = get_call_model() call_tools = get_call_tools() @@ -1643,16 +1632,15 @@ def check_workflow_budget(self) -> None: if call_tools: check_req["tools"] = list(call_tools) - # Разрыв 3 / 2026-07-28: forward cached MCP tool class + - # annotations when the SDK recognises an MCP server. Both - # fields are optional — `None` means "I don't know", and the - # gate treats absent values as unknown rather than false. - # The honest-SDK trust boundary (CLAUDE.md §22): a - # malicious SDK could lie about annotations to bypass the - # destructive block. We accept that trade-off (matches the - # existing model-string trust model) — server-side discovery - # for verification is in the v3.31 Phase C scope and - # requires HTTP-transport MCP servers only. + # Forward cached MCP tool class + annotations when the SDK + # recognises an MCP server. Both fields are optional -- + # `None` means "I don't know", and the gate treats absent + # values as unknown rather than false. The trust boundary + # is honest: a malicious SDK could lie about annotations to + # bypass the destructive block. We accept that trade-off + # (matches the existing model-string trust model) -- + # server-side discovery for verification requires + # HTTP-transport MCP servers only. mcp_class = get_call_mcp_class() if mcp_class is not None: check_req["tool_class"] = mcp_class @@ -1676,8 +1664,8 @@ def check_workflow_budget(self) -> None: # an idempotency_key without an extra round-trip. check_req["idempotency_key"] = check_req["operation_id"] - # 2026-07-04 (BUG #5): in-process gate cache for chain-mode. - # See module-top comment on _GATE_CACHE for full rationale. + # In-process gate cache for chain-mode invocations. See + # module-top comment on _GATE_CACHE for full rationale. response: dict[str, Any] cache_key: tuple[str, str | None, str | None] | None = None cache_enabled = ( @@ -1709,9 +1697,9 @@ def check_workflow_budget(self) -> None: try: response = self._transport.check(check_req) except (httpx.HTTPError, NullRunError) as exc: - # Narrow catch (Phase 6 H5): fail-OPEN only on - # transport + classified SDK errors. Internal - # bugs (KeyError, AttributeError) should surface + # Narrow catch: fail-OPEN only on transport + + # classified SDK errors. Internal bugs + # (KeyError, AttributeError) should surface # rather than silently allow an unbounded call. logger.warning(f"check_workflow_budget: /gate unavailable, failing open: {exc}") return @@ -1752,11 +1740,11 @@ def check_workflow_budget(self) -> None: decision = response.get("decision", "allow") decision_source = response.get("decision_source", DecisionSource.GATEWAY) - # Round 3 (Phase 0.4.0): only fail-OPEN on EXPLICIT synthetic - # responses (decision_source starts with "fallback" or is one - # of the classified TransportErrorSource values). Real - # backend decisions (decision_source="gateway", or missing - # for backward compat) are honoured. + # Only fail-OPEN on EXPLICIT synthetic responses + # (decision_source starts with "fallback" or is one of the + # classified TransportErrorSource values). Real backend + # decisions (decision_source="gateway", or missing for + # backward compat) are honoured. if decision_source.startswith("fallback") or decision_source in { TransportErrorSource.NETWORK_ERROR, TransportErrorSource.GATEWAY_ERROR, @@ -1782,11 +1770,11 @@ def check_workflow_budget(self) -> None: reasons = response.get("explanations") or ( [response["explanation"]] if response.get("explanation") else ["block"] ) - # Sprint 3 follow-up (B23): bump ``cost_limit_exceeded`` - # when the pre-flight blocks the workflow. The counter - # is the operator's primary signal for "the budget - # cap is biting" — distinct from loop / retry / rate - # which have their own counters. + # Bump ``cost_limit_exceeded`` when the pre-flight + # blocks the workflow. The counter is the operator's + # primary signal for "the budget cap is biting" -- + # distinct from loop / retry / rate which have their + # own counters. metrics.inc_runtime("cost_limit_exceeded") raise WorkflowKilledInterrupt( workflow_id=workflow_id, @@ -1800,36 +1788,63 @@ def check_workflow_budget(self) -> None: workflow_id=workflow_id, reason="; ".join(reasons), ) - if decision == "throttle": - reasons = response.get("explanations") or ( - [response["explanation"]] if response.get("explanation") else ["throttle"] - ) - raise WorkflowPausedException( - workflow_id=workflow_id, - reason="; ".join(reasons), + if decision == "soft_pass": + # Soft-mode call proceeded via the chain's overdraft cap + # (CLAUDE.md §5). The body MUST execute — soft_pass is + # semantically distinct from `block`; rejecting here + # would silently disable the soft-mode escape hatch for + # every agent. We log at INFO so the operator sees the + # overdraft is biting and track the cents-burned for + # telemetry, but we do NOT raise -- execution continues. + # + # Wire shape (backend::gate::internal.rs + # GateResponse::soft_pass): + # { + # decision: "soft_pass", + # decision_source: "gateway", + # explanation: "Budget exhausted, overdraft cap covers request", + # overdraft_used_cents: 50, # incremented on backend + # max_overdraft_cents: 200, + # remaining_overdraft_cents: 150, + # details: {...} + # } + overdraft_used = response.get("overdraft_used_cents") + max_overdraft = response.get("max_overdraft_cents") + remaining = response.get("remaining_overdraft_cents") + explanation = response.get("explanation") or "soft_pass" + # Counter name parallels ``cost_limit_exceeded`` for hard + # blocks — operators can graph "soft overdraft pressure" + # alongside "hard cap hits" via the same dashboard panel. + metrics.inc_runtime("soft_overdraft_used") + logger.warning( + "check_workflow_budget: soft_pass -- %s " + "(overdraft_used=%s, max=%s, remaining=%s)", + explanation, + overdraft_used, + max_overdraft, + remaining, ) + return if decision == "require_approval": - # Drift section 7 (2026-07-06) + Разрыв 1c (2026-07-21): - # the gate requires a human-approval before the call + # The gate requires a human-approval before the call # may proceed. Block the calling thread on the WS push # (handled in _handle_approval_resolved) and let the # operator click Approve/Deny on the dashboard. On # timeout (WS push silent for the configured duration) # we fall through and the caller is expected to treat - # the call as blocked — the same fail-CLOSED semantics + # the call as blocked -- the same fail-CLOSED semantics # as a regular block. # - # Разрыв 1c: prefer the server-authoritative + # Prefer the server-authoritative # `approval_timeout_seconds` value from the response - # (added in commit 0ad03b9) over the env default + # over the env default # `NULLRUN_APPROVAL_TIMEOUT_SECONDS`. This prevents the - # SDK/backend desync that the Разрыв 3 sweeper was - # written to fix on the backend side. We fall back to - # the env default only when the field is missing or - # non-positive — both signal "backend without Разрыв 1c - # field" and we preserve the pre-Разрыв 1c behaviour - # for those callers. + # SDK/backend desync that the backend expiry sweeper + # was written to fix. We fall back to the env default + # only when the field is missing or non-positive -- + # both signal "backend without that field" and we + # preserve the legacy behaviour for those callers. approval_id = response.get("approval_id", "") or "" if not approval_id: logger.warning( @@ -1844,10 +1859,10 @@ def check_workflow_budget(self) -> None: # `approval_expires_at` (ISO8601 string) are exposed; # we prefer the integer field because it's directly # usable in `event.wait(timeout=...)`. If the backend - # only sent the ISO8601 string (e.g. older Разрыв 1c - # draft or proxy rewriting the field), fall through to - # the env default rather than try to parse it inline — - # the field is documented as informational for UI/logs + # only sent the ISO8601 string (e.g. an older proxy + # rewriting the field), fall through to the env + # default rather than try to parse it inline -- the + # field is documented as informational for UI/logs # and isn't required for the SDK's wait math. server_timeout = _validate_approval_timeout( response.get("approval_timeout_seconds"), @@ -1934,7 +1949,7 @@ def ping_chain( if interval < 10.0 or interval > 120.0: raise ValueError( f"ping_chain interval must be in [10, 120] seconds per " - f"CLAUDE.md §26, got {interval}" + f"the chain heartbeat spec, got {interval}" ) stop_event = _threading.Event() @@ -2077,14 +2092,14 @@ def shutdown(self, flush: bool = True) -> None: # Stop the HTTP poller (legacy path) if it was started. self._poll_running = False if self._poll_thread and self._poll_thread.is_alive(): - # Phase 6 #6.3: cap to 0.5s (was 2.0s) so a SIGTERM - # handler returns quickly. The HTTP-poll is best-effort - # and the WS push channel is the authoritative source. + # Cap to 0.5s so a SIGTERM handler returns quickly. + # The HTTP-poll is best-effort and the WS push channel + # is the authoritative source. self._poll_thread.join(timeout=0.5) - # Stop the WS control plane listener (Phase B). Closing the - # connection causes the receive task to unblock, the loop to - # exit, and the thread to terminate. + # Stop the WS control plane listener. Closing the + # connection causes the receive task to unblock, the loop + # to exit, and the thread to terminate. self._ws_stop_event.set() conn = self._ws_connection if conn is not None and self._ws_loop is not None: @@ -2147,10 +2162,11 @@ def track( """ logger.debug(f"Tracking event: {event.get('event_type', 'unknown')}") - # Phase D: dedup gate. The httpx transport, LangChain callback, and - # OpenAI Agents tracer can all fire for the same LLM call. We drop - # repeats keyed by `_fingerprint` (set by the observation path) so - # each unique call produces exactly one /api/v1/track POST. + # Dedup gate. The httpx transport, LangChain callback, and + # OpenAI Agents tracer can all fire for the same LLM call. + # We drop repeats keyed by `_fingerprint` (set by the + # observation path) so each unique call produces exactly + # one /api/v1/track POST. fp = event.get("_fingerprint") if fp: from nullrun.instrumentation.auto import _fingerprint_is_seen @@ -2200,7 +2216,7 @@ def track( if workflow_id: self._remote_state_for(workflow_id) - # Phase 0.3.1: the local cost / loop / retry-storm check + # The local cost / loop / retry-storm check # (``_check_local_limits``) has been removed. It read # ``event.get("cost_cents", 0)`` and accumulated into a # per-workflow counter, but ``track_llm`` / @@ -2296,7 +2312,7 @@ def _trigger_action( # Let the exception propagate # ============================================================================= - # Phase 1.4: Pre-Execution Enforcement (SDK Boundary Fix) + # Pre-Execution Enforcement (SDK Boundary) # ============================================================================= def is_sensitive_tool(self, tool_name: str) -> bool: @@ -2326,12 +2342,12 @@ def is_sensitive_tool(self, tool_name: str) -> bool: ``add_sensitive_tool``. The lock is uncontended under CPython's GIL, so the cost is negligible. """ - # Phase 4 (2026-07-05): O(1) lookup against the - # pre-lowercased frozenset snapshot. The lock is still - # taken to keep the snapshot coherent with the live - # set during concurrent add/remove_sensitive_tool calls - # (the snapshot is rebuilt under the lock), but the - # read itself is a single frozenset membership check. + # O(1) lookup against the pre-lowercased frozenset + # snapshot. The lock is still taken to keep the snapshot + # coherent with the live set during concurrent + # add/remove_sensitive_tool calls (the snapshot is rebuilt + # under the lock), but the read itself is a single + # frozenset membership check. needle = tool_name.lower() with self._tools_lock: return needle in self._sensitive_tools_lower or needle in self._strict_mode_tools_lower @@ -2339,9 +2355,8 @@ def is_sensitive_tool(self, tool_name: str) -> bool: def get_org_status(self, org_id: str | None = None) -> dict[str, Any]: """Public helper for reading ``/api/v1/orgs/{org_id}/status``. - Phase 8 #8.1: routes through ``self._transport._client`` so - the shared connection pool, retry policy, and circuit breaker - apply. Used by ``examples/cost_dashboard.py``. + Routes through ``self._transport._client`` so the shared + connection pool, retry policy, and circuit breaker apply. Args: org_id: Optional organisation ID. Defaults to the runtime's @@ -2397,8 +2412,8 @@ def add_sensitive_tool(self, tool_name: str) -> None: """ with self._tools_lock: self._strict_mode_tools.add(tool_name) - # Phase 4: rebuild the lowercase snapshot so the - # hot-path is_sensitive_tool sees the new entry. + # Rebuild the lowercase snapshot so the hot-path + # is_sensitive_tool sees the new entry. self._strict_mode_tools_lower = frozenset(t.lower() for t in self._strict_mode_tools) def remove_sensitive_tool(self, tool_name: str) -> None: @@ -2416,7 +2431,7 @@ def remove_sensitive_tool(self, tool_name: str) -> None: """ with self._tools_lock: self._strict_mode_tools.discard(tool_name) - # Phase 4: rebuild the lowercase snapshot. + # Rebuild the lowercase snapshot. self._strict_mode_tools_lower = frozenset(t.lower() for t in self._strict_mode_tools) def register_sensitive_tools(self, tool_names: list[str]) -> None: @@ -2437,9 +2452,9 @@ def register_sensitive_tools(self, tool_names: list[str]) -> None: with self._tools_lock: for tool_name in tool_names: self._strict_mode_tools.add(tool_name) - # Phase 4: rebuild the lowercase snapshot once - # after the batch insert (a single set comprehension - # beats N rebuilds in the loop). + # Rebuild the lowercase snapshot once after the batch + # insert (a single set comprehension beats N rebuilds + # in the loop). self._strict_mode_tools_lower = frozenset(t.lower() for t in self._strict_mode_tools) def get_sensitive_tools(self) -> set[str]: @@ -2473,11 +2488,11 @@ def execute( - "auto": auto-select based on tool risk on_transport_error: Optional callback for transport-error handling (legacy); prefer the typed exception path. - business_impact: Phase 1 / MVP 1.0 typed action payload - (Money impact for now). When supplied, the backend - uses it to evaluate rule predicates AND stamps the - approval row's `action_digest` so the post-approval - /execute re-check can refuse tampered payloads. + business_impact: Typed action payload (Money impact for + now). When supplied, the backend uses it to evaluate + rule predicates AND stamps the approval row's + `action_digest` so the post-approval /execute re-check + can refuse tampered payloads. action_digest: SHA-256 hex of the canonicalised impact JSON. Computed client-side (Python helper in ``nullrun.business_impact.compute_action_digest``) @@ -2556,13 +2571,13 @@ def execute( "operation_id": operation_id, "on_transport_error": on_transport_error, } - # Phase 1 / MVP 1.0: digest-bound approval. Forward the - # typed impact + digest to the wire when supplied. The - # backend stamps the approval row with the digest and - # verifies it on the post-approval re-check. When the - # caller did NOT supply them (legacy Phase 0 path), the - # fields are absent from the wire; the backend falls - # back to approval_id-only grant consume. + # Digest-bound approval: forward the typed impact + digest + # to the wire when supplied. The backend stamps the approval + # row with the digest and verifies it on the post-approval + # re-check. When the caller did NOT supply them (the + # legacy approval_id-only path), the fields are absent from + # the wire; the backend falls back to approval_id-only + # grant consume. if business_impact is not None: execute_kwargs["business_impact"] = business_impact if action_digest is not None: @@ -2627,32 +2642,52 @@ def execute( # Check if execution is allowed if result.get("decision") == "block": metrics.inc_runtime("execute_blocked") - # Layer 1: best-effort error_code mapping from the - # backend's ``explanation`` string. The backend does not - # yet stamp a structured block_reason on /execute - # responses (planned for the next round), so we match on - # keywords in the free-text explanation. Anything we - # cannot classify falls back to ``NR-X001`` (generic - # block). The mapping is intentionally conservative — - # false positives give the user the wrong code, false - # negatives just fall back to the generic code. + # Layer 1: best-effort error_code mapping. + # + # The backend stamps a structured ``details.error_code`` + # on every block response, alongside the existing + # BUDGET_* / RATE_LIMIT_* family. When the backend + # provides one, we use it verbatim -- no string + # parsing, no false positives. Falls back to the + # legacy keyword-on-explanation mapping for older + # backends that pre-date the structured code (the + # keyword path stays for back-compat -- an older + # SDK still classifies budget/loop/rate/tool blocks + # correctly). explanation = result.get("explanation", "policy violation") - explanation_lower = explanation.lower() - if "budget" in explanation_lower or "exhausted" in explanation_lower: - block_code, block_action = "NR-B004", "block" - block_cls = "NullRunBudgetError" - elif "loop" in explanation_lower or "repetition" in explanation_lower: - block_code, block_action = "NR-L001", "block" - block_cls = "NullRunBlockedException" - elif "rate" in explanation_lower or "too many" in explanation_lower: - block_code, block_action = "NR-R001", "block" + wire_details = result.get("details") or {} + if not isinstance(wire_details, dict): + wire_details = {} + wire_error_code = wire_details.get("error_code") + if wire_error_code and isinstance(wire_error_code, str): + # Backend-supplied structured code wins. The + # catalogue exception class is mapped via + # ``_V3_ERROR_CODE_MAP`` on the transport path; on + # this /execute path we only have the SCREAMING_SNAKE + # backend code, so we surface it as-is in the + # ``error_code`` slot and let the caller branch on + # the catalog subclass if it has imported one. The + # block_code -> SDK exception-class mapping is done + # via the catalogue in nullrun.breaker.exceptions. + block_code, block_action = wire_error_code, "block" block_cls = "NullRunBlockedException" - elif "tool" in explanation_lower and "block" in explanation_lower: - block_code, block_action = "NR-T001", "block" - block_cls = "NullRunToolBlockedError" else: - block_code, block_action = "NR-X001", "block" - block_cls = "NullRunBlockedException" + explanation_lower = explanation.lower() + if "budget" in explanation_lower or "exhausted" in explanation_lower: + block_code, block_action = "NR-B004", "block" + block_cls = "NullRunBudgetError" + elif "loop" in explanation_lower or "repetition" in explanation_lower: + block_code, block_action = "NR-L001", "block" + block_cls = "NullRunBlockedException" + elif "rate" in explanation_lower or "too many" in explanation_lower: + block_code, block_action = "NR-R001", "block" + block_cls = "NullRunBlockedException" + elif "tool" in explanation_lower and "block" in explanation_lower: + block_code, block_action = "NR-T001", "block" + block_cls = "NullRunToolBlockedError" + else: + block_code, block_action = "NR-X001", "block" + block_cls = "NullRunBlockedException" # Note: we still raise the base ``NullRunBlockedException`` # for non-budget/tool cases to keep the construction # shape simple — the catalogue code is what the user @@ -2662,13 +2697,22 @@ def execute( # subclass per branch above; keeping one raise here is # easier to reason about and matches the way the rest of # the codebase handles backend blocks. + # + # ``details`` carries the wire ``details`` payload so the + # caller can introspect ``exc.details["error_code"]`` and + # ``exc.details["decision_source"]`` for diagnostic + # routing. ``mapped_class`` is preserved as a backwards- + # compat shim for callers that branched on the keyword + # path; new code should branch on ``exc.error_code``. + merged_details = dict(wire_details) + merged_details["mapped_class"] = block_cls err = NullRunBlockedException( workflow_id=workflow_id or UNKNOWN_WORKFLOW_ID, reason=explanation, action=block_action, tool_name=tool_name, error_code=block_code, - details={"mapped_class": block_cls}, + details=merged_details, ) # Layer 2: fire the on_error hook. The hook sees the # same exception the caller will catch plus the @@ -2739,10 +2783,9 @@ def _enrich_event(self, event: dict[str, Any]) -> dict[str, Any]: """Add context fields to event.""" enriched = dict(event) # Don't modify original - # Phase 139+: workflow_id from context, else from the API - # key's binding (set in _authenticate). Stays unset on legacy - # keys -- emitted events then carry no workflow_id (orphan, as - # before this change). + # workflow_id from context, else from the API key's + # binding (set in _authenticate). Stays unset on legacy + # keys -- emitted events then carry no workflow_id (orphan). if "workflow_id" not in enriched: wf_id = self._resolve_workflow_id(get_workflow_id()) if wf_id: @@ -2837,7 +2880,46 @@ def _enrich_event(self, event: dict[str, Any]) -> dict[str, Any]: idem_key = get_server_minted_idempotency_key() if idem_key: - enriched["idempotency_key"] = idem_key + # 2026-08-06 (DEF-SDKWRAP-CHAIN-SOFT-EXECUTION-ID-REUSE-01, + # Session 6 TC-SDKWRAP-05/07/16): the captured /check + # operation_id is reused across every llm_call event + # within the same chain-context cache window + # (``_GATE_CACHE_TTL_SECONDS=5s``). The backend's v3 + # /track idempotency layer + # (``backend/src/proxy/handlers.rs::finalize_track_idempotency``) + # hashes the request body against the stored body for + # the same key — every event after the FIRST one in + # the cache window shares the same idempotency_key but + # has a DIFFERENT body (tokens, model, latency, etc.) + # → 409 ``IDEMPOTENCY_KEY_MISMATCH`` and the event is + # silently dropped. Per CLAUDE.md §22 (Trust model): + # "losing actual token counts means downstream billing + # sees tokens=0 instead of the real cost" — billing- + # integrity regression. + # + # Fix: derive a per-event idempotency_key by combining + # the captured /check operation_id (so retries of the + # same event still hit the same server-side cache slot + # and the backend returns 200 + ``idempotent_replay: + # true``) with a per-event discriminator (span_id is + # minted once per @protect invocation, see + # ``decorators.py::_next_span`` — unique per event, + # stable across retries of the same event). Format: + # ``:`` where ``span_short`` is the + # first 16 hex chars of span_id — collision-free for + # distinct span_ids (122 bits of entropy in the source + # UUID v4) and short enough to keep the key under 80 + # chars for backend storage. The discriminator only + # applies when a captured /check key is in scope — + # caller-supplied keys (above) and legacy batch-path + # keys (no /check involved) are unaffected. + span_id = enriched.get("span_id") + if span_id and ":" not in idem_key: + enriched["idempotency_key"] = ( + f"{idem_key}:{str(span_id)[:16]}" + ) + else: + enriched["idempotency_key"] = idem_key # 2026-07-12 (multi-agent span attachment — SDK counterpart at # nullrun-sdk-python release/0.13.5 commit efff530): @@ -3111,9 +3193,9 @@ def track_event( """ event = {"type": event_type, **kwargs} event.setdefault("tokens", 0) - # Phase 3: emit a stable fingerprint so the dedup LRU at - # the track sink can collapse repeat emissions of the - # same event (e.g. when the user calls track_event manually + # Emit a stable fingerprint so the dedup LRU at the + # track sink can collapse repeat emissions of the same + # event (e.g. when the user calls track_event manually # AND the httpx transport hook fires for the same LLM # call). Field is stripped before wire send (see # ``_strip_wire_only_fields``). @@ -3202,10 +3284,10 @@ def _post_auth_with_retry( # Module-level convenience functions. -# Phase 3 (2026-07-05): the legacy _runtime module slot is now a -# proxy over the registry (see __getattr__ below). Reads and -# writes route through :class:`nullrun._registry.RuntimeRegistry`, -# which is the single source of truth. External code that imports +# The legacy _runtime module slot is now a proxy over the +# registry (see __getattr__ below). Reads and writes route +# through :class:`nullrun._registry.RuntimeRegistry`, which is +# the single source of truth. External code that imports # nullrun.runtime._runtime keeps working unchanged. @@ -3217,12 +3299,11 @@ def __getattr__(name): raise AttributeError(f"module {__name__!r} has no attribute {name!r}") -# Phase 3 (2026-07-05): the module-level slot is a -# proxy over the registry. The PEP 562 above handles -# reads; writes go through the proxy class installed by -# . See the long-form comment in -# nullrun._singleton for why a plain does not work -# on module instances. +# The module-level slot is a proxy over the registry. The +# PEP 562 __getattr__ above handles reads; writes go through the +# proxy class installed by install_runtime_proxy. See the +# long-form comment in nullrun._singleton for why a plain +# assignment does not work on module instances. # 2026-07-04 (v0.12.0 wiring fix — ): @@ -3487,11 +3568,11 @@ def _build_v3_track_payload( def get_runtime() -> NullRunRuntime: """Get or create the global runtime instance. - Phase 3 (2026-07-05): prefer the registry. We keep the - legacy global _runtime slot as a backwards-compat cache so - external code that imports nullrun.runtime._runtime still - works, but the canonical source of truth is the registry - (see nullrun._registry.RuntimeRegistry). + Prefers the registry. The legacy global _runtime slot is + kept as a backwards-compat cache so external code that + imports nullrun.runtime._runtime still works, but the + canonical source of truth is the registry (see + nullrun._registry.RuntimeRegistry). """ cached = get_active_runtime() if cached is not None: @@ -3514,9 +3595,10 @@ def track(event: dict[str, Any]) -> dict[str, Any]: return get_runtime().track(event) -# Phase 3.4: explicit alias for `track ` -- same call signature, friendlier -# name for users who reach for `track_event` first. Both names share the -# same callable object, so `nullrun.track is nullrun.track_event` is True. +# Explicit alias for `track` -- same call signature, friendlier +# name for users who reach for `track_event` first. Both names +# share the same callable object, so `nullrun.track is +# nullrun.track_event` is True. track_event = track @@ -3561,12 +3643,12 @@ def track_tool( return get_runtime().track_tool(tool_name, duration_ms=duration_ms, **kwargs) -# Phase 3 (2026-07-05): install the registry-backed proxy on the -# module class so reads AND writes to ``runtime._runtime`` route -# through the registry. PEP 562 ``__getattr__`` alone covers the -# read path; writes need a real data descriptor on the module's -# metaclass — see ``nullrun._singleton._RuntimeProxyModule`` for -# the long-form rationale. +# Install the registry-backed proxy on the module class so +# reads AND writes to ``runtime._runtime`` route through the +# registry. PEP 562 ``__getattr__`` alone covers the read +# path; writes need a real data descriptor on the module's +# metaclass -- see ``nullrun._singleton._RuntimeProxyModule`` +# for the long-form rationale. from nullrun._singleton import install_runtime_proxy install_runtime_proxy(__name__) diff --git a/src/nullrun/toolbox/langgraph.py b/src/nullrun/toolbox/langgraph.py index 439b3f2..0bf2976 100644 --- a/src/nullrun/toolbox/langgraph.py +++ b/src/nullrun/toolbox/langgraph.py @@ -21,8 +21,8 @@ to from the LangGraph integration docs. The previous location `nullrun.instrumentation.langgraph.instrument` -is removed as of Phase 1 Commit 6. Users who imported it should -switch to `nullrun.toolbox.langgraph.wrapper`. +has been removed. Users who imported it should switch to +`nullrun.toolbox.langgraph.wrapper`. """ from __future__ import annotations diff --git a/src/nullrun/toolbox/mcp.py b/src/nullrun/toolbox/mcp.py index 5ee9800..27e96a7 100644 --- a/src/nullrun/toolbox/mcp.py +++ b/src/nullrun/toolbox/mcp.py @@ -2,7 +2,7 @@ Wraps a connected MCP server so every tool invocation forwards the cached canonical class + per-tool `annotations` to the gate -on `/check`. The v3.31 (Разрыв 3) gate honors +on `/check`. The v3.31 gate honors `mcp_destructive_policy` / `mcp_readonly_policy` against these annotations — without an adapter, no SDK on the planet calls ``set_mcp_tool_context()`` and the umbrella policies stay @@ -18,8 +18,7 @@ Scope (kept deliberately small): * Cache `tools/list` for ``MCP_ADAPTER_CACHE_SECONDS`` - (default 300s, matches the gate's ``heartbeat`` cadence in - CLAUDE.md §6). + (default 300s, matches the gate's ``heartbeat`` cadence). * On every ``call_tool(name, args)``, set ``call_mcp_class='mcp'`` + ``call_mcp_annotations=...`` via the public ``context`` helpers so the runtime's @@ -40,15 +39,14 @@ * Negotiating JSON-RPC frames. The user brings their own MCP client (e.g. ``mcp`` PyPI, or the official ``modelcontextprotocol/python-sdk``). - * Server-side discovery polling. That's NULLRUN's Phase C - cron worker (table migration 239 already exists, the - worker itself is a follow-up PR). + * Server-side discovery polling. That's NULLRUN's cron + worker responsibility (table migration 239 already exists, + the worker itself is a follow-up PR). * Tools / Resources / Prompts distinction — only ``tools`` is forwarded. Resources (``mcp://server/resource/...``) and Prompts (``mcp://server/prompt/...``) are MCP - primitives we don't model on the wire yet - (CLAUDE.md §8 / v3.31 still classifies them by string - shape). + primitives we don't model on the wire yet; v3.31 still + classifies them by string shape. """ from __future__ import annotations @@ -68,10 +66,10 @@ # Cache TTL for the ``tools/list`` discovery response. Matches -# the v3.31 gate's ``heartbeat`` cadence (CLAUDE.md §6) so the -# adapter and the gate see roughly the same version of the -# server's tool inventory over time. Operators who need a -# tighter or looser TTL can override it via the constructor. +# the v3.31 gate's ``heartbeat`` cadence so the adapter and the +# gate see roughly the same version of the server's tool inventory +# over time. Operators who need a tighter or looser TTL can +# override it via the constructor. DEFAULT_CACHE_SECONDS = 300 diff --git a/src/nullrun/tracing.py b/src/nullrun/tracing.py index 233c1c1..1394de1 100644 --- a/src/nullrun/tracing.py +++ b/src/nullrun/tracing.py @@ -1,19 +1,16 @@ """ Trace/span context management via Python contextvars. -This module is the core of the new trace/span system (Phase 2 of -the SDK cleanup plan). The previous `nullrun.context` module -exposed loose `_trace_id` and `_span_id` contextvars — fine for -attaching IDs to events, but it didn't model the parent/child -hierarchy that a trace timeline needs. +The previous `nullrun.context` module exposed loose `_trace_id` and +`_span_id` contextvars — fine for attaching IDs to events, but it +didn't model the parent/child hierarchy that a trace timeline needs. `SpanContext` is a structured value: a single contextvar holds the *current* span, and child spans are derived from it via `create_child_span(parent)`. This is the same pattern OpenTelemetry uses for its Python SDK (`opentelemetry.context.get_current`) and -gives `@protect` (Commit 4) and `track_*` (Commit 5) a uniform -way to attach `trace_id` / `span_id` / `parent_span_id` / `depth` -to every emitted event. +gives `@protect` and `track_*` a uniform way to attach `trace_id` / +`span_id` / `parent_span_id` / `depth` to every emitted event. Thread/async safety: `ContextVar` is thread-local by default but PEP 567 guarantees the right value is restored across `await` @@ -97,11 +94,11 @@ def create_child_span(parent: SpanContext) -> SpanContext: ValueError: if `parent` is ``None``. The function does NOT silently degrade to creating a root span — that would hide bugs in the caller where a parent was expected. - Sprint 2.6 (B5): pre-fix this raised - ``TypeError: unsupported operand for None + 1`` on - ``parent.depth + 1`` which crashed the entire - ``@protect`` / track_* pipeline. Raise a clear - ``ValueError`` instead so the caller can fix the bug. + Pre-fix this raised ``TypeError: unsupported operand + for None + 1`` on ``parent.depth + 1`` (B5) which + crashed the entire ``@protect`` / track_* pipeline. + Raise a clear ``ValueError`` instead so the caller + can fix the bug. """ if parent is None: raise ValueError( diff --git a/src/nullrun/transport.py b/src/nullrun/transport.py index 0d87833..b8fcb95 100644 --- a/src/nullrun/transport.py +++ b/src/nullrun/transport.py @@ -76,10 +76,11 @@ # # Bumping `NULLRUN_PROTOCOL_VERSION` here must be coordinated with # the backend's `proxy::http::gate::protocol` constant and the -# `/health` endpoint's `current_protocol_version`. /health also -# publishes `min_protocol_version` (the floor — older SDKs get -# `PROTOCOL_TOO_OLD`) and `max_protocol_version` (the ceiling — -# newer SDKs get `PROTOCOL_TOO_NEW`). +# `/api/v1/capabilities` endpoint's `protocol_version`. +# `/api/v1/capabilities` also publishes `min_protocol_version` +# (the floor — older SDKs get `PROTOCOL_TOO_OLD`) and +# `max_protocol_version` (the ceiling — newer SDKs get +# `PROTOCOL_TOO_NEW`). NULLRUN_PROTOCOL_VERSION: int = 3 HEADER_PROTOCOL: str = "X-NULLRUN-PROTOCOL" @@ -328,11 +329,11 @@ def _retry_with_backoff( ) raise err if result.status_code >= 500 and on_transport_error == "raise": - # Round 3 (Phase 0.4.0): 5xx is a classified - # GATEWAY_ERROR. Don't retry -- this is a server - # bug, not a network blip. Only raise when the - # caller has opted into the typed-error contract - # via on_transport_error="raise". + # 5xx is a classified GATEWAY_ERROR. Don't + # retry -- this is a server bug, not a network + # blip. Only raise when the caller has opted + # into the typed-error contract via + # on_transport_error="raise". from nullrun.breaker.exceptions import NullRunBackendError err = NullRunBackendError( @@ -357,11 +358,11 @@ def _retry_with_backoff( except Exception as exc: last_exc = exc - # Sprint 3 follow-up (B24): bump ``last_error`` so the - # operator can read the most recent failure type without - # grepping logs. The string is the exception class - # name plus the message — short, searchable, and - # doesn't leak request bodies. + # Bump ``last_error`` so the operator can read the + # most recent failure type without grepping logs. + # The string is the exception class name plus the + # message -- short, searchable, and doesn't leak + # request bodies. metrics.set_transport("last_error", f"{type(exc).__name__}: {exc}") # ``timeouts`` is a specific subcategory of retry # trigger — distinguished so an SRE can alert on @@ -412,7 +413,7 @@ def _retry_with_backoff( # ============================================================================= -# Fallback Modes (Phase 1 - SDK Resilience) +# Fallback Modes (SDK Resilience) # ============================================================================= @@ -526,9 +527,9 @@ def __init__( self.api_key = api_key self.secret_key = secret_key # HMAC signing key self.config = config or FlushConfig() - # Phase 8 #8.4: allow env-var override of batch size and - # flush interval. Useful for tuning high-throughput agents - # without subclassing. + # Allow env-var override of batch size and flush interval. + # Useful for tuning high-throughput agents without + # subclassing. if "NULLRUN_BATCH_SIZE" in os.environ: try: self.config.batch_size = int(os.environ["NULLRUN_BATCH_SIZE"]) @@ -941,8 +942,8 @@ def send_batch(): metrics.inc_transport("batches_failed") def _drain_batch(self) -> list[dict[str, Any]] | None: - """Round 2 (Phase 0.4.0): public, lock-acquiring snapshot of - the current buffer. Returns ``None`` when empty. + """Public, lock-acquiring snapshot of the current buffer. + Returns ``None`` when empty. Used by ``tests/test_buffer_invariants.py``. The full flush logic (CB, re-queue, metrics) lives in ``_do_flush_locked`` @@ -1060,9 +1061,9 @@ def _build_signed_headers( ) -> dict[str, str]: """Build the canonical signed-headers dict for a request. - Round 2 (Phase 0.4.0): the canonical one-call helper used - by every signed POST. Mirrors the contract the test - framework in ``tests/test_hmac_signing.py`` expects. + The canonical one-call helper used by every signed POST. + Mirrors the contract the test framework in + ``tests/test_hmac_signing.py`` expects. Always includes: - Content-Type: application/json @@ -1319,7 +1320,7 @@ def flush_now(self) -> None: self._do_flush() # ============================================================================= - # Execute (Strict Mode) - Phase 1 + # Execute (Strict Mode) # ============================================================================= def execute( @@ -1333,26 +1334,20 @@ def execute( fallback_mode: str = FallbackMode.PERMISSIVE, operation_id: str | None = None, approval_id: str | None = None, - # Phase 1 / MVP 1.0: typed-impact + digest-bound approval. - # The runtime.execute() helper builds these kwargs and the - # transport includes them on the wire so the backend can - # stamp the approval row with the digest and verify it on - # the post-approval re-check. Pre-fix these kwargs were - # constructed in runtime.execute but never accepted by - # Transport.execute (which raised TypeError and was - # classified as a transport error by the on_transport_error - # arm below — the body was blocked even though no real - # policy violation happened). + # Typed-impact + digest-bound approval. The runtime.execute() + # helper builds these kwargs and the transport includes them + # on the wire so the backend can stamp the approval row with + # the digest and verify it on the post-approval re-check. + # These kwargs must be accepted by Transport.execute so the + # typed payload reaches the wire; otherwise the call would be + # classified as a transport error. business_impact: dict[str, Any] | None = None, action_digest: str | None = None, - # Разрыв 4 (T5.6, 2026-07-31): tool-call - # argument bag forwarded on /execute so the - # gate can compute a schema fingerprint and - # write it to mcp_tool_signatures. Optional - # — legacy SDKs (≤ 0.14.4) do not pass this; - # the gate's T5.6 fallback chain reads - # `tool_params` (Разрыв 2) when this is - # absent. + # Tool-call argument bag forwarded on /execute so the gate + # can compute a schema fingerprint and write it to + # mcp_tool_signatures. Optional -- legacy SDKs do not pass + # this; the gate's fallback chain reads `tool_params` when + # this is absent. tool_arguments: dict[str, Any] | None = None, on_transport_error: Callable[[Exception], dict[str, Any]] | None = None, ) -> dict[str, Any]: @@ -1379,12 +1374,12 @@ def execute( fallback_mode: What to do if Gateway unavailable operation_id: Optional idempotency key on_transport_error: Optional callback invoked on - ``BreakerTransportError`` (Phase 5 #5.10). When set, the - callback's return value is returned verbatim; otherwise - the request falls through to the ``fallback_mode`` + ``BreakerTransportError``. When set, the callback's + return value is returned verbatim; otherwise the + request falls through to the ``fallback_mode`` default. The decorator's ``_enforce_sensitive_tool`` - sets this to a closure that converts the error into a - ``NullRunBlockedException`` (fail-CLOSED). + sets this to a closure that converts the error into + a ``NullRunBlockedException`` (fail-CLOSED). Returns: Dict with: @@ -1414,22 +1409,20 @@ def execute( } if approval_id is not None: gate_request["approval_id"] = approval_id - # Phase 1 / MVP 1.0: typed-impact + digest-bound approval. - # Forward both fields on the wire when supplied. The backend - # stamps the approval row with the digest and verifies it on - # the post-approval re-check. The keys are only included - # when the runtime layer actually built them (i.e. when - # ``@sensitive(impact=...)`` was applied) so the wire stays - # quiet for legacy Phase 0 callers. + # Typed-impact + digest-bound approval. Forward both + # fields on the wire when supplied. The backend stamps the + # approval row with the digest and verifies it on the + # post-approval re-check. The keys are only included when + # the runtime layer actually built them (i.e. when + # ``@sensitive(impact=...)`` was applied) so the wire + # stays quiet for callers that don't use the typed payload. if business_impact is not None: gate_request["business_impact"] = business_impact if action_digest is not None: gate_request["action_digest"] = action_digest - # Разрыв 4 (T5.6, 2026-07-31): same forwarding - # on the /execute path. The tool_arguments - # field is the same wire-shape as on /check. - # Re-using the call site identity so the - # field name is the canonical one across all + # Tool-call argument bag forwarded on /execute. The + # tool_arguments field uses the same wire shape as on + # /check so the field name stays canonical across all # gate endpoints. if tool_arguments is not None: gate_request["tool_arguments"] = tool_arguments @@ -1482,9 +1475,9 @@ def do_execute_request() -> httpx.Response: } except BreakerTransportError as exc: - # Phase 5 #5.10: ADR-008 lets callers opt into a - # classified-error handler. Round 3 (Phase 0.4.0): - # on_transport_error accepts both callables AND strings: + # ADR-008 lets callers opt into a classified-error + # handler. on_transport_error accepts both callables + # AND strings: # "raise" -> raise NullRunTransportError (classified) # "open" -> return synthetic allow with FALLBACK_* source # "closed" -> return synthetic block with FALLBACK_* source @@ -1521,7 +1514,7 @@ def do_execute_request() -> httpx.Response: except NullRunTransportError: raise # Already classified -- propagate as-is except httpx.RequestError as exc: - # Round 3: classify httpx network errors at the call site. + # Classify httpx network errors at the call site. # isinstance guard narrows the type so the second string # comparison below no longer overlaps with Callable | None. if callable(on_transport_error): @@ -1536,11 +1529,10 @@ def do_execute_request() -> httpx.Response: except NullRunAuthenticationError: raise # Don't fall back on auth errors - # All attempts failed - apply fallback mode - # Sprint 3 follow-up (B24): bump ``fallback_mode_activations`` - # every time we reach this branch (gateway unreachable). - # The operator alerts on a spike here as a proxy for - # backend unavailability. + # All attempts failed - apply fallback mode. + # Bump ``fallback_mode_activations`` every time we reach + # this branch (gateway unreachable). The operator alerts + # on a spike here as a proxy for backend unavailability. metrics.inc_transport("fallback_mode_activations") if fallback_mode == FallbackMode.STRICT: return { @@ -1561,6 +1553,7 @@ def check( self, check_request: dict[str, Any], on_transport_error: Callable[[Exception], dict[str, Any]] | str | None = None, + parent_execution_id: str | None = None, ) -> dict[str, Any]: """ Call /api/v1/gate endpoint for pre-execution budget checking. @@ -1600,20 +1593,19 @@ def check( "model": check_request.get("model"), "estimated_tokens": check_request.get("estimated_tokens"), "operation_id": check_request.get("operation_id") or str(uuid.uuid4()), - # T4 (2026-06-27): forward the per-call `tools` list so the - # backend's `gate/internal.rs::check_tool_block` can match - # each tool against the workflow's effective `blocked_tools` - # aggregate. Pre-T4 this key was silently dropped here, so - # `set_call_context(tools=[...])` had no effect on /gate. - # When unset (None) we omit the key entirely — the backend - # distinguishes "no tools sent" from "explicit []". + # Forward the per-call `tools` list so the backend's + # `gate/internal.rs::check_tool_block` can match each + # tool against the workflow's effective `blocked_tools` + # aggregate. When unset (None) we omit the key entirely + # -- the backend distinguishes "no tools sent" from + # "explicit []". **({"tools": check_request["tools"]} if "tools" in check_request else {}), } - # 2026-07-02 (v0.11.0): wire-protocol v3 fields ( - # ). Forwarded only when present so legacy /gate callers - # (which never set chain_id) keep their previous payload - # shape. The backend treats missing as "single-shot Hard". + # Wire-protocol v3 fields. Forwarded only when present so + # legacy /gate callers (which never set chain_id) keep + # their previous payload shape. The backend treats missing + # as "single-shot Hard". if check_request.get("chain_id") is not None: gate_request["chain_id"] = check_request["chain_id"] if check_request.get("chain_op") is not None: @@ -1622,20 +1614,39 @@ def check( gate_request["idempotency_key"] = check_request["idempotency_key"] if "stream" in check_request: gate_request["stream"] = bool(check_request["stream"]) - # Разрыв 4 (T5.6, 2026-07-31): forward the - # `tool_arguments` bag alongside `tool` so the - # gate can hash it via `signature::compute_schema_hash` - # and write the fingerprint into - # `mcp_tool_signatures` (T5.6). Pre-T5.6 SDKs - # never set this; the backend's gate falls - # back to `tool_params` (Разрыв 2) when the - # field is missing, so legacy callers do not - # regress. The shape is `Optional[dict[str, - # Any]]` — the backend canonicalises the JSON - # before hashing, so field ordering inside - # the dict does not affect the fingerprint. + # Forward the `tool_arguments` bag alongside `tool` so + # the gate can hash it via `signature::compute_schema_hash` + # and write the fingerprint into `mcp_tool_signatures`. + # Legacy SDKs never set this; the backend's gate falls + # back to `tool_params` when the field is missing, so + # legacy callers do not regress. The shape is + # `Optional[dict[str, Any]]` -- the backend + # canonicalises the JSON before hashing, so field + # ordering inside the dict does not affect the + # fingerprint. if "tool_arguments" in check_request and check_request["tool_arguments"] is not None: gate_request["tool_arguments"] = check_request["tool_arguments"] + # Execution Graph v0 (2026-08-06, backend): additive + # `parent_execution_id` wire field on /gate. A sub-agent SDK + # call to a child execution names the parent execution here; + # the backend validates ownership against the parent's + # `execution:{id}` Redis binding (mirrors the /cancel + # ownership check at `backend/src/proxy/http/cancel.rs:258-329`) + # and rejects cross-org / cross-key / not-found with 403 + # PARENT_EXECUTION_*. Forwarded only when the caller passes + # a non-None string -- unset (legacy / single-shot) callers + # keep the previous payload shape. Resolution order: + # 1. `check_request["parent_execution_id"]` (preferred -- + # lets the runtime layer stamp it from a captured + # server-minted execution_id via + # `nullrun.capture_current_execution_id()`). + # 2. `parent_execution_id` kwarg (caller-supplied; useful + # for fan-out where the parent is not the current + # execution). + # 3. None / omitted entirely (legacy / single-shot). + _parent_execution_id = check_request.get("parent_execution_id", parent_execution_id) + if _parent_execution_id is not None: + gate_request["parent_execution_id"] = _parent_execution_id # 2026-07-02 (v0.11.0 refactor): route through the canonical # signed-headers helper — produces Content-Type + X-API-Key + @@ -1678,9 +1689,9 @@ def check( "suggestions": ["Check API availability"], } except httpx.RequestError as e: - # Round 3: classify network errors. By default fall - # through to synthetic block (legacy); raise only when - # the caller opted in via on_transport_error="raise". + # Classify network errors. By default fall through + # to synthetic block (legacy); raise only when the + # caller opted in via on_transport_error="raise". if on_transport_error == "raise": raise NullRunTransportError( f"Network error on /check: {e}", @@ -1699,7 +1710,7 @@ def check( } # ============================================================================= - # WebSocket Connection (Task 6 - WebSocket Push) + # WebSocket Connection # ============================================================================= async def connect_websocket( @@ -1733,8 +1744,8 @@ async def connect_websocket( Raises: ConnectionError: If WebSocket connection fails """ - # Phase 6 #6.6: build the WS URL via urllib.parse instead of - # string replace. Reject unknown schemes with a clear error. + # Build the WS URL via urllib.parse instead of string + # replace. Reject unknown schemes with a clear error. from urllib.parse import urlparse, urlunparse from nullrun.transport_websocket import WebSocketConnection @@ -1811,14 +1822,14 @@ async def _refetch_credentials(self) -> None: our HMAC secret_key has been rotated. We need to get the new secret_key from the /auth/verify endpoint. - Sprint 2.4 (B20): the previous implementation used - ``import requests`` and bypassed every transport-layer - invariant — the shared ``httpx.Client`` (mTLS, connection - pool), the circuit breaker, the HMAC body signature, and - the retry policy. It also pulled in ``requests`` as a new - dependency that is not in ``pyproject.toml`` (a runtime - ImportError waiting to happen on any environment where - ``requests`` is not installed transitively). + The previous implementation used ``import requests`` and + bypassed every transport-layer invariant -- the shared + ``httpx.Client`` (mTLS, connection pool), the circuit + breaker, the HMAC body signature, and the retry policy. + It also pulled in ``requests`` as a new dependency that + is not in ``pyproject.toml`` (a runtime ImportError + waiting to happen on any environment where ``requests`` + is not installed transitively). Post-fix: route through ``self._client`` so the same TLS configuration, connection pool, and HMAC signing path @@ -2422,6 +2433,7 @@ def _parse_v3_error_envelope( # would create a cycle. The price is one extra import # non-2xx response — irrelevant for the failure path. from nullrun.breaker.exceptions import ( + NullRunAuthError, NullRunBackendError, NullRunBudgetError, NullRunChainError, @@ -2583,8 +2595,35 @@ def _parse_v3_error_envelope( return catalog(full_message) if catalog is NullRunProtocolError: return catalog(full_message) + # NullRunAuthError — surface the wire error_code (one of + # v3.38's API_KEY_REVOKED / API_KEY_EXPIRED / API_KEY_DISABLED + # / API_KEY_INVALID / API_KEY_MISSING / API_KEY_MALFORMED) on + # ``self.wire_code`` so callers can branch on granular + # lifecycle state without clobbering the SDK-side + # ``error_code`` taxonomy (NR-A003). Mirrors the + # ``NullRunChainError.backend_code`` pattern. + # + # Filter ``details`` to the kwargs the base NullRunError + # constructor accepts — the envelope's ``details`` dict can + # carry arbitrary keys (``expires_at``, ``ttl_seconds``, ...) + # and the base class rejects unknown kwargs with TypeError. + # Unknown fields are stored on ``self.details`` for caller + # introspection instead. + if catalog is NullRunAuthError: + allowed = {"error_code", "user_action", "retryable", "docs_url", "cause"} + forwarded = {k: v for k, v in details.items() if k in allowed} + extra = {k: v for k, v in details.items() if k not in allowed} + instance = NullRunAuthError( + full_message, + wire_code=backend_code, + **forwarded, + ) + if extra: + instance.details = extra # type: ignore[attr-defined] + return cast(Exception, instance) # Final fallback for catalog classes with a generic - # (message, **details) signature (NullRunAuthError). + # (message, **details) signature (NullRunWorkflowInactiveError + # and any future addition). # The details payload is forwarded as a positional kwarg # via **details (typed as Any to satisfy mypy since # type[BaseException] does not expose the kwargs the @@ -2595,7 +2634,9 @@ def _parse_v3_error_envelope( # _V3_ERROR_CODE_MAP is a real Exception subclass. Cast # to Exception so mypy stops flagging the return value # as BaseException (the helper declares -> Exception). - instance = catalog(full_message, **details) # type: ignore[call-arg] + allowed = {"error_code", "user_action", "retryable", "docs_url", "cause"} + forwarded = {k: v for k, v in details.items() if k in allowed} + instance = catalog(full_message, **forwarded) # type: ignore[call-arg] return cast(Exception, instance) # Fallback — use HTTP status. The catalog may not yet cover @@ -2643,6 +2684,7 @@ def _build_v3_error_code_map() -> dict[str, type[BaseException]]: from nullrun.breaker.exceptions import ( NullRunAuthError, NullRunBackendError, + NullRunBlockedException, NullRunBudgetError, NullRunChainError, NullRunConsumeOverbudgetError, @@ -2667,9 +2709,33 @@ def _build_v3_error_code_map() -> dict[str, type[BaseException]]: # 403 — chain security + workflow state "CHAIN_CROSS_ORG": NullRunChainError, "CHAIN_ORG_MISMATCH": NullRunChainError, + # 403 — Execution Graph v0 (2026-08-06, backend). Sub-agent + # ownership validation against the parent's + # `execution:{id}` Redis binding (mirrors the /cancel + # ownership check). Fail-CLOSED — the sub-agent call does + # NOT proceed. Same diagnostic class as CHAIN_CROSS_ORG / + # CHAIN_ORG_MISMATCH: 403-class security errors with + # `(org_id, api_key_id)` ownership semantics. Diagnostic + # clarity wins over a new exception class per CLAUDE.md §13 + # philosophy. + "PARENT_EXECUTION_NOT_FOUND": NullRunChainError, + "PARENT_EXECUTION_ORG_MISMATCH": NullRunChainError, + "PARENT_EXECUTION_KEY_MISMATCH": NullRunChainError, "WORKFLOW_INACTIVE": NullRunWorkflowInactiveError, - # 401/403 — auth + # 401/403 — auth (v3.38 distinct lifecycle states). + # The backend splits the v3.36 ``API_KEY_REVOKED`` bucket into + # five distinct wire codes so SDKs can branch on each state + # (e.g. surface "rotate this key" vs "this key was admin- + # disabled" vs "no Authorization header was sent"). All map + # to NullRunAuthError — diagnostic class is preserved; the + # granular codes live in ``details.error_code`` and are + # surfaced via NullRunAuthError.code for handler dispatch. "API_KEY_REVOKED": NullRunAuthError, + "API_KEY_EXPIRED": NullRunAuthError, + "API_KEY_DISABLED": NullRunAuthError, + "API_KEY_INVALID": NullRunAuthError, + "API_KEY_MISSING": NullRunAuthError, + "API_KEY_MALFORMED": NullRunAuthError, # 422 — consume invariant violation "CONSUME_OVERBUDGET": NullRunConsumeOverbudgetError, # 429 — rate limit @@ -2677,6 +2743,22 @@ def _build_v3_error_code_map() -> dict[str, type[BaseException]]: # 503 — backend availability "RATE_LIMIT_REDIS_UNAVAILABLE": NullRunRateLimitRedisError, "BUDGET_DATA_UNAVAILABLE": NullRunBackendError, + # 402 — approval-create failure family (DEF-ARFLOW-TOOLNAME-01, + # E2E 2026-08-05). Backend's + # ``classify_approval_create_error`` exposes these as + # ``details.error_code`` on the gate response so operators + # can tell a Postgres outage (retry-friendly) from a data + # integrity bug (rebuild-and-retry) from a config bug + # (operator fix). All map to ``NullRunBlockedException`` + # because they are hard-rejects -- the body did NOT run, + # the approval row could NOT be created, and the + # fail-CLOSED posture is preserved. + "APPROVAL_DB_UNAVAILABLE": NullRunBlockedException, + "APPROVAL_PERSISTENCE_FAILED": NullRunBlockedException, + "APPROVAL_VALIDATION_FAILED": NullRunBlockedException, + "APPROVAL_CONFLICT": NullRunBlockedException, + "APPROVAL_NOT_FOUND": NullRunBlockedException, + "APPROVAL_CREATE_FAILED": NullRunBlockedException, } diff --git a/src/nullrun/transport_websocket.py b/src/nullrun/transport_websocket.py index 7699790..e5a479f 100644 --- a/src/nullrun/transport_websocket.py +++ b/src/nullrun/transport_websocket.py @@ -225,16 +225,17 @@ def __init__( # survive the at-least-once delivery semantics of the WS # channel. # - # Sprint 1.4 (B2): the previous sentinel of 0 dropped incoming + # The previous sentinel of 0 dropped incoming # ``version == 0`` on first receive because ``0 <= 0`` is # True. The server uses ``version: 0`` for the very first - # ``initial_state`` frame after a (re)connect, so the SDK was - # silently discarding the server's initial view — meaning a - # ``Killed``/``Paused`` state delivered in that first frame - # was lost. Sentinel is now -1 so any non-negative version - # passes the guard on the first message; subsequent stale - # ``version == 0`` re-deliveries are still dropped because - # ``last_seen`` will be ``>= 1`` for that workflow. + # ``initial_state`` frame after a (re)connect, so the SDK + # was silently discarding the server's initial view -- + # meaning a ``Killed``/``Paused`` state delivered in that + # first frame was lost. Sentinel is now -1 so any + # non-negative version passes the guard on the first + # message; subsequent stale ``version == 0`` re-deliveries + # are still dropped because ``last_seen`` will be ``>= 1`` + # for that workflow. self._last_version: dict[str, int] = {} async def _reconnect_loop(self) -> None: @@ -454,18 +455,18 @@ async def _handle_message(self, message: str) -> None: signature, max_age_seconds=300, ): - # Sprint 1.5 (B13): pre-fix this logged at - # WARNING and dropped the message silently. For a - # safety layer whose core contract is "the - # server can always KILL a workflow", a failed - # signature verification on a control plane - # message is a first-class incident — promote to - # ERROR and bump the counter so an SRE can - # alert on ``hmac_verify_failures_total > 0``. - # A signed-but-invalid message means either + # Pre-fix this logged at WARNING and dropped + # the message silently. For a safety layer + # whose core contract is "the server can always + # KILL a workflow", a failed signature + # verification on a control plane message is a + # first-class incident -- promote to ERROR and + # bump the counter so an SRE can alert on + # ``hmac_verify_failures_total > 0``. A + # signed-but-invalid message means either # (a) the secret_key is out of sync (server - # rotated, client missed the rotation event), or - # (b) something is forging traffic. Both are + # rotated, client missed the rotation event), + # or (b) something is forging traffic. Both are # actionable and the operator needs to know. logger.error( f"Invalid HMAC signature for {msg_type} message - " @@ -567,13 +568,13 @@ async def _handle_message(self, message: str) -> None: logger.warning(f"Key rotation callback error: {e}") elif msg_type == "approval_resolved": - # Drift section 7 (2026-07-06): human-approval - # resolution notification. The dashboard operator - # approved or denied a pending approval; the SDK - # uses this to release the gate reservation - # (approved) or surface WorkflowKilledInterrupt - # (denied) so the agent can resume from the same - # execution_id without polling /status. + # Human-approval resolution notification. The + # dashboard operator approved or denied a pending + # approval; the SDK uses this to release the gate + # reservation (approved) or surface + # WorkflowKilledInterrupt (denied) so the agent + # can resume from the same execution_id without + # polling /status. # # Wire shape (backend WsMessage::ApprovalResolved): # { @@ -834,8 +835,8 @@ def _dispatch_state(self, state: dict[str, Any]) -> None: workflow_id = state.get("workflow_id", "") incoming_version = state.get("version", 0) if workflow_id: - # Sprint 1.4 (B2): default -1 (not 0) so version=0 is - # accepted on first receive. See __init__ for rationale. + # Default -1 (not 0) so version=0 is accepted on first + # receive. See __init__ for rationale. last = self._last_version.get(workflow_id, -1) if incoming_version <= last: logger.debug( diff --git a/tests/conftest.py b/tests/conftest.py index 0caf9f4..7a40dfd 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -24,7 +24,7 @@ def reset_runtime(): # Disable polling for all tests via the runtime's internal `polling` flag # (see make_runtime below — passes polling=False by default). The legacy - # NULLRUN_DISABLE_POLLING env var is gone as of Commit 5. + # NULLRUN_DISABLE_POLLING env var is no longer consulted. # Reset before test only - don't call shutdown in teardown # because mock_api fixture already cleaned up its respx context @@ -134,8 +134,26 @@ def mock_api(): ) # 0.7.0: SDK no longer fetches /policies on init (backend # owns all policy state; SDK is a thin client). - # Health endpoint - respx.get(f"{BASE_URL}/health").mock(return_value=Response(200, json={"status": "ok"})) + # Capabilities endpoint (canonical /api/v1/capabilities, + # mirrors backend/src/proxy/http/protocol.rs:189). + # Empty capabilities object — SDK treats this as a non-v3 + # backend and continues in compatibility mode. + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( + return_value=Response( + 200, + json={ + "min_protocol_version": 1, + "max_protocol_version": 1, + "protocol_version": 1, + "capabilities": { + "server_minted_execution_id": False, + "per_execution_reservations": False, + "enforcement_modes_soft": False, + "heartbeat_time_based": False, + }, + }, + ) + ) yield @@ -217,7 +235,7 @@ def _factory(**overrides): @pytest.fixture(autouse=True) def _fast_sleep(monkeypatch, request): - # Sprint 0 (coverage): neutralise time.sleep in test code so the suite + # (coverage): neutralise time.sleep in test code so the suite # is no longer gated on the retry loop's real wall-clock wait. The # three TestCircuitBreaker tests in tests/test_transport.py # (test_open_transitions_to_half_open_after_timeout and its two @@ -280,7 +298,7 @@ def _fast_sleep(seconds): @pytest.fixture(autouse=True) def _isolated_wal(monkeypatch, tmp_path): - # CI flakefix (Sprint 0 follow-up): every test gets a private + # CI flakefix: every test gets a private # ``NULLRUN_WAL_PATH`` so ``Transport._replay_from_wal`` cannot # replay events from a previous run / parallel xdist worker / # failed teardown against the real backend. diff --git a/tests/test_actions.py b/tests/test_actions.py index 949e986..c69a973 100644 --- a/tests/test_actions.py +++ b/tests/test_actions.py @@ -261,7 +261,7 @@ def test_block_does_not_propagate_exception(self): # =========================================================================== -# Sprint 1.5 (B14): unknown action type must NOT silently BLOCK +# B14: unknown action type must NOT silently BLOCK # =========================================================================== # Pre-fix: an unknown action type (e.g. server schema regression # version mismatch, or attacker-controlled input) silently degraded diff --git a/tests/test_actions_context_init.py b/tests/test_actions_context_init.py index f14cec7..146096b 100644 --- a/tests/test_actions_context_init.py +++ b/tests/test_actions_context_init.py @@ -76,7 +76,7 @@ def test_clear_history_empties_list(): def test_handle_unknown_action_does_not_invoke_handler(): - """Sprint 1.5 (B14): unknown action logs ERROR + records BLOCK but + """B14: unknown action logs ERROR + records BLOCK but does NOT invoke any handler (fail-open). Pre-fix this degraded to BLOCK → DoS amplifier. """ diff --git a/tests/test_approval_money_flow.py b/tests/test_approval_money_flow.py index f497b15..0bc69fc 100644 --- a/tests/test_approval_money_flow.py +++ b/tests/test_approval_money_flow.py @@ -1,15 +1,15 @@ -"""Phase 1 / MVP 1.0 — 5 DoD scenarios for the Money approval flow. +"""Typed impact + digest-bound approval — 5 DoD scenarios for the Money approval flow. The exact 5 scenarios Anatolii requested (2026-07-23): 1. Refund $40 -> Allow (no approval needed) 2. Refund $1200 -> Require Approval -> Approve -> Execute (success) 3. Refund $1200 -> Approve -> Modify amount to $1300 -> Block on - digest mismatch (the headline security invariant of Phase 1) + digest mismatch (the headline security invariant of typed impact) 4. Approve -> Execute -> Second Execute -> Block on replay - (Phase 0 grant-consume invariant, still must hold) + (grant-consume invariant, still must hold) 5. Approve -> Wait expiry -> Execute -> Block on expiry - (Phase 0 expiry invariant, still must hold) + (expiry invariant, still must hold) These are SDK-level tests, not end-to-end HTTP tests. We: @@ -56,8 +56,8 @@ class that returns exactly what `gate_internal` would return # A minimal in-process simulator that mirrors gate_internal's # decisions without spinning up the backend. Verified against the # backend's grant-consume path in `db.rs::consume_approved` -# (Phase 0 contract: status, execution_id binding, expiry, -# consumed_at IS NULL) plus the Phase 1 digest compare. The +# (grant-consume contract: status, execution_id binding, expiry, +# consumed_at IS NULL) plus the digest compare. The # simulator exposes the failure modes so the tests can pin which # one fired — different error codes belong to different DoD # scenarios. @@ -86,7 +86,7 @@ def decide(self, business_impact: BusinessImpact | None) -> str: raises. The tests assert on the return value's prefix to map to each of the 5 DoD scenarios. """ - # Phase 0 path: missing-replay / wrong-execution / wrong-status. + # legacy path: missing-replay / wrong-execution / wrong-status. if self.status != "APPROVED": self.last_decision = "block:status-not-approved" return self.last_decision @@ -96,12 +96,12 @@ def decide(self, business_impact: BusinessImpact | None) -> str: if time.monotonic() > self.expires_at: self.last_decision = "block:expired" return self.last_decision - # Phase 1 digest check (live: `gate_internal::digest re-check`). + # digest check (live: `gate_internal::digest re-check`). if business_impact is not None: live_digest = compute_action_digest(business_impact) stored = self.stored_digest if stored is None: - # Legacy Phase 0 row: digest-empty approvals cannot + # Legacy digest-empty approvals cannot # be re-checked against an impact. Backend falls back # to approval_id-only grant, simulator mirrors. self.last_decision = "allow" @@ -109,7 +109,7 @@ def decide(self, business_impact: BusinessImpact | None) -> str: if stored != live_digest: self.last_decision = "block:digest-mismatch" return self.last_decision - # Phase 0 consume path: stamp consumed_at (we mark in-memory + # grant-consume path: stamp consumed_at (we mark in-memory # once per decision, so a second call triggers replay). self.consumed = True self.last_decision = "allow" @@ -121,7 +121,7 @@ def decide(self, business_impact: BusinessImpact | None) -> str: @pytest.fixture(autouse=True) def reset_observability() -> None: - """Phase 0 SDK policy: tests must not leak metrics across runs.""" + """SDK policy: tests must not leak metrics across runs.""" from nullrun.observability import metrics metrics.reset() @@ -163,7 +163,7 @@ def _make(argument: str, currency: str = "USD") -> MoneyImpactExtractor: class TestBusinessImpactRoundTrip: """1. Test the digest primitive itself before wiring it up. - Phase 1 / MVP 1.0 security invariant: any drift between + Typed impact + digest-bound approval security invariant: any drift between SDK-computed and backend-computed digests is a P0 bug. We pin the digest by encoding a known fixture and asserting the exact 64-char hex. @@ -274,7 +274,7 @@ def test_extractor_rejects_unknown_argument(self, extractor_factory): ex.impact_for(_refund_call, (1_000,), {}) def test_extractor_rejects_wrong_type(self, extractor_factory): - # Phase 1.1 (Decimal support): a string is not a Decimal + # Decimal support: a string is not a Decimal # and not an int, so the discriminator rejects it. The # exact error message names the unit discriminator so the # operator can fix the call site. @@ -285,7 +285,7 @@ def test_extractor_rejects_wrong_type(self, extractor_factory): ) def test_extractor_rejects_bool_amount(self, extractor_factory): - # Phase 1.1 (Decimal support): ``bool`` is a subclass + # Decimal support: ``bool`` is a subclass # of ``int`` in Python; the discriminator explicitly # rejects ``bool`` so a hostile caller can't smuggle # ``True`` as ``amount=1`` cent. The unit-discriminator @@ -306,7 +306,7 @@ def test_1_refund_40_dollars_is_allowed( ): # Scenario 1: Refund $40 -> Allow (no approval needed). # - # The MVP-1.0 rule fires on `outflow > 50 USD cents = $50`. + # The 50 USD cents threshold rule fires on `outflow > 50 USD cents = $50`. # Refund $40 is below threshold → no approval → /gate # returns 'allow' without invoking the approval cycle. ex = extractor_factory("amount_cents") @@ -314,7 +314,7 @@ def test_1_refund_40_dollars_is_allowed( sim = ApprovalSimulator( stored_digest=None, # /gate path: never even reaches grant ) - # /gate path: refund of 4000 cents ($40) is below the MVP + # /gate path: refund of 4000 cents ($40) is below the # threshold; the simulator's grant-consume path is not # invoked. We assert the SDK's decision is "no approval # needed" by checking the impact is below the rule @@ -367,7 +367,7 @@ def test_3_refund_1200_then_modify_to_1300_blocks_on_digest( def test_4_replay_after_approved_execute_blocks(self, extractor_factory): # Scenario 4: Approved -> Execute -> Second Execute -> - # Block on replay. Phase 0 grant-consume contract. + # Block on replay. grant-consume contract. ex = extractor_factory("amount_cents") impact = ex.impact_for(_refund_call, (50_000,), {}) sim = ApprovalSimulator( diff --git a/tests/test_approval_timeout_field.py b/tests/test_approval_timeout_field.py index 4fe25bf..914bfbd 100644 --- a/tests/test_approval_timeout_field.py +++ b/tests/test_approval_timeout_field.py @@ -1,24 +1,24 @@ """ -Разрыв 1c (2026-07-21) — SDK reads `approval_timeout_seconds` +Server-timeout-vs-env-default (2026-07-21) — SDK reads `approval_timeout_seconds` from the /gate response, not its own env default. -До этой правки SDK использовал `NULLRUN_APPROVAL_TIMEOUT_SECONDS` -env default (300s) как единственный источник wait duration. Если -backend row имел другой `expires_in_seconds` (например, 20s для -коротких approval-правил или 1800s для длинных), SDK timeout'ил -раньше или позже чем backend sweeper — exactly the Разрыв 3 desync +Before this fix the SDK used `NULLRUN_APPROVAL_TIMEOUT_SECONDS` +env default (300s) as the only source of wait duration. If +a backend row had a different `expires_in_seconds` (e.g. 20s for +short approval rules or 1800s for long ones), the SDK timed out +earlier or later than the backend sweeper — the same desync class of bug. -Backend commit 0ad03b9 добавил `approval_timeout_seconds: Option` -поле в `GateResponse`. SDK теперь: +Backend commit 0ad03b9 added `approval_timeout_seconds: Option` +field in `GateResponse`. The SDK now: - prefers `response["approval_timeout_seconds"]` (server-authoritative) -- falls back to env default только когда поле отсутствует/невалидно +- falls back to env default only when the field is missing/invalid -Тесты ниже пинят этот контракт: при валидном server timeout -используется он, при отсутствующем — env default, при -невалидном — env default + WARN log. +The tests below pin this contract: a valid server timeout +is used as-is, a missing one falls back to env default, and an +invalid one falls back to env default + WARN log. -# Test mechanics (Разрыв 1c, 2026-07-21) +# Test mechanics `_wait_for_approval_resolution` creates a NEW `threading.Event()` inside the function and waits on it. Pre-setting an event from @@ -132,7 +132,7 @@ def target() -> None: class TestApprovalTimeoutResolution: - """Pin the Разрыв 1c contract: server timeout wins, env is fallback.""" + """Pin the server-timeout-vs-env-default contract: server timeout wins, env is fallback.""" def test_server_timeout_used_when_response_has_valid_value(self): """DoD #1: server-supplied timeout=15s is the value passed @@ -155,7 +155,7 @@ def test_server_timeout_used_when_response_has_valid_value(self): "wait should have released on the WS push, not timed out" ) assert result_box["result"]["timeout_seconds"] == 15.0, ( - "Разрыв 1c: server timeout (15s) must be stored on the " + "Server timeout (15s) must be stored on the " f"entry; got {result_box['result']['timeout_seconds']}" ) # Sanity: the wait did NOT consume 15s. @@ -193,7 +193,7 @@ def test_env_fallback_when_server_value_is_zero(self): # on the very first event.wait(), so we explicitly reject # non-positive values. # - # Sprint 0 (coverage): this test was rare-flaky under + # (coverage): this test was rare-flaky under # pytest-xdist on CI (linux, Python 3.12) — the spawned # wait thread occasionally missed the 50ms release window # when the main thread was mid-test-collection, and the @@ -208,7 +208,7 @@ def test_env_fallback_when_server_value_is_zero(self): # thread missed the 200ms release window twice in a # row on the shared Linux runner. # 2. ``release_after_ms=400`` widens the release window - # from 200ms (Sprint 0) to 400ms — still well below + # from 200ms to 400ms — still well below # the 120s env default timeout so the test runs fast # on CI, but enough headroom that the spawned thread # reliably reaches ``event.wait()`` before the release @@ -262,7 +262,7 @@ def test_timeout_sentinel_returned_when_no_ws_push(self): a fresh dict, not the entry) — only `outcome`, `timed_out`, `approval_id`. - Phase 0 review (2026-07-23): the test used + Initial review (2026-07-23): the test used `timeout_seconds=0.1` to keep the suite fast. After the clamp to `[MIN_APPROVAL_TIMEOUT_SECONDS=1, MAX_APPROVAL_TIMEOUT_SECONDS=3600]`, sub-1s values now @@ -308,7 +308,7 @@ def test_diverging_server_value_logs_at_debug(self, caplog): if r.levelname == "DEBUG" and "using server timeout" in r.message ] assert len(debug_messages) >= 1, ( - "Разрыв 1c: diverging server timeout should emit a DEBUG log. " + "Diverging server timeout should emit a DEBUG log. " f"Got caplog records: {[r.message for r in caplog.records]}" ) finally: @@ -316,7 +316,7 @@ def test_diverging_server_value_logs_at_debug(self, caplog): # --------------------------------------------------------------------------- -# Phase 0 review (2026-07-23): server-timeout clamp to +# Initial review (2026-07-23): server-timeout clamp to # [MIN_APPROVAL_TIMEOUT_SECONDS, MAX_APPROVAL_TIMEOUT_SECONDS]. # Pre-fix only `> 0` was rejected, so a server advertising # 1e9 seconds would lock the calling thread for years. The diff --git a/tests/test_blocked_exception.py b/tests/test_blocked_exception.py index 8a7f9ff..0f0b832 100644 --- a/tests/test_blocked_exception.py +++ b/tests/test_blocked_exception.py @@ -10,7 +10,7 @@ Backwards compat: `tool_name` is optional and defaults to `None`, so all existing raise sites that do not pass it still work. -Sprint 2.2: the previously-tested subclasses ``LoopDetectedException`` +Removed (previously-tested) subclasses ``LoopDetectedException`` ``RetryStormException``, and ``RateLimitExceededException`` were removed because they had no in-tree callers. The base-class attribute surface tests below still pin the contract for any future diff --git a/tests/test_blocker_fixes.py b/tests/test_blocker_fixes.py index 16f4ee2..e6e1fac 100644 --- a/tests/test_blocker_fixes.py +++ b/tests/test_blocker_fixes.py @@ -1,7 +1,6 @@ """ Regression tests for BLOCKER fixes in 0.4.0. -Phase 2 of the production-readiness plan: - #1 First-`track ` AttributeError on `_workflow_costs` (removed in 0.3.1). - #3 `_safe_bump_coverage` missing — `auto_requests.py` was unimportable. - #4 `auto_instrument ` did not call `patch_requests`. diff --git a/tests/test_business_impact.py b/tests/test_business_impact.py index a3728f9..e1205dd 100644 --- a/tests/test_business_impact.py +++ b/tests/test_business_impact.py @@ -57,8 +57,8 @@ "dfc96387ca539b7130caebe705e042f2e34e52ab44352ae5e527bcef64f0df27" ) -# Cross-language parity pin for the Tier 2 / Разрыв 2 -# ``ToolCall`` impact (Разрыв 2 / 2026-07-27). The Rust backend +# Cross-language parity pin for the +# ``ToolCall`` impact (2026-07-27). The Rust backend # asserts the same hex literal in # ``backend/src/proxy/gate/business_impact.rs::tests:: # tool_call_digest_golden_value_stripe_charge_500``. Any drift @@ -262,7 +262,7 @@ class TestExtractorFailureModes: def test_negative_amount_raises_value_error(self) -> None: ext = money_outflow(argument="amount_cents") - # Phase 1.1: hardening pass added ``InvalidMoneyAmountError`` + # Decimal support hardening pass added ``InvalidMoneyAmountError`` # which subclasses ``ValueError``; the legacy matcher # still works for ``except ValueError`` callers. with pytest.raises(ValueError, match="rejected negative"): @@ -285,7 +285,7 @@ def test_bool_amount_rejected_even_though_bool_is_int_in_python(self) -> None: # --------------------------------------------------------------------------- -# 1b. Tier 2 / Разрыв 2 — ToolCall impact cross-language parity +# 1b. ToolCall impact cross-language parity # --------------------------------------------------------------------------- # # Pins the SDK digest to the SAME hex literal the Rust backend diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index a78b7e1..221d4de 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -4,7 +4,7 @@ - parse_capabilities: tolerant parsing with default-false fallbacks - validate_sdk_version: returns warnings for version mismatch - is_v3_ready: True only when ALL three v3 capabilities are set -- probe_capabilities: /health fetch with respx (network failure paths) +- probe_capabilities: /api/v1/capabilities fetch with respx (network failure paths) """ from __future__ import annotations @@ -160,7 +160,7 @@ def test_sdk_min_version_constant(): # --------------------------------------------------------------------------- -# probe_capabilities — /health fetch (network failure paths) +# probe_capabilities — /api/v1/capabilities fetch (network failure paths) # --------------------------------------------------------------------------- # These cover the ``logger.debug`` branches in probe_capabilities that the # pure-data tests above cannot reach: non-2xx responses and transport @@ -169,17 +169,19 @@ def test_sdk_min_version_constant(): def test_probe_capabilities_returns_caps_on_2xx(): - """A successful /health response parses into a ServerCapabilities.""" + """A successful /api/v1/capabilities response parses into a ServerCapabilities.""" payload = { "min_protocol_version": 3, "max_protocol_version": 3, - "server_minted_execution_id": True, - "per_execution_reservations": True, - "enforcement_modes_soft": False, - "heartbeat_time_based": True, + "capabilities": { + "server_minted_execution_id": True, + "per_execution_reservations": True, + "enforcement_modes_soft": False, + "heartbeat_time_based": True, + }, } with respx.mock: - respx.get(f"{BASE_URL}/health").mock( + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( return_value=httpx.Response(200, json=payload) ) caps = probe_capabilities(BASE_URL) @@ -189,14 +191,14 @@ def test_probe_capabilities_returns_caps_on_2xx(): def test_probe_capabilities_returns_none_on_non_2xx(): - """A non-2xx /health response returns None (advisory, not fatal). + """A non-2xx /api/v1/capabilities response returns None (advisory, not fatal). Pins the ``logger.debug("... returned %d",...)` branch in probe_capabilities so a future refactor can't silently swallow the response code without a test catching it. """ with respx.mock: - respx.get(f"{BASE_URL}/health").mock( + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( return_value=httpx.Response(503, text="service unavailable") ) caps = probe_capabilities(BASE_URL) @@ -211,7 +213,7 @@ def test_probe_capabilities_returns_none_on_network_error(): branch (transport-level exception path). """ with respx.mock: - respx.get(f"{BASE_URL}/health").mock( + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( side_effect=httpx.ConnectError("connection refused") ) caps = probe_capabilities(BASE_URL) @@ -223,7 +225,7 @@ def test_probe_capabilities_returns_none_on_malformed_json(): contract as a transport error: best-effort, not fatal. """ with respx.mock: - respx.get(f"{BASE_URL}/health").mock( + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( return_value=httpx.Response(200, text="not-json{") ) caps = probe_capabilities(BASE_URL) diff --git a/tests/test_cb_halfopen_publish.py b/tests/test_cb_halfopen_publish.py index 29344f5..4847be4 100644 --- a/tests/test_cb_halfopen_publish.py +++ b/tests/test_cb_halfopen_publish.py @@ -69,7 +69,7 @@ def test_publish_half_open_state_noop_when_already_closed(self): # =========================================================================== -# Sprint 2.5 (B3): HALF_OPEN call-allocation under concurrent load +# HALF_OPEN call-allocation under concurrent load (B3) # =========================================================================== # Pins the invariant: when the breaker is HALF_OPEN, at most # ``half_open_max_calls`` concurrent calls are allowed to probe diff --git a/tests/test_dead_code_removed.py b/tests/test_dead_code_removed.py index c2e1530..3ec9204 100644 --- a/tests/test_dead_code_removed.py +++ b/tests/test_dead_code_removed.py @@ -116,7 +116,7 @@ def test_workflow_contextmanager_still_works(): with workflow("explicit-id") as wid: assert wid == "explicit-id" - # Phase 5 #5.6: workflow now emits a real UUID4 (matching the + # workflow now emits a real UUID4 (matching the # rest of the SDK's id generation). with workflow() as wid: _uuid.UUID(wid) # raises ValueError if not a UUID @@ -160,7 +160,7 @@ def test_adaptive_pool_removed(): # =========================================================================== # Decision-history removals # =========================================================================== -# Sprint 2.1: the entire ``nullrun.decision_history`` module was +# The entire ``nullrun.decision_history`` module was # deleted because the feature moved to the backend dashboard. The # SDK does not (and cannot) replay LLM calls because the platform # does not store request/response payloads. The ``start_recording`` @@ -172,7 +172,7 @@ def test_decision_history_module_removed(): """The entire ``nullrun.decision_history`` module was deleted in 0.4.0. Previously a separate ``test_event_recorder_removed`` tested that - a single symbol was gone; after Sprint 2.1 the whole module is + a single symbol was gone; after this deletion the whole module is gone, so the import fails at the module level (not the attribute level). Both ``from nullrun.decision_history import X`` and ``import nullrun.decision_history`` must now raise. @@ -188,7 +188,7 @@ def test_decision_history_module_removed(): # =========================================================================== -# Sprint 2.2: zombie exception classes removed +# Zombie exception classes removed # =========================================================================== # Six exception classes had zero in-tree callers — they were defined # but never raised. They were public surface, so external callers @@ -219,7 +219,7 @@ def test_zombie_exception_removed_from_breaker(name: str): assert not hasattr(exceptions, name), ( f"{name} is still defined in nullrun.breaker.exceptions. " - "It was marked as a zombie class in Sprint 2.2 — it has " + "It was marked as a zombie class — it has " "no in-tree callers. Re-add it only when a real use case " "appears, with a regression test for the raise path." ) @@ -245,7 +245,7 @@ def test_zombie_exception_not_in_lazy_exports(name: str): # =========================================================================== -# Sprint 2.7 (B27): dead tenant contextvars / getters +# B27: dead tenant contextvars / getters # =========================================================================== # Pre-fix: ``_organization_id_var`` and ``_api_key_id_var`` were # defined but never written, so ``get_organization_id `` and @@ -293,7 +293,7 @@ def test_dir_size_unchanged(): import is a regression. History: - * Phase 3.4 — surface was 6: ``__version__``, ``init`` + * Initial curated surface was 6: ``__version__``, ``init`` ``protect``, ``track_event``, ``track_llm``, ``track_tool``. * Layer 2 (``on_error``) and Layer 3 (``status``) — added because users need to know they exist (discoverability @@ -312,7 +312,7 @@ def test_dir_size_unchanged(): # contains — no auto-imported submodules, no lazy-resolved # names bleeding in. assert nullrun.__all__[0] == "__version__" - # The five Phase-3.4 anchors are still on the surface. + # The five original anchors are still on the surface. for anchor in ("init", "protect", "track_event", "track_llm", "track_tool"): assert anchor in nullrun.__all__, f"{anchor} missing from __all__" @@ -324,7 +324,7 @@ def test_wrap_symbol_absent(): # =========================================================================== -# Sprint 1.2 (B11, B12): patch_openai / unpatch_openai lazy exports +# B11, B12: patch_openai / unpatch_openai lazy exports # =========================================================================== # These were entries in `_LAZY_EXPORTS` pointing at # `("nullrun.instrumentation", "patch_openai")` / diff --git a/tests/test_dedup.py b/tests/test_dedup.py index 067bc97..b535c4c 100644 --- a/tests/test_dedup.py +++ b/tests/test_dedup.py @@ -250,7 +250,7 @@ class _Rt: # --------------------------------------------------------------------------- -# Phase 3 production-readiness: track_event emits a stable _fingerprint +# track_event emits a stable _fingerprint # --------------------------------------------------------------------------- diff --git a/tests/test_e2e_observation.py b/tests/test_e2e_observation.py index 589447e..8bf7774 100644 --- a/tests/test_e2e_observation.py +++ b/tests/test_e2e_observation.py @@ -1,5 +1,5 @@ """ -Phase 2: real e2e observation test. +Real e2e observation test. The previous suite used respx to mock the NULLRUN backend. That's fine for unit coverage, but it doesn't prove the SDK actually diff --git a/tests/test_error_envelope.py b/tests/test_error_envelope.py index d92b096..b76ee4e 100644 --- a/tests/test_error_envelope.py +++ b/tests/test_error_envelope.py @@ -1,5 +1,5 @@ """ -tests/test_error_envelope.py — Phase 4 production-readiness. +tests/test_error_envelope.py. Verifies ``_parse_error_envelope`` maps 4xx / 5xx / 429 to the right exception subclass per the canonical ``contracts/errors.ts`` diff --git a/tests/test_execute_approval_flow.py b/tests/test_execute_approval_flow.py index b8d03cd..fab3e1b 100644 --- a/tests/test_execute_approval_flow.py +++ b/tests/test_execute_approval_flow.py @@ -1,4 +1,4 @@ -"""Phase 0 regression tests for human approval on the live /execute path.""" +"""Regression tests for human approval on the live /execute path.""" from __future__ import annotations diff --git a/tests/test_extractors.py b/tests/test_extractors.py index bfe4973..0a665e8 100644 --- a/tests/test_extractors.py +++ b/tests/test_extractors.py @@ -566,16 +566,16 @@ def test_provider_table_covers_seven_hosts(): # --------------------------------------------------------------------------- -# Phase 4.1: new fields (cache / reasoning / finish / tool_names) and +# New fields (cache / reasoning / finish / tool_names) and # the privacy boundary that strips them at the wire. # --------------------------------------------------------------------------- def test_openai_no_tool_calls_returns_empty_list(): """A response without tool_calls must not break the extractor — we - get an empty list and a normalized finish_reason. Pre-Phase-4.1 - this would have KeyError'd on `tool_calls` because the loop - iterated over None.""" + get an empty list and a normalized finish_reason. Before these + extractor additions this would have KeyError'd on `tool_calls` + because the loop iterated over None.""" body = json.dumps( { "choices": [ diff --git a/tests/test_framework_patches.py b/tests/test_framework_patches.py index 2b75663..39734a7 100644 --- a/tests/test_framework_patches.py +++ b/tests/test_framework_patches.py @@ -2,7 +2,7 @@ Regression tests for the new framework auto-instrumentation patches in 0.4.0. -Phase 7 of the production-readiness plan adds three new patches: +Adds three new patches: - llama-index (LLMChatEndEvent + FunctionCallEvent via Dispatcher) - crewai (Crew.kickoff + Crew.kickoff_async + post-run usage_metrics) - autogen (BaseChatAgent.on_messages + OpenAIChatCompletionClient.create) @@ -12,7 +12,7 @@ provided no coverage and gave a false sense of green-on-arrival. Real coverage for these frameworks lives in the framework-specific integration suites (one per repo, gated on the framework being -installed). See Sprint 2.9 ticket. +installed). """ from __future__ import annotations @@ -90,7 +90,7 @@ def test_new_framework_modules_importable(): # =========================================================================== -# Sprint 2.9 (B47): safe_patch wrapper for centralised error visibility +# B47: safe_patch wrapper for centralised error visibility # =========================================================================== # Pre-fix: the auto-instrumentation modules had 25+ scattered # ``try/except Exception: pass # pragma: no cover`` blocks. A diff --git a/tests/test_high_reliability_fixes.py b/tests/test_high_reliability_fixes.py index 591e785..604f597 100644 --- a/tests/test_high_reliability_fixes.py +++ b/tests/test_high_reliability_fixes.py @@ -1,16 +1,15 @@ """ Regression tests for HIGH-reliability fixes in 0.4.0. -Phase 5 of the production-readiness plan: -- #5.1: _remote_state_for / _set_remote_state / _states_lock helpers. -- #5.2: PolicyCache policy_version is its own field, not ttl_seconds. -- #5.3: get_instance atomic credential rotation. -- #5.5: _fetch_remote_state uses shared transport client. -- #5.6: workflow emits UUID4 (was wf-{hex32}). -- #5.7: @sensitive fails CLOSED on registration error (wraps original - # exception as RuntimeError with chained __cause__). -- #5.8: Custom-host KILL reach. -- #5.10: Transport.execute on_transport_error callback. +- _remote_state_for / _set_remote_state / _states_lock helpers. +- PolicyCache policy_version is its own field, not ttl_seconds. +- get_instance atomic credential rotation. +- _fetch_remote_state uses shared transport client. +- workflow emits UUID4 (was wf-{hex32}). +- @sensitive fails CLOSED on registration error (wraps original + exception as RuntimeError with chained __cause__). +- Custom-host KILL reach. +- Transport.execute on_transport_error callback. """ from __future__ import annotations @@ -255,7 +254,7 @@ def callback(exc): received.append(exc) return {"decision": "block", "decision_source": "FALLBACK"} - # Round 3 (Phase 0.4.0): runtime.execute raises NullRunBlockedException + # runtime.execute raises NullRunBlockedException # when the result has decision="block". The callback was already invoked # by Transport.execute before the result propagated up. import pytest diff --git a/tests/test_hmac_byte_equality.py b/tests/test_hmac_byte_equality.py index 74587d8..7bff1a8 100644 --- a/tests/test_hmac_byte_equality.py +++ b/tests/test_hmac_byte_equality.py @@ -9,7 +9,7 @@ signed `/gate` and `/check` calls were rejected with 401 when `secret_key` was configured. -Phase 4 introduces `_signed_request_body` (canonical JSON bytes) and +Introduces `_signed_request_body` (canonical JSON bytes) and moves all three signed POSTs to `content=body`. """ diff --git a/tests/test_hmac_signing.py b/tests/test_hmac_signing.py index 1b1ec6f..179b5f3 100644 --- a/tests/test_hmac_signing.py +++ b/tests/test_hmac_signing.py @@ -1,8 +1,7 @@ """ -tests/test_hmac_signing.py — Phase 1 production-readiness. +tests/test_hmac_signing.py. -Verifies the HMAC always-on contract from the production-readiness -plan: every POST that has a body and a ``secret_key`` produces a +Verifies the HMAC always-on contract: every POST that has a body and a ``secret_key`` produces a canonical ``X-Signature`` + ``X-Signature-Timestamp`` pair. Without ``secret_key`` no signature headers are emitted (preserves the dev/legacy path). Tampered bodies and stale timestamps are rejected @@ -324,7 +323,7 @@ def test_track_batch_request_is_signed(self, transport_factory): # (This is a smoke test for the wire format. The actual # _send_batch_with_retry_info path is integration-tested # in test_transport.py — that file has pre-existing - # structural issues unrelated to Phase 1.) + # structural issues unrelated to HMAC.) assert sig is not None assert len(sig) == 64 diff --git a/tests/test_httpx_patch.py b/tests/test_httpx_patch.py index 1d1ab3d..bccaae2 100644 --- a/tests/test_httpx_patch.py +++ b/tests/test_httpx_patch.py @@ -17,7 +17,7 @@ header — otherwise the downstream openai/anthropic client tries to decompress an already-decompressed body and raises `zlib.error: Error -3 while decompressing data: incorrect header check`. Regression - test for the bug that broke Phase 3 of `policy_e2e_demo.py`. + test for the bug that broke the policy demo's end-to-end gzip path. """ from __future__ import annotations diff --git a/tests/test_init_contract.py b/tests/test_init_contract.py index 5499955..427dc30 100644 --- a/tests/test_init_contract.py +++ b/tests/test_init_contract.py @@ -84,10 +84,9 @@ class TestInitWritesAllSingletonSlots: """ def test_init_writes_all_three_singleton_slots(self, monkeypatch, mock_api): - # Phase 3 (2026-07-05): the three slots - # (`runtime._runtime`, `NullRunRuntime._instance`, + # The three slots (`runtime._runtime`, `NullRunRuntime._instance`, # `decorators._runtime`) all route through the - # RuntimeRegistry. We assert the registry pointer directly + # RuntimeRegistry (as of 2026-07-05). We assert the registry pointer directly # and also confirm the legacy read paths see the same # instance (backwards compat). from nullrun._registry import get_active_runtime @@ -114,7 +113,7 @@ def test_init_is_thread_safe(self, monkeypatch, mock_api): slots — that directly tests the locking primitive without the noise of background WS threads. - Phase 3 (2026-07-05): the worker writes through the + As of 2026-07-05, the worker writes through the RuntimeRegistry (the canonical store). The NullRunRuntime._instance descriptor routes to the registry, and the module-level `_runtime` proxies re-resolve @@ -165,10 +164,11 @@ def worker(rt: NullRunRuntime) -> None: class TestInitCapabilityProbeLogging: """Pins the ``logger.warning/info/debug`` branches added in 0.12.0 - when ``init `` runs the /health capability probe. These tests - exist to keep the new logging paths covered so a refactor that - accidentally drops one (e.g. replacing ``logger.info`` with - ``print``) gets caught in CI rather than at first production init. + when ``init `` runs the /api/v1/capabilities capability probe. + These tests exist to keep the new logging paths covered so a + refactor that accidentally drops one (e.g. replacing + ``logger.info`` with ``print``) gets caught in CI rather than + at first production init. """ def test_init_with_debug_true_sets_log_level( @@ -238,11 +238,11 @@ def test_init_replaces_existing_runtime_logs_warning( def test_init_logs_info_when_probe_unreachable( self, monkeypatch, mock_api, caplog ): - """When ``/health`` is unreachable, ``init `` logs at INFO - that the probe was skipped (does NOT fail init). + """When ``/api/v1/capabilities`` is unreachable, ``init `` + logs at INFO that the probe was skipped (does NOT fail init). - Pins the ``logger.info("nullrun.init: could not probe %s/health...")`` - branch on lines 358-362. + Pins the ``logger.info("nullrun.init: could not probe + %s/api/v1/capabilities...")`` branch on lines 358-362. """ import logging @@ -251,11 +251,12 @@ def test_init_logs_info_when_probe_unreachable( monkeypatch.setenv("NULLRUN_API_KEY", "test-key-12345678") monkeypatch.setenv("NULLRUN_API_URL", "https://api.test.nullrun.io") - # Override the /health mock from `mock_api` to fail. We have - # to do this inside the respx.mock context that mock_api opened - # so we route through respx again rather than nesting. + # Override the /api/v1/capabilities mock from `mock_api` to + # fail. We have to do this inside the respx.mock context that + # mock_api opened so we route through respx again rather than + # nesting. with respx.mock: - respx.get("https://api.test.nullrun.io/health").mock( + respx.get("https://api.test.nullrun.io/api/v1/capabilities").mock( return_value=httpx.Response(503) ) # Re-mock the other endpoints that init hits so the diff --git a/tests/test_instrumentation_phase41.py b/tests/test_instrumentation_phase41.py index 6d092a1..4e6aaeb 100644 --- a/tests/test_instrumentation_phase41.py +++ b/tests/test_instrumentation_phase41.py @@ -1,4 +1,4 @@ -"""Coverage padding for Phase 4.1 instrumentation additions. +"""Coverage padding for instrumentation additions (finish_reason normaliser, cache/reasoning/tool-name extraction). The PR adds a finish_reason normaliser + cache / reasoning / tool-name extraction in two places: @@ -7,7 +7,7 @@ new branches in ``_openai_extractor`` / ``_anthropic_extractor`` / etc. * ``nullrun.instrumentation.langgraph._safe_get_gen_message`` - ``_get_finish_reason``, and the Phase 4.1 second-tier fields of + ``_get_finish_reason``, and the second-tier fields of ``extract_usage_from_response``. The functions are pure (or near-pure) — feed them a representative @@ -94,9 +94,9 @@ def test_empty_string_returns_none(self) -> None: # --------------------------------------------------------------------------- -# _openai_extractor — Phase 4.1 second-tier fields +# _openai_extractor — second-tier fields # --------------------------------------------------------------------------- -class TestOpenAIPhase41Fields: +class TestOpenAISecondTierFields: def test_cache_read_and_reasoning_tokens_extracted(self) -> None: # OpenAI's o-series responses nest cache + reasoning under # prompt_tokens_details / completion_tokens_details. @@ -123,7 +123,7 @@ def test_cache_read_and_reasoning_tokens_extracted(self) -> None: assert out["cache_write_tokens"] == 0 def test_finish_reason_normalised(self) -> None: - # The Phase 4.1 extractor pulls ``finish_reason`` off the + # The extractor pulls ``finish_reason`` off the # first choice and routes it through the normaliser. body = json.dumps( { @@ -170,9 +170,9 @@ def test_tool_names_collected_from_choices(self) -> None: # --------------------------------------------------------------------------- -# _anthropic_extractor — Phase 4.1 cache_read + cache_write +# _anthropic_extractor — cache_read + cache_write # --------------------------------------------------------------------------- -class TestAnthropicPhase41Fields: +class TestAnthropicSecondTierFields: def test_cache_read_and_write_tokens(self) -> None: # Anthropic exposes BOTH cache_read_input_tokens and # cache_creation_input_tokens — the SDK surfaces both. @@ -273,9 +273,9 @@ def test_returns_none_when_no_source_has_value(self) -> None: # --------------------------------------------------------------------------- -# extract_usage_from_response — Phase 4.1 second-tier fields +# extract_usage_from_response — second-tier fields # --------------------------------------------------------------------------- -class TestExtractUsagePhase41: +class TestExtractUsageSecondTier: def test_cache_read_tokens_from_anthropic(self) -> None: # Anthropic exposes cache_read_input_tokens directly on the # usage block; the SDK mirrors it as cache_read_tokens. diff --git a/tests/test_legacy_key_warning.py b/tests/test_legacy_key_warning.py index b8af8c8..bfbb92c 100644 --- a/tests/test_legacy_key_warning.py +++ b/tests/test_legacy_key_warning.py @@ -1,7 +1,7 @@ """ Regression test for the legacy-API-key kill-switch warning. -Pre-Phase-139 API keys do not return ``workflow_id`` from +Pre-0.3.x API keys do not return ``workflow_id`` from ``/auth/verify``. When the SDK has no workflow bound, every ``check_control_plane`` call is a silent no-op — the dashboard's KILL/PAUSE button has no effect on the running agent. This is a @@ -26,7 +26,7 @@ class TestLegacyApiKeyWarning: def test_legacy_key_emits_kill_switch_warning(self, monkeypatch, caplog): - """A pre-Phase-139 key (no workflow_id in auth response) + """A pre-0.3.x key (no workflow_id in auth response) must emit a WARNING explaining that kill/pause will not be honoured. """ @@ -37,7 +37,7 @@ def test_legacy_key_emits_kill_switch_warning(self, monkeypatch, caplog): 200, json={ "organization_id": "00000000-0000-0000-0000-000000000000", - # NO workflow_id — pre-Phase-139 key + # NO workflow_id — pre-0.3.x key "plan": "pro", "features": [], "limits": {"max_cost_cents": 10000}, diff --git a/tests/test_mcp_adapter.py b/tests/test_mcp_adapter.py index 046e960..995852e 100644 --- a/tests/test_mcp_adapter.py +++ b/tests/test_mcp_adapter.py @@ -261,7 +261,7 @@ def test_call_tool_unknown_tool_stamps_class_invalid(): assert get_call_mcp_class() == "invalid" ann = get_call_mcp_annotations() - # Per Разрыв 3 / wire contract, all three hints are + # Per wire contract, all three hints are # explicitly ``None`` (= unknown) rather than ``False`` # so the gate cannot accidentally bypass a destructive # block because the adapter lied. diff --git a/tests/test_mcp_context.py b/tests/test_mcp_context.py index 586496b..eec0ed8 100644 --- a/tests/test_mcp_context.py +++ b/tests/test_mcp_context.py @@ -1,4 +1,4 @@ -"""Tests for the v3.31 (Разрыв 3) MCP tool-context helpers. +"""Tests for the v3.31 MCP tool-context helpers. Pure contextvar plumbing — no network involved. These tests are the SDK-side contract pin for the wire fields the backend expects: @@ -138,7 +138,7 @@ class alone.""" def test_annotations_partial_dict_allowed(self): # Operators may forward only the keys they have. The # backend treats absent keys as "unknown" rather than - # false (per Разрыв 3 / wire contract). The SDK does + # false (per wire contract). The SDK does # the same — partial dicts are accepted verbatim. set_mcp_tool_context(annotations={"destructive": True}) ann = get_call_mcp_annotations() diff --git a/tests/test_medium_hygiene_fixes.py b/tests/test_medium_hygiene_fixes.py index 97bb05b..80bf0b6 100644 --- a/tests/test_medium_hygiene_fixes.py +++ b/tests/test_medium_hygiene_fixes.py @@ -1,12 +1,11 @@ """ Regression tests for MEDIUM-hygiene fixes in 0.4.0. -Phase 6: -- #6.1: NULLRUN_FALLBACK_MODE env var override. -- #6.2: _rebuild strips Transfer-Encoding alongside Content-Encoding. -- #6.3: shutdown join caps (0.5s) for signal-handler safety. -- #6.6: WS URL built via urllib.parse. -- #6.7: DEDUP_LRU_MAX raised 512 -> 4096. +- NULLRUN_FALLBACK_MODE env var override. +- _rebuild strips Transfer-Encoding alongside Content-Encoding. +- shutdown join caps (0.5s) for signal-handler safety. +- WS URL built via urllib.parse. +- DEDUP_LRU_MAX raised 512 -> 4096. """ from __future__ import annotations diff --git a/tests/test_money_hardening.py b/tests/test_money_hardening.py index a85f24f..57510f9 100644 --- a/tests/test_money_hardening.py +++ b/tests/test_money_hardening.py @@ -1,4 +1,4 @@ -"""Phase 1.1 hardening tests for the money contract. +"""Decimal support hardening tests for the money contract. This module is the dedicated hardening suite for the ``MoneyImpactExtractor`` hardening pass that closed the diff --git a/tests/test_observability.py b/tests/test_observability.py index 6c93107..197b105 100644 --- a/tests/test_observability.py +++ b/tests/test_observability.py @@ -190,9 +190,9 @@ def reader(): # =========================================================================== -# Sprint 3 follow-up (B23/B24): every metric field must be wired up +# B23/B24: every metric field must be wired up # =========================================================================== -# Pre-Sprint-3-follow-up: 6 fields were defined on the dataclasses +# Before the B23/B24 follow-up: 6 fields were defined on the dataclasses # but never incremented: # - TransportMetrics: retries_total, circuit_breaker_opens # fallback_mode_activations, timeouts, last_error diff --git a/tests/test_preflight_fail_policy.py b/tests/test_preflight_fail_policy.py index 2d63100..16cdd24 100644 --- a/tests/test_preflight_fail_policy.py +++ b/tests/test_preflight_fail_policy.py @@ -346,7 +346,7 @@ def test_real_block_still_honored(self, make_runtime, mock_api): class TestProtectCallsControlPlaneFirst: @pytest.mark.skip( reason=( - "Round 3 (Phase 0.4.0): @protect unifies WorkflowKilledInterrupt " + "@protect unifies WorkflowKilledInterrupt " "into NullRunBlockedException at the decorator boundary. This test " "expects the original WorkflowKilledInterrupt type, which is the " "direct-call contract preserved by check_workflow_budget(). Both " @@ -412,7 +412,7 @@ def agent(q): @pytest.mark.skip( reason=( - "Round 3 (Phase 0.4.0): @protect unifies WorkflowKilledInterrupt " + "@protect unifies WorkflowKilledInterrupt " "into NullRunBlockedException. This test asserts span_end is emitted " "with the original WorkflowKilledInterrupt type, but the decorator " "now raises NullRunBlockedException. Re-enable when span_end payload " @@ -462,7 +462,7 @@ def agent(q): class TestTransportClassification: @pytest.mark.skip( reason=( - "Round 3 (Phase 0.4.0): Transport.check() now requires " + "Transport.check() now requires " 'on_transport_error="raise" to surface classified errors ' "(preserves legacy fail-OPEN behaviour by default so " "check_workflow_budget can treat network errors as transient). " diff --git a/tests/test_protect.py b/tests/test_protect.py index efa1207..e2b541b 100644 --- a/tests/test_protect.py +++ b/tests/test_protect.py @@ -1,5 +1,5 @@ """ -Tests for `@protect` with automatic span hierarchy (Phase 2 Commit 4). +Tests for `@protect` with automatic span hierarchy. The decorator must: - Create a root span (parent_span_id=None, depth=0) on the outermost call diff --git a/tests/test_protect_branches.py b/tests/test_protect_branches.py index 36cc2ba..5cc0962 100644 --- a/tests/test_protect_branches.py +++ b/tests/test_protect_branches.py @@ -2,7 +2,7 @@ Additional tests for ``nullrun.decorators`` — branch coverage for the ``_safe_args`` / ``_strip_details_balanced`` / ``_enforce_sensitive_tool`` helpers, the fail-CLOSED / fail-OPEN contract, the KILL→BlockedException -unification (Round 3), and the ``@protect `` paren-form. +unification, and the ``@protect `` paren-form. """ from __future__ import annotations @@ -400,7 +400,7 @@ def f(x): assert f(3) == 6 -# ─── KILL→BlockedException unification (Round 3) ────────────────────── +# ─── KILL→BlockedException unification ────────────────────── def test_protect_sync_kill_raises_NullRunBlockedException(test_runtime): diff --git a/tests/test_registry.py b/tests/test_registry.py index 93de33a..751c00b 100644 --- a/tests/test_registry.py +++ b/tests/test_registry.py @@ -1,4 +1,4 @@ -"""Tests for the Phase 3 RuntimeRegistry. +"""Tests for the RuntimeRegistry. Covers the single-source-of-truth contract: @@ -39,9 +39,8 @@ def test_registry_set_returns_previous_instance(): The contract lets `init()` shut down the old runtime before installing a new one without holding the lock across the - swap. (Phase 3 commit message: this avoids the deadlock - pattern where a stale instance keeps the lock during its - own shutdown.) + swap. This avoids the deadlock pattern where a stale + instance keeps the lock during its own shutdown. """ from nullrun._registry import RuntimeRegistry @@ -58,7 +57,7 @@ def test_registry_set_returns_previous_instance(): def test_registry_clear_does_not_shutdown(): """clear() drops the pointer without calling any teardown. - Phase 3 rationale: the registry never owns the lifetime + The registry never owns the lifetime of the runtime it stores. Callers that want a real shutdown call `runtime.shutdown()` (which itself calls `registry.clear()` on success). Conflating the two would @@ -139,7 +138,7 @@ def test_metaclass_descriptor_routes_through_registry(): registry, so the class attribute is always the same object the registry holds. - The Phase 3 metaclass proxy is the only path that touches + The metaclass proxy is the only path that touches the singleton; legacy code that imports `NullRunRuntime._instance` keeps working without importing the registry directly. @@ -201,7 +200,7 @@ def test_legacy_globals_set_on_runtime_module_does_not_shadow(): """Backwards-compat: a test fixture that does `runtime._runtime = None` (the historical reset idiom) goes through the proxy and clears the registry, NOT a regular - attribute. This is the regression we fixed in Phase 3. + attribute. This is the regression the proxy fixed. """ import nullrun.runtime as rt_mod from nullrun._registry import get_registry diff --git a/tests/test_release_polish.py b/tests/test_release_polish.py index 3f83c6f..59dc612 100644 --- a/tests/test_release_polish.py +++ b/tests/test_release_polish.py @@ -1,10 +1,9 @@ """ -Regression tests for Phase 8 release polish. +Regression tests for release polish. -Phase 8: -- #8.1: get_org_status public method on NullRunRuntime. -- #8.4: NULLRUN_BATCH_SIZE / NULLRUN_FLUSH_INTERVAL_MS env vars. -- #8.6: RecordingSession does not persist _fingerprint. +- get_org_status public method on NullRunRuntime. +- NULLRUN_BATCH_SIZE / NULLRUN_FLUSH_INTERVAL_MS env vars. +- RecordingSession does not persist _fingerprint. - Circuit-breaker sleep capped at 5s. """ @@ -13,7 +12,7 @@ import pytest # =========================================================================== -# 8.1: get_org_status +# get_org_status # =========================================================================== @@ -81,7 +80,7 @@ def get(self, url, headers=None, timeout=None): # =========================================================================== -# 8.4: env vars +# env vars # =========================================================================== @@ -115,9 +114,9 @@ def test_batch_size_env_invalid_ignored(monkeypatch): # =========================================================================== -# 8.6: _fingerprint not persisted +# _fingerprint not persisted # =========================================================================== -# Sprint 2.1: the local decision-history recorder was deleted (the +# The local decision-history recorder was deleted (the # feature moved to the backend dashboard; the SDK does not store # request/response payloads). The ``start_recording`` / ``stop_recording`` # methods on ``NullRunRuntime`` are kept as no-op stubs for one minor @@ -129,7 +128,7 @@ def test_batch_size_env_invalid_ignored(monkeypatch): def test_start_stop_recording_are_noop_stubs(): """``start_recording`` returns "" and ``stop_recording`` returns None. - Pre-Sprint-2.1 these returned a ``RecordingSession`` / + Before this change these returned a ``RecordingSession`` / ``session_id`` and persisted events to disk. The recorder itself was deleted, so the methods are now no-op stubs. This test pins the new contract. diff --git a/tests/test_runtime.py b/tests/test_runtime.py index be01c93..752b3c3 100644 --- a/tests/test_runtime.py +++ b/tests/test_runtime.py @@ -90,7 +90,7 @@ def test_track_does_not_raise_on_server_error(self, make_runtime, mock_api): rt.track({"event_type": "test"}) def test_wire_payload_strips_sensitive_fields(self, make_runtime): - """Phase 4.1 privacy boundary: ``raw_usage``, ``_fingerprint`` + """Privacy boundary: ``raw_usage``, ``_fingerprint`` and ``cost_cents`` MUST NOT appear in the dict that lands on the transport buffer (i.e. what /api/v1/track/batch would serialise). Normalised fields pass through unchanged. @@ -195,9 +195,53 @@ def test_execute_blocked_raises(self, make_runtime, mock_api): with pytest.raises(NullRunBlockedException): rt.execute(tool_name="gpt-4", input_data={}, mode="strict") + def test_execute_blocked_surfaces_wire_error_code(self, make_runtime, mock_api): + # DEF-ARFLOW-TOOLNAME-01 (E2E 2026-08-05): the backend now stamps + # ``details.error_code`` on block responses via + # ``classify_approval_create_error``. The SDK must surface the + # structured code verbatim instead of falling back to the + # keyword-on-explanation path (which would have classified + # "Approval infrastructure unavailable: validation error during + # approval row creation" as the generic NR-X001 — the very + # bug the journal test surfaced). + respx.post(f"{BASE_URL}/api/v1/execute").mock( + return_value=httpx.Response( + 200, + json={ + "decision": "block", + "explanation": ( + "Approval infrastructure unavailable: validation " + "error during approval row creation — failing closed " + "per Hard-always policy" + ), + "decision_source": "gateway", + "policy_version": 1, + "details": { + "error_code": "APPROVAL_VALIDATION_FAILED", + "decision_source": "approval_create_failed", + }, + }, + ) + ) + rt = make_runtime() + with pytest.raises(NullRunBlockedException) as exc_info: + rt.execute(tool_name="refund_customer", input_data={}, mode="strict") + # The wire code wins — no keyword guessing. + assert exc_info.value.error_code == "APPROVAL_VALIDATION_FAILED" + # The structured payload is preserved on details so a caller + # can introspect ``decision_source`` for routing/alerting. + # ``NullRunBlockedException.__init__`` wraps ``**details`` so + # the dict lands under the "details" key on self.details. + wire_details = exc_info.value.details.get("details") or {} + assert wire_details.get("error_code") == "APPROVAL_VALIDATION_FAILED" + assert wire_details.get("decision_source") == "approval_create_failed" + # Back-compat shim: the legacy ``mapped_class`` field is still + # populated so any caller that branched on it pre-fix keeps working. + assert wire_details.get("mapped_class") == "NullRunBlockedException" + @pytest.mark.skip( reason=( - "Round 3 (Phase 0.4.0): runtime.execute now requires " + "runtime.execute now requires " 'on_transport_error="raise" to surface classified errors ' "(preserves legacy fail-OPEN behaviour by default so " "check_workflow_budget can treat network errors as transient). " diff --git a/tests/test_sensitive_extractor.py b/tests/test_sensitive_extractor.py index 7bcac9f..9938892 100644 --- a/tests/test_sensitive_extractor.py +++ b/tests/test_sensitive_extractor.py @@ -1,4 +1,4 @@ -"""Phase 1 / MVP 1.0 -- SDK e2e for the @sensitive(impact=...) path. +"""Typed impact + digest-bound approval -- SDK e2e for the @sensitive(impact=...) path. These tests pin the wire shape produced by the auto-wire path: when a sensitive tool decorated with ``@sensitive(impact=...)`` @@ -99,7 +99,7 @@ def captured_payload(captured_runtime) -> _PayloadCapture: # --------------------------------------------------------------------------- -# Phase 1 typed tools: built manually instead of via the @sensitive +# Typed-impact tools: built manually instead of via the @sensitive # decorator. The decorator wiring is exercised by # ``test_decorator_factory_form_attaches_extractor`` below. # --------------------------------------------------------------------------- @@ -110,7 +110,7 @@ def _refund_customer_impl(amount_cents: int, customer_id: str = "c-1") -> dict[s def _register_refund_tool(rt: NullRunRuntime) -> Any: - """Bind ``_refund_customer_impl`` with the Phase 1 extractor + """Bind ``_refund_customer_impl`` with the typed-impact extractor and register it as a sensitive tool. Mirrors what ``@sensitive(impact=money_outflow(argument="amount_cents"))`` would do at decorator-application time, but without paying @@ -124,8 +124,8 @@ def _register_refund_tool(rt: NullRunRuntime) -> Any: def _register_legacy_tool(rt: NullRunRuntime) -> Any: - """Bind a sensitive tool WITHOUT a Phase 1 extractor (legacy - Phase 0 path). The wrapper must NOT attach business_impact + """Bind a sensitive tool WITHOUT a typed-impact extractor (legacy + approval_id-only path). The wrapper must NOT attach business_impact or action_digest to the wire. """ def search_docs(query: str) -> list[str]: @@ -141,7 +141,7 @@ def search_docs(query: str) -> list[str]: class TestSensitiveExtractorWirePayload: - """Pin the wire shape produced by the Phase 1 auto-wire path. + """Pin the wire shape produced by the typed-impact auto-wire path. These tests replace the SDK's transport with a recorder and invoke ``_enforce_sensitive_tool`` directly. The capture @@ -163,14 +163,14 @@ def test_refund_customer_50_dollars_sends_typed_business_impact( assert captured_payload.last_kwargs is not None kwargs = captured_payload.last_kwargs - # Both Phase 1 fields must be present because the function + # Both typed-impact fields must be present because the function # has the extractor attribute set. assert "business_impact" in kwargs, ( - f"Phase 1 contract broken: business_impact missing from " + f"Typed-impact contract broken: business_impact missing from " f"wire kwargs: {sorted(kwargs.keys())}" ) assert "action_digest" in kwargs, ( - f"Phase 1 contract broken: action_digest missing from " + f"Typed-impact contract broken: action_digest missing from " f"wire kwargs: {sorted(kwargs.keys())}" ) @@ -189,7 +189,7 @@ def test_refund_customer_50_dollars_sends_typed_business_impact( def test_legacy_sensitive_tool_sends_no_business_impact( self, captured_payload: _PayloadCapture, captured_runtime: NullRunRuntime ) -> None: - """Phase 0 path: tool is sensitive but has no impact + """Legacy path: tool is sensitive but has no impact extractor. The SDK MUST NOT attach business_impact or action_digest to the wire -- the backend falls back to approval_id-only grant consume. @@ -206,7 +206,7 @@ def test_legacy_sensitive_tool_sends_no_business_impact( def test_extractor_rejects_unknown_argument_at_call_time( self, captured_runtime: NullRunRuntime ) -> None: - """Phase 1 fail-CLOSED: if the extractor raises (e.g. + """Typed-impact fail-CLOSED: if the extractor raises (e.g. argument name mismatch), the pre-check MUST fail. The body NEVER runs. """ @@ -229,7 +229,7 @@ def bad_tool(amount: int) -> dict[str, Any]: def test_extractor_rejects_negative_amount( self, captured_runtime: NullRunRuntime ) -> None: - """Phase 1 fail-CLOSED: a negative amount must NOT pass + """Typed-impact fail-CLOSED: a negative amount must NOT pass the pre-check. Without this, a hostile SDK caller could subtract their way past the rule threshold by passing a negative number. @@ -241,7 +241,7 @@ def test_extractor_rejects_negative_amount( with pytest.raises(NullRunBlockedException) as exc_info: _enforce_sensitive_tool(captured_runtime, fn, (-1,), {"customer_id": "c-1"}) assert exc_info.value.error_code == "NR-B003" - # Phase 1.1 hardening: the negative-amount guard now + # Decimal support hardening: the negative-amount guard now # lives in ``_to_minor_units`` (not in # ``MoneyImpact.validate``), so the reason text # matches the new "rejected negative" message. The diff --git a/tests/test_tool_params_extractor.py b/tests/test_tool_params_extractor.py index 4bbb682..837303d 100644 --- a/tests/test_tool_params_extractor.py +++ b/tests/test_tool_params_extractor.py @@ -1,7 +1,7 @@ -"""Phase 1 / MVP 1.1 -- SDK e2e for the ToolParameters path. +"""Typed impact + digest-bound approval -- SDK e2e for the ToolParameters path. These tests pin the wire shape produced by ``@sensitive`` when -paired with ``ToolParamsExtractor`` (the Tier 2 / Razryv 2 +paired with ``ToolParamsExtractor`` (the tool-parameters follow-up to ``MoneyImpactExtractor``). Mirrors the structure of ``test_sensitive_extractor.py`` so a reader who knows one file knows the other. @@ -279,7 +279,7 @@ def charge_card(pan: str, amount: int) -> None: def test_float_arg_is_silently_dropped( self, captured_payload: _PayloadCapture, captured_runtime: NullRunRuntime ) -> None: - """Phase 1 filter-not-block: a kwarg whose type is not + """Typed-impact filter-not-block: a kwarg whose type is not JSON-roundtrippable (``float``) is silently dropped from the wire payload rather than failing the pre-check. @@ -547,7 +547,7 @@ def test_business_impact_kind_dispatch(self) -> None: # --------------------------------------------------------------------------- -# 5. Regression: explicit-extractor-vs-auto-attach priority (Phase 1 / MVP 1.1) +# 5. Regression: explicit-extractor-vs-auto-attach priority (typed impact + tool-params) # --------------------------------------------------------------------------- # # Bug found via ad-hoc verification after the initial auto-attach @@ -633,9 +633,8 @@ def test_explicit_money_outflow_chain_walk_preserved( self, captured_runtime: NullRunRuntime ) -> None: """``@sensitive(impact=money_outflow(...))`` also survives - the auto-attach path (the original Phase 1 / MVP 1.0 - money variant must NOT be overwritten by the Tier 2 - auto-attach). + the auto-attach path (the original money variant must NOT be + overwritten by the tool-parameters auto-attach). """ def tool_fn(amount_cents: int) -> None: diff --git a/tests/test_toolbox_langgraph.py b/tests/test_toolbox_langgraph.py index bab8dcf..6c014da 100644 --- a/tests/test_toolbox_langgraph.py +++ b/tests/test_toolbox_langgraph.py @@ -105,6 +105,6 @@ def test_old_instrument_path_is_removed(): import nullrun.instrumentation.langgraph as mod assert not hasattr(mod, "instrument"), ( - "Phase 1 Commit 6: `instrument` should be removed; " + "`instrument` should be removed; " "use `nullrun.toolbox.langgraph.wrapper` instead." ) diff --git a/tests/test_tracing.py b/tests/test_tracing.py index 88688a7..a7272d8 100644 --- a/tests/test_tracing.py +++ b/tests/test_tracing.py @@ -139,7 +139,7 @@ def test_span_context_is_immutable(): # =========================================================================== -# Sprint 2.6 (B5): create_child_span must reject None parent clearly +# B5: create_child_span must reject None parent clearly # =========================================================================== # Pre-fix: ``create_child_span(None)`` raised # ``TypeError: unsupported operand for None + 1`` on the diff --git a/tests/test_track_span_context.py b/tests/test_track_span_context.py index c9c9bf8..b9347b9 100644 --- a/tests/test_track_span_context.py +++ b/tests/test_track_span_context.py @@ -1,7 +1,7 @@ """ Tests for span-context attachment in track_llm / track_tool. -Phase 2 Commit 5: track_llm and track_tool must auto-include +track_llm and track_tool must auto-include `trace_id` / `span_id` (and `parent_span_id` / `depth`) from the active SpanContext set by `@protect` or a manual `set_span`. This lets the backend render LLM/tool calls under the right node of the diff --git a/tests/test_transport.py b/tests/test_transport.py index 6cab408..b3734fc 100644 --- a/tests/test_transport.py +++ b/tests/test_transport.py @@ -283,7 +283,7 @@ def test_execute_success_does_not_cache_decision(self, transport): @respx.mock def test_check_endpoint_returns_block_on_error(self, transport): """Check endpoint returns block decision on error.""" - # Round 3 (Phase 0.4.0): check now uses the unified + # Check now uses the unified # /api/v1/gate endpoint (was /api/v1/check). respx.post("https://api.test.nullrun.io/api/v1/gate").mock( return_value=httpx.Response(500, text="Server Error") @@ -710,7 +710,7 @@ def test_verify_hmac_signature_expired(self): # =========================================================================== -# Sprint 2.4 (B20): _refetch_credentials must use the shared httpx client +# B20: _refetch_credentials must use the shared httpx client # =========================================================================== # Pre-fix the implementation did ``import requests; requests.post(...)`` # inside the function body, which: @@ -819,7 +819,7 @@ def test_refetch_does_not_import_requests(self): class TestToolArgumentsForwarding: - """Разрыв 4 / T5.6 (2026-07-31) wire-shape pins for + """T5.6 (2026-07-31) wire-shape pins for the `tool_arguments` field on the /execute and /check endpoints. The backend (T5.6) reads `tool_arguments` from the request, computes a schema fingerprint, and @@ -827,7 +827,7 @@ class TestToolArgumentsForwarding: authenticated MCP /check. Pre-T5.6 SDKs (≤ 0.14.4) never set this field; the - backend falls back to the Разрыв 2 `tool_params` + backend falls back to the `tool_params` field. The wire change is additive-only. """ @@ -939,3 +939,117 @@ def capture(request: httpx.Request) -> httpx.Response: # Wire contract: tool_arguments round-trips # verbatim from check_request → wire JSON. assert captured.get("tool_arguments") == {"repo": "acme/api"} + + @respx.mock + def test_check_forwards_parent_execution_id_when_present(self, transport): + """Execution Graph v0 (2026-08-06, backend): additive + `parent_execution_id` on /gate. A sub-agent SDK call to a + child execution names the parent execution here; the + backend validates ownership against the parent's + ``execution:{id}`` Redis binding. Forwarded only when the + caller passes a non-None string -- legacy / single-shot + callers keep the previous payload shape (see + ``test_check_omits_parent_execution_id_when_absent``). + """ + captured: dict = {} + + def capture(request: httpx.Request) -> httpx.Response: + import json + captured.update(json.loads(request.content)) + return httpx.Response( + 200, + json={ + "decision": "allow", + "policy_id": "policy-eg", + "policy_version": 1, + "explanation": "sub-agent call; parent lineage OK", + }, + ) + + respx.post("https://api.test.nullrun.io/api/v1/gate").mock( + side_effect=capture + ) + parent_id = "00000000-0000-0000-0000-000000000099" + child_id = "00000000-0000-0000-0000-0000000000aa" + result = transport.check( + check_request={ + "organization_id": "ws-123", + "execution_id": child_id, + "tool": "mcp://github/create_issue", + "parent_execution_id": parent_id, + } + ) + assert result["decision"] == "allow" + # Wire contract: parent_execution_id round-trips + # verbatim from check_request → wire JSON. + # Field name matches the backend's wire schema at + # ``backend/src/proxy/http/gate/schemas.rs:73``. + assert captured.get("parent_execution_id") == parent_id + + @respx.mock + def test_check_omits_parent_execution_id_when_absent(self, transport): + """Default `parent_execution_id=None` (or absent from + ``check_request``) MUST NOT appear on the wire. Legacy / + single-shot SDKs round-trip cleanly because the field is + absent -- the backend's ``skip_serializing_if = "Option::is_none"`` + contract is mirrored SDK-side by the conditional forward + at ``transport.py:`` (after the ``tool_arguments`` block). + The wire change is additive-only; pre-Execution-Graph SDKs + never wrote the key. + """ + captured: dict = {} + + def capture(request: httpx.Request) -> httpx.Response: + import json + captured.update(json.loads(request.content)) + return httpx.Response( + 200, + json={"decision": "allow", "policy_id": "p", "policy_version": 1}, + ) + + respx.post("https://api.test.nullrun.io/api/v1/gate").mock( + side_effect=capture + ) + transport.check( + check_request={ + "organization_id": "ws-123", + "execution_id": "exec-456", + "tool": "mcp://github/create_issue", + } + ) + # `parent_execution_id` MUST be absent when caller + # didn't pass it. The wire change is additive-only. + assert "parent_execution_id" not in captured + + @respx.mock + def test_check_omits_parent_execution_id_when_none_explicit(self, transport): + """Explicit ``parent_execution_id=None`` in + ``check_request`` (vs. key absent) MUST also be omitted. + Guards against SDK callers that build their + ``check_request`` programmatically and set the field to + ``None`` for clarity. + """ + captured: dict = {} + + def capture(request: httpx.Request) -> httpx.Response: + import json + captured.update(json.loads(request.content)) + return httpx.Response( + 200, + json={"decision": "allow", "policy_id": "p", "policy_version": 1}, + ) + + respx.post("https://api.test.nullrun.io/api/v1/gate").mock( + side_effect=capture + ) + transport.check( + check_request={ + "organization_id": "ws-123", + "execution_id": "exec-456", + "tool": "mcp://github/create_issue", + "parent_execution_id": None, + } + ) + # Explicit None must NOT be forwarded -- the SDK + # treats None as "no parent" (single-shot semantics). + assert "parent_execution_id" not in captured diff --git a/tests/test_units_discriminator.py b/tests/test_units_discriminator.py index bec8449..3286b3e 100644 --- a/tests/test_units_discriminator.py +++ b/tests/test_units_discriminator.py @@ -1,4 +1,4 @@ -"""Phase 1.1 UX follow-up: explicit units discriminator + Decimal support. +"""Decimal support follow-up: explicit units discriminator + Decimal support. These tests pin the behavior the previous review explicitly called out: the unit semantics (major / minor) must be @@ -86,7 +86,7 @@ def test_decimal_50_minor_5000(self) -> None: assert impact.impact.amount_minor == 5_000 def test_decimal_50_005_rejected_for_usd(self) -> None: - # Phase 1.1 production-grade contract: precision must be + # Production-grade contract: precision must be # supplied correctly by the caller. ``Decimal("50.005")`` # is a sub-cent precision that USD does not support, so # the SDK raises ``ValueError`` rather than silently @@ -321,7 +321,7 @@ def test_minor_int_5000_matches_golden(self) -> None: ) impact = ext.impact_for(_refund_cents, (5_000,), {}) # The canonical wire form is identical to the legacy - # Phase 0 path. The golden hex is the SAME on the + # pre-Decimal path. The golden hex is the SAME on the # backend side (see ``business_impact.rs::tests:: # action_digest_golden_usd_outflow_5000_cents``). from nullrun.business_impact import compute_action_digest @@ -481,7 +481,7 @@ def test_unknown_units_is_defensive_branch(self) -> None: class TestDirectionIsUnaffected: """``units`` does not interact with ``direction`` (outflow / inflow). The default direction is OUTFLOW, matching the - Phase 0 / pre-Decimal path.""" + pre-Decimal path.""" def test_major_units_default_direction_is_outflow(self) -> None: ext = money_outflow( diff --git a/tests/test_v3_38_drift_fixes.py b/tests/test_v3_38_drift_fixes.py new file mode 100644 index 0000000..929fce6 --- /dev/null +++ b/tests/test_v3_38_drift_fixes.py @@ -0,0 +1,295 @@ +"""Regression tests for the v3.38 wire-drift fixes (2026-08-07). + +These pin three contract-level fixes that were verified against +backend source code, not against comments or documentation: + +* **capabilities probe route** — the SDK was probing + ``/health`` (a generic liveness payload) instead of the + canonical ``/api/v1/capabilities`` route. Pre-fix, every + ``is_v3_ready()`` returned False because the probe never saw + a v3 capability payload, leaving every flag a runtime no-op. + +* **API_KEY_* error code granularity (v3.38)** — the backend + split the v3.36 ``API_KEY_REVOKED`` bucket into five distinct + wire codes (``API_KEY_EXPIRED`` / ``API_KEY_DISABLED`` / + ``API_KEY_INVALID`` / ``API_KEY_MISSING`` / + ``API_KEY_MALFORMED``) so SDKs can branch on each lifecycle + state. Pre-fix, only ``API_KEY_REVOKED`` was mapped in + ``_V3_ERROR_CODE_MAP`` — the other five silently fell through + to the generic HTTP-status fallback (``NullRunAuthentication + Error``) without ever becoming ``NullRunAuthError``, losing + the diagnostic class. Wire codes are now surfaced on + ``NullRunAuthError.wire_code``. + +* **decision == "soft_pass" handling** — the backend returns + ``soft_pass`` for soft-mode calls that proceed via the chain's + overdraft cap (CLAUDE.md §5). Pre-fix, the runtime's + ``check_workflow_budget`` had no branch for ``soft_pass`` — + the ``decision == "allow"`` default fall-through meant the + body proceeded (correct) but the operator saw no log line + and no overdraft counter incremented (silent budget drift). + +The tests pin the fixed behaviour so a future refactor that +breaks any of these three contracts gets caught in CI rather +than at first production /check. +""" + +from __future__ import annotations + +from pathlib import Path + +import httpx +import pytest +import respx + +from nullrun.breaker import exceptions as exc +from nullrun.capabilities import ( + CAPABILITIES_PATH, + probe_capabilities, +) +from nullrun.transport import _V3_ERROR_CODE_MAP, _parse_v3_error_envelope + +BASE_URL = "https://api.test.nullrun.io" + +_RUNTIME_SRC_PATH = ( + Path(__file__).parent.parent / "src" / "nullrun" / "runtime.py" +) + + +# --------------------------------------------------------------------------- +# Fix #1 — capabilities probe route (/api/v1/capabilities, not /health) +# --------------------------------------------------------------------------- + + +def test_capabilities_path_constant_is_canonical_route(): + """``CAPABILITIES_PATH`` must point at ``/api/v1/capabilities``. + + The constant is the single source of truth — every + ``probe_capabilities`` call builds ``{api_url}{CAPABILITIES_PATH}`` + (capabilities.py:290). Pinning the constant here catches a + refactor that re-introduces the legacy ``/health`` route. + """ + assert CAPABILITIES_PATH == "/api/v1/capabilities" + + +def test_probe_capabilities_against_canonical_route_with_v3_payload(): + """A v3 backend responding at /api/v1/capabilities with the + nested ``capabilities:`` payload yields ``is_v3_ready() == True``. + + Pins the entire probe → parse → flag chain against the canonical + route. Pre-fix the SDK probed /health and never saw this payload, + so ``is_v3_ready()`` was always False. + """ + payload = { + "min_protocol_version": 3, + "max_protocol_version": 3, + "protocol_version": 3, + "capabilities": { + "server_minted_execution_id": True, + "per_execution_reservations": True, + "enforcement_modes_soft": True, + "heartbeat_time_based": True, + }, + } + with respx.mock: + respx.get(f"{BASE_URL}/api/v1/capabilities").mock( + return_value=httpx.Response(200, json=payload) + ) + # Negative pin — a stale /health mock returning 200 must + # NOT satisfy the probe. This catches regressions where + # someone re-adds /health as a fallback. + respx.get(f"{BASE_URL}/health").mock( + return_value=httpx.Response(200, json={"status": "ok"}) + ) + parsed = probe_capabilities(BASE_URL) + assert parsed is not None + assert parsed.is_v3_ready() + assert parsed.server_minted_execution_id is True + assert parsed.per_execution_reservations is True + assert parsed.heartbeat_time_based is True + + +# --------------------------------------------------------------------------- +# Fix #2 — v3.38 API_KEY_* codes in _V3_ERROR_CODE_MAP + wire_code attr +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "wire_code", + [ + "API_KEY_REVOKED", + "API_KEY_EXPIRED", + "API_KEY_DISABLED", + "API_KEY_INVALID", + "API_KEY_MISSING", + "API_KEY_MALFORMED", + ], +) +def test_v3_error_code_map_covers_all_api_key_states(wire_code): + """All six v3.38 API_KEY_* wire codes must map to NullRunAuthError. + + Pre-fix the map only covered ``API_KEY_REVOKED`` — the other + five silently fell through to the generic HTTP-status fallback + (line ~2616 in transport.py), losing the diagnostic class. + Pinning the map catches a refactor that drops any of the five + new entries. + """ + assert wire_code in _V3_ERROR_CODE_MAP + assert _V3_ERROR_CODE_MAP[wire_code] is exc.NullRunAuthError + + +def test_parse_v3_error_envelope_surfaces_wire_code_on_auth_error(): + """A 401 with error_code=API_KEY_EXPIRED yields NullRunAuthError + whose ``wire_code`` attribute exposes the granular backend code. + + Without ``wire_code``, callers have only the SDK-side NR-A003 + taxonomy and lose the granular lifecycle signal. Mirrors + NullRunChainError.backend_code pattern (exceptions.py:448). + """ + response = httpx.Response( + 401, + json={ + "error_code": "API_KEY_EXPIRED", + "error_message": "key TTL elapsed", + "details": {"expires_at": "2026-08-01T00:00:00Z"}, + }, + ) + err = _parse_v3_error_envelope(response, "gate") + assert isinstance(err, exc.NullRunAuthError) + # SDK-side taxonomy preserved (NR-A003) — the fix adds wire_code + # instead of clobbering error_code. + assert err.error_code == "NR-A003" + # Granular wire code surfaced for handler dispatch. + assert err.wire_code == "API_KEY_EXPIRED" + + +def test_parse_v3_error_envelope_preserves_default_wire_code_for_revoked(): + """API_KEY_REVOKED continues to work — wire_code defaults to it + when the constructor is called without an explicit value (e.g. + a future refactor that bypasses the catalog dispatch). + """ + err = exc.NullRunAuthError("revoked") + assert err.wire_code == "API_KEY_REVOKED" + assert err.error_code == "NR-A003" + + +def test_parse_v3_error_envelope_auth_error_does_not_clobber_unrelated_details(): + """The fix to filter ``details`` to known kwargs must not lose + extras silently — unknown keys (e.g. ``expires_at``) must land + on ``self.details`` for caller introspection. Pre-fix the + envelope parser forwarded every detail as a kwarg, which threw + TypeError on the first unknown key (e.g. when the backend + started emitting ``expires_at`` for v3.38 EXPIRED responses). + """ + response = httpx.Response( + 401, + json={ + "error_code": "API_KEY_DISABLED", + "error_message": "admin disabled this key", + "details": { + "disabled_at": "2026-08-01T00:00:00Z", + "disabled_by": "admin@nullrun.io", + }, + }, + ) + err = _parse_v3_error_envelope(response, "gate") + assert isinstance(err, exc.NullRunAuthError) + assert err.wire_code == "API_KEY_DISABLED" + # The disabled_at / disabled_by fields land on self.details + # (not lost, not raised). + details = getattr(err, "details", {}) or {} + assert details.get("disabled_at") == "2026-08-01T00:00:00Z" + assert details.get("disabled_by") == "admin@nullrun.io" + + +# --------------------------------------------------------------------------- +# Fix #3 — decision == "soft_pass" handling in check_workflow_budget +# --------------------------------------------------------------------------- +# +# ``check_workflow_budget(self) -> None`` builds its own ``check_req`` +# dict and fetches via ``self._transport.check()`` — the signature +# has no way to inject a response fixture without a full transport +# mock. The soft_pass branch is a pure decision switch (runtime.py +# ~1799-1830) so a source-level scan is the most reliable pin, +# matching the migration_drift_tests pattern used elsewhere in the +# SDK and backend. + + +def test_check_workflow_budget_handles_soft_pass_decision(): + """``check_workflow_budget`` must contain a ``decision == + "soft_pass"`` branch. + + Pre-fix, ``soft_pass`` fell through the ``decision == "allow"`` + default — body executed (correct) but no log line, no counter. + Operators had zero visibility into "budget soft cap is biting". + + Static scan pins the runtime.py structure so a future refactor + that drops the branch gets caught in CI rather than at first + production /check. + """ + runtime_src = _RUNTIME_SRC_PATH.read_text(encoding="utf-8") + + assert 'decision == "soft_pass"' in runtime_src, ( + "check_workflow_budget must branch on `decision == \"soft_pass\"`. " + "Pre-fix the branch was missing — soft_pass fell through the " + "default allow path and operators got no overdraft telemetry." + ) + + +def test_check_workflow_budget_soft_pass_branch_increments_overdraft_counter(): + """The soft_pass branch must increment ``soft_overdraft_used`` + so operators can graph soft-cap pressure in the dashboard — + silent budget drift is the regression we are preventing. + """ + runtime_src = _RUNTIME_SRC_PATH.read_text(encoding="utf-8") + + # Slice the soft_pass branch out of the file by anchoring on + # the literal and the next known decision branch. The slice + # must contain the counter increment. + soft_pass_idx = runtime_src.find('decision == "soft_pass"') + assert soft_pass_idx >= 0, "soft_pass branch not found" + require_approval_idx = runtime_src.find( + 'decision == "require_approval"', soft_pass_idx + ) + assert require_approval_idx >= 0, ( + "decision == require_approval marker not found after soft_pass — " + "the runtime source structure has drifted from this pin's anchor." + ) + branch_slice = runtime_src[soft_pass_idx:require_approval_idx] + + assert "soft_overdraft_used" in branch_slice, ( + "soft_pass branch must increment `soft_overdraft_used` so the " + "dashboard can graph soft-cap pressure." + ) + assert "metrics.inc_runtime" in branch_slice, ( + "soft_pass branch must call `metrics.inc_runtime(...)` to record " + "the counter." + ) + + +def test_check_workflow_budget_soft_pass_branch_logs_overdraft_telemetry(): + """The soft_pass branch must log at WARNING level with the + backend's ``overdraft_used_cents`` value — that's the operator's + primary signal that the chain's overdraft cap is burning. + """ + runtime_src = _RUNTIME_SRC_PATH.read_text(encoding="utf-8") + + soft_pass_idx = runtime_src.find('decision == "soft_pass"') + require_approval_idx = runtime_src.find( + 'decision == "require_approval"', soft_pass_idx + ) + branch_slice = runtime_src[soft_pass_idx:require_approval_idx] + + assert "overdraft_used_cents" in branch_slice, ( + "soft_pass branch must surface `overdraft_used_cents` from the " + "backend response — silent loss of this value means operators " + "have no visibility into which chains are burning overdraft." + ) + assert "logger.warning" in branch_slice, ( + "soft_pass branch must log at WARNING level — overdraft pressure " + "is operator-actionable, not informational." + ) + + +if __name__ == "__main__": + pytest.main([__file__, "-v"]) \ No newline at end of file diff --git a/tests/test_v3_wire_contract.py b/tests/test_v3_wire_contract.py index 3ed2860..1e8863a 100644 --- a/tests/test_v3_wire_contract.py +++ b/tests/test_v3_wire_contract.py @@ -643,7 +643,7 @@ def test_ping_chain_emits_heartbeats_on_time_schedule(self): # real 10s interval — turns a 10s test into a sub-second one # without changing the production scheduler code. # - # Sprint 0 (coverage): this test depends on the real + # (coverage): this test depends on the real # wall clock to accumulate scheduler iterations within the # 500ms ``time.sleep`` window. ``@pytest.mark.slow_sleep`` # on the enclosing class opts out of the conftest autouse diff --git a/tests/test_ws_push.py b/tests/test_ws_push.py index f4c44d6..3014415 100644 --- a/tests/test_ws_push.py +++ b/tests/test_ws_push.py @@ -1,5 +1,5 @@ """ -Tests for the SDK WebSocket push path (Phase B of the hardening plan). +Tests for the SDK WebSocket push path. The push contract: when the server pushes a `state_change` message with `state: "Killed"`, the runtime's `on_state_change` callback writes the @@ -458,7 +458,7 @@ def test_dispatch_state_drops_older_versions_after_seen_higher(): # --------------------------------------------------------------------------- -# 5. Sprint 1.5 (B13): HMAC verify failure on signed messages +# 5. B13: HMAC verify failure on signed messages # --------------------------------------------------------------------------- # Pre-fix: a signed WS message with a bad signature was logged at # WARNING and dropped silently. For a safety-layer product, a