From cedcf2f461ce35ed1884f0561a21dab831f5a655 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 18:20:31 -0400 Subject: [PATCH 01/64] IMP-098: record bounded SDK runtime implementation lease --- cli/protocol/OWNERSHIP.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index db19d387..2cadfa7e 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -908,3 +908,9 @@ PR47 Linux admission failure. Publish the fixture's observed descendantPID before unrelated logging and validate a nonzero numericPID before procfs. Preserve300ms/3s bounds and killed-or-zombie assertion. Production logic, cleanup deadlines and frozen qualified RC3 binaries remain unchanged. + +## IMP-098 packaged SDK runtime — October 6, 2026 + +Lead: Codex `/root`, session `01a1133b-e47b-7b32-af8e-2655afa0259c`, branch `codex/imp-098-packaged-sdk`, base `e780725`. Root owns all Git/index operations and integration. User authorizes production-readiness implementation and qualification; merge/publication remain separate. + +`/root/sdk_runtime` exact writable paths: `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` (new). Scope: reuse Python SDK loop; public search/retrieval and fresh bounded child contexts; safe credential setup/errors, shared aggregate bounds/usage, bounded tool-output memory, cancellation and honest permissions. No contract interpretation/company procedures. No dependency/manifests, adapters, shared files, model calls, Git/index or formatting outside lease. Dependency additions require root approval; public search uses existing SDK-supported hosted tool. Provider-free mocks precede behavior; retain cancellation/failure controls. Other lanes await exact leases and design freeze. From c1c191c4da4bf7d37e6666c16cc21311614fa1a3 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 18:21:48 -0400 Subject: [PATCH 02/64] IMP-098: select package-owned SDK helper after peer comparison --- cli/protocol/OWNERSHIP.md | 2 ++ docs/packaged-sdk-design.md | 28 ++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 docs/packaged-sdk-design.md diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 2cadfa7e..4fc5474f 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -914,3 +914,5 @@ cleanup deadlines and frozen qualified RC3 binaries remain unchanged. Lead: Codex `/root`, session `01a1133b-e47b-7b32-af8e-2655afa0259c`, branch `codex/imp-098-packaged-sdk`, base `e780725`. Root owns all Git/index operations and integration. User authorizes production-readiness implementation and qualification; merge/publication remain separate. `/root/sdk_runtime` exact writable paths: `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` (new). Scope: reuse Python SDK loop; public search/retrieval and fresh bounded child contexts; safe credential setup/errors, shared aggregate bounds/usage, bounded tool-output memory, cancellation and honest permissions. No contract interpretation/company procedures. No dependency/manifests, adapters, shared files, model calls, Git/index or formatting outside lease. Dependency additions require root approval; public search uses existing SDK-supported hosted tool. Provider-free mocks precede behavior; retain cancellation/failure controls. Other lanes await exact leases and design freeze. + +`/root/operational_design` packaging lease (October 6): exact paths `harnesses/agents-sdk/requirements-build.txt` (new), `cli/ci/build_agents_sdk.py` (new), `cli/ci/test_build_agents_sdk.py` (new), `cli/ci/package_local.py`, `cli/ci/test_package_local.py`, `cli/ci/build_local.py`, `cli/ci/test_check_build_local.py`, `cli/ci/build_kernel_rc.py`, `cli/ci/test_build_kernel_rc.py`, `cli/ci/homebrew_rehearsal.py`, `cli/ci/test_homebrew_rehearsal.py`, `cli/ci/dependency_evidence.py`. Scope: package-owned frozen helper, transitive hash lock, platform build/package/rehearsal and evidence; preserve archive safety, existing modes/fixtures and offline gates. Other evidence schemas/workflows require root-recorded extension before edits. No product trees, runtime run.py, Git/index or model calls. Root owns workflow/schema integration and operational design doc. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md new file mode 100644 index 00000000..f3cf7c4c --- /dev/null +++ b/docs/packaged-sdk-design.md @@ -0,0 +1,28 @@ +# Packaged SDK operational design + +IMP-098, checked October 6, 2026; applies IMP-099. The user selects `~/.prose/cli.toml` and explicit overrides under IMP-097. + +## Peer comparison + +- [Codex launcher source](https://raw.githubusercontent.com/openai/codex/main/codex-cli/bin/codex.js): package-owned optional platform payload, resolved from installed package. +- [Claude setup](https://code.claude.com/docs/en/setup): native executable through standalone and npm platform payloads; settings remain user-owned. +- [OpenCode installation](https://opencode.ai/docs/): standalone, npm and Homebrew installation. +- [Prime source](https://raw.githubusercontent.com/PrimeIntellect-ai/prime-agent/main/README.md): versioned installer and integrated Python tooling; a different runtime design. + +The peers support package-owned platform runtimes and installation-independent preferences, without a universal Python layout. Retain the CLI's existing four native platform payloads. Freeze the existing Python SDK harness into a sibling `prose-agents-sdk` executable using pinned PyInstaller onefile; no user Python, source, provisioning or PATH changes. Archives contain sibling regular executables; npm uses `bin/`; Homebrew installs both to `bin`. Resolve the helper beside the canonical installed CLI executable. Explicit existing test/developer seams remain separate. + +## Choice and costs + +[PyInstaller usage](https://pyinstaller.org/en/stable/usage.html) documents onefile extraction. [POSIX bundle guidance](https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html) documents onedir symlinks and sanitizing dynamic-library variables for child tools. Onefile preserves the existing regular-file archive safety boundary; onedir or bundled interpreter trees would require broader packaging changes. Cost: larger payload/cold start, native build on every supported platform, temporary executable filesystem requirements, and possible leftover extraction directories after forced death. Do not claim forced-death temporary cleanup. Verify frozen-helper supervision including shell descendants. + +Build CPython 3.10.20, existing SDK0.22.2/OpenAI3.13.0, and a hash-locked transitive dependency graph with source identities, SBOM/notices and helper hashes in package evidence. Qualified release targets remain macOS/Linux ARM64/x64, with no expansion to unadmitted Windows execution. Verify actual packaged imports, HTTPS certificates, cold discovery, cancellation and tools on native target builders. Preserve existing glibc/platform admission policy. + +## Configuration integration + +IMP-097 owns canonical discovery, migration, explicit override editing, exact-command explanation and harness-aware defaults. Its initial implementation retains the old harness identifier until IMP-098 switches the built-in harness to `agents-sdk`. SDK contextual defaults are `gpt-6.1-sol` and `openai-api-key`; other selected harnesses receive their own route, never these implicit SDK selections. Explicit incompatible inherited model/auth choices fail before readiness rather than switch account. IMP-098 stacks runtime selection integration on the reviewed IMP-097 contract and retains separate PR ownership. + +## Runtime and qualification + +The generic Python harness transports opaque instructions/tasks, offers bounded public search/retrieval, local tools and fresh child contexts without interpreting contracts. Shared aggregate bounds cover parent and child model calls, tools, usage and cancellation. Environment filtering is not filesystem isolation; permissions and effective authority must be explicit and tested. Configure API client retries as zero for qualification, so callers own retry allocation. + +The workspace assignment allocates $50 total and twelve top-level attempts, one at a time, including children/evaluators/failures. Freeze a matrix and conservative reservation before live calls; unknown terminal accounting retains reservation. No merge or publication is authorized. Release readiness requires independent Rust/Bun/shared/installed-route and live artifact evidence, not process completion alone. From 3418b7b729a63804816393108791ea8a112d7de7 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 18:50:48 -0400 Subject: [PATCH 03/64] Package locked Agents SDK runtime and bind release custody --- .github/workflows/cli-ci.yml | 6 + .github/workflows/cli-kernel-rc.yml | 8 +- cli/benchmarks/installed/benchmark.py | 53 ++- cli/benchmarks/installed/test_benchmark.py | 44 +++ cli/ci/assemble_kernel_rc.py | 4 + cli/ci/build_agents_sdk.py | 249 ++++++++++++++ cli/ci/build_kernel_rc.py | 38 ++- cli/ci/build_local.py | 27 ++ cli/ci/check_workflows.py | 2 +- cli/ci/dependency_evidence.py | 32 +- cli/ci/homebrew_rehearsal.py | 21 +- cli/ci/kernel_rc_evidence.py | 96 +++++- cli/ci/package_local.py | 169 ++++++++-- cli/ci/publication.py | 36 ++- cli/ci/release_package_admission.py | 14 + cli/ci/reproducible_release.py | 33 +- cli/ci/sign_macos.py | 48 ++- cli/ci/test_assemble_kernel_rc.py | 35 +- cli/ci/test_build_agents_sdk.py | 58 ++++ cli/ci/test_build_kernel_rc.py | 19 ++ cli/ci/test_check_build_local.py | 24 ++ cli/ci/test_dependency_evidence.py | 20 ++ cli/ci/test_homebrew_rehearsal.py | 19 ++ cli/ci/test_kernel_rc_evidence.py | 21 ++ cli/ci/test_package_local.py | 73 +++++ cli/ci/test_publication.py | 18 +- cli/ci/test_release_package_admission.py | 6 + cli/ci/test_sign_macos.py | 30 ++ cli/protocol/OWNERSHIP.md | 16 + docs/agents-sdk-adapter.md | 86 ++--- harnesses/agents-sdk/README.md | 47 +-- harnesses/agents-sdk/requirements-build.txt | 230 +++++++++++++ harnesses/agents-sdk/run.py | 340 +++++++++++++++++--- harnesses/agents-sdk/test_run.py | 299 ++++++++++++++++- 34 files changed, 2039 insertions(+), 182 deletions(-) create mode 100644 cli/ci/build_agents_sdk.py create mode 100644 cli/ci/test_build_agents_sdk.py create mode 100644 harnesses/agents-sdk/requirements-build.txt diff --git a/.github/workflows/cli-ci.yml b/.github/workflows/cli-ci.yml index e2d2805b..717c69e6 100644 --- a/.github/workflows/cli-ci.yml +++ b/.github/workflows/cli-ci.yml @@ -61,6 +61,12 @@ jobs: python3 -m pip install --require-hashes --only-binary=:all: -r cli/ci/requirements-test.txt cd cli/bun bun install --frozen-lockfile --ignore-scripts + - name: Prepare the separately locked Agents SDK build interpreter + run: | + uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python" + "$RUNNER_TEMP/agents-sdk-python/bin/python3" -m pip install --require-hashes --only-binary=:all: -r harnesses/agents-sdk/requirements-build.txt + - name: Test the actual pinned Agents SDK runtime without provider credentials + run: '"$RUNNER_TEMP/agents-sdk-python/bin/python3" -m unittest discover -s harnesses/agents-sdk -p test_run.py' - name: Admit current source without provider credentials shell: bash run: python3 cli/ci/run_local.py 2>&1 | tee "$RUNNER_TEMP/cli-admission.log" diff --git a/.github/workflows/cli-kernel-rc.yml b/.github/workflows/cli-kernel-rc.yml index df121453..b2915a37 100644 --- a/.github/workflows/cli-kernel-rc.yml +++ b/.github/workflows/cli-kernel-rc.yml @@ -67,6 +67,12 @@ jobs: python3 -m pip install --require-hashes --only-binary=:all: -r cli/ci/requirements-test.txt cd cli/bun bun install --frozen-lockfile --ignore-scripts + - name: Prepare the separately locked Agents SDK build interpreter + run: | + uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python" + "$RUNNER_TEMP/agents-sdk-python/bin/python3" -m pip install --require-hashes --only-binary=:all: -r harnesses/agents-sdk/requirements-build.txt + - name: Test the actual pinned Agents SDK runtime without provider credentials + run: '"$RUNNER_TEMP/agents-sdk-python/bin/python3" -m unittest discover -s harnesses/agents-sdk -p test_run.py' - name: Test kernel RC build guards run: | python3 -m unittest discover -s cli/ci -p test_build_kernel_rc.py @@ -74,7 +80,7 @@ jobs: - name: Build and verify fresh standalone and npm installations env: RC_VERSION: ${{ inputs.version || '0.15.0-rc.1' }} - run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" + run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" - uses: Homebrew/actions/setup-homebrew@dc7099b3e807f1e2ecc61f3ecabc840eedd5586a - name: Rehearse Homebrew against these exact release archives env: diff --git a/cli/benchmarks/installed/benchmark.py b/cli/benchmarks/installed/benchmark.py index fdc44094..fe7a9409 100644 --- a/cli/benchmarks/installed/benchmark.py +++ b/cli/benchmarks/installed/benchmark.py @@ -378,7 +378,7 @@ def validate_release_manifest( "promotion", "artifacts", "dependencyEvidence", - }, + } | ({"agentsSdk"} if "agentsSdk" in release else set()), "release manifest", ) if release.get("schema") != RELEASE_SCHEMA: @@ -738,7 +738,7 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: ): fail("EVIDENCE_MALFORMED", f"invalid dependency source length: {path}") require_sha(source.get("sha256"), f"dependency source sha256 {path}") - if tuple(source_paths) != EXPECTED_DEPENDENCY_SOURCES: + if tuple(source_paths) not in (EXPECTED_DEPENDENCY_SOURCES, EXPECTED_DEPENDENCY_SOURCES + ("harnesses/agents-sdk/requirements-build.txt",)): fail( "EVIDENCE_MALFORMED", "dependency sources differ from the closed stable source set", @@ -749,7 +749,7 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: ) require_exact_keys( inventories, - {"bun", "cargo", "windowsProcessHostCargo"}, + {"bun", "cargo", "windowsProcessHostCargo"} | ({"agentsSdkPython"} if "agentsSdkPython" in inventories else set()), "dependency inventories", ) inventory_shapes = { @@ -775,7 +775,10 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: "package-manifest-direct-kind-plus-lockfile-reachability", ), } - allowed_scopes = {"runtime", "development", "build", "workspace", "component"} + if "agentsSdkPython" in inventories: + inventory_shapes["agentsSdkPython"] = ({"lockfileVersion", "scopeBasis", "packages"}, None, None, 1, + "hash-locked-four-platform-wheel-candidates") + allowed_scopes = {"runtime", "development", "build", "workspace", "component", "frozen-sdk-build"} for inventory_name, ( keys, expected_component, @@ -958,6 +961,8 @@ def validate_sbom( "cargo": "rust-cli", "windowsProcessHostCargo": "windows-process-host", } + if "agentsSdkPython" in dependency.get("inventories", {}): + inventory_groups["agentsSdkPython"] = "agents-sdk-python" expected_packages: set[ tuple[str, str, str, str, str, str, str | None, str | None] ] = set() @@ -1256,6 +1261,36 @@ def verify_package_output( artifacts, dependency_digest, ) + if release['mode'] == 'release': + import sys + ci_path = str(Path(__file__).resolve().parents[2] / 'ci') + if ci_path not in sys.path: + sys.path.insert(0, ci_path) + import kernel_rc_evidence as custody + try: + sdk = custody.validate_sdk_archives(release, lambda name: { + path: data for path, (data, mode) in decode_archive_members(encoded[name], name).items()}) + except (ValueError, KeyError, TypeError) as error: + fail('IDENTITY_DIVERGENCE', str(error)) + if 'agentsSdkPython' not in dependency['inventories']: + fail('IDENTITY_DIVERGENCE', 'Release lacks SDK Python dependency inventory') + source = next((p for p in dependency['sources'] if p['path'] == 'harnesses/agents-sdk/requirements-build.txt'), {}) + if source.get('sha256') != sdk['dependencyLockSha256']: + fail('IDENTITY_DIVERGENCE', 'Release SDK lock differs from dependency source') + provenance = json_no_duplicates(encoded['provenance.json'], 'provenance.json') + definition = provenance['predicate']['buildDefinition'] + if definition.get('externalParameters', {}).get('agentsSdk') != sdk: + fail('IDENTITY_DIVERGENCE', 'Provenance SDK identity differs') + for uri, digest in (('openprose:agents-sdk-helper', sdk['sha256']), + ('openprose:agents-sdk-build-receipt', sdk['receiptSha256']), + ('openprose:agents-sdk-python-lock', sdk['dependencyLockSha256'])): + matches = [p for p in definition['resolvedDependencies'] if p.get('uri') == uri] + if matches != [{'uri': uri, 'digest': {'sha256': digest}}]: + fail('IDENTITY_DIVERGENCE', 'Provenance SDK dependency differs: ' + uri) + sbom = json_no_duplicates(encoded['sbom.cdx.json'], 'sbom.cdx.json') + matches = [p for p in sbom['components'] if p.get('name') == 'prose-agents-sdk'] + if len(matches) != 1 or matches[0].get('hashes') != [{'alg': 'SHA-256', 'content': sdk['sha256']}]: + fail('IDENTITY_DIVERGENCE', 'SBOM lacks exact frozen SDK helper binding') return { "root": package_output, "purpose": purpose, @@ -1553,6 +1588,8 @@ def validate_npm_packages( } if platform_value.startswith("win32-"): expected_platform_members.add("package/bin/openprose-windows-process-host.exe") + if isinstance(release.get('agentsSdk'), dict): + expected_platform_members.update('package/bin/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) if set(platform_members) != expected_platform_members: fail( "MEMBERSHIP_MALFORMED", @@ -2779,6 +2816,10 @@ def validate_package_payloads(context: Mapping[str, Any]) -> dict[str, Any]: context["encoded"][artifact_name], artifact_name ) root_name, expected = expected_standalone_members(artifact_name, platform_value) + if isinstance(context['release'].get('agentsSdk'), dict): + expected.update(root_name + '/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) + if members.get(root_name + '/prose-agents-sdk', (b'', 0))[1] & 0o111 == 0: + fail('ARCHIVE_UNSAFE', 'SDK helper is not executable') if set(members) != expected: fail( "MEMBERSHIP_MALFORMED", @@ -3115,6 +3156,10 @@ def run_benchmark( context["encoded"][artifact_name], artifact_name ) root_name, expected = expected_standalone_members(artifact_name, platform_value) + if isinstance(context['release'].get('agentsSdk'), dict): + expected.update(root_name + '/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) + if members.get(root_name + '/prose-agents-sdk', (b'', 0))[1] & 0o111 == 0: + fail('ARCHIVE_UNSAFE', 'SDK helper is not executable') if set(members) != expected: fail( "MEMBERSHIP_MALFORMED", diff --git a/cli/benchmarks/installed/test_benchmark.py b/cli/benchmarks/installed/test_benchmark.py index cd30d7e9..5d2fb4cf 100644 --- a/cli/benchmarks/installed/test_benchmark.py +++ b/cli/benchmarks/installed/test_benchmark.py @@ -332,6 +332,7 @@ def refresh_evidence(output: Path) -> None: "bun": "bun-cli", "cargo": "rust-cli", "windowsProcessHostCargo": "windows-process-host", + "agentsSdkPython": "agents-sdk-python", } for inventory_name, inventory in dependency_value.get("inventories", {}).items(): group = groups.get(inventory_name) @@ -426,6 +427,20 @@ def refresh_evidence(output: Path) -> None: ), "utf-8", ) + if isinstance(release.get('agentsSdk'), dict): + sdk = release['agentsSdk'] + sbom_path = output / 'sbom.cdx.json'; sbom = json.loads(sbom_path.read_text()) + sbom['components'].append({'type': 'file', 'name': 'prose-agents-sdk', + 'hashes': [{'alg': 'SHA-256', 'content': sdk['sha256']}]}) + sbom_path.write_text(json.dumps(sbom)) + provenance_path = output / 'provenance.json'; provenance = json.loads(provenance_path.read_text()) + definition = provenance['predicate']['buildDefinition'] + definition['externalParameters'] = {'agentsSdk': sdk} + definition['resolvedDependencies'] += [ + {'uri': 'openprose:agents-sdk-helper', 'digest': {'sha256': sdk['sha256']}}, + {'uri': 'openprose:agents-sdk-build-receipt', 'digest': {'sha256': sdk['receiptSha256']}}, + {'uri': 'openprose:agents-sdk-python-lock', 'digest': {'sha256': sdk['dependencyLockSha256']}}] + provenance_path.write_text(json.dumps(provenance)) paths = sorted(path for path in output.iterdir() if path.name != "SHA256SUMS") (output / "SHA256SUMS").write_text( "".join(f"{digest(path.read_bytes())} {path.name}\n" for path in paths), @@ -688,6 +703,27 @@ def release_platform_manifest(members): members[index] = (name, canonical(package), mode) rewrite_tar(platform_package, release_platform_manifest) + import sys + ci_path = str(ROOT / 'cli/ci') + if ci_path not in sys.path: + sys.path.insert(0, ci_path) + from test_assemble_kernel_rc import sdk_fixture + sdk, sdk_members = sdk_fixture(platform_value) + manifest['agentsSdk'] = sdk + for artifact in manifest['artifacts']: + if artifact['kind'] in ('standalone-archive', 'npm-platform'): + def append_sdk(members): + cli_name = next(n for n, d, m in members if n.endswith('/prose') or n.endswith('/prose.exe')) + prefix = cli_name.rsplit('/', 1)[0] + '/' + members.extend((prefix + n, d, m) for n, d, m in sdk_members) + rewrite_tar(output / artifact['path'], append_sdk) + dependency_path = output / 'dependency-evidence.json'; dependency = json.loads(dependency_path.read_text()) + dependency['sources'].append({'path': 'harnesses/agents-sdk/requirements-build.txt', 'byteLength': 1, 'sha256': sdk['dependencyLockSha256']}) + dependency['inventories']['agentsSdkPython'] = { + 'lockfileVersion': 1, 'scopeBasis': 'hash-locked-four-platform-wheel-candidates', + 'packages': [{'name': 'fixture', 'version': '1.0.0', 'source': 'pypi:wheel-sha256:' + 'e'*64, + 'scopes': ['frozen-sdk-build'], 'integrity': {'status': 'declared', 'algorithm': 'sha256', 'digest': 'e'*64}}]} + dependency_path.write_text(json.dumps(dependency)) manifest_path.write_text(json.dumps(manifest, sort_keys=True), "utf-8") refresh_evidence(output) return output @@ -758,6 +794,14 @@ def rewrite_tar(path: Path, mutate) -> None: class InstalledPackageBenchmarkTests(unittest.TestCase): + def test_release_cannot_omit_sdk_identity_after_rehashing(self): + with tempfile.TemporaryDirectory() as temporary: + output = make_release_package_output(Path(temporary)) + path = output / 'release-manifest.json'; manifest = json.loads(path.read_text()) + manifest.pop('agentsSdk'); path.write_text(json.dumps(manifest)); refresh_evidence(output) + with self.assertRaisesRegex(BENCHMARK.BenchmarkError, 'SDK identity'): + BENCHMARK.verify_package_output(output, purpose='release-invariants') + def test_package_verification_purpose_separates_mock_and_release_invariants( self, ) -> None: diff --git a/cli/ci/assemble_kernel_rc.py b/cli/ci/assemble_kernel_rc.py index 3f293512..92b39728 100644 --- a/cli/ci/assemble_kernel_rc.py +++ b/cli/ci/assemble_kernel_rc.py @@ -52,6 +52,10 @@ def assemble(roots, output, evidence, live_smoke=None): inventory[name] = (actual, record) artifact_names = {a['path'] for a in manifest['artifacts']} pub.require(set(custody.CHECK_PATHS).issubset(report['evidence']), 'Required structured evidence is missing') + custody.validate_sdk_archives(manifest, lambda name: pub.archive_members(package / name)) + for relative in custody.SDK_PROBES: + pub.require(relative in report['evidence'], 'Missing installed SDK evidence') + pub.require(pub.read_json(root / relative) == (custody.SDK_TOOL_TEST if 'sdk-tools-' in relative else custody.SDK_IMPORT_TEST), 'Installed SDK probe differs') native_hashes = {} launcher_hash = None for artifact in manifest['artifacts']: diff --git a/cli/ci/build_agents_sdk.py b/cli/ci/build_agents_sdk.py new file mode 100644 index 00000000..b93b6081 --- /dev/null +++ b/cli/ci/build_agents_sdk.py @@ -0,0 +1,249 @@ +#!/usr/bin/env python3 +"""Freeze the package-owned SDK helper without provider calls or ambient credentials.""" +from __future__ import annotations +import argparse +import hashlib +import importlib.metadata as metadata +import json +import os +from pathlib import Path +import platform +import re +import shutil +import subprocess +import sys +import tempfile + +ROOT = Path(__file__).resolve().parents[2] +LOCK = ROOT / 'harnesses/agents-sdk/requirements-build.txt' +SOURCE = ROOT / 'harnesses/agents-sdk/run.py' +NAME = 'prose-agents-sdk' +RECEIPT = 'agents-sdk-build.json' +NOTICES = 'AGENTS-SDK-NOTICES.txt' +MAX_BYTES = 256 * 1024 * 1024 +TOOL_SELF_TEST_SOURCE = r''' +import asyncio +import json +from pathlib import Path +import socket +import tempfile +from unittest.mock import AsyncMock, patch +import prose_sdk_runtime as runtime + +class Writer: + def write(self, data): + assert data.startswith(b'GET /fixture HTTP/1.1\r\n') + async def drain(self): pass + def close(self): pass + async def wait_closed(self): pass + +async def check(): + with tempfile.TemporaryDirectory(prefix='sdk-tool-self-test-') as directory: + root = Path(directory) + env = {'PATH': '/usr/bin:/bin'} + result = await runtime.shell('printf fixture > effect; cat effect', directory, 2, env) + assert result['exit_code'] == 0 and result['stdout'] == 'fixture' + assert (root / 'effect').read_text() == 'fixture' + result = await runtime.shell('head -c 100000 /dev/zero', directory, 2, env, output_limit=4096) + assert len(result['stdout']) == 4096 and result['stdout_truncated'] + task = asyncio.create_task(runtime.shell('(sleep .2; touch late) & wait', directory, 2, env)) + await asyncio.sleep(.04) + task.cancel() + try: + await task + raise AssertionError('shell cancellation did not cancel') + except asyncio.CancelledError: + pass + await asyncio.sleep(.25) + assert not (root / 'late').exists() + reader = asyncio.StreamReader() + reader.feed_data(b'HTTP/1.1 200 OK\r\nContent-Length: 7\r\nContent-Type: text/plain\r\n\r\nfixture') + reader.feed_eof() + loop = asyncio.get_running_loop() + addresses = [(socket.AF_INET, socket.SOCK_STREAM, 6, '', ('93.184.216.34', 443))] + with patch.object(loop, 'getaddrinfo', AsyncMock(return_value=addresses)), patch.object(asyncio, 'open_connection', AsyncMock(return_value=(reader, Writer()))): + retrieved = await runtime.retrieve_public('https://example.invalid/fixture', 2) + assert retrieved['status'] == 200 and retrieved['content'] == 'fixture' and not retrieved['truncated'] + truncated = asyncio.StreamReader() + truncated.feed_data(b'HTTP/1.1 200 OK\r\nContent-Length: 20\r\n\r\nshort') + truncated.feed_eof() + with patch.object(loop, 'getaddrinfo', AsyncMock(return_value=addresses)), patch.object(asyncio, 'open_connection', AsyncMock(return_value=(truncated, Writer()))): + try: + await runtime.retrieve_public('https://example.invalid/fixture', 2) + raise AssertionError('truncated HTTP content was accepted') + except (ValueError, asyncio.IncompleteReadError): + pass + return {'schema': 'openprose.sdk-packaged-tools-self-test/1', 'shellEffects': True, + 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, + 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} + +def main(): + print(json.dumps(asyncio.run(check()), sort_keys=True)) +''' + + +def require(condition, message): + if not condition: + raise ValueError(message) + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def lock_packages(path=LOCK): + rows = path.read_text('utf-8').replace('\\\n', '').splitlines() + result = [] + for row in rows: + row = row.strip() + if not row or row.startswith('#'): + continue + match = re.fullmatch(r'([A-Za-z0-9_.-]+)==([A-Za-z0-9_.+-]+)((?:\s+--hash=sha256:[0-9a-f]{64})+)', row) + require(match is not None, 'SDK lock must contain only exact hash-locked requirements') + name, version, hashes = match.groups() + result.append({'name': name.lower().replace('_', '-'), 'version': version, + 'wheelSha256': sorted(set(re.findall(r'sha256:([0-9a-f]{64})', hashes)))}) + require(len(result) > 0 and len({r['name'] for r in result}) == len(result), 'Empty or duplicate SDK lock') + return result + + +def environment(output, ambient): + env = {k: ambient[k] for k in ('PATH', 'DEVELOPER_DIR', 'SDKROOT', 'SYSTEMROOT') if k in ambient} + home = output / 'home'; home.mkdir() + tmp = output / 'tmp'; tmp.mkdir() + env.update(HOME=str(home), TMPDIR=str(tmp), LANG='C.UTF-8', PYTHONHASHSEED='0', + PYTHONNOUSERSITE='1', PYTHONDONTWRITEBYTECODE='1', SOURCE_DATE_EPOCH='0', + PYINSTALLER_CONFIG_DIR=str(output / 'pyinstaller-cache')) + return env + + +def installed_inventory(packages): + notices = ['OpenProse packaged Agents SDK helper: third-party notices', + 'Python ' + platform.python_version() + ' (PSF License); https://docs.python.org/3/license.html', ''] + import builtins + builtins.license._Printer__setup() + notices += builtins.license._Printer__lines + for package in packages: + dist = metadata.distribution(package['name']) + require(dist.version == package['version'], 'Installed SDK distribution differs from lock: ' + package['name']) + notices += [package['name'] + ' ' + package['version'], + dist.metadata.get('License-Expression') or dist.metadata.get('License') or 'License: see included distribution license files'] + for entry in sorted(dist.files or [], key=str): + if ('license' in str(entry).lower() or 'copying' in str(entry).lower()) and '.dist-info/' in str(entry): + path = dist.locate_file(entry) + if path.is_file() and path.stat().st_size <= 1024 * 1024: + notices += ['--- ' + str(entry) + ' ---', path.read_text('utf-8', errors='replace')] + notices += [''] + return '\n'.join(notices).encode('utf-8') + + +def run(command, *, env, cwd, log, timeout=900): + with log.open('wb') as stream: + completed = subprocess.run([str(x) for x in command], env=env, cwd=cwd, stdin=subprocess.DEVNULL, + stdout=stream, stderr=subprocess.STDOUT, timeout=timeout) + require(completed.returncode == 0, 'SDK build command failed; inspect ' + str(log)) + + +def build(output, *, epoch=0, codesign_identity=None): + require(platform.python_version() == '3.10.20', 'SDK release builder requires Python 3.10.20') + require(sys.platform in ('darwin', 'linux') and platform.machine() in ('arm64', 'aarch64', 'x86_64'), 'SDK builder requires native supported POSIX host') + if codesign_identity is not None: + require(sys.platform == 'darwin' and re.fullmatch(r'Developer ID Application: [^\r\n]+ \([A-Z0-9]{10}\)', codesign_identity), 'SDK signing requires a macOS Developer ID Application identity') + require(not output.exists() and not output.is_symlink(), 'SDK output must be fresh') + require(epoch >= 0, 'Source date epoch must be non-negative') + output.parent.mkdir(parents=True, exist_ok=True) + require(shutil.disk_usage(output.parent).free >= 512 * 1024 * 1024, + 'SDK freeze requires at least 512 MiB free scratch space') + packages = lock_packages() + notices = installed_inventory(packages) + output.mkdir(parents=True) + env = environment(output, os.environ); env['SOURCE_DATE_EPOCH'] = str(epoch) + (output / NOTICES).write_bytes(notices) + (output / 'sdk_tool_selftest.py').write_text(TOOL_SELF_TEST_SOURCE) + # A frozen module cannot run python -c; this explicit self-test entrypoint is bundled + # separately and exercises imported SDK modules/certificate resources without API calls. + entry = output / 'sdk-entry.py' + entry.write_text("import sys\nif '--packaged-self-test' in sys.argv:\n" + " import json, ssl, pathlib, certifi, agents, openai, pydantic, jiter\n" + " from importlib.metadata import version\n" + " ssl.create_default_context(cafile=certifi.where())\n" + " print(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':version('openai-agents'),'openai':version('openai'),'certificates':pathlib.Path(certifi.where()).is_file(),'modelCalls':0},sort_keys=True))\n" + "elif '--packaged-tool-self-test' in sys.argv:\n" + " import sdk_tool_selftest\n sdk_tool_selftest.main()\n" + "elif '--packaged-library-test' in sys.argv:\n" + " import json, pathlib, re\n" + " versions=[]; count=0\n" + " for path in pathlib.Path(sys._MEIPASS).rglob('*'):\n" + " if path.is_file() and not path.is_symlink() and path.stat().st_size<=268435456:\n" + " with path.open('rb') as stream:\n" + " if stream.read(4)!=b'\\x7fELF': continue\n" + " data=stream.read()\n" + " count+=1\n" + " versions.extend(tuple(int(x) for x in v.split(b'.')) for v in re.findall(rb'GLIBC_([0-9]+\\.[0-9]+(?:\\.[0-9]+)?)',data))\n" + " print(json.dumps({'schema':'openprose.sdk-packaged-libraries/1','elfCount':count,'requiredGlibcMaximum':'.'.join(str(x) for x in max(versions)) if versions else None,'modelCalls':0},sort_keys=True))\n" + "else:\n import runpy\n runpy.run_module('prose_sdk_runtime',run_name='__main__')\n") + # Snapshot the concurrently maintained runtime before freezing. Receipt binds exact bytes. + source_copy = output / 'prose_sdk_runtime.py'; source_copy.write_bytes(SOURCE.read_bytes()) + source_sha = digest(source_copy) + args = [sys.executable, '-m', 'PyInstaller', '--clean', '--noconfirm', '--onefile', '--noupx', + '--name', NAME, '--distpath', output / 'dist', '--workpath', output / 'work', + '--specpath', output, '--paths', output, '--hidden-import', 'prose_sdk_runtime', + '--collect-all', 'agents', '--collect-all', 'openai', '--collect-all', 'certifi', + '--copy-metadata', 'openai-agents', '--copy-metadata', 'openai', + '--add-data', str(output / NOTICES) + os.pathsep + '.', entry] + if codesign_identity is not None: + args.extend(['--codesign-identity', codesign_identity]) + for package in packages: + args.extend(['--copy-metadata', package['name']]) + run(args, env=env, cwd=output, log=output / 'freeze.log') + helper = output / NAME; shutil.copyfile(output / 'dist' / NAME, helper); helper.chmod(0o755) + require(0 < helper.stat().st_size <= MAX_BYTES, 'Frozen SDK helper exceeds size limit') + # Disposable build caches duplicate the frozen payload and need not accompany receipts. + for directory in ('dist', 'work', 'pyinstaller-cache'): + shutil.rmtree(output / directory) + run([helper, '--packaged-self-test'], env=env, cwd=output, log=output / 'self-test.json', timeout=30) + self_test = json.loads((output / 'self-test.json').read_text()) + require(self_test == {'schema': 'openprose.sdk-packaged-self-test/1', 'openaiAgents': '0.22.2', 'openai': '3.13.0', 'certificates': True, 'modelCalls': 0}, 'Frozen SDK self-test mismatch') + run([helper, '--packaged-tool-self-test'], env=env, cwd=output, log=output / 'tool-self-test.json', timeout=30) + tool_self_test = json.loads((output / 'tool-self-test.json').read_text()) + require(tool_self_test == {'schema': 'openprose.sdk-packaged-tools-self-test/1', 'shellEffects': True, + 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, + 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False}, 'Frozen SDK tool self-test mismatch') + run([helper, '--version'], env=env, cwd=output, log=output / 'version.txt', timeout=5) + require((output / 'version.txt').read_text().strip() == 'prose-agents-sdk 0.1.0', 'Frozen SDK version mismatch') + signing = 'not-applicable' + linux_libraries = 'not-applicable' + if sys.platform == 'linux': + run([helper, '--packaged-library-test'], env=env, cwd=output, log=output / 'libraries.json', timeout=30) + linux_libraries = json.loads((output / 'libraries.json').read_text()) + require(linux_libraries.get('elfCount', 0) > 0 and linux_libraries.get('requiredGlibcMaximum') is not None, + 'Missing frozen Linux library inspection') + maximum = tuple(int(x) for x in linux_libraries['requiredGlibcMaximum'].split('.')) + require(maximum <= (2, 34), 'Frozen SDK libraries require glibc newer than 2.34') + if sys.platform == 'darwin': + run(['/usr/bin/codesign', '--verify', '--deep', '--strict', helper], env=env, cwd=output, log=output / 'codesign.log', timeout=30) + signing = 'developer-id-embedded' if codesign_identity is not None else 'ad-hoc-integrity-only' + receipt = {'schema': 'openprose.agents-sdk-build/1', 'helper': {'path': NAME, 'sha256': digest(helper), 'byteLength': helper.stat().st_size}, + 'python': platform.python_version(), 'pyinstaller': metadata.version('pyinstaller'), + 'platform': sys.platform, 'architecture': platform.machine(), 'sourceDateEpoch': epoch, + 'sources': {'harnesses/agents-sdk/run.py': source_sha, 'harnesses/agents-sdk/requirements-build.txt': digest(LOCK)}, + 'dependencies': packages, 'notices': {'path': NOTICES, 'sha256': digest(output / NOTICES), 'byteLength': len(notices)}, + 'selfTest': self_test, 'signing': signing, 'modelCalls': 0, + 'embeddedSigning': {'identity': codesign_identity, 'verification': 'pyinstaller-inner-binaries-and-frozen-self-tests'} if codesign_identity is not None else 'ad-hoc-integrity-only', + 'toolSelfTest': tool_self_test, + 'linuxLibraries': linux_libraries, 'pythonExecutableSha256': digest(Path(sys.executable).resolve()), + 'authority': 'local-build-and-imports-only', 'publicationAuthorized': False} + (output / RECEIPT).write_text(json.dumps(receipt, indent=2, sort_keys=True) + '\n') + return receipt + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--out', type=Path, required=True) + parser.add_argument('--codesign-identity', help='macOS Developer ID identity for every embedded binary; requires prepared signing keychain') + parser.add_argument('--source-date-epoch', type=int, default=0) + args = parser.parse_args() + try: + print(json.dumps(build(args.out.absolute(), epoch=args.source_date_epoch, codesign_identity=args.codesign_identity), sort_keys=True)) + except (ValueError, OSError, subprocess.SubprocessError, metadata.PackageNotFoundError) as error: + parser.exit(2, str(error) + '\n') diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index d0922292..439283ba 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -72,12 +72,12 @@ def prepare_macos_binary(binary, env, logs): cwd=ROOT, log=logs / 'rust-ad-hoc-verify.log', timeout=60) -def extract_binary(archive, output): +def extract_binary(archive, output, *, require_sdk=False): """Extract only one regular executable; reject unsafe archive metadata first.""" with tarfile.open(archive) as source: members = source.getmembers() require(len(members) <= 100, 'Too many archive members') - names = set(); binaries = []; total = 0 + names = set(); binaries = []; sdk = []; total = 0 for member in members: name = PurePosixPath(member.name) require(not name.is_absolute() and '..' not in name.parts and '\\' not in member.name, @@ -87,11 +87,22 @@ def extract_binary(archive, output): require(0 <= member.size <= MAX_BYTES and total <= MAX_BYTES, 'Archive exceeds size limit') if name.name == 'prose': binaries.append(member) + if name.name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + sdk.append(member) require(len(binaries) == 1 and binaries[0].mode & 0o111, 'Missing unique executable') + if require_sdk: + require({Path(m.name).name for m in sdk} == {'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'} and len(sdk) == 3, + 'Missing unique packaged SDK runtime/receipt/notices') + require(all(Path(m.name).parent == Path(binaries[0].name).parent for m in sdk), 'SDK members must be CLI siblings') output.parent.mkdir(parents=True, exist_ok=False) with source.extractfile(binaries[0]) as src, output.open('xb') as dst: shutil.copyfileobj(src, dst) output.chmod(0o755) + for member in sdk: + target = output.parent / Path(member.name).name + with source.extractfile(member) as src, target.open('xb') as dst: + shutil.copyfileobj(src, dst) + target.chmod(0o755 if target.name == 'prose-agents-sdk' else 0o644) return output @@ -116,7 +127,7 @@ def verified_artifacts(package): return manifest -def build(version, output): +def build(version, output, *, agents_sdk_python=None): require(RC.fullmatch(version) is not None, 'Use an exact X.Y.Z-rc.N version') require(not output.exists() and not output.is_symlink(), 'Output must be fresh') revision = subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip() @@ -129,6 +140,9 @@ def build(version, output): env.update(OPENPROSE_BUILD_COMMIT=revision, OPENPROSE_BUILD_VERSION=version, OPENPROSE_REQUIRE_RELEASE_IMAGE='1', CARGO_TARGET_DIR=str(output / 'cargo-target')) binaries = output / 'binaries'; binaries.mkdir() + sdk_output = output / 'agents-sdk' + command([agents_sdk_python or sys.executable, ROOT / 'cli/ci/build_agents_sdk.py', '--out', sdk_output, + '--source-date-epoch', epoch], env=env, cwd=ROOT, log=logs / 'build-sdk.log') command(['bun', '--no-env-file', 'scripts/image-bundle.ts', 'build', '--require-release-eligible', '--outfile', binaries / 'prose-bun'], env=env, cwd=ROOT / 'cli/bun', log=logs / 'build-bun.log') command(['cargo', 'build', '--manifest-path', ROOT / 'cli/rust/Cargo.toml', '--release', '--locked', @@ -147,6 +161,7 @@ def check(binary, runner, label, node=None): args = [sys.executable, ROOT / 'cli/ci/package_local.py', '--mode', 'kernel-rc', '--publication-platforms', 'posix-four', '--version', version, '--source-revision', revision, '--source-date-epoch', epoch, '--rust-binary', binaries / 'prose-rust', '--bun-binary', binaries / 'prose-bun', + '--agents-sdk-build', sdk_output, '--image-manifest', ROOT / 'cli/shared/image/echo-v0/manifest.json', '--out', package] if sys.platform.startswith('linux'): args += ['--readelf', executable_tool('readelf', env)] @@ -155,8 +170,12 @@ def check(binary, runner, label, node=None): for item in manifest['artifacts']: if item['kind'] == 'standalone-archive': runner = item['implementation'] - binary = extract_binary(package / item['path'], output / 'installed' / runner / 'prose') + binary = extract_binary(package / item['path'], output / 'installed' / runner / 'prose', require_sdk=True) check(binary, runner, 'installed-' + runner) + command([binary.parent / 'prose-agents-sdk', '--packaged-self-test'], env=env, cwd=output, + log=logs / ('installed-sdk-' + runner + '.json'), timeout=30) + command([binary.parent / 'prose-agents-sdk', '--packaged-tool-self-test'], env=env, cwd=output, + log=logs / ('installed-sdk-tools-' + runner + '.json'), timeout=30) prefix = output / 'npm-prefix' from npm_alias_install import install as install_alias meta = next(package / a['path'] for a in manifest['artifacts'] if a['kind'] == 'npm-meta') @@ -166,6 +185,14 @@ def check(binary, runner, label, node=None): (logs / 'npm-alias-install.json').write_text(json.dumps(alias_install, sort_keys=True) + '\n') node = shutil.which('node', path=env.get('PATH')); require(node, 'Node is required for npm launcher') check(prefix / 'bin/prose', 'bun', 'installed-npm', node) + installed_helpers = list((prefix / 'lib/node_modules').rglob('prose-agents-sdk')) + require(len(installed_helpers) == 1 and installed_helpers[0].is_file() and not installed_helpers[0].is_symlink(), + 'npm installation requires one package-owned SDK helper') + require(digest(installed_helpers[0]) == manifest['agentsSdk']['sha256'], 'Installed npm SDK helper differs') + command([installed_helpers[0], '--packaged-self-test'], env=env, cwd=output, + log=logs / 'installed-sdk-npm.json', timeout=30) + command([installed_helpers[0], '--packaged-tool-self-test'], env=env, cwd=output, + log=logs / 'installed-sdk-tools-npm.json', timeout=30) report = {'schema': 'openprose.kernel-rc-build/1', 'version': version, 'sourceRevision': revision, 'platform': manifest['platform'], 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'signing': 'unsigned', 'modelCalls': 0, 'kernelFetches': 0, @@ -181,8 +208,9 @@ def check(binary, runner, label, node=None): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('--version', required=True) parser.add_argument('--out', type=Path, required=True) + parser.add_argument('--agents-sdk-python', type=Path, help='Separate hash-locked SDK build interpreter (defaults to current interpreter)') args = parser.parse_args() try: - print(json.dumps(build(args.version, args.out.absolute()), sort_keys=True)) + print(json.dumps(build(args.version, args.out.absolute(), agents_sdk_python=args.agents_sdk_python), sort_keys=True)) except (ValueError, OSError, subprocess.SubprocessError) as error: parser.exit(2, str(error) + '\n') diff --git a/cli/ci/build_local.py b/cli/ci/build_local.py index a4c4ab9d..457142dd 100644 --- a/cli/ci/build_local.py +++ b/cli/ci/build_local.py @@ -607,6 +607,7 @@ def _build( executor: Executor = execute, package_purpose: str = ORDINARY_PACKAGE_PURPOSE, internal_packager: Callable[[Sequence[str]], None] | None = None, + agents_sdk_build: Path | None = None, ) -> dict[str, object]: if package_purpose not in PACKAGE_PURPOSES: raise LocalBuildError("package purpose is unsupported") @@ -619,6 +620,22 @@ def _build( if install_dir is not None and os.path.lexists(install_dir): raise LocalBuildError("--install-dir must not already exist") environment = clean_environment(ambient) + sdk_members = [] + if agents_sdk_build is not None: + sdk_snapshots = owned_root / "sdk-inputs" + sdk_snapshots.mkdir(mode=0o700) + sdk_readelf = None + if sys.platform.startswith("linux"): + discovered = shutil.which("readelf", path=environment.get("PATH")) + if discovered is None: + raise LocalBuildError("Local SDK installation requires readelf") + sdk_readelf = Path(discovered).resolve(strict=True) + try: + sdk_members, _sdk_record = package_local.snapshot_sdk( + agents_sdk_build, sdk_snapshots, package_local.current_platform_id(), "development", sdk_readelf + ) + except package_local.PackageError as error: + raise LocalBuildError(f"Local SDK input verification failed: {error}") from error windows_host_record: dict[str, object] | None = None owned_windows_host: Path | None = None if IS_WINDOWS: @@ -950,6 +967,8 @@ def _build( raise LocalBuildError("Local Linux packaging requires readelf") exact_readelf = Path(discovered_readelf).resolve(strict=True) packaging_argv.extend(("--readelf", str(exact_readelf))) + if agents_sdk_build is not None: + packaging_argv.extend(("--agents-sdk-build", str(agents_sdk_build))) if package_purpose == ORDINARY_PACKAGE_PURPOSE: run_checked( "Local packaging", @@ -1007,6 +1026,10 @@ def _build( install_record: dict[str, object] | None = None if install_dir is not None: install_dir.mkdir(parents=True) + for name, data, mode in sdk_members: + destination = install_dir / name + destination.write_bytes(data) + destination.chmod(mode) install_record = {} for candidate in candidates: verify_candidate_snapshot(candidate) @@ -1057,6 +1080,7 @@ def build( executor: Executor = execute, package_purpose: str = ORDINARY_PACKAGE_PURPOSE, internal_packager: Callable[[Sequence[str]], None] | None = None, + agents_sdk_build: Path | None = None, ) -> dict[str, object]: with tempfile.TemporaryDirectory(prefix="openprose-local-build-") as directory: return _build( @@ -1069,6 +1093,7 @@ def build( executor=executor, package_purpose=package_purpose, internal_packager=internal_packager, + agents_sdk_build=agents_sdk_build, ) @@ -1078,6 +1103,7 @@ def parser() -> argparse.ArgumentParser: result.add_argument("--smoke", action="store_true") result.add_argument("--package", type=Path) result.add_argument("--install-dir", type=Path) + result.add_argument("--agents-sdk-build", type=Path) result.add_argument("--json", action="store_true") return result @@ -1091,6 +1117,7 @@ def main(argv: Sequence[str] | None = None) -> int: package=args.package, install_dir=args.install_dir, ambient=os.environ, + agents_sdk_build=args.agents_sdk_build, ) except (LocalBuildError, OSError, KeyError, json.JSONDecodeError) as error: print(f"local-build: {error}", file=sys.stderr) diff --git a/cli/ci/check_workflows.py b/cli/ci/check_workflows.py index dfc91211..ae112ab3 100644 --- a/cli/ci/check_workflows.py +++ b/cli/ci/check_workflows.py @@ -294,7 +294,7 @@ def require(condition: bool, message: str) -> None: ), "cli-kernel-rc.yml": ( "Build and verify fresh standalone and npm installations", - 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc"', + 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3"', ), }[name] require( diff --git a/cli/ci/dependency_evidence.py b/cli/ci/dependency_evidence.py index bed951a6..6129b7e7 100644 --- a/cli/ci/dependency_evidence.py +++ b/cli/ci/dependency_evidence.py @@ -857,9 +857,38 @@ def build_report(root: Path) -> dict[str, Any]: cargo, cargo_sources = cargo_inventory(root) windows_host, windows_host_sources = windows_host_cargo_inventory(root) bun, bun_sources = bun_inventory(root) + python_lock = root / "harnesses/agents-sdk/requirements-build.txt" + python_sources = [] + python_inventory = None + if python_lock.exists(): + lock_bytes = safe_read_under(root, python_lock, MAX_LOCK_BYTES) + text = decode_utf8(lock_bytes, "SDK Python lock").replace("\\\n", "") + packages = [] + names = set() + for row in text.splitlines(): + row = row.strip() + if not row or row.startswith("#"): + continue + match = re.fullmatch(r"([A-Za-z0-9_.-]+)==([A-Za-z0-9_.+-]+)((?:\s+--hash=sha256:[0-9a-f]{64})+)", row) + if match is None: + fail("SOURCE_MALFORMED", "SDK Python requirements must be exact and hash locked") + name, version, hashes = match.groups() + name = name.lower().replace("_", "-") + if name in names: + fail("SOURCE_DUPLICATE", "duplicate SDK Python requirement") + names.add(name) + for digest in sorted(set(re.findall(r"sha256:([0-9a-f]{64})", hashes))): + packages.append({"name": name, "version": version, + "source": "pypi:wheel-sha256:" + digest, + "scopes": ["frozen-sdk-build"], + "integrity": {"status": "declared", "algorithm": "sha256", "digest": digest}}) + if not packages: + fail("SOURCE_MALFORMED", "SDK Python requirements are empty") + python_inventory = {"scopeBasis": "hash-locked-four-platform-wheel-candidates", "packages": packages} + python_sources = [(python_lock, lock_bytes)] source_values = [ source_record(root, path, data) - for path, data in cargo_sources + windows_host_sources + bun_sources + for path, data in cargo_sources + windows_host_sources + bun_sources + python_sources ] source_values.sort(key=lambda source: source["path"]) blockers = [ @@ -881,6 +910,7 @@ def build_report(root: Path) -> dict[str, Any]: "cargo": cargo, "bun": bun, "windowsProcessHostCargo": windows_host, + **({"agentsSdkPython": python_inventory} if python_inventory is not None else {}), }, "authority": { "licenses": { diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 1d581733..48c61577 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -63,6 +63,15 @@ def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, executables = [m for m in members if Path(m.name).name == 'prose' and m.isfile()] if len(executables) != 1 or len(Path(executables[0].name).parts) != 2: raise ValueError('Archive requires one prose executable under one package root') + sdk_record = manifest.get('agentsSdk') + if isinstance(sdk_record, dict): + for filename, digest_key in (('prose-agents-sdk', 'sha256'), ('agents-sdk-build.json', 'receiptSha256'), ('AGENTS-SDK-NOTICES.txt', 'noticesSha256')): + selected_members = [m for m in members if Path(m.name).name == filename and m.isfile()] + if len(selected_members) != 1 or Path(selected_members[0].name).parent != Path(executables[0].name).parent: + raise ValueError('Archive requires unique SDK helper/receipt/notices siblings') + member = selected_members[0] + if not 0 < member.size <= 256 * 1024 * 1024 or hashlib.sha256(contents.extractfile(member).read()).hexdigest() != sdk_record.get(digest_key): + raise ValueError('Archive SDK member differs from package identity') selected[implementation] = item if set(selected) != {'bun', 'rust'}: raise ValueError('Both native implementations are required') @@ -75,7 +84,10 @@ def formula(version: str, implementation: str, archive: Path, sha256: str) -> st ' desc "Development-only Prose packaging rehearsal"', ' homepage "https://prose.md"', f' version "{version}"', ' license "MIT"', f' url "{archive.as_uri()}"', f' sha256 "{sha256}"', '', - ' def install', ' bin.install "prose"', ' end', '', + ' def install', ' bin.install "prose"', + ' bin.install "prose-agents-sdk" if File.exist?("prose-agents-sdk")', + ' pkgshare.install "agents-sdk-build.json", "AGENTS-SDK-NOTICES.txt" if File.exist?("agents-sdk-build.json")', + ' end', '', ' test do', f' assert_equal "prose {version} ({implementation})", shell_output("#{{bin}}/prose --version").strip', ' end', 'end', '', ]) @@ -223,6 +235,13 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s digest = hashlib.sha256(active.read_bytes()).hexdigest() if digest != verified['packageIdentity'][f'{implementation}BinarySha256']: raise ValueError('Homebrew executable differs from the verified candidate') + sdk_record = manifest.get('agentsSdk') + if isinstance(sdk_record, dict): + helper = active.resolve().parent / 'prose-agents-sdk' + if not helper.is_file() or hashlib.sha256(helper.read_bytes()).hexdigest() != sdk_record.get('sha256'): + raise ValueError('Homebrew SDK helper differs from the verified package') + command(f'sdk-self-test-{implementation}', [str(helper), '--packaged-self-test']) + command(f'sdk-tools-self-test-{implementation}', [str(helper), '--packaged-tool-self-test']) command(f'unlink-{implementation}', [brew, 'unlink', name]) command('link-bun', [brew, 'link', f'{TAP}/prose-bun']) original = hashlib.sha256(active.read_bytes()).hexdigest() diff --git a/cli/ci/kernel_rc_evidence.py b/cli/ci/kernel_rc_evidence.py index 698c1df0..adba8745 100644 --- a/cli/ci/kernel_rc_evidence.py +++ b/cli/ci/kernel_rc_evidence.py @@ -24,6 +24,94 @@ def asset_name(platform, relative, artifacts): return platform + '-logs-' + path.name +SDK_NAMES = ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt') +SDK_PROBES = tuple('logs/installed-sdk' + suffix + '-' + runner + '.json' + for runner in ('bun', 'rust', 'npm') for suffix in ('', '-tools')) +SDK_IMPORT_TEST = {'schema': 'openprose.sdk-packaged-self-test/1', 'openaiAgents': '0.22.2', + 'openai': '3.13.0', 'certificates': True, 'modelCalls': 0} +SDK_TOOL_TEST = {'schema': 'openprose.sdk-packaged-tools-self-test/1', 'shellEffects': True, + 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, + 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} + + +def validate_sdk_members(manifest, members, prefix, platform): + """Bind the frozen helper and its source/dependency receipt without executing it.""" + import json + import publication as pub + sdk = manifest.get('agentsSdk') + keys = {'path', 'byteLength', 'sha256', 'receiptSha256', 'noticesSha256', 'python', + 'pyinstaller', 'version', 'discovery', 'selfTest', 'toolSelfTest', 'dependencyLockSha256'} + require(isinstance(sdk, dict) and set(sdk) == keys, 'Production package lacks closed SDK identity') + require(sdk['path'] == SDK_NAMES[0] and sdk['python'] == '3.10.20' + and sdk['pyinstaller'] == '6.22.3' and sdk['version'] == '0.1.0' + and sdk['discovery'] == 'canonical-cli-sibling' + and sdk['selfTest'] == SDK_IMPORT_TEST and sdk['toolSelfTest'] == SDK_TOOL_TEST, + 'Packaged SDK policy or tool qualification differs') + require(all(prefix + name in members for name in SDK_NAMES), 'Missing packaged SDK siblings') + helper, encoded, notices = (members[prefix + name] for name in SDK_NAMES) + require(type(sdk['byteLength']) is int and 0 < sdk['byteLength'] == len(helper) + and hashlib.sha256(helper).hexdigest() == sdk['sha256'] + and hashlib.sha256(encoded).hexdigest() == sdk['receiptSha256'] + and hashlib.sha256(notices).hexdigest() == sdk['noticesSha256'], + 'Packaged SDK sibling bytes differ from manifest') + require(0 < len(encoded) <= 2 * 1024 * 1024 and 0 < len(notices) <= 8 * 1024 * 1024, + 'Packaged SDK receipt/notices exceed bounds') + receipt = json.loads(encoded, object_pairs_hook=pub.object_pairs) + require(isinstance(receipt, dict), 'SDK build receipt must be an object') + os_name, architecture = platform.split('-')[:2] + architecture = {'darwin-arm64': 'arm64', 'darwin-x64': 'x86_64', 'linux-arm64-gnu': 'aarch64', 'linux-x64-gnu': 'x86_64'}.get(platform, architecture) + require(receipt.get('schema') == 'openprose.agents-sdk-build/1' + and receipt.get('platform') == os_name and receipt.get('architecture') == architecture + and receipt.get('python') == sdk['python'] and receipt.get('pyinstaller') == sdk['pyinstaller'] + and receipt.get('helper') == {k: sdk[k] for k in ('path', 'byteLength', 'sha256')} + and receipt.get('selfTest') == SDK_IMPORT_TEST and receipt.get('toolSelfTest') == SDK_TOOL_TEST + and type(receipt.get('modelCalls')) is int and receipt['modelCalls'] == 0 + and receipt.get('publicationAuthorized') is False, + 'Packaged SDK build receipt identity differs') + sources = receipt.get('sources') + require(isinstance(sources, dict) and set(sources) == { + 'harnesses/agents-sdk/run.py', 'harnesses/agents-sdk/requirements-build.txt'} + and all(isinstance(v, str) and re.fullmatch(r'[0-9a-f]{64}', v) for v in sources.values()) + and sources['harnesses/agents-sdk/requirements-build.txt'] == sdk['dependencyLockSha256'], + 'Packaged SDK source/lock binding differs') + require(receipt.get('notices') == {'path': SDK_NAMES[2], 'byteLength': len(notices), + 'sha256': sdk['noticesSha256']}, 'SDK notices receipt differs') + dependencies = receipt.get('dependencies') + require(isinstance(dependencies, list) and dependencies, 'SDK dependency receipt is missing') + seen = set() + for package in dependencies: + require(isinstance(package, dict) and set(package) == {'name', 'version', 'wheelSha256'} + and isinstance(package['name'], str) and package['name'] not in seen + and isinstance(package['version'], str) and isinstance(package['wheelSha256'], list) + and package['wheelSha256'] and all(isinstance(h, str) and re.fullmatch(r'[0-9a-f]{64}', h) for h in package['wheelSha256']), + 'SDK dependency receipt is malformed') + seen.add(package['name']) + if os_name == 'linux': + libraries = receipt.get('linuxLibraries', {}) + maximum = libraries.get('requiredGlibcMaximum') if isinstance(libraries, dict) else None + require(isinstance(maximum, str) and re.fullmatch(r'[0-9]+\.[0-9]+(?:\.[0-9]+)?', maximum) + and tuple(map(int, maximum.split('.'))) + (0,) * (3 - len(maximum.split('.'))) <= (2, 34, 0), 'SDK Linux glibc floor differs') + else: + require(receipt.get('linuxLibraries') == 'not-applicable', 'SDK platform library receipt differs') + return sdk + + +def validate_sdk_archives(manifest, archive_reader): + """All three platform payloads must contain the identical bound helper.""" + platform = manifest['platform'] + observed = [] + for artifact in manifest['artifacts']: + if artifact['kind'] not in ('standalone-archive', 'npm-platform'): + continue + members = archive_reader(artifact['path']) + cli_names = [name for name in members if name.endswith('/prose') or name.endswith('/prose.exe')] + require(len(cli_names) == 1, 'Expected one packaged CLI for SDK sibling binding') + prefix = cli_names[0].rsplit('/', 1)[0] + '/' + observed.append(validate_sdk_members(manifest, members, prefix, platform)) + require(len(observed) == 3, 'Three platform SDK payloads are required') + return manifest['agentsSdk'] + + def validate_native(report, manifest, checks, final_hashes, launcher_hash): platform = report.get('platform') source, version = report.get('sourceRevision'), report.get('version') @@ -90,7 +178,7 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): require(manifest.get('embeddedDiagnosticImage') == preflight.get('embeddedDiagnosticImage') and manifest.get('kernelPolicy') == preflight.get('kernelPolicy'), 'Native manifest kernel policy differs from qualification') artifact_names = {a['path'] for a in manifest.get('artifacts', [])} evidence = report.get('evidence', {}) - require(isinstance(evidence, dict) and set(CHECK_PATHS).union({'package/release-manifest.json'}).issubset(evidence), 'Required structured evidence is missing') + require(isinstance(evidence, dict) and set(CHECK_PATHS + SDK_PROBES).union({'package/release-manifest.json'}).issubset(evidence), 'Required structured evidence is missing') seen = set() for relative, record in evidence.items(): name = asset_name(platform, relative, artifact_names) @@ -110,6 +198,12 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): require(item['sha256'] == artifact['sha256'] and item['size'] == artifact['byteLength'] and item['kind'] == kind and item['implementation'] == artifact['implementation'] and item['platform'] == (artifact['platform'] or 'all'), 'Native artifact differs from final reviewed inventory') + require(manifest.get('platform') == platform, 'Native manifest identity mismatch') + validate_sdk_archives(manifest, lambda name: pub.archive_members(root / name)) + for relative in SDK_PROBES: + probe = pub.read_json(root / asset_name(platform, relative, artifact_names)) + require(probe == (SDK_TOOL_TEST if 'sdk-tools-' in relative else SDK_IMPORT_TEST), + 'Installed SDK probe differs: ' + relative) meta = next((a for a in manifest['artifacts'] if a['kind'] == 'npm-meta'), None) require(meta is not None, 'Missing root npm package') members = pub.archive_members(root / meta['path']) diff --git a/cli/ci/package_local.py b/cli/ci/package_local.py index 0417941c..466b1b07 100644 --- a/cli/ci/package_local.py +++ b/cli/ci/package_local.py @@ -49,6 +49,9 @@ RUST_LOCK = CLI / "rust" / "Cargo.lock" BUN_LOCK = CLI / "bun" / "bun.lock" WINDOWS_HOST_NAME = "openprose-windows-process-host.exe" +SDK_NAME = "prose-agents-sdk" +SDK_RECEIPT = "agents-sdk-build.json" +SDK_NOTICES = "AGENTS-SDK-NOTICES.txt" LINUX_MINIMUM_GLIBC = "2.34" LINUX_EXECUTION_EVIDENCE = "ubuntu-22.04-only" NODE_MINIMUM = "22.22.3" @@ -73,6 +76,7 @@ COMMAND_CLEANUP_SECONDS = 3.0 NOT_APPLICABLE_INTEGRITY_REASONS = {"local-source-package", "workspace-package"} DEPENDENCY_SOURCE_PATHS = { + "harnesses/agents-sdk/requirements-build.txt", "cli/bun/bun.lock", "cli/bun/package.json", "cli/platform/windows-process-host/Cargo.lock", @@ -1407,6 +1411,28 @@ def standalone_readme( "Inspect the sibling local package evidence: release-manifest.json, " "provenance.json, sbom.cdx.json, and dependency-evidence.json.\n" ) + if mode in {"kernel-rc", "release"}: + first_use = ( + "First run with the packaged Agents SDK default:\n" + " Keep prose and its sibling prose-agents-sdk together. No Python installation is required.\n" + " Supply OPENAI_API_KEY in your environment; requests use your OpenAI API account.\n" + " The fresh-install model default is gpt-6.1-sol. Saved explicit harness choices remain effective.\n" + f" {prose_command} cli doctor\n" + f' {prose_command} run "$PWD/{example_path}"\n\n' + " Running contacts the selected provider and may incur charges.\n" + ) + else: + first_use = ( + "First run with Codex 0.149.0-alpha.4.1 (install that exact version and " + "complete Codex sign-in before you continue):\n" + f"{PROVIDER_CHARGE_BOUNDARY}\n" + " npm install --global @openai/codex@0.149.0-alpha.4.1\n" + " codex login\n" + f" {prose_command} cli harness list\n" + f" {prose_command} cli harness use codex\n" + f" {prose_command} cli doctor\n" + f' {prose_command} run "$PWD/{example_path}"\n\n' + ) return ( f"OpenProse CLI {version} ({implementation}, {platform_identifier})\n" f"Release channel: {channel}\n\n" @@ -1420,15 +1446,7 @@ def standalone_readme( " macOS: awk -v name=\"$ASSET\" '$2 == name' SHA256SUMS | shasum -a 256 -c -\n\n" "Install:\n" f"{installation}\n" - "First run with Codex 0.149.0-alpha.4.1 (install that exact version and " - "complete Codex sign-in before you continue):\n" - f"{PROVIDER_CHARGE_BOUNDARY}\n" - " npm install --global @openai/codex@0.149.0-alpha.4.1\n" - " codex login\n" - f" {prose_command} cli harness list\n" - f" {prose_command} cli harness use codex\n" - f" {prose_command} cli doctor\n" - f' {prose_command} run "$PWD/{example_path}"\n\n' + f"{first_use}" "Upgrade:\n" f"{upgrade_guidance}" "Repair this same exact version:\n" @@ -1531,7 +1549,11 @@ def npm_readme( "Do not install a platform payload version as the top-level CLI. " "Lifecycle scripts are unnecessary. npm installs Bun; Rust has separate standalone downloads.\n\n" "A CLI version pins executable bytes; normal startup resolves the latest published kernel per run. " - "Harness installation and authentication are separate. No mock execution is admitted in release binaries.\n\n" + "The Agents SDK helper is included and selected by default; no Python installation is required. " + "Supply OPENAI_API_KEY in your environment to use your OpenAI API account with the fresh-install gpt-6.1-sol default. " + "Saved explicit harness choices remain effective. No mock execution is admitted in release binaries.\n\n" + f'First run: `"$HOME/.local/openprose-cli-{version}/bin/prose" run "$HOME/.local/openprose-cli-{version}/lib/node_modules/@openprose/prose-cli/{HELLO_EXAMPLE_MEMBER}"`. ' + "Running contacts the selected provider and may incur charges.\n\n" f'Uninstall: `npm uninstall --global --prefix "$HOME/.local/openprose-cli-{version}" @openprose/prose-cli`.\n' "For offline installation, populate an isolated npm cache with the exact root and matching payload " "versions from a trusted registry first; then install the root with --offline. " @@ -1621,6 +1643,26 @@ def npm_readme( if gatekeeper_guidance else "" ) + first_run = ( + "## First run with Codex 0.149.0-alpha.4.1\n\n" + "Install that exact version and complete Codex sign-in before you continue.\n\n" + f"{PROVIDER_CHARGE_BOUNDARY}\n\n" + " npm install --global @openai/codex@0.149.0-alpha.4.1\n" + " codex login\n" + f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli harness list\n' + f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli harness use codex\n' + f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli doctor\n' + f' "$HOME/.local/openprose-cli-{version}/bin/prose" run "$HOME/.local/openprose-cli-{version}/lib/node_modules/@openprose/prose-cli/examples/hello.prose.md"\n\n' + ) + if mode == "release": + first_run = ( + "## First run with the packaged Agents SDK default\n\n" + "The platform package includes the SDK helper and Python runtime. " + "Supply OPENAI_API_KEY in your environment to use your OpenAI API account with gpt-6.1-sol. " + "Saved explicit harness choices remain effective. Running contacts the selected provider and may incur charges.\n\n" + f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli doctor\n' + f' "$HOME/.local/openprose-cli-{version}/bin/prose" run "$HOME/.local/openprose-cli-{version}/lib/node_modules/@openprose/prose-cli/examples/hello.prose.md"\n\n' + ) return ( f"# OpenProse CLI {version}\n\n" f"Release channel: {channel}.\n\n" @@ -1663,15 +1705,7 @@ def npm_readme( "The x64 packages use Bun's baseline CPU runtime variants " "(`bun-darwin-x64-baseline` and `bun-linux-x64-baseline`); ARM64 packages " "use the native `bun-darwin-arm64` and `bun-linux-arm64` targets.\n\n" - "## First run with Codex 0.149.0-alpha.4.1\n\n" - "Install that exact version and complete Codex sign-in before you continue.\n\n" - f"{PROVIDER_CHARGE_BOUNDARY}\n\n" - " npm install --global @openai/codex@0.149.0-alpha.4.1\n" - " codex login\n" - f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli harness list\n' - f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli harness use codex\n' - f' "$HOME/.local/openprose-cli-{version}/bin/prose" cli doctor\n' - f' "$HOME/.local/openprose-cli-{version}/bin/prose" run "$HOME/.local/openprose-cli-{version}/lib/node_modules/@openprose/prose-cli/examples/hello.prose.md"\n\n' + f"{first_run}" "The meta package contains the exact source contract at " "`examples/hello.prose.md`. " f"{runtime_guidance}" @@ -1715,6 +1749,7 @@ def standalone_archive( mode: str, hello_example: bytes, windows_host: bytes | None = None, + sdk_members: list[tuple[str, bytes, int]] | None = None, ) -> Path: root = f"openprose-prose-cli-{implementation}-{version}-{platform_identifier}" executable_name = ( @@ -1744,6 +1779,7 @@ def standalone_archive( ] if windows_host is not None: members.append((f"{root}/{WINDOWS_HOST_NAME}", windows_host, 0o755)) + members.extend((f"{root}/{name}", data, mode) for name, data, mode in (sdk_members or [])) tar_gz(destination, members, epoch) return destination @@ -1882,6 +1918,7 @@ def npm_packages( *, package_name: str = "@openprose/prose-cli", publication_platforms: str | None = None, + sdk_members: list[tuple[str, bytes, int]] | None = None, ) -> tuple[Path, Path]: if package_name not in {"@openprose/prose-cli", "@openprose/prose"}: raise PackageError("npm identity must be explicitly supported") @@ -1963,6 +2000,7 @@ def named_manifest(value: dict[str, Any]) -> dict[str, Any]: 0o644, ), (f"package/bin/{executable_name}", bun_binary, 0o755), + *((f"package/bin/{name}", data, mode) for name, data, mode in (sdk_members or [])), *( [(f"package/bin/{WINDOWS_HOST_NAME}", windows_host, 0o755)] if windows_host is not None @@ -2285,6 +2323,7 @@ def dependency_evidence() -> tuple[bytes, dict[str, Any]]: "bun", "cargo", "windowsProcessHostCargo", + "agentsSdkPython", }: raise PackageError("dependency evidence component inventory is not closed") expected_components = { @@ -2322,8 +2361,11 @@ def dependency_sbom_components(report: dict[str, Any]) -> list[dict[str, Any]]: "cargo": "rust-cli", "windowsProcessHostCargo": "windows-process-host", "bun": "bun-cli", + "agentsSdkPython": "agents-sdk-python", } - for inventory_name in ("cargo", "windowsProcessHostCargo", "bun"): + for inventory_name in ("cargo", "windowsProcessHostCargo", "bun", "agentsSdkPython"): + if inventory_name not in report["inventories"]: + continue component_name = component_names[inventory_name] for package in report["inventories"][inventory_name]["packages"]: if not isinstance(package, dict) or set(package) != { @@ -2410,6 +2452,72 @@ def dependency_sbom_components(report: dict[str, Any]) -> list[dict[str, Any]]: return components +def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, + mode: str, readelf: Path | None) -> tuple[list[tuple[str, bytes, int]], dict[str, Any]]: + """Bind the package-owned helper to its native build receipt and exact source.""" + if directory.is_symlink() or not directory.is_dir(): + raise PackageError("SDK build directory must be a real directory") + receipt_bytes = read_static_asset(directory / SDK_RECEIPT, "SDK receipt", 4 * 1024 * 1024) + receipt = strict_json_object(receipt_bytes, "SDK receipt") + if receipt.get("schema") != "openprose.agents-sdk-build/1" or receipt.get("modelCalls") != 0: + raise PackageError("Unsupported SDK build receipt") + expected_os = "darwin" if platform_identifier.startswith("darwin-") else "linux" + expected_arch = {"darwin-arm64": "arm64", "darwin-x64": "x86_64", "linux-arm64-gnu": "aarch64", "linux-x64-gnu": "x86_64"}.get(platform_identifier) + if receipt.get("platform") != expected_os or receipt.get("architecture") != expected_arch: + raise PackageError("SDK helper native target differs from package") + if receipt.get("python") != "3.10.20" or receipt.get("pyinstaller") != "6.22.3": + raise PackageError("SDK builder identity differs from pinned toolchain") + expected_sources = {path: sha256_file(ROOT / path) for path in + ("harnesses/agents-sdk/run.py", "harnesses/agents-sdk/requirements-build.txt")} + if receipt.get("sources") != expected_sources: + raise PackageError("SDK build sources differ from current package source") + helper, length, digest = snapshot_binary(directory / SDK_NAME, snapshots / SDK_NAME, "Agents SDK") + if receipt.get("helper") != {"path": SDK_NAME, "sha256": digest, "byteLength": length}: + raise PackageError("SDK helper differs from build receipt") + notices = read_static_asset(directory / SDK_NOTICES, "SDK notices", 16 * 1024 * 1024) + if receipt.get("notices") != {"path": SDK_NOTICES, "sha256": sha256_bytes(notices), "byteLength": len(notices)}: + raise PackageError("SDK notices differ from build receipt") + if expected_os == "darwin" and mode in {"kernel-rc", "release", "alpha"}: + verify_darwin_code_signature(helper, "Agents SDK") + if readelf is not None: + inspect_linux_glibc(helper, "Agents SDK", readelf) + environment = {"PATH": "/usr/bin:/bin", "PYTHONDONTWRITEBYTECODE": "1"} + completed = run_bounded([str(helper), "--packaged-self-test"], cwd=snapshots, + environment=environment, timeout_seconds=30, label="packaged SDK imports") + if completed.returncode != 0 or completed.stderr: + raise PackageError("Packaged SDK import self-test did not settle cleanly") + self_test = strict_json_object(completed.stdout, "SDK self-test") + expected_test = {"schema": "openprose.sdk-packaged-self-test/1", "openaiAgents": "0.22.2", "openai": "3.13.0", "certificates": True, "modelCalls": 0} + if self_test != expected_test or receipt.get("selfTest") != expected_test: + raise PackageError("Packaged SDK import self-test differs") + expected_tools = {'schema': 'openprose.sdk-packaged-tools-self-test/1', 'shellEffects': True, + 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, + 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} + tool_test = run_bounded([str(helper), "--packaged-tool-self-test"], cwd=snapshots, + environment=environment, timeout_seconds=30, label="packaged SDK tools") + if tool_test.returncode != 0 or tool_test.stderr or strict_json_object(tool_test.stdout, "SDK tool self-test") != expected_tools or receipt.get('toolSelfTest') != expected_tools: + raise PackageError("Packaged SDK tool self-test differs") + if expected_os == "linux": + inspected = run_bounded([str(helper), "--packaged-library-test"], cwd=snapshots, + environment=environment, timeout_seconds=30, label="packaged SDK libraries") + if inspected.returncode != 0 or inspected.stderr: + raise PackageError("Packaged SDK Linux library inspection failed") + libraries = strict_json_object(inspected.stdout, "SDK Linux libraries") + maximum = libraries.get("requiredGlibcMaximum") + if libraries != receipt.get("linuxLibraries") or not isinstance(maximum, str) or re.fullmatch(r"[0-9]+\.[0-9]+(?:\.[0-9]+)?", maximum) is None: + raise PackageError("Packaged SDK Linux library receipt differs") + if version_tuple(maximum) > version_tuple(LINUX_MINIMUM_GLIBC): + raise PackageError("Packaged SDK libraries require newer than the admitted glibc floor") + helper_bytes = verified_snapshot_bytes(helper, length, digest, "Agents SDK") + return [(SDK_NAME, helper_bytes, 0o755), (SDK_RECEIPT, receipt_bytes, 0o644), (SDK_NOTICES, notices, 0o644)], { + "path": SDK_NAME, "byteLength": length, "sha256": digest, + "receiptSha256": sha256_bytes(receipt_bytes), "noticesSha256": sha256_bytes(notices), + "python": "3.10.20", "pyinstaller": "6.22.3", "version": "0.1.0", + "discovery": "canonical-cli-sibling", "selfTest": self_test, + "toolSelfTest": expected_tools, + "dependencyLockSha256": expected_sources["harnesses/agents-sdk/requirements-build.txt"]} + + def build( args: argparse.Namespace, *, internal_package_purpose: str = "ordinary-development" ) -> None: @@ -2428,6 +2536,9 @@ def build( ) if args.source_date_epoch < 0: raise PackageError("--source-date-epoch must be non-negative") + sdk_directory = getattr(args, "agents_sdk_build", None) + if args.mode in {"kernel-rc", "release"} and sdk_directory is None: + raise PackageError("production packaging requires --agents-sdk-build DIRECTORY") image, image_manifest_sha256 = read_image_manifest(args.image_manifest) image_record = image_identity(image, image_manifest_sha256) publication_platforms = getattr(args, "publication_platforms", None) @@ -2620,6 +2731,10 @@ def build( else "not-applicable" ) + sdk_members, sdk_record = (snapshot_sdk(sdk_directory, snapshots, platform_identifier, + args.mode, readelf_record[0] if readelf_record else None) + if sdk_directory is not None else ([], "not-packaged-development-fixture")) + staging = temporary_root / "result" staging.mkdir() rust_archive = standalone_archive( @@ -2633,6 +2748,7 @@ def build( args.mode, hello_example, windows_host_bytes, + sdk_members, ) bun_archive = standalone_archive( staging, @@ -2645,6 +2761,7 @@ def build( args.mode, hello_example, windows_host_bytes, + sdk_members, ) meta_package, platform_package = npm_packages( staging, @@ -2660,6 +2777,7 @@ def build( windows_host_bytes, package_name=getattr(args, "npm_package_name", "@openprose/prose-cli"), publication_platforms=publication_platforms, + sdk_members=sdk_members, ) artifacts = [ artifact_record( @@ -2711,6 +2829,7 @@ def build( "kernelPolicy": dict(PUBLISHED_KERNEL_POLICY), } if args.mode == "kernel-rc" else {"image": image_record}), "windowsProcessHost": windows_host_record, + "agentsSdk": sdk_record, "windowsJobObjectReleaseAdmission": False, "toolchains": toolchains, "lockfiles": { @@ -2783,6 +2902,10 @@ def build( if windows_host_bytes is not None else [] ) + + ([{"type": "file", "name": SDK_NAME, "version": "0.1.0", + "hashes": [{"alg": "SHA-256", "content": sdk_record["sha256"]}], + "properties": [{"name": "openprose:kind", "value": "packaged-agents-sdk-helper"}]}] + if isinstance(sdk_record, dict) else []) + dependency_sbom_components(dependency_report), "properties": [ { @@ -2823,6 +2946,7 @@ def build( "bunRuntime": release_manifest["bunRuntime"], "linuxRuntime": linux_runtime, "windowsProcessHost": windows_host_record, + "agentsSdk": sdk_record, "windowsJobObjectReleaseAdmission": False, }, "resolvedDependencies": [ @@ -2843,6 +2967,10 @@ def build( "digest": {"sha256": dependency_record["sha256"]}, }, ] + + ([{"uri": "openprose:agents-sdk-helper", "digest": {"sha256": sdk_record["sha256"]}}, + {"uri": "openprose:agents-sdk-build-receipt", "digest": {"sha256": sdk_record["receiptSha256"]}}, + {"uri": "openprose:agents-sdk-python-lock", "digest": {"sha256": sdk_record["dependencyLockSha256"]}}] + if isinstance(sdk_record, dict) else []) + ( [ { @@ -2894,6 +3022,7 @@ def parser() -> argparse.ArgumentParser: result.add_argument("--rust-binary", type=Path, required=True) result.add_argument("--bun-binary", type=Path, required=True) result.add_argument("--windows-process-host", type=Path) + result.add_argument("--agents-sdk-build", type=Path, help="Verified native frozen SDK build directory; required for production packaging") result.add_argument("--readelf", type=Path) result.add_argument("--image-manifest", type=Path, required=True) result.add_argument("--canonical-profile", type=Path) diff --git a/cli/ci/publication.py b/cli/ci/publication.py index 35840150..81c29892 100644 --- a/cli/ci/publication.py +++ b/cli/ci/publication.py @@ -145,6 +145,23 @@ def verify_local(plan, root): if item['kind'] not in ('npm', 'standalone'): continue members = archive_members(root / item['name']) + if item['kind'] in ('standalone', 'npm') and item['platform'] != 'all': + platform = item['platform'] + manifest_name = platform + '-release-manifest.json' + require(any(a['name'] == manifest_name and a['kind'] == 'evidence' for a in plan['artifacts']), 'Production SDK manifest is not retained') + manifest = read_json(root / manifest_name) + require(manifest.get('mode') in ('release', 'kernel-rc') and manifest.get('platform') == platform + and manifest.get('version') == plan['version'] + and manifest.get('source') == {'revision': plan['source'], 'verification': 'matched-product-doctor'}, + 'Production SDK source identity differs') + cli_names = [name for name in members if name.endswith('/prose')] + require(len(cli_names) == 1, 'Expected one CLI for SDK binding') + prefix = cli_names[0].rsplit('/', 1)[0] + '/' + from kernel_rc_evidence import validate_sdk_members + sdk = validate_sdk_members(manifest, members, prefix, platform) + old = binary_hashes.get(('agents-sdk', platform)) + require(old is None or old == sdk['sha256'], 'SDK helper differs between installation routes') + binary_hashes[('agents-sdk', platform)] = sdk['sha256'] if item['kind'] == 'standalone': binaries = [b for n, b in members.items() if n.endswith('/prose')] require(len(binaries) == 1, 'Expected one standalone binary') @@ -236,15 +253,28 @@ def verify_macos(plan, root, key, key_id, issuer): receipt = read_json(root / ref['receipt']) require(receipt.get('schema') == 'openprose.macos-signing/1' and receipt.get('teamId') == ref['teamId'], 'Wrong signing identity') require(receipt.get('notarization', {}).get('sha256') == digest(root / ref['zip']), 'Wrong notarization ZIP') - for implementation in ('bun', 'rust'): + require(set(receipt.get('binaries', {})) == {'bun', 'rust', 'agents-sdk'}, 'Signing receipt lacks production SDK helper') + for implementation in ('bun', 'rust', 'agents-sdk'): require(receipt.get('binaries', {}).get(implementation, {}).get('signedSha256') == binaries[(implementation, platform)], 'Signed binary differs from packaged binary') + manifest = read_json(root / (platform + '-release-manifest.json')) + sdk_archive = next(a for a in plan['artifacts'] if a['platform'] == platform and a['kind'] == 'standalone') + members = archive_members(root / sdk_archive['name']) + sdk_receipts = [data for name, data in members.items() if name.endswith('/agents-sdk-build.json')] + require(len(sdk_receipts) == 1, 'Missing signed SDK build receipt') + sdk_build = json.loads(sdk_receipts[0], object_pairs_hook=object_pairs) + require(sdk_build.get('signing') == 'developer-id-notarized' + and sdk_build.get('signingReceiptSha256') == digest(root / ref['receipt']) + and sdk_build.get('embeddedSigning') == {'identity': receipt['identity'], 'verification': 'pyinstaller-inner-binaries-and-frozen-self-tests'} + and sdk_build.get('unsignedHelper', {}).get('sha256') == receipt['binaries']['agents-sdk']['inputSha256'] + and sdk_build.get('helper', {}).get('sha256') == binaries[('agents-sdk', platform)], + 'SDK signing stages do not bind notarized helper') with tempfile.TemporaryDirectory(prefix='prose-signature-check-') as temporary: directory = Path(temporary) (directory / 'receipt.json').write_bytes((root / ref['receipt']).read_bytes()) (directory / 'notarization.zip').write_bytes((root / ref['zip']).read_bytes()) with zipfile.ZipFile(directory / 'notarization.zip') as archive: - require(set(archive.namelist()) == {'prose-bun', 'prose-rust'} and len(archive.infolist()) == 2, 'Unexpected notarization archive') - for name in ('prose-bun', 'prose-rust'): + require(set(archive.namelist()) == {'prose-bun', 'prose-rust', 'prose-agents-sdk'} and len(archive.infolist()) == 3, 'Unexpected notarization archive') + for name in ('prose-bun', 'prose-rust', 'prose-agents-sdk'): require(0 < archive.getinfo(name).file_size <= MAX_BYTES, 'Oversized signed executable') (directory / name).write_bytes(archive.read(name)) (directory / name).chmod(0o755) diff --git a/cli/ci/release_package_admission.py b/cli/ci/release_package_admission.py index a920cd7a..831f2236 100644 --- a/cli/ci/release_package_admission.py +++ b/cli/ci/release_package_admission.py @@ -258,6 +258,7 @@ def validate_identity_inputs( "native-manifest.json", f"prose-rust{suffix}", f"prose-bun{suffix}", + "prose-agents-sdk", "agents-sdk-build.json", "AGENTS-SDK-NOTICES.txt", } if target_id == "win-x64": expected_files.add("openprose-windows-process-host.exe") @@ -758,6 +759,18 @@ def native_lineage( "nativeSha256": files[path]["sha256"], "packagedBinarySha256": packaged, } + sdk = payloads['release'].get('agentsSdk') + if not isinstance(sdk, dict): + fail('NATIVE_LINEAGE_MISMATCH', 'Release lacks packaged SDK identity') + npm = payloads['npmPackage'] + encoded = payloads['context']['encoded'][npm['platformName']] + import io, tarfile + with tarfile.open(fileobj=io.BytesIO(encoded), mode='r:gz') as archive: + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + member = archive.getmember('package/bin/' + name) + data = archive.extractfile(member).read() + if files.get(name) != {'byteLength': len(data), 'sha256': sha256(data)}: + fail('NATIVE_LINEAGE_MISMATCH', 'Packaged SDK differs from native artifact: ' + name) windows = native["windowsProcessHost"] if target_id == "win-x64": if windows != payloads["release"]["windowsProcessHost"]: @@ -779,6 +792,7 @@ def native_lineage( "nativeManifestSha256": native["nativeManifestSha256"], "files": files, "products": products, + "agentsSdk": sdk, "windowsProcessHost": windows if windows is not None else "not-applicable", } diff --git a/cli/ci/reproducible_release.py b/cli/ci/reproducible_release.py index 651df44e..e6486277 100644 --- a/cli/ci/reproducible_release.py +++ b/cli/ci/reproducible_release.py @@ -687,6 +687,31 @@ def _validate_package_checksums( ) + +def _archive_members_for_sdk(data, name): + members = {}; portable = set(); total = 0 + try: + with tarfile.open(fileobj=io.BytesIO(data), mode='r:gz') as archive: + for member in archive: + if (len(members) >= MAX_ARCHIVE_MEMBERS or not _safe_archive_name(member.name) + or member.name.casefold() in portable or not member.isfile() + or not 0 <= member.size <= MAX_FILE_BYTES): + raise _error('UNSAFE_ARCHIVE', 'Unsafe SDK archive member: ' + name) + portable.add(member.name.casefold()); total += member.size + if total > MAX_ARCHIVE_BYTES: + raise _error('UNSAFE_ARCHIVE', 'SDK archive exceeds expanded bound: ' + name) + stream = archive.extractfile(member) + if stream is None: + raise _error('UNSAFE_ARCHIVE', 'SDK archive member cannot be read: ' + name) + content = stream.read(MAX_FILE_BYTES + 1) + if len(content) != member.size: + raise _error('UNSAFE_ARCHIVE', 'SDK archive member length differs: ' + name) + members[member.name] = content + except tarfile.TarError: + raise _error('UNSAFE_ARCHIVE', 'Malformed SDK archive: ' + name) from None + return members + + def _capture_image_and_channel( snapshot: dict[str, bytes], *, @@ -697,7 +722,7 @@ def _capture_image_and_channel( manifest = _parse_package_json( snapshot["release-manifest.json"], "release-manifest.json" ) - if set(manifest) != RELEASE_MANIFEST_KEYS: + if set(manifest) not in (RELEASE_MANIFEST_KEYS, RELEASE_MANIFEST_KEYS | {"agentsSdk"}): raise _error( "MALFORMED_PACKAGE_METADATA", "release-manifest.json top-level keys are not exact", @@ -713,6 +738,12 @@ def _capture_image_and_channel( "PACKAGE_IDENTITY_MISMATCH", "capture accepts only alpha or release package modes", ) + if mode == "release": + import kernel_rc_evidence as custody + try: + custody.validate_sdk_archives(manifest, lambda name: _archive_members_for_sdk(snapshot[name], name)) + except (ValueError, KeyError, TypeError) as error: + raise _error("PACKAGE_IDENTITY_MISMATCH", str(error)) from None for field, supplied in ( ("version", version), ("platform", platform_id), diff --git a/cli/ci/sign_macos.py b/cli/ci/sign_macos.py index aae4a61c..120fa7e0 100644 --- a/cli/ci/sign_macos.py +++ b/cli/ci/sign_macos.py @@ -118,7 +118,7 @@ def verify_existing(output: Path, identity: str, team_id: str, if (receipt["schema"] != "openprose.macos-signing/1" or receipt["identity"] != identity or receipt["teamId"] != team_id or receipt["bunEntitlements"] != BUN_ENTITLEMENTS - or set(receipt["binaries"]) != {"bun", "rust"} + or set(receipt["binaries"]) not in ({"bun", "rust"}, {"bun", "rust", "agents-sdk"}) or receipt["notarization"]["archive"] != "notarization.zip" or receipt["notarization"]["status"] != "Accepted"): raise SigningError("Signing receipt does not match release policy") @@ -136,7 +136,7 @@ def verify_existing(output: Path, identity: str, team_id: str, if archive_hash != receipt["notarization"]["sha256"]: raise SigningError("Notarization archive digest mismatch") with zipfile.ZipFile(archive) as zipped: - if sorted(zipped.namelist()) != ["prose-bun", "prose-rust"]: + if sorted(zipped.namelist()) != sorted(record["file"] for record in receipt["binaries"].values()) or len(zipped.infolist()) != len(receipt["binaries"]): raise SigningError("Notarization archive must contain exactly both binaries") for record in receipt["binaries"].values(): info = zipped.getinfo(record["file"]) @@ -165,7 +165,7 @@ def verify_existing(output: Path, identity: str, team_id: str, def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, notary_key: Path, notary_key_id: str, notary_issuer: str, keychain: Path | None = None, timeout_seconds: int = 900, - *, commands=None) -> dict: + *, commands=None, agents_sdk: Path | None = None) -> dict: if platform.system() != "Darwin": raise SigningError("Developer ID signing requires macOS") if not re.fullmatch(r"[A-Z0-9]{10}", team_id): @@ -181,6 +181,25 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, if type(timeout_seconds) is not int or not 30 <= timeout_seconds <= 3600: raise SigningError("Timeout must be between 30 and 3600 seconds") inputs = {"bun": regular_file(bun), "rust": regular_file(rust)} + sdk_build = None + if agents_sdk is not None: + inputs["agents-sdk"] = regular_file(agents_sdk) + build_path = regular_file(agents_sdk.parent / 'agents-sdk-build.json') + notices = regular_file(agents_sdk.parent / 'AGENTS-SDK-NOTICES.txt') + if build_path.stat().st_size > 2 * 1024 * 1024 or notices.stat().st_size > 8 * 1024 * 1024: + raise SigningError('SDK signing metadata exceeds bounds') + try: + sdk_build = json.loads(build_path.read_text()) + except ValueError as error: + raise SigningError('SDK build receipt is malformed') from error + if not isinstance(sdk_build, dict): + raise SigningError('SDK build receipt must be an object') + if sdk_build.get('embeddedSigning') != {'identity': identity, 'verification': 'pyinstaller-inner-binaries-and-frozen-self-tests'}: + raise SigningError('SDK signing requires same-identity signed embedded binaries') + if sdk_build.get('helper') != {'path': 'prose-agents-sdk', 'byteLength': agents_sdk.stat().st_size, 'sha256': sha256(agents_sdk)}: + raise SigningError('SDK input differs from frozen build receipt') + if sdk_build.get('notices') != {'path': notices.name, 'byteLength': notices.stat().st_size, 'sha256': sha256(notices)}: + raise SigningError('SDK notices differ from frozen build receipt') notary_key = regular_file(notary_key) if keychain is not None: keychain = regular_file(keychain) @@ -210,6 +229,11 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, details = run(["/usr/bin/codesign", "--display", "--verbose=4", str(target)]) signature_details(details, identity, team_id) verify_entitlements(run, target, BUN_ENTITLEMENTS if name == "bun" else {}) + if name == 'agents-sdk': + from kernel_rc_evidence import SDK_IMPORT_TEST, SDK_TOOL_TEST + for flag, expected in (('--packaged-self-test', SDK_IMPORT_TEST), ('--packaged-tool-self-test', SDK_TOOL_TEST)): + if json.loads(run([str(target), flag])) != expected: + raise SigningError('Signed SDK self-test differs from frozen tool qualification') if sha256(source) != original_hash: raise SigningError("Source binary changed during signing") binaries[name] = {"file": target.name, "inputSha256": original_hash, @@ -239,6 +263,10 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, sha256(output / record["file"]) != record["signedSha256"] for record in binaries.values() ): raise SigningError("Signed artifacts changed during notarization") + if sdk_build is not None: + if (json.loads(build_path.read_text()) != sdk_build + or sha256(notices) != sdk_build['notices']['sha256']): + raise SigningError('SDK build metadata changed during signing') # Store a bounded projection, not the raw service log, which can contain # local paths/account data. Native service logs remain available by ID. receipt = {"schema": "openprose.macos-signing/1", "teamId": team_id, @@ -248,12 +276,22 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, "archive": archive.name, "sha256": archive_hash}, "stapled": False} (output / "receipt.json").write_text(json.dumps(receipt, indent=2) + "\n") + if sdk_build is not None: + build = sdk_build + signed = binaries['agents-sdk'] + build['unsignedHelper'] = build['helper'] + build['helper'] = {'path': 'prose-agents-sdk', 'byteLength': (output / signed['file']).stat().st_size, + 'sha256': signed['signedSha256']} + build['signing'] = 'developer-id-notarized' + build['signingReceiptSha256'] = sha256(output / 'receipt.json') + (output / 'agents-sdk-build.json').write_text(json.dumps(build, indent=2) + '\n') + shutil.copyfile(notices, output / notices.name) return receipt def main() -> int: parser = argparse.ArgumentParser(description=__doc__) - for name in ("bun", "rust", "verify-existing"): + for name in ("bun", "rust", "agents-sdk", "verify-existing"): parser.add_argument("--" + name, type=Path) for name in ("output", "notary-key"): parser.add_argument("--" + name, type=Path, required=name == "notary-key") @@ -265,7 +303,7 @@ def main() -> int: args = vars(parser.parse_args()) existing = args.pop("verify_existing") if existing is not None: - if any([args.pop(name) is not None for name in ("bun", "rust", "output")]): + if any([args.pop(name) is not None for name in ("bun", "rust", "agents_sdk", "output")]): raise SigningError("Verification mode cannot accept build inputs") args.pop("keychain") verify_existing(existing, **args) diff --git a/cli/ci/test_assemble_kernel_rc.py b/cli/ci/test_assemble_kernel_rc.py index e2c0230b..af84b28a 100644 --- a/cli/ci/test_assemble_kernel_rc.py +++ b/cli/ci/test_assemble_kernel_rc.py @@ -11,6 +11,31 @@ import kernel_rc_evidence as custody +def sdk_fixture(platform): + """Hermetic bytes with production-shaped custody; never executable release evidence.""" + helper = ('fixture-sdk-' + platform).encode(); notices = b'fixture-notices' + sha = lambda data: hashlib.sha256(data).hexdigest() + sdk = {'path': 'prose-agents-sdk', 'byteLength': len(helper), 'sha256': sha(helper), + 'noticesSha256': sha(notices), 'python': '3.10.20', 'pyinstaller': '6.22.3', 'version': '0.1.0', + 'discovery': 'canonical-cli-sibling', 'selfTest': custody.SDK_IMPORT_TEST, + 'toolSelfTest': custody.SDK_TOOL_TEST, 'dependencyLockSha256': 'e'*64} + os_name, arch = platform.split('-')[:2] + arch = {'darwin-arm64': 'arm64', 'darwin-x64': 'x86_64', 'linux-arm64-gnu': 'aarch64', 'linux-x64-gnu': 'x86_64'}.get(platform, arch) + receipt = {'schema': 'openprose.agents-sdk-build/1', 'platform': os_name, 'architecture': arch, + 'python': sdk['python'], 'pyinstaller': sdk['pyinstaller'], + 'helper': {k:sdk[k] for k in ('path','byteLength','sha256')}, + 'notices': {'path': 'AGENTS-SDK-NOTICES.txt', 'byteLength': len(notices), 'sha256': sha(notices)}, + 'sources': {'harnesses/agents-sdk/run.py': 'f'*64, 'harnesses/agents-sdk/requirements-build.txt': 'e'*64}, + 'dependencies': [{'name': 'fixture', 'version': '1.0.0', 'wheelSha256': ['e'*64]}], + 'selfTest': custody.SDK_IMPORT_TEST, 'toolSelfTest': custody.SDK_TOOL_TEST, + 'modelCalls': 0, 'publicationAuthorized': False, + 'authority': 'hermetic-test-fixture-not-release-evidence', + 'linuxLibraries': {'requiredGlibcMaximum': '2.34'} if os_name == 'linux' else 'not-applicable'} + encoded = json.dumps(receipt).encode(); sdk['receiptSha256'] = sha(encoded) + return sdk, [('prose-agents-sdk', helper, 0o755), ('agents-sdk-build.json', encoded, 0o644), + ('AGENTS-SDK-NOTICES.txt', notices, 0o644)] + + class AssemblyTests(unittest.TestCase): def setUp(self): self.temp = tempfile.TemporaryDirectory() @@ -27,6 +52,7 @@ def setUp(self): output.mkdir(parents=True) self.roots.append(root) artifacts = [] + sdk, sdk_members = sdk_fixture(platform) for implementation in ('bun', 'rust'): name = implementation + '-' + platform + '.tgz' data = (implementation + platform).encode() @@ -34,11 +60,14 @@ def setUp(self): member = tarfile.TarInfo('root/prose') member.size = len(data) archive.addfile(member, io.BytesIO(data)) + for sdk_name, sdk_data, sdk_mode in sdk_members: + item = tarfile.TarInfo('root/' + sdk_name); item.size = len(sdk_data); item.mode = sdk_mode + archive.addfile(item, io.BytesIO(sdk_data)) artifacts.append(self.record(output / name, implementation, 'standalone-archive', platform)) runtime = {'minimumGlibc': '2.34', 'requiredGlibcMaximum': {'rust': '2.34', 'bun': '2.34'}, 'executionEvidence': 'ubuntu-22.04-only'} if platform.startswith('linux-') else 'not-applicable' - meta, native = package.npm_packages(output, ('bun' + platform).encode(), self.version, platform, 0, self.source, self.image, runtime, 'kernel-rc', package.HELLO_EXAMPLE.read_bytes(), publication_platforms='posix-four') + meta, native = package.npm_packages(output, ('bun' + platform).encode(), self.version, platform, 0, self.source, self.image, runtime, 'kernel-rc', package.HELLO_EXAMPLE.read_bytes(), publication_platforms='posix-four', sdk_members=sdk_members) artifacts += [self.record(meta, 'bun', 'npm-meta', None), self.record(native, 'bun', 'npm-platform', platform)] - manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', 'platform': platform, 'version': self.version, 'source': {'revision': self.source, 'verification': 'matched-product-doctor'}, 'releaseEligible': False, 'publicationAuthorized': False, 'buildProfiles': {r: {'profile': 'release', 'testSeamsEnabled': False} for r in ('bun', 'rust')}, 'imageSource': 'published-on-run', 'embeddedDiagnosticImage': {k:v for k,v in self.image.items() if k != 'releaseEligible'}, 'kernelPolicy': package.PUBLISHED_KERNEL_POLICY, 'artifacts': artifacts} + manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', 'platform': platform, 'version': self.version, 'source': {'revision': self.source, 'verification': 'matched-product-doctor'}, 'releaseEligible': False, 'publicationAuthorized': False, 'buildProfiles': {r: {'profile': 'release', 'testSeamsEnabled': False} for r in ('bun', 'rust')}, 'imageSource': 'published-on-run', 'embeddedDiagnosticImage': {k:v for k,v in self.image.items() if k != 'releaseEligible'}, 'kernelPolicy': package.PUBLISHED_KERNEL_POLICY, 'artifacts': artifacts, 'agentsSdk': sdk} (output / 'release-manifest.json').write_text(json.dumps(manifest)) logs = root / 'logs'; logs.mkdir() with tarfile.open(meta) as archive: @@ -49,6 +78,8 @@ def setUp(self): if name == 'installed-npm': check['nodeInterpreterSha256'] = 'd'*64 (logs / (name + '.json')).write_text(json.dumps(check)) + for relative in custody.SDK_PROBES: + (root / relative).write_text(json.dumps(custody.SDK_TOOL_TEST if 'sdk-tools-' in relative else custody.SDK_IMPORT_TEST)) evidence = {str(f.relative_to(root)): {'sha256': p.digest(f), 'byteLength': f.stat().st_size} for directory in (output, logs) for f in directory.iterdir()} report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, 'version': self.version, 'sourceRevision': self.source, 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'qualification': 'offline-install-only', 'publicationAuthorized': False, 'modelCalls': 0, 'kernelFetches': 0, 'checks': [{'name': n, 'status': 'passed'} for n in ('built-bun','built-rust','installed-bun','installed-rust','installed-npm')], 'evidence': evidence} (root / 'build-report.json').write_text(json.dumps(report)) diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py new file mode 100644 index 00000000..a71fccd1 --- /dev/null +++ b/cli/ci/test_build_agents_sdk.py @@ -0,0 +1,58 @@ +import importlib.util +import os +from pathlib import Path +import tempfile +import unittest +from unittest import mock + +import build_agents_sdk as sdk + + +class BuildSdkTests(unittest.TestCase): + def test_lock_is_closed_and_contains_all_runtime_and_build_requirements(self): + packages = sdk.lock_packages() + self.assertEqual(len(packages), 46) + versions = {p['name']: p['version'] for p in packages} + self.assertEqual(versions['openai-agents'], '0.22.2') + self.assertEqual(versions['openai'], '3.13.0') + self.assertEqual(versions['pyinstaller'], '6.22.3') + self.assertEqual(versions['cryptography'], '48.0.1') + self.assertTrue(all(p['wheelSha256'] for p in packages)) + + def test_unpinned_hashless_duplicate_and_extra_options_are_rejected(self): + with tempfile.TemporaryDirectory() as temp: + path = Path(temp) / 'lock' + for text in ('openai>=3\n', 'openai==3.13.0\n', '--index-url https://example.invalid\n', + 'openai==3.13.0 --hash=sha256:' + 'a' * 64 + '\n' + 'openai==3.13.0 --hash=sha256:' + 'b' * 64): + path.write_text(text) + with self.assertRaises(ValueError): + sdk.lock_packages(path) + + def test_build_environment_excludes_credentials_configuration_and_runtime_injection(self): + with tempfile.TemporaryDirectory() as temp: + env = sdk.environment(Path(temp), {'PATH': '/usr/bin:/bin', 'OPENAI_API_KEY': 'fixture-secret', + 'PYTHONPATH': '/untrusted', 'LD_LIBRARY_PATH': '/untrusted', 'HOME': '/real-home'}) + self.assertNotIn('OPENAI_API_KEY', env) + self.assertNotIn('PYTHONPATH', env) + self.assertNotIn('LD_LIBRARY_PATH', env) + self.assertEqual(env['HOME'], str(Path(temp) / 'home')) + + def test_release_builder_rejects_wrong_python_before_writing(self): + with tempfile.TemporaryDirectory() as temp: + path = Path(temp) / 'out' + with mock.patch.object(sdk.platform, 'python_version', return_value='3.11.0'): + with self.assertRaisesRegex(ValueError, 'Python 3.10.20'): + sdk.build(path) + self.assertFalse(path.exists()) + + def test_installed_dependency_mismatch_prevents_freezing(self): + with mock.patch.object(sdk.metadata, 'distribution', return_value=mock.Mock(version='0.0.0')): + with self.assertRaisesRegex(ValueError, 'differs from lock'): + sdk.installed_inventory([{'name': 'openai', 'version': '3.13.0'}]) + + def test_sdk_signing_rejects_non_developer_id_before_freezing(self): + with tempfile.TemporaryDirectory() as temp: + for identity in ('-', 'Apple Development: example', 'Developer ID Application: bad\n(ABCDE12345)'): + with self.assertRaisesRegex(ValueError, 'Developer ID'): + sdk.build(Path(temp) / 'out', codesign_identity=identity) + self.assertFalse((Path(temp) / 'out').exists()) diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index c627840e..0807d63e 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -112,3 +112,22 @@ def test_tampered_artifact_is_rejected(self): if __name__ == '__main__': unittest.main() + +class KernelRCSdkExtractionTests(unittest.TestCase): + archive = KernelRCBuildTests.archive + def test_production_extraction_requires_sdk_siblings_before_writing(self): + with tempfile.TemporaryDirectory() as d: + root = Path(d); archive = root / 'source.tgz' + self.archive(archive, [('release/prose', 'file')]) + with self.assertRaisesRegex(ValueError, 'SDK'): + rc.extract_binary(archive, root / 'install/prose', require_sdk=True) + self.assertFalse((root / 'install').exists()) + + def test_production_extracts_closed_sdk_payload(self): + with tempfile.TemporaryDirectory() as d: + root = Path(d); archive = root / 'source.tgz' + names = ('prose', 'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt') + self.archive(archive, [('release/' + name, 'file') for name in names]) + executable = rc.extract_binary(archive, root / 'install/prose', require_sdk=True) + self.assertEqual({p.name for p in executable.parent.iterdir()}, set(names)) + self.assertTrue(os.access(executable.parent / 'prose-agents-sdk', os.X_OK)) diff --git a/cli/ci/test_check_build_local.py b/cli/ci/test_check_build_local.py index 60956611..5f4f6cff 100644 --- a/cli/ci/test_check_build_local.py +++ b/cli/ci/test_check_build_local.py @@ -9,11 +9,35 @@ from tempfile import TemporaryDirectory import time import unittest +from unittest import mock import build_local class LocalBuildDriverTest(unittest.TestCase): + def test_sdk_install_uses_verified_snapshot_and_passes_build_to_packager(self): + root, _rust, _bun = self.fixture() + sdk = root / 'sdk'; sdk.mkdir() + original = sdk / 'prose-agents-sdk'; original.write_bytes(b'verified-helper') + package = root / 'package-sdk'; install = root / 'installed-sdk' + members = [('prose-agents-sdk', b'verified-helper', 0o755), + ('agents-sdk-build.json', b'{}', 0o644), ('AGENTS-SDK-NOTICES.txt', b'notices', 0o644)] + calls = [] + def execute(argv, _cwd, _environment): + calls.append(list(argv)) + original.write_bytes(b'changed-after-verification') + if 'package_local.py' in ' '.join(argv): + self.write_package_evidence(package) + return subprocess.CompletedProcess(argv, 0, b'', b'') + with mock.patch.object(build_local.package_local, 'snapshot_sdk', return_value=(members, {})) as snapshot: + build_local.build(selection='both', smoke=False, package=package, install_dir=install, + ambient={}, executor=execute, agents_sdk_build=sdk) + self.assertEqual(snapshot.call_args.args[0], sdk) + packaging = next(argv for argv in calls if 'package_local.py' in ' '.join(argv)) + self.assertEqual(packaging[packaging.index('--agents-sdk-build') + 1], str(sdk)) + self.assertEqual((install / 'prose-agents-sdk').read_bytes(), b'verified-helper') + self.assertEqual((install / 'prose-agents-sdk').stat().st_mode & 0o777, 0o755) + def test_clean_environment_removes_case_variant_credentials_and_overrides( self, ) -> None: diff --git a/cli/ci/test_dependency_evidence.py b/cli/ci/test_dependency_evidence.py index 72866a04..e7425a77 100644 --- a/cli/ci/test_dependency_evidence.py +++ b/cli/ci/test_dependency_evidence.py @@ -199,6 +199,7 @@ def test_real_stable_locks_emit_complete_deterministic_inventory(self) -> None: "cli/rust/crates/prose-cli/Cargo.toml", "cli/rust/crates/prose-process-supervisor/Cargo.toml", "cli/rust/crates/prose-runner-core/Cargo.toml", + "harnesses/agents-sdk/requirements-build.txt", ], ) self.assertGreater(len(report["inventories"]["cargo"]["packages"]), 50) @@ -211,6 +212,10 @@ def test_real_stable_locks_emit_complete_deterministic_inventory(self) -> None: len(report["inventories"]["windowsProcessHostCargo"]["packages"]), 14 ) self.assertEqual(len(report["inventories"]["bun"]["packages"]), 13) + python = report["inventories"]["agentsSdkPython"]["packages"] + self.assertEqual(len({p["name"] for p in python}), 46) + self.assertEqual(len(python), 67) + self.assertTrue(all(p["integrity"]["status"] == "declared" for p in python)) for source in report["sources"]: self.assertRegex(source["sha256"], r"^[0-9a-f]{64}$") self.assertGreater(source["byteLength"], 0) @@ -483,3 +488,18 @@ def test_malformed_cli_is_machine_readable_and_has_no_override_switch(self) -> N if __name__ == "__main__": unittest.main() + +class PythonSdkDependencyEvidenceTests(unittest.TestCase): + def test_python_lock_candidates_bind_hashes_and_reject_unpinned_input(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary); make_fixture(root) + lock = root / 'harnesses/agents-sdk/requirements-build.txt'; lock.parent.mkdir(parents=True) + lock.write_text('openai==3.13.0 --hash=sha256:' + SHA256_A + ' --hash=sha256:' + SHA256_B + '\n') + report = EVIDENCE.build_report(root) + packages = report['inventories']['agentsSdkPython']['packages'] + self.assertEqual([p['integrity']['digest'] for p in packages], [SHA256_A, SHA256_B]) + self.assertIn('harnesses/agents-sdk/requirements-build.txt', [s['path'] for s in report['sources']]) + for text in ('openai>=3\n', 'openai==3.13.0\n', '--extra-index-url https://example.invalid\n'): + lock.write_text(text) + with self.assertRaises(EVIDENCE.EvidenceError): + EVIDENCE.build_report(root) diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index f4c4e0bd..c44c588e 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -40,9 +40,28 @@ def test_binds_both_native_archives_and_version(self): self.assertEqual(manifest['version'], '0.15.0-dev.0') rendered = rehearsal.formula(manifest['version'], 'bun', self.package / 'bun.tar.gz', selected['bun']['sha256']) self.assertIn('bin.install "prose"', rendered) + self.assertIn('bin.install "prose-agents-sdk"', rendered) self.assertIn((self.package / 'bun.tar.gz').as_uri(), rendered) self.assertNotIn('pkg.prose.md', rendered) + def test_sdk_archive_identity_is_required_before_homebrew_mutation(self): + sdk = [('prose-agents-sdk', b'helper', 'sha256'), ('agents-sdk-build.json', b'{}', 'receiptSha256'), + ('AGENTS-SDK-NOTICES.txt', b'notices', 'noticesSha256')] + self.manifest['agentsSdk'] = {key: hashlib.sha256(data).hexdigest() for _, data, key in sdk} + with self.assertRaisesRegex(ValueError, 'SDK'): + self.select() + for item in self.manifest['artifacts']: + path = self.package / item['path'] + with tarfile.open(path, 'w:gz') as archive: + for name, data in [('prose', item['implementation'].encode()), *[(n, d) for n, d, _ in sdk]]: + member = tarfile.TarInfo('package/' + name); member.size = len(data) + archive.addfile(member, io.BytesIO(data)) + item.update(byteLength=path.stat().st_size, sha256=hashlib.sha256(path.read_bytes()).hexdigest()) + self.select() + self.manifest['agentsSdk']['sha256'] = '0' * 64 + with self.assertRaisesRegex(ValueError, 'SDK member differs'): + self.select() + def test_changed_bytes_and_size_are_rejected(self): with (self.package / 'bun.tar.gz').open('ab') as stream: stream.write(b'changed') diff --git a/cli/ci/test_kernel_rc_evidence.py b/cli/ci/test_kernel_rc_evidence.py index a28c4659..7072a35d 100644 --- a/cli/ci/test_kernel_rc_evidence.py +++ b/cli/ci/test_kernel_rc_evidence.py @@ -43,6 +43,27 @@ def test_all_structured_logs_retained_and_verified(self): for check in custody.CHECKS: self.assertIn(platform + '-logs-' + check + '.json', names) + def test_production_manifest_cannot_omit_or_rehash_sdk_identity(self): + for mutation in ('omit', 'digest'): + with self.subTest(mutation=mutation): + plan, output, hashes = self.assembled() + manifest = pub.read_json(output / 'darwin-arm64-release-manifest.json') + if mutation == 'omit': + manifest.pop('agentsSdk') + else: + manifest['agentsSdk']['sha256'] = '0'*64 + self.update_bound(plan, output, 'package/release-manifest.json', manifest) + with self.assertRaisesRegex(ValueError, 'SDK'): + self.verify(plan, output, hashes) + import shutil + shutil.rmtree(output) + + def test_rehashed_installed_sdk_probe_cannot_claim_skipped_tools(self): + plan, output, hashes = self.assembled() + self.update_bound(plan, output, 'logs/installed-sdk-tools-bun.json', {}) + with self.assertRaisesRegex(ValueError, 'SDK probe'): + self.verify(plan, output, hashes) + def test_rehashed_passing_log_cannot_describe_other_binary(self): plan, output, hashes = self.assembled() check = pub.read_json(output / 'darwin-arm64-logs-installed-bun.json') diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index b81e17d2..149f54b4 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -2318,6 +2318,34 @@ def write_candidate( path.chmod(0o755) return source + def write_sdk_candidate(self): + """Native provider-free helper fixture; never qualified release evidence.""" + directory = self.root / "sdk-fixture"; directory.mkdir() + helper = directory / PACKAGE_LOCAL.SDK_NAME + expected = {"schema": "openprose.sdk-packaged-self-test/1", "openaiAgents": "0.22.2", "openai": "3.13.0", "certificates": True, "modelCalls": 0} + tools = {'schema': 'openprose.sdk-packaged-tools-self-test/1', 'shellEffects': True, + 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, + 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} + libraries = {"schema": "openprose.sdk-packaged-libraries/1", "elfCount": 1, "requiredGlibcMaximum": "2.34", "modelCalls": 0} + c_source = directory / "fixture.c" + c_source.write_text("#include \n#include \nint main(int argc,char **argv){if(argc!=2)return 2; if(strcmp(argv[1],\"--packaged-self-test\")==0) puts(" + json.dumps(json.dumps(expected)) + "); else if(strcmp(argv[1],\"--packaged-tool-self-test\")==0) puts(" + json.dumps(json.dumps(tools)) + "); else if(strcmp(argv[1],\"--packaged-library-test\")==0) puts(" + json.dumps(json.dumps(libraries)) + "); else if(strcmp(argv[1],\"--version\")==0) puts(\"prose-agents-sdk 0.1.0\"); else return 2; return 0;}") + compiler = shutil.which("cc"); self.assertIsNotNone(compiler) + subprocess.run([compiler, "-O0", "-o", str(helper), str(c_source)], check=True, timeout=30, capture_output=True) + if platform.system() == "Darwin": + subprocess.run(["/usr/bin/codesign", "--force", "--sign", "-", str(helper)], check=True, timeout=30, capture_output=True) + notices = b"Hermetic native fixture; no SDK dependencies or provider calls." + (directory / PACKAGE_LOCAL.SDK_NOTICES).write_bytes(notices) + record = {"schema": "openprose.agents-sdk-build/1", "modelCalls": 0, + "platform": "darwin" if platform.system() == "Darwin" else "linux", + "architecture": platform.machine(), "python": "3.10.20", "pyinstaller": "6.22.3", + "sources": {p: PACKAGE_LOCAL.sha256_file(ROOT / p) for p in ("harnesses/agents-sdk/run.py", "harnesses/agents-sdk/requirements-build.txt")}, + "helper": {"path": PACKAGE_LOCAL.SDK_NAME, "sha256": PACKAGE_LOCAL.sha256_file(helper), "byteLength": helper.stat().st_size}, + "notices": {"path": PACKAGE_LOCAL.SDK_NOTICES, "sha256": hashlib.sha256(notices).hexdigest(), "byteLength": len(notices)}, + "selfTest": expected, "toolSelfTest": tools, "linuxLibraries": libraries if platform.system() == "Linux" else "not-applicable", + "authority": "hermetic-test-fixture-not-release-evidence"} + (directory / PACKAGE_LOCAL.SDK_RECEIPT).write_text(json.dumps(record)) + return directory + def install_npm(self, name: str, include_platform: bool) -> Path: npm = shutil.which("npm") self.assertIsNotNone(npm) @@ -4694,6 +4722,8 @@ def test_release_mode_refuses_each_development_or_test_seam_candidate(self) -> N ), "--image-manifest", str(eligible), + "--agents-sdk-build", + str(self.root / "uninspected-sdk"), "--canonical-profile", str(canonical_profile), "--release-evidence", @@ -4744,6 +4774,7 @@ def test_arbitrary_gate_files_only_create_an_unpublishable_release_candidate( release_evidence = self.root / "arbitrary-evidence.txt" canonical_profile.write_text("not authority\n", "utf-8") release_evidence.write_text("not validation\n", "utf-8") + sdk = self.write_sdk_candidate() output = self.root / "candidate-only-release" completed = subprocess.run( [ @@ -4761,6 +4792,8 @@ def test_arbitrary_gate_files_only_create_an_unpublishable_release_candidate( str(binaries["bun"]), "--image-manifest", str(eligible), + "--agents-sdk-build", + str(sdk), "--canonical-profile", str(canonical_profile), "--release-evidence", @@ -4793,3 +4826,43 @@ def test_arbitrary_gate_files_only_create_an_unpublishable_release_candidate( if __name__ == "__main__": unittest.main() + +class PackagedSdkArtifactTests(unittest.TestCase): + def test_release_requires_packaged_helper_before_other_work(self): + args = argparse.Namespace(mode='kernel-rc', version='0.15.0-rc.4', source_revision='fixture', source_date_epoch=0) + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError, '--agents-sdk-build'): + PACKAGE_LOCAL.build(args) + + def test_sdk_build_receipt_rejects_changed_source_before_executing(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory); sdk = root / 'sdk'; sdk.mkdir(); snapshots = root / 'snapshots'; snapshots.mkdir() + receipt = {'schema': 'openprose.agents-sdk-build/1', 'modelCalls': 0, + 'platform': 'darwin', 'architecture': 'arm64', 'python': '3.10.20', 'pyinstaller': '6.22.3', + 'sources': {'harnesses/agents-sdk/run.py': 'a' * 64}} + (sdk / PACKAGE_LOCAL.SDK_RECEIPT).write_text(json.dumps(receipt)) + with mock.patch.object(PACKAGE_LOCAL, 'run_bounded') as execute: + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError, 'sources differ'): + PACKAGE_LOCAL.snapshot_sdk(sdk, snapshots, 'darwin-arm64', 'kernel-rc', None) + execute.assert_not_called() + + def test_sdk_payload_is_sibling_in_standalone_and_npm(self): + sdk_members = [('prose-agents-sdk', b'fixture-helper', 0o755), + ('agents-sdk-build.json', b'{}', 0o644), ('AGENTS-SDK-NOTICES.txt', b'notices', 0o644)] + image, image_sha = PACKAGE_LOCAL.read_image_manifest(ECHO_IMAGE_MANIFEST) + identity = PACKAGE_LOCAL.image_identity(image, image_sha) + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + archive = PACKAGE_LOCAL.standalone_archive(root, 'rust', b'fixture-cli', '0.1.0', 'darwin-arm64', 0, + 'not-applicable', 'development', b'example', sdk_members=sdk_members) + with tarfile.open(archive) as source: + names = source.getnames() + self.assertTrue(any(n.endswith('/prose-agents-sdk') for n in names)) + self.assertTrue(all(n.split('/')[0] == names[0].split('/')[0] for n in names)) + _, payload = PACKAGE_LOCAL.npm_packages(root, b'fixture-cli', '0.1.0', 'darwin-arm64', 0, + 'fixture', identity, 'not-applicable', 'development', b'example', sdk_members=sdk_members) + with tarfile.open(payload) as source: + for name, data, mode in sdk_members: + member = source.getmember('package/bin/' + name) + self.assertTrue(member.isfile()) + self.assertEqual(member.mode, mode) + self.assertEqual(source.extractfile(member).read(), data) diff --git a/cli/ci/test_publication.py b/cli/ci/test_publication.py index 21bed62e..20e04b36 100644 --- a/cli/ci/test_publication.py +++ b/cli/ci/test_publication.py @@ -8,6 +8,7 @@ import unittest from unittest.mock import patch import publication as p +from test_assemble_kernel_rc import sdk_fixture class PublicationTests(unittest.TestCase): @@ -19,8 +20,10 @@ def setUp(self): self.image = {'formatVersion': 1, 'version': 'test', 'sha256': 'c'*64, 'manifestSha256': 'd'*64, 'purpose': 'canonical-language-runtime', 'releaseEligible': True} self.add('preflight.json', json.dumps({'schema': 'openprose.release-preflight-report/1', 'status': 'pass', 'failures': [], 'sourceSha': 'a'*40, 'version': self.plan['version'], 'protectedAuthority': {'status': 'pass'}, 'image': {'imageSha256': 'c'*64, 'manifestSha256': 'd'*64, 'version': 'test', 'purpose': 'canonical-language-runtime', 'releaseEligible': True}}).encode()) for platform in p.PLATFORMS: + sdk, sdk_members = sdk_fixture(platform) + self.add(platform + '-release-manifest.json', json.dumps({'mode': 'release', 'platform': platform, 'version': self.plan['version'], 'source': {'revision': self.plan['source'], 'verification': 'matched-product-doctor'}, 'agentsSdk': sdk}).encode()) for implementation in ('bun', 'rust'): - self.tar(implementation + '-' + platform + '.tgz', {'root/prose': (implementation+platform).encode()}, 'standalone', platform, implementation) + self.tar(implementation + '-' + platform + '.tgz', {'root/prose': (implementation+platform).encode(), **{'root/' + n: d for n, d, m in sdk_members}}, 'standalone', platform, implementation) if platform.startswith('darwin'): self.add(platform + '-receipt.json', b'{}') self.add(platform + '-notarization.zip', b'not a real signature') @@ -35,6 +38,8 @@ def setUp(self): platform = name.removeprefix('@openprose/prose-cli-') meta.update(openproseSourceRevision='a'*40, openproseImage=self.image) members['package/bin/prose'] = ('bun'+platform).encode() + _, sdk_members = sdk_fixture(platform) + members.update({'package/bin/' + n: d for n, d, m in sdk_members}) members['package/package.json'] = json.dumps(meta).encode() self.tar(name.split('/')[1] + '.tgz', members, 'npm', platform, 'bun') self.path = self.root / 'plan.json' @@ -60,10 +65,19 @@ def test_complete_inventory_validates_but_is_not_signing_evidence(self): plan = p.load_plan(self.path) packages, hashes = p.verify_local(plan, self.root) self.assertEqual(set(packages), set(p.PACKAGES)) - self.assertEqual(len(hashes), 12) + self.assertEqual(len(hashes), 16) with self.assertRaisesRegex(ValueError, 'Wrong signing identity'): p.verify_macos(plan, self.root, Path('unused'), 'unused', 'unused') + def test_production_cannot_publish_unbound_sdk_siblings(self): + manifest_path = self.root / 'darwin-arm64-release-manifest.json' + manifest = json.loads(manifest_path.read_text()); manifest.pop('agentsSdk') + manifest_path.write_text(json.dumps(manifest)) + item = next(a for a in self.plan['artifacts'] if a['name'] == manifest_path.name) + item.update(sha256=p.digest(manifest_path), size=manifest_path.stat().st_size) + with self.assertRaisesRegex(ValueError, 'SDK identity'): + p.verify_local(self.plan, self.root) + def test_development_plan_rejected(self): self.plan['qualification']['status'] = 'development' self.save() diff --git a/cli/ci/test_release_package_admission.py b/cli/ci/test_release_package_admission.py index 22d81fdb..a4e77398 100644 --- a/cli/ci/test_release_package_admission.py +++ b/cli/ci/test_release_package_admission.py @@ -8,6 +8,7 @@ import shutil import subprocess import sys +import tarfile import tempfile import time import unittest @@ -240,6 +241,11 @@ def authority_inputs( "sha256": payloads["extracted"]["bun"]["binarySha256"], }, } + sdk_archive = packages / f'openprose-prose-cli-{release["platform"]}-{release["version"]}.tgz' + with tarfile.open(sdk_archive) as archive: + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + data = archive.extractfile('package/bin/' + name).read() + files[name] = {'byteLength': len(data), 'sha256': hashlib.sha256(data).hexdigest()} windows = None if target_id == "win-x64": windows = release["windowsProcessHost"] diff --git a/cli/ci/test_sign_macos.py b/cli/ci/test_sign_macos.py index 0807baf7..09687319 100644 --- a/cli/ci/test_sign_macos.py +++ b/cli/ci/test_sign_macos.py @@ -43,6 +43,12 @@ def native(self, args): return f"Authority={IDENTITY}\nTeamIdentifier={self.detail_team}\nTimestamp=2026-09-17\nCodeDirectory v=20500 flags=0x10000(runtime)\n" if "--entitlements" in args and "--display" in args: return plistlib.dumps(self.entitlements).decode() if args[-1].endswith("prose-bun") else "Executable=prose-rust\n" + if '--packaged-self-test' in args: + import kernel_rc_evidence as custody + return json.dumps(custody.SDK_IMPORT_TEST) + if '--packaged-tool-self-test' in args: + import kernel_rc_evidence as custody + return json.dumps(custody.SDK_TOOL_TEST) if "submit" in args: return json.dumps({"id": SUBMISSION, "status": "Accepted"}) if "log" in args: @@ -73,6 +79,30 @@ def test_success_preserves_inputs_and_binds_exact_signed_archive(self): self.assertIn("runtime", call) self.assertEqual("--entitlements" in call, call[-1].endswith("prose-bun")) + def test_sdk_missing_inner_signing_receipt_fails_before_native_commands(self): + helper = self.root / 'sdk'; helper.write_bytes(b'sdk') + with self.assertRaises(signing.SigningError): + self.sign(agents_sdk=helper) + self.assertEqual(self.calls, []) + self.assertFalse(self.output.exists()) + + def test_sdk_helper_signed_and_notarized_with_both_runners(self): + helper = self.root / 'sdk'; helper.write_bytes(b'sdk') + notices = self.root / 'AGENTS-SDK-NOTICES.txt'; notices.write_bytes(b'notices') + build = {'helper': {'path': 'prose-agents-sdk', 'byteLength': 3, 'sha256': signing.sha256(helper)}, + 'notices': {'path': notices.name, 'byteLength': 7, 'sha256': signing.sha256(notices)}, + 'embeddedSigning': {'identity': IDENTITY, 'verification': 'pyinstaller-inner-binaries-and-frozen-self-tests'}} + (self.root / 'agents-sdk-build.json').write_text(json.dumps(build)) + receipt = self.sign(agents_sdk=helper) + bound = json.loads((self.output / 'agents-sdk-build.json').read_text()) + self.assertEqual(bound['helper']['sha256'], receipt['binaries']['agents-sdk']['signedSha256']) + self.assertEqual(bound['signingReceiptSha256'], signing.sha256(self.output / 'receipt.json')) + self.assertEqual(set(receipt['binaries']), {'bun', 'rust', 'agents-sdk'}) + self.assertEqual(receipt['binaries']['agents-sdk']['signedSha256'], hashlib.sha256(b'sdk-signed').hexdigest()) + self.assertEqual(signing.verify_existing(self.output, **self.kwargs()), receipt) + with zipfile.ZipFile(self.output / 'notarization.zip') as archive: + self.assertEqual(set(archive.namelist()), {'prose-bun', 'prose-rust', 'prose-agents-sdk'}) + def test_non_macos_makes_no_native_calls(self): with patch.object(signing.platform, "system", return_value="Linux"): with self.assertRaises(signing.SigningError): diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 4fc5474f..53b1cc7f 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -916,3 +916,19 @@ Lead: Codex `/root`, session `01a1133b-e47b-7b32-af8e-2655afa0259c`, branch `cod `/root/sdk_runtime` exact writable paths: `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` (new). Scope: reuse Python SDK loop; public search/retrieval and fresh bounded child contexts; safe credential setup/errors, shared aggregate bounds/usage, bounded tool-output memory, cancellation and honest permissions. No contract interpretation/company procedures. No dependency/manifests, adapters, shared files, model calls, Git/index or formatting outside lease. Dependency additions require root approval; public search uses existing SDK-supported hosted tool. Provider-free mocks precede behavior; retain cancellation/failure controls. Other lanes await exact leases and design freeze. `/root/operational_design` packaging lease (October 6): exact paths `harnesses/agents-sdk/requirements-build.txt` (new), `cli/ci/build_agents_sdk.py` (new), `cli/ci/test_build_agents_sdk.py` (new), `cli/ci/package_local.py`, `cli/ci/test_package_local.py`, `cli/ci/build_local.py`, `cli/ci/test_check_build_local.py`, `cli/ci/build_kernel_rc.py`, `cli/ci/test_build_kernel_rc.py`, `cli/ci/homebrew_rehearsal.py`, `cli/ci/test_homebrew_rehearsal.py`, `cli/ci/dependency_evidence.py`. Scope: package-owned frozen helper, transitive hash lock, platform build/package/rehearsal and evidence; preserve archive safety, existing modes/fixtures and offline gates. Other evidence schemas/workflows require root-recorded extension before edits. No product trees, runtime run.py, Git/index or model calls. Root owns workflow/schema integration and operational design doc. + +IMP-098 packaging lease extension: `/root/operational_design` may edit exactly `cli/ci/test_dependency_evidence.py` for Python lock/inventory controls. Root reserves `cli/ci/check_architecture.py`, required release/evidence schemas and workflows for integration; no changes to packaging safety allowlists without tested justification. + +IMP-098 SDK runtime lane released after 23 provider-free checks; root owns final integration edits to `harnesses/agents-sdk/run.py`, `test_run.py`, `README.md`. Root architecture integration lease: `cli/ci/check_architecture.py`, `cli/ci/test_check_architecture.py`, `.github/workflows/cli-kernel-rc.yml`, `.github/workflows/cli-ci.yml`, `.github/workflows/cli-distribution-check.yml`, `.github/workflows/cli-publish.yml` and required shared release-manifest/dependency evidence schemas after exact names are inspected. No gate weakening or public dispatch. + +IMP-098 independent runtime review repair: `/root/conformance_plumbing` exact writable paths `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md`. Root runtime integration lane released to this agent. Scope: independently found Python3.10 deadline incompatibility, HTTP framing completeness/safe errors and SDK internal hidden retries; add real SDK transport-level controls and run pinned Python3.10. No packaging/build helper/dependency edits/Git/model calls. Original SDK builder remains available for separate follow-up review after repair. + +IMP-098 packaging wave extension: `/root/operational_design` owns `docs/agents-sdk-adapter.md` for current packaged setup and explicitly labelled developer recipe, following independent review. Schema leases await exact filenames. + +IMP-098 release custody integration wave: `/root/operational_design` owns `cli/ci/reproducible_release.py`, `cli/ci/test_reproducible_release.py`, `cli/ci/publication.py`, `cli/ci/test_publication.py`, `cli/ci/kernel_rc_evidence.py`, `cli/ci/test_kernel_rc_evidence.py`, `cli/ci/assemble_kernel_rc.py`, `cli/ci/test_assemble_kernel_rc.py`, `cli/ci/release_package_admission.py`, `cli/ci/test_release_package_admission.py` for end-to-end SDK sibling identity/trust/source binding and preserved historical fixture behavior. Root retains architecture/workflows/default adapter integration. No new schemas, Git or live calls. + +IMP-098 custody signing extension: `/root/operational_design` owns `cli/ci/sign_macos.py` and `cli/ci/test_sign_macos.py` to include packaged SDK helper in notarization/ad-hoc verification authority. Preserve historical alpha path; root owns workflow signing inputs. + +IMP-098 custody installed-validator extension: `/root/operational_design` owns `cli/benchmarks/installed/benchmark.py` and `cli/benchmarks/installed/test_benchmark.py` for closed SDK payload/dependency/source metadata validation before installed admission. New production kernel-rc/release requires SDK; retain explicitly historical alpha only. + +IMP-098 root workflow authority extension: `cli/ci/check_workflows.py` and `cli/ci/test_check_workflows.py` solely for separate hash-locked SDK interpreter setup, runtime provider-free gate and unsigned kernel build argument verification. diff --git a/docs/agents-sdk-adapter.md b/docs/agents-sdk-adapter.md index 65badb16..e17a11d0 100644 --- a/docs/agents-sdk-adapter.md +++ b/docs/agents-sdk-adapter.md @@ -1,87 +1,41 @@ # Agents SDK adapter -Both runners accept `--harness agents-sdk --model MODEL --auth-profile openai-api-key --output-contract native`. They discover an executable named `prose-agents-sdk` on the explicitly selected PATH. This optional Python harness receives opaque instructions, a task prompt and a working directory. It exposes an ordinary shell tool; neither the adapter nor harness interprets Contracts or OpenProse syntax. +## Packaged setup -## Provision a private launcher +CLI release packages include `prose-agents-sdk`, Python 3.10.20, the pinned Agents SDK and HTTPS certificates. Standalone archives contain the helper beside `prose`; npm platform packages contain both in `bin/`; Homebrew installs both executables. Install the complete package and retain the helper with its CLI. Normal use needs no Python installation, source checkout, virtual environment or PATH provisioning. -Use a reviewed **full CLI source checkout** containing `harnesses/agents-sdk/run.py` and `requirements.txt`. A private weave review bundle does not include those installation sources. Record the selected checkout revision and file hashes. Current native admission is `prose-agents-sdk 0.1.0` on macOS arm64; other environments need separate qualification. +Both runners select `agents-sdk` by default, with `gpt-6.1-sol`, the `openai-api-key` profile and native output. Supply `OPENAI_API_KEY` through your trusted process environment. Do not put it in command arguments, configuration files or copied support reports. The SDK has no cached-login route and does not switch to another account after an authentication failure. -Choose an installed Python 3.10 or later and fresh private directories. Replace each absolute placeholder below. The virtual-environment path must contain no whitespace and must be short enough for a direct shebang; the launcher generator checks this conservatively. Do not resolve its `bin/python` symlink to the system interpreter: the virtual-environment path is needed to load the installed dependencies. +User choices in `~/.prose/cli.toml` and explicit command overrides take precedence over built-in defaults. An existing saved harness selection remains selected. Use `prose config explain` to inspect the effective choices and their origins, and `prose harness list` to inspect supported adapters. Explicit selection remains available as `--harness agents-sdk --model gpt-6.1-sol --auth-profile openai-api-key --output-contract native`. -```sh -PROSE_CLI_SOURCE=/absolute/reviewed/prose-cli -PROSE_PYTHON=/absolute/python3 -PROSE_SDK_VENV=/absolute/new-private-venv -PROSE_HARNESS_BIN=/absolute/new-private-harness-bin -``` - -The next commands create the environment and install dependencies. Dependency installation may access your configured package index; run it only as an explicit provisioning action. They do not invoke a model. The requirements pin direct dependencies, not the entire transitive graph; retain the resolved package versions for your installation. +The default helper is resolved beside the canonical installed CLI executable. An unrelated `prose-agents-sdk` on PATH does not replace the packaged helper. A missing helper means the installation is incomplete: reinstall the complete matching CLI package. Do not repair it by downloading an unbound executable or provisioning a source launcher. -```sh -umask 077 -test ! -e "$PROSE_SDK_VENV" && test ! -L "$PROSE_SDK_VENV" && \ - "$PROSE_PYTHON" -m venv "$PROSE_SDK_VENV" && \ -"$PROSE_SDK_VENV/bin/python" -m pip install \ - --requirement "$PROSE_CLI_SOURCE/harnesses/agents-sdk/requirements.txt" && \ -"$PROSE_SDK_VENV/bin/python" -m pip freeze -``` +## Verify before a provider run -Run each step only after the preceding step succeeds. The requirements currently select `openai-agents==0.22.2`, `openai==3.13.0` and `python-dotenv==1.2.3`. For offline provisioning, use separately reviewed cached packages; this guide does not download or manage a cache automatically. +Use the installed CLI's provider-free doctor and dry-run routes with the working directory and bounds planned for the action. Readiness reports can establish local selection and availability; key presence does not establish successful authentication or model availability. Dry runs do not invoke a model. See [SDK execution budgets](sdk-budgets.md) for inner and outer deadlines. -Create the launcher with the virtual environment's absolute interpreter. The generator copies the reviewed harness bytes, changing only its first shebang line. It refuses an existing launcher directory and never executes the harness. Its parent directory must already exist. A partial directory after an error is not an accepted installation; inspect it and choose a fresh destination. +For a standalone installation, local helper checks are available using its exact installed path: ```sh -"$PROSE_SDK_VENV/bin/python" - "$PROSE_CLI_SOURCE" "$PROSE_SDK_VENV/bin/python" "$PROSE_HARNESS_BIN" <<'PY' -import hashlib, os, pathlib, sys -source_root, interpreter, destination = sys.argv[1:] -source = pathlib.Path(source_root) / 'harnesses/agents-sdk/run.py' -folder = pathlib.Path(destination) -if not all(pathlib.Path(p).is_absolute() for p in (source_root, interpreter, destination)): - raise SystemExit('Use absolute source, interpreter and launcher paths.') -if any(c.isspace() for c in interpreter) or '\0' in interpreter or not os.access(interpreter, os.X_OK): - raise SystemExit('Use an executable virtual-environment interpreter without whitespace.') -shebang = ('#!' + interpreter + '\n').encode('utf-8') -if len(shebang) > 127: - raise SystemExit('Use a shorter virtual-environment interpreter path.') -original = source.read_bytes() -first, separator, body = original.partition(b'\n') -if first != b'#!/usr/bin/env python3' or not separator: - raise SystemExit('Review the changed source launcher format before proceeding.') -folder.mkdir(mode=0o700) -launcher = folder / 'prose-agents-sdk' -data = shebang + body -fd = os.open(launcher, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o700) -with os.fdopen(fd, 'wb') as stream: - stream.write(data) - stream.flush() - os.fsync(stream.fileno()) -print('launcher:', launcher) -print('source-sha256:', hashlib.sha256(original).hexdigest()) -print('launcher-sha256:', hashlib.sha256(data).hexdigest()) -PY +env -i /absolute/installation/prose-agents-sdk --version +env -i /absolute/installation/prose-agents-sdk --packaged-self-test +env -i /absolute/installation/prose-agents-sdk --packaged-tool-self-test ``` -Keep that virtual environment and launcher at their selected paths. Moving or changing either requires a reviewed replacement; the native CLI executable digest does not attest all harness dependencies. Nothing above edits a PATH profile or installs globally. +Version output is `prose-agents-sdk 0.1.0`. The packaged self-tests check imports, HTTPS certificates, local shell effects, bounded output, cancellation and mocked retrieval. They load no credentials and make no model or network requests. These checks do not qualify a provider account or prove a program's fulfillment. -## Verify locally before a provider run +## Development and specialized integrations -These two checks use an empty environment and do not invoke a model or load credentials: +Development from a full CLI source checkout is separate from the installed release route. Use the pinned Python and hash-locked build graph in `harnesses/agents-sdk/requirements-build.txt` through the [packaged SDK build](packaged-sdk-design.md). The release build records the source and dependency hashes, helper identity and notices in package evidence. A developer or test launcher requires an explicitly selected supported override; PATH discovery is not the consumer default. -```sh -env -i "$PROSE_HARNESS_BIN/prose-agents-sdk" --version -env -i "$PROSE_HARNESS_BIN/prose-agents-sdk" --help -``` - -Version output must be `prose-agents-sdk 0.1.0`. Help must show the model, cwd, prompt and native budget options. Successful import/help does not prove provider access or model availability. - -For native CLI readiness, select the launcher directory explicitly in the process PATH; for example, `PATH="$PROSE_HARNESS_BIN:/usr/bin:/bin"`. Supply `OPENAI_API_KEY` through your trusted process environment, never a source file or copied support report. Use the selected CLI's provider-free doctor and dry-run routes with the same model, cwd and bounds planned for the action; see the [native actor setup](../experiments/weave-seed/integration/native-actor/README.md#explicit-setup). The weave actor requires a matching **fixed-image CLI with test seams disabled**, not an ordinary published-on-run build. Follow [fixed-image staging](../experiments/weave-seed/getting-started/FIXED-IMAGE.md) before configuring that actor. - -When using the weave bridge, its host binding, the coordinator and the actor each filter environment names. Admit PATH and OPENAI_API_KEY at every required layer, and admit the separate assessor key in the bridge/coordinator if selected. A successful offline `check` does not execute these native readiness checks or establish provider authentication. The ordinary CLI defaults to the unavailable hosted route, so keep `--harness agents-sdk` and the API-key profile explicit. +The weave native actor requires a matching fixed-image CLI with test seams disabled, rather than the ordinary published-kernel loader. Follow [fixed-image staging](../experiments/weave-seed/getting-started/FIXED-IMAGE.md) and [native actor setup](../experiments/weave-seed/integration/native-actor/README.md#explicit-setup). The bridge, coordinator and actor each filter environment names: admit the selected API credential at every required layer, and admit the separate assessor credential where selected. An offline weave check does not establish native provider authentication. ## Effects, limits and evidence -There is no cached-login route in this harness. The shell receives a scrubbed environment without credential-like variables, but it can read host files allowed by the operating system: this is not filesystem confinement or an OS sandbox. Native events are start, tool_call, tool_result, final or error. Only final settles a successful native run; final text and exit zero do not establish program fulfillment. +The harness receives opaque instructions, a task prompt and a working directory. It offers a bounded shell for local file operations, public retrieval/search tools and fresh child contexts. Neither the adapter nor harness interprets Contracts or OpenProse syntax or assesses fulfillment. + +The shell receives a scrubbed environment without credential-like variables, but can read host files allowed by the operating system. Environment filtering is not filesystem confinement or an OS sandbox. Native events are `start`, `tool_call`, `tool_result`, `final` or `error`. Only `final` settles a successful native run; final text and exit zero do not establish program fulfillment. -Use the [SDK execution budgets](sdk-budgets.md) for the separate inner and outer deadlines. Turn and token limits are not an aggregate dollar cap. A model may make prohibited operations through its available shell; artifact-only assessment cannot certify procedural compliance. +Shared budgets account for parent and child calls, tools, usage and cancellation. Turn and token limits are not an aggregate dollar cap. The SDK's automatic model and HTTP retries are disabled; callers own retry allocation. A model can still perform prohibited actions through available tools, and artifact-only assessment cannot certify procedural compliance. -The launcher recipe was checked with an already installed Python environment and dependencies on macOS arm64: exclusive creation, exact copied body, private executable permissions, and empty-environment help/version. No package installation, network or provider request was used for that check. It does not qualify a fresh dependency install or a complete live BYOK run. Earlier development canaries exercised native reads and writes; their evidence does not generalize to every program or prove language conformance. +The frozen helper extracts its runtime into temporary storage at startup. It needs a writable temporary filesystem that permits execution. Startup interruption or forced death can leave extraction directories; do not assume cleanup after every signal. Native platform and installed-route qualification, retained package evidence and live program assessment remain distinct from these local self-tests. diff --git a/harnesses/agents-sdk/README.md b/harnesses/agents-sdk/README.md index 6bed02de..94d252a5 100644 --- a/harnesses/agents-sdk/README.md +++ b/harnesses/agents-sdk/README.md @@ -1,41 +1,42 @@ -# Generic OpenAI Agents SDK local harness +# OpenAI Agents SDK local harness -A normal `Agent` + `Runner` loop with one general shell tool. The model chooses every tool action. This harness treats instruction-file contents as opaque text and has no knowledge of Contracts, references, programs, acceptance, state, or libraries. It is an optional bring-your-own-harness test implementation, not an OpenProse dependency. +This generic `Agent` + `Runner` loop transports the caller's instructions as opaque text. It does not interpret OpenProse contracts, discover instruction files, assess fulfillment or supply company procedures. The CLI release packages a native `prose-agents-sdk` sibling executable with Python and its pinned dependencies; normal users do not provision a virtual environment. Development from source uses the release-pinned Python 3.10.20 and `pip install -r requirements.txt`. -## Install and run - -Use Python 3.10 or later: +Supply an OpenAI API key through `OPENAI_API_KEY` or `--env-file FILE`. Only `OPENAI_API_KEY` is read from that file, without changing the parent environment. Missing credentials fail before a model request. Rejected credentials and unavailable models produce actionable setup errors, with no credential/model fallback or exception bodies. The standalone harness model default is `gpt-6.1-sol`; the outer CLI passes its resolved model explicitly. Tracing export is disabled, and the OpenAI client and SDK runner have retries explicitly disabled, including SDK conversation-lock compatibility retries. ```sh -python3 -m venv .venv -.venv/bin/pip install -r requirements.txt -.venv/bin/python run.py --model MODEL --cwd /path/to/workspace --prompt 'Read README.md and perform the requested task.' +prose-agents-sdk --cwd /path/to/workspace --prompt 'Read README.md and perform the requested task.' ``` -Supply `OPENAI_API_KEY` in the process environment or pass `--env-file /private/path/.env`. Only OPENAI_API_KEY is loaded from that file. An optional `--instructions FILE` appends opaque instructions to a generic coding-agent instruction. The working directory is identified in instructions. No file is automatically discovered or interpreted. +`--instructions FILE` appends opaque instructions to the generic agent instruction; the selected working directory is identified separately. No other file is automatically loaded. Stdout is the existing version 0.1.0 JSONL event protocol: start, actual function-tool calls/results, final response/usage, or safe error. Exit zero establishes completion of the agent loop; inspect artifacts and assess requested requirements independently. -JSON lines on stdout record starts, actual tool commands and results, final text and token usage, or error type. Exit 0 means Runner returned a final response, **not that the requested task passed acceptance**. Observers must inspect results independently. Tracing export is disabled. Exceptions expose types rather than potentially sensitive request bodies. +The model has four tools: -Defaults: 180 second overall timeout, 30 seconds per shell action, 20 Runner turns, 12,000 output tokens per model request. Shell timeouts and cancellation terminate the shell process group. Tool stdout/stderr are each truncated to 30,000 characters. No hidden retries, output repair, adjudication, or program-specific policy is supplied by this wrapper; SDK/provider transport behavior remains upstream behavior. +- `execute_shell`: ordinary local Bash for reading/writing files and other host operations. +- `web_search`: SDK-supported hosted OpenAI search with low search context. Responses limits hosted tool calls to one per model request. +- `retrieve_url`: bounded public HTTPS GET. It rejects credentials, non-443 ports, private/non-global DNS addresses and redirects. Connections pin the vetted address while validating TLS against the original hostname. Responses support identity encoding and bounded Content-Length, EOF-delimited and chunked bodies. Declared-length and chunk framing must be complete; partial bodies fail rather than being presented as complete content. Failed or unsupported retrieval returns a safe tool error. +- `delegate(task)`: a new independent `Agent` and `Runner` receives only the explicit task string plus the same opaque instruction file and working directory. Parent conversation, tool history and final answers are not passed automatically. Up to eight children execute serially; children cannot delegate. Files remain shared, so conversation isolation does not isolate artifacts. -## Environment boundary +## Permissions and lifecycle -The shell is ordinary local bash with a supplied working directory. **This is not an OS sandbox:** use a separately isolated machine/container for untrusted programs. Keys/tokens/secrets/password variables are excluded from the shell environment, but the shell can still read host files within OS permissions. This wrapper adds no filesystem confinement. Tests use synthetic data in separate copied workspaces. +Shell commands have the current user's OS permissions. **This is not an OS sandbox.** Commands can read host files, write outside the selected working directory and use the network. Environment names containing KEY/TOKEN/SECRET/PASSWORD are removed from shell inheritance; this cannot prevent commands from reading credentials in host files. A separately isolated machine/container is necessary for untrusted programs. Public-URL restrictions apply to `retrieve_url`, not to arbitrary shell commands. The wrapper never claims filesystem or network confinement. -No web, subagent, or notification tool is provided in this initial profile. Unsupported program requirements must be handled by the model rather than silently provided by an evaluator. +Default limits are 180 seconds for the entire parent/children invocation, 30 seconds per shell/retrieval action, 20 aggregate model requests (also the individual Runner turn cap), 80 aggregate function-tool calls and 12,000 output tokens per model request. `--max-turns` sets the shared request cap. `--max-tools` controls function tools separately; hosted calls are bounded by the request cap and one hosted call per request. `--max-input-bytes` defaults to 256,000 UTF-8 bytes per request, measured on serialized instructions, full task/history input, tools and output schema at the actual SDK Responses boundary, before transmission. Oversized instruction files fail during setup. This is a request-byte limit, not a provider token or billable-search-context measurement. `--max-total-tokens` defaults to 500,000 observed total tokens and stops the next request after the observed ceiling is reached. This observation limit does not interrupt a response already in flight and does not establish a billing ceiling. Unobserved/cancelled provider work remains unknown. Callers conducting bounded paid qualification must reserve spending conservatively before launch. -## Validation +Shell stdout/stderr capture drains pipes while retaining at most 30,000 bytes each; results disclose truncation. Retrieval retains 60,000 body bytes with a truncation flag. Child input/output is bounded to 60,000 bytes. Oversized child output fails the invocation rather than silently presenting it as complete. Shell timeout/cancellation kills its process group and waits for the direct process; descendants deliberately escaping that group are outside this guarantee. Frozen builds restore original dynamic-loader environment settings for subprocesses rather than exposing their bundled-library directory. -```sh -.venv/bin/python test_run.py -``` +SIGINT/SIGTERM cancel the running invocation, including active children and tool awaits. Errors, timeouts, exhaustion and cancellation preserve observed completed usage in a terminal error when the process is allowed to settle. External force-kill can prevent terminal reporting. SDK/provider and hosted-tool internals remain upstream behavior. -Local tests exercise actual read/write/exit behavior, process-group timeout cleanup, budget forwarding, safe failures, and provider-free usage accounting. A mocked provider request comparison verifies that raw-usage preservation changes neither request fields nor the tool schema. Initial real-model observations are in `EVIDENCE.md`. No language acceptance tests are embedded in the harness. +## Usage and validation -Implementation followed the official [Agents SDK migration example](https://developers.openai.com/cookbook/examples/agents_sdk/migrate-from-claude-agent-sdk/readme) and inspected installed SDK signatures. Pinned direct dependencies: OpenAI Agents SDK 0.22.2, OpenAI Python 3.13.0, python-dotenv 1.2.3. +Successful `final.usage` sums the normalized usage from the parent and each independently completed child once. It overlaps `usageObservation`; do not add them together. Observations sum only allowlisted nonnegative integer counters present in raw usage on unique completed model responses across parent and children: input/output/total, cached/cache-write and reasoning tokens. Missing/invalid fields remain absent. Event `modelIdentity` records the requested model and allowlisted observed model IDs from completed provider Responses without exposing response contents. `fieldResponseCounts` shows coverage. Provider IDs stay internal to deduplication and never appear in observation records. -The outer runners expose explicit `--native-max-turns` and `--native-timeout` selections; see [SDK execution budgets](../../docs/sdk-budgets.md). Native start/error records report configured limits. Safe native error types distinguish turn exhaustion and timeout; all other exception classes become `ExecutionError`, without exception bodies. +Logical hook starts/completions do not establish provider transport billing. `outstandingProviderRequestCount` is null and `totalRunUsageKnown` remains false, including success. Preserve failed/cancelled spending reservations unless independent accounting closes them. + +```sh +python test_run.py +``` -Final and error records also carry an additive `usageObservation`, collected through public SDK `RunHooks` with `ModelSettings(preserve_raw_usage=True)`. Existing successful `final.usage` remains the SDK's normalized aggregate. The new observation sums only allowlisted nonnegative integer counters actually present in raw usage on unique completed model responses: input/output/total tokens, cached/cache-write input tokens, and reasoning output tokens. Missing or invalid counters are absent, never inferred as zero. `fieldResponseCounts` reports how many completed responses supplied each counter; observed totals can therefore cover only part of a run. These totals overlap legacy `final.usage`: **do not add them together**. `aggregationScope` identifies the per-run completed-response aggregate, excluding cumulative context snapshots. Provider response/request IDs are used only for internal deduplication and are not emitted; identical callbacks without IDs are deduplicated by object identity. +Provider-free tests exercise actual local file effects, byte-bounded output capture, process-group timeout/cancellation, missing credentials before inference, safe auth/model failures, aggregate request/tool/observed-token limits, the actual transmitted request byte guard including tools and history, private URL rejection, raw-usage deduplication and genuine SDK-loop child isolation. Mocked Responses transport verifies one hosted-tool allowance per request and no transport retries. Live artifact fulfillment and installed release readiness are separate qualification gates, retained by the owning task. -`startedCallCount`, `completedResponseCount`, and `outstandingCallCount` refer to logical SDK hook calls, not provider transport requests. `duplicateResponseCallbackCount` records ignored duplicate completion callbacks. Transport retries, responses without preserved raw usage, cancelled requests, and any provider work invoked separately from shell are outside this observation. Accordingly `outstandingProviderRequestCount` is null and `totalRunUsageKnown` is always false, including successful runs; these observations cannot establish total billing or justify releasing a failed run's spending reservation. An error before any observed response yields empty token totals, not measured zero usage. Ordinary SDK failure, turn exhaustion, or overall timeout preserves completed-response observations in the error record. External process termination before final/error emits no such summary. No new JSON event type, tracing export, model instruction, tool, budget, or protocol version is introduced. +The integration reuses the pinned SDK 0.22.2 public Agent/Runner/RunHooks/ModelSettings/WebSearchTool APIs. OpenAI Python and dotenv versions are pinned in [requirements.txt](requirements.txt); packaging retains transitive dependency identities. See [SDK budgets](../../docs/sdk-budgets.md) for the outer-runner budgets and historical [evidence](EVIDENCE.md) for earlier source-only observations. diff --git a/harnesses/agents-sdk/requirements-build.txt b/harnesses/agents-sdk/requirements-build.txt new file mode 100644 index 00000000..955c8187 --- /dev/null +++ b/harnesses/agents-sdk/requirements-build.txt @@ -0,0 +1,230 @@ +# CPython3.10.20; native macOS/Linux ARM64/x64 frozen SDK build. +# Resolved October6,2026; PyPI wheel bytes for four release targets. +# cryptography48.0.1 retains macOSIntel wheels; Linux wheels require glibc2.34. +# Install with --require-hashes --only-binary=:all:; no sdists. + +# altgraph-0.17.5-py2.py3-none-any.whl +altgraph==0.17.5 \ + --hash=sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597 + +# annotated_types-0.8.0-py3-none-any.whl +annotated-types==0.8.0 \ + --hash=sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0 + +# anyio-4.15.1-py3-none-any.whl +anyio==4.15.1 \ + --hash=sha256:6152fdbbf9a77fdec97731721bebf7c4c44f7c29b424b0065826173efc7ed101 + +# attrs-26.1.0-py3-none-any.whl +attrs==26.1.0 \ + --hash=sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309 + +# certifi-2026.7.22-py3-none-any.whl +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 + +# cffi-2.1.1-cp310-cp310-macosx_10_15_x86_64.whl +# cffi-2.1.1-cp310-cp310-macosx_11_0_arm64.whl +# cffi-2.1.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl +# cffi-2.1.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl +cffi==2.1.1 \ + --hash=sha256:194cffa889098ced9976c3fc6340305e43f6303657d298da55366907c05c22d6 \ + --hash=sha256:5a59cc1c4442bc3d5c703bf720b51138d0bfc173618807c9ee2490a7541dd3d9 \ + --hash=sha256:baed1e86cc735622097354b9d1281406caf42ff42a886d29faa8e8d1630333be \ + --hash=sha256:ca82be1a1d406ecfe1d25dc16cb33488e5a16bf4438c9fb590484ea29d92478b + +# charset_normalizer-3.5.2-cp310-cp310-macosx_10_9_universal2.whl +# charset_normalizer-3.5.2-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl +# charset_normalizer-3.5.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl +charset-normalizer==3.5.2 \ + --hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 \ + --hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 \ + --hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e + +# click-8.5.0-py3-none-any.whl +click==8.5.0 \ + --hash=sha256:255bc9599cf7748b4b1a446ccc735421bd08a2ae529a8b88597d3de5664ee360 + +# cryptography-48.0.1-cp39-abi3-macosx_10_9_universal2.whl +# cryptography-48.0.1-cp39-abi3-manylinux_2_34_aarch64.whl +# cryptography-48.0.1-cp39-abi3-manylinux_2_34_x86_64.whl +cryptography==48.0.1 \ + --hash=sha256:4fdc69f8e4316bcf0c8c8ec1f26f285d12e8142d88d96c876a59a03be3f6ae67 \ + --hash=sha256:b74ca3b8e5ecdd833bf6a002ca41b4793bb27fb8f1c06ffaf2643c9e9140e31b \ + --hash=sha256:fe0180af5bf9236518a087e35bf2d9a347d5f5f51e63c579d683ddff424e3d46 + +# exceptiongroup-1.3.1-py3-none-any.whl +exceptiongroup==1.3.1 \ + --hash=sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598 + +# griffelib-2.3.0-py3-none-any.whl +griffelib==2.3.0 \ + --hash=sha256:1b8f9cd525681c26b1d6d574faa1371651e8459ca51d209684f50b8096ae06e0 + +# h11-0.16.0-py3-none-any.whl +h11==0.16.0 \ + --hash=sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86 + +# httpcore2-2.13.1-py3-none-any.whl +httpcore2==2.13.1 \ + --hash=sha256:e1e05d4f25f7d7d496bfb96748f6f4b67657b03da069b3a68c36069f3db73d0a + +# httpx2-2.13.1-py3-none-any.whl +httpx2==2.13.1 \ + --hash=sha256:6dff50fabc270ee5fd25d845d0b078ed20564579744d6d962850975996d2f9a4 + +# idna-3.20-py3-none-any.whl +idna==3.20 \ + --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c + +# jiter-0.17.0-cp310-cp310-macosx_10_12_x86_64.whl +# jiter-0.17.0-cp310-cp310-macosx_11_0_arm64.whl +# jiter-0.17.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl +# jiter-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl +jiter==0.17.0 \ + --hash=sha256:7b68d3495d95da120651a5628c7ebadee84ed001a1b76e6afc325c42482f15b5 \ + --hash=sha256:826871c42cebaae22f0a2b5673a4a1a75c851bb2d13b3c17764a630a6b298984 \ + --hash=sha256:b847b18d066c46b3b7ae49d6c94a7634c5e4a8983146ee25562a092000f5e3ad \ + --hash=sha256:ed1a24005daac667d577402d75a2922f9775a165b146b883ff1ad3602d8be689 + +# jsonschema-4.26.0-py3-none-any.whl +jsonschema==4.26.0 \ + --hash=sha256:d489f15263b8d200f8387e64b4c3a75f06629559fb73deb8fdfb525f2dab50ce + +# jsonschema_specifications-2025.9.1-py3-none-any.whl +jsonschema-specifications==2025.9.1 \ + --hash=sha256:98802fee3a11ee76ecaca44429fda8a41bff98b00a0f2838151b113f210cc6fe + +# macholib-1.16.4-py2.py3-none-any.whl +macholib==1.16.4 \ + --hash=sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea + +# mcp-2.3.0-py3-none-any.whl +mcp==2.3.0 \ + --hash=sha256:dd0c44c089d16453e8ae31a3877a0054d7a2314caaa81f5e0541b9b1734b2377 + +# mcp_types-2.3.0-py3-none-any.whl +mcp-types==2.3.0 \ + --hash=sha256:968efdbdaedfab06adae40d378a34395f1090c5921d4be3c9cde283aaf76d91d + +# openai-3.13.0-py3-none-any.whl +openai==3.13.0 \ + --hash=sha256:e35b1f6fe99245e86e37504d9fad1ad2a363807307c424232c1d849bd0666c8e + +# openai_agents-0.22.2-py3-none-any.whl +openai-agents==0.22.2 \ + --hash=sha256:685ae918878be4cf52b62d9c732a6d7b8fae7b634268945a20e1a6d2107bd31e + +# opentelemetry_api-1.45.1-py3-none-any.whl +opentelemetry-api==1.45.1 \ + --hash=sha256:b31553efa588ae44bc306f863c785c5333a9ecc091248c6ee68b4b6c87fdedfb + +# packaging-26.3-py3-none-any.whl +packaging==26.3 \ + --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c + +# pycparser-3.0-py3-none-any.whl +pycparser==3.0 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 + +# pydantic-2.13.5-py3-none-any.whl +pydantic==2.13.5 \ + --hash=sha256:346a034f080da3755d8e9cb5e00e8b07de1d39e4f6e2c87d8ab7cafa0b269a73 + +# pydantic_core-2.46.5-cp310-cp310-macosx_10_12_x86_64.whl +# pydantic_core-2.46.5-cp310-cp310-macosx_11_0_arm64.whl +# pydantic_core-2.46.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl +# pydantic_core-2.46.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl +pydantic_core==2.46.5 \ + --hash=sha256:4dedce55295becb61921e386b99d4f2706045306e7fa52249a33004c837379fb \ + --hash=sha256:657b40d6240c0a7b6a64b30f22d1e3aa631c7e846c621b0c0f6d1d75e2e15ea6 \ + --hash=sha256:850a08d167dde16db8702c274f320c7be9d7da6f6dff2b58b18f9e815bd94f5b \ + --hash=sha256:ecb42011e12ee19cafbc312887cbf3546959fe02fbad44f272d4be5baa997615 + +# pyinstaller-6.22.3-py3-none-macosx_10_13_universal2.whl +# pyinstaller-6.22.3-py3-none-manylinux2014_aarch64.whl +# pyinstaller-6.22.3-py3-none-manylinux2014_x86_64.whl +pyinstaller==6.22.3 \ + --hash=sha256:052f4a1cd4f81092ccb7a18fe8ebc9ecf832a5917317e39fb3775af4e959b253 \ + --hash=sha256:312da84b4b31ab9750066a823734c11b7c68757fdd67c9f038bf19c330a954e4 \ + --hash=sha256:451a4ae14b719365bf1a2f0a99dae7b3463060061c3a394c70d5264cfb439528 + +# pyinstaller_hooks_contrib-2026.8-py3-none-any.whl +pyinstaller-hooks-contrib==2026.8 \ + --hash=sha256:6d21d65323ea4467753afc21f52d7d0e69416ae8101dd88269bf273ad48db084 + +# pyjwt-2.15.1-py3-none-any.whl +PyJWT==2.15.1 \ + --hash=sha256:42d59d631f7768a1028a64c7ff581a9bf7519804daf91fc5b6c56e30eec5e193 + +# python_dotenv-1.2.3-py3-none-any.whl +python-dotenv==1.2.3 \ + --hash=sha256:904552145e8bfed22162c09dab1c2b9b54fefa7b23ba780f4f26ca0316b0f0d9 + +# python_multipart-0.0.32-py3-none-any.whl +python-multipart==0.0.32 \ + --hash=sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23 + +# referencing-0.37.0-py3-none-any.whl +referencing==0.37.0 \ + --hash=sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231 + +# requests-2.34.2-py3-none-any.whl +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 + +# rpds_py-0.30.0-cp310-cp310-macosx_10_12_x86_64.whl +# rpds_py-0.30.0-cp310-cp310-macosx_11_0_arm64.whl +# rpds_py-0.30.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl +# rpds_py-0.30.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl +rpds-py==0.30.0 \ + --hash=sha256:0c0e95f6819a19965ff420f65578bacb0b00f251fefe2c8b23347c37174271f3 \ + --hash=sha256:389a2d49eded1896c3d48b0136ead37c48e221b391c052fba3f4055c367f60a6 \ + --hash=sha256:4cc2206b76b4f576934f0ed374b10d7ca5f457858b157ca52064bdfc26b9fc00 \ + --hash=sha256:679ae98e00c0e8d68a7fda324e16b90fd5260945b45d3b824c892cec9eea3288 + +# setuptools-84.0.0-py3-none-any.whl +setuptools==84.0.0 \ + --hash=sha256:51a52592b3b99e102b609654876bd65f19f999935166d1352678931132b0c670 + +# sniffio-1.3.1-py3-none-any.whl +sniffio==1.3.1 \ + --hash=sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2 + +# sse_starlette-3.5.0-py3-none-any.whl +sse-starlette==3.5.0 \ + --hash=sha256:3e6e1070df3f0f5d9cea81496de92dbb72f6721871d99748ece67441dd8b7997 + +# starlette-1.7.0-py3-none-any.whl +starlette==1.7.0 \ + --hash=sha256:67f8e99895493dd2911a03f11314af6ceebeae4e704bb9f43dfc6a9db151c93e + +# truststore-0.10.4-py3-none-any.whl +truststore==0.10.4 \ + --hash=sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981 + +# typing_inspection-0.4.4-py3-none-any.whl +typing-inspection==0.4.4 \ + --hash=sha256:65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147 + +# typing_extensions-4.16.0-py3-none-any.whl +typing_extensions==4.16.0 \ + --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 + +# urllib3-2.8.0-py3-none-any.whl +urllib3==2.8.0 \ + --hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3 + +# uvicorn-0.54.0-py3-none-any.whl +uvicorn==0.54.0 \ + --hash=sha256:505bdb0f318731d45f1f712071fc781a8981f6847a31c902c9f5e652d4f67faf + +# websockets-16.1.1-cp310-cp310-macosx_10_9_x86_64.whl +# websockets-16.1.1-cp310-cp310-macosx_11_0_arm64.whl +# websockets-16.1.1-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl +# websockets-16.1.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl +websockets==16.1.1 \ + --hash=sha256:1214e673c404684b9bf7154f5cf43b45025b1a6160fac3a9e438e9c1a97e22cb \ + --hash=sha256:5bfd1ac19b1b9986a9c95a82d5e23a391ebb09e12c34d7be6094b86efcc35731 \ + --hash=sha256:90001d893bc368e302ef168d82130b4e4fdd27b85fa094682df9b667c2d48838 \ + --hash=sha256:9246a0d063cfcbcc85f2359dd6876d681213f4790832272aa16641b4ed5d64d4 diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index 24fab1a3..3ef64447 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -4,12 +4,21 @@ import asyncio import json import math +import re import os from pathlib import Path import signal +import sys import time +import ipaddress +import socket +import ssl +from urllib.parse import urlsplit -from agents import Agent, Runner, RunConfig, RunHooks, ModelSettings, function_tool +from openai import AsyncOpenAI +from agents.models.openai_provider import OpenAIProvider + +from agents import Agent, Runner, RunConfig, RunHooks, ModelSettings, ModelRetrySettings, function_tool, WebSearchTool from dotenv import dotenv_values @@ -20,7 +29,11 @@ class UsageObservation(RunHooks): 'input_tokens_details.cache_write_tokens', 'output_tokens_details.reasoning_tokens') - def __init__(self): + def __init__(self, max_requests=None, max_tools=None, max_total_tokens=None): + self.max_requests = max_requests + self.max_tools = max_tools + self.max_total_tokens = max_total_tokens + self.tool_calls = 0 self.started = 0 self.completed = 0 self.duplicates = 0 @@ -30,6 +43,10 @@ def __init__(self): self.coverage = {} async def on_llm_start(self, context, agent, system_prompt, input_items): + if self.max_requests is not None and self.started >= self.max_requests: + raise BudgetExceeded() + if self.max_total_tokens is not None and self.totals.get("total_tokens", 0) >= self.max_total_tokens: + raise BudgetExceeded() self.started += 1 async def on_llm_end(self, context, agent, response): @@ -55,10 +72,15 @@ async def on_llm_end(self, context, agent, response): self.totals[field] = self.totals.get(field, 0) + value self.coverage[field] = self.coverage.get(field, 0) + 1 + async def on_tool_start(self, context, agent, tool): + if self.max_tools is not None and self.tool_calls >= self.max_tools: + raise BudgetExceeded() + self.tool_calls += 1 + def summary(self): return { 'source': 'sdk_completed_response_raw_usage', - 'aggregationScope': 'unique_completed_responses_in_this_runner_run', + 'aggregationScope': 'unique_completed_responses_in_parent_and_children', 'startedCallCount': self.started, 'completedResponseCount': self.completed, 'duplicateResponseCallbackCount': self.duplicates, @@ -70,83 +92,300 @@ def summary(self): } -async def shell(command, cwd, timeout, env): +class BudgetExceeded(Exception): + """An aggregate invocation budget was exhausted.""" + + +async def shell(command, cwd, timeout, env, output_limit=30000): process = await asyncio.create_subprocess_exec( '/bin/bash', '-c', command, cwd=cwd, env=env, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, start_new_session=True, ) + async def capture(stream): + kept = bytearray() + total = 0 + while True: + chunk = await stream.read(8192) + if not chunk: + return bytes(kept), total > output_limit + total += len(chunk) + kept.extend(chunk[:max(0, output_limit - len(kept))]) + readers = [asyncio.create_task(capture(stream)) for stream in (process.stdout, process.stderr)] + async def complete(): + captured = await asyncio.gather(*readers) + await process.wait() + return captured try: - stdout, stderr = await asyncio.wait_for(process.communicate(), timeout) - except (asyncio.TimeoutError, asyncio.CancelledError): + stdout, stderr = await asyncio.wait_for(complete(), timeout) + except BaseException: try: os.killpg(process.pid, signal.SIGKILL) except ProcessLookupError: pass + for reader in readers: + reader.cancel() + await asyncio.gather(*readers, return_exceptions=True) await process.wait() raise return {'exit_code': process.returncode, - 'stdout': stdout.decode(errors='replace')[:30000], - 'stderr': stderr.decode(errors='replace')[:30000]} + 'stdout': stdout[0].decode(errors='replace'), + 'stderr': stderr[0].decode(errors='replace'), + 'stdout_truncated': stdout[1], 'stderr_truncated': stderr[1]} + + +async def retrieve_public(url, timeout, output_limit=60000): + """HTTPS GET with DNS-pinned public addresses, bounded body, no redirects.""" + parsed = urlsplit(url) + if (parsed.scheme != 'https' or not parsed.hostname or parsed.username or + parsed.password or parsed.port not in (None, 443) or len(url) > 4096 or + any(ord(c) < 33 for c in url)): + raise ValueError('unsupported public URL') + host = parsed.hostname.encode('idna').decode('ascii') + async def exchange(): + addresses = await asyncio.get_running_loop().getaddrinfo(host, 443, type=socket.SOCK_STREAM) + if not addresses or any(not ipaddress.ip_address(a[4][0]).is_global for a in addresses): + raise ValueError('non-public address') + # Pin the vetted IP; TLS still validates the original hostname. + reader, writer = await asyncio.open_connection(addresses[0][4][0], 443, + ssl=ssl.create_default_context(), server_hostname=host, limit=16384) + try: + target = parsed.path or '/' + if parsed.query: + target += '?' + parsed.query + authority = '[' + host + ']' if ':' in host else host + writer.write(('GET ' + target + ' HTTP/1.1\r\nHost: ' + authority + + '\r\nUser-Agent: prose-agents-sdk/0.1.0\r\nAccept: text/html,text/plain,application/json\r\nAccept-Encoding: identity\r\nConnection: close\r\n\r\n').encode('ascii')) + await writer.drain() + headers = await reader.readuntil(b'\r\n\r\n') + lines = headers.decode('iso-8859-1').split('\r\n') + if not re.fullmatch(r'HTTP/1\.[01] [0-9]{3}(?: [^\r\n]*)?', lines[0]): + raise ValueError('invalid HTTP status') + status = int(lines[0].split(' ')[1]) + fields = {} + for line in lines[1:-2]: + if ':' not in line or line.startswith((' ', '\t')): + raise ValueError('invalid HTTP header') + name, value = line.split(':', 1) + if not re.fullmatch(r"[!#$%&'*+.^_`|~0-9A-Za-z-]+", name): + raise ValueError('invalid HTTP header name') + name = name.lower() + if name in fields and name in ('content-length', 'transfer-encoding', 'content-encoding'): + raise ValueError('duplicate HTTP framing header') + fields[name] = value.strip().lower() + if status != 200: + return {'status': status, 'error': 'HTTP response is not 200; redirects are not followed'} + if fields.get('content-encoding', 'identity') != 'identity': + return {'status': status, 'error': 'compressed response is unsupported'} + transfer = fields.get('transfer-encoding') + length = fields.get('content-length') + if transfer is not None and (transfer != 'chunked' or length is not None): + raise ValueError('unsupported or conflicting HTTP framing') + if length is not None and not re.fullmatch(r'[0-9]+', length): + raise ValueError('invalid HTTP content length') + body = bytearray() + if transfer == 'chunked': + while len(body) <= output_limit: + size_line = await reader.readline() + size_text = size_line.split(b';', 1)[0].strip() + if not size_line.endswith(b'\r\n') or not re.fullmatch(rb'[0-9A-Fa-f]+', size_text): + raise ValueError('invalid HTTP chunk size') + size = int(size_text, 16) + if size == 0: + trailer_bytes = 0 + while True: + trailer = await reader.readline() + trailer_bytes += len(trailer) + if not trailer.endswith(b'\r\n') or trailer_bytes > 16384: + raise ValueError('invalid or oversized HTTP chunk trailer') + if trailer == b'\r\n': + break + if b':' not in trailer or trailer.startswith((b' ', b'\t')): + raise ValueError('invalid HTTP chunk trailer') + break + take = min(size, output_limit + 1 - len(body)) + body.extend(await reader.readexactly(take)) + if take < size: + break + if await reader.readexactly(2) != b'\r\n': + raise ValueError('invalid HTTP chunk') + elif length is not None: + body.extend(await reader.readexactly(min(int(length), output_limit + 1))) + else: + while len(body) <= output_limit: + chunk = await reader.read(min(8192, output_limit + 1 - len(body))) + if not chunk: + break + body.extend(chunk) + return {'status': status, 'content': bytes(body[:output_limit]).decode('utf-8', errors='replace'), + 'truncated': len(body) > output_limit} + finally: + writer.close() + try: + await asyncio.wait_for(writer.wait_closed(), 0.25) + except (asyncio.TimeoutError, OSError): + pass + return await asyncio.wait_for(exchange(), timeout) + + +class GuardedResponses: + """Guard the actual SDK request after instruction/history/tool conversion.""" + def __init__(self, resource, max_input_bytes, observed_models): + self.resource = resource + self.max_input_bytes = max_input_bytes + self.observed_models = observed_models + + async def create(self, **kwargs): + payload = {field: kwargs[field] for field in ('instructions', 'input', 'tools', 'text') + if field in kwargs and isinstance(kwargs[field], (str, list, dict))} + if len(json.dumps(payload, ensure_ascii=False).encode('utf-8')) > self.max_input_bytes: + raise BudgetExceeded() + response = await self.resource.create(**kwargs) + model = getattr(response, 'model', None) + if isinstance(model, str) and 0 < len(model) <= 128 and all(c.isalnum() or c in '._-/' for c in model): + self.observed_models.add(model) + return response async def run(args): - # Capture tool environment before loading provider credential; never give it to shell. + # This filters environment inheritance, not filesystem/network permissions. tool_env = {k: v for k, v in os.environ.items() if not any(s in k.upper() for s in ('KEY', 'TOKEN', 'SECRET', 'PASSWORD'))} - if args.env_file: - key = dotenv_values(args.env_file).get('OPENAI_API_KEY') - if key: - os.environ['OPENAI_API_KEY'] = key - cwd = str(Path(args.cwd).resolve()) - instructions = 'You are a helpful coding agent. Use available tools to complete the user request.' - if args.instructions: - instructions += '\n\n' + Path(args.instructions).read_text() - instructions += '\nYour working directory is: ' + cwd + for name in ('LD_LIBRARY_PATH', 'DYLD_LIBRARY_PATH'): + original = os.environ.get(name + '_ORIG') + tool_env.pop(name, None) + tool_env.pop(name + '_ORIG', None) + if getattr(sys, "frozen", False) and original: + tool_env[name] = original start = time.monotonic() + observed_models = set() def emit(kind, **data): + data['modelIdentity'] = {'requested': args.model, 'observed': sorted(observed_models)} print(json.dumps({'type': kind, 'event': kind, 'elapsed_seconds': round(time.monotonic()-start, 3), **data}), flush=True) - - @function_tool + max_input_bytes = getattr(args, 'max_input_bytes', 256000) + max_tools = getattr(args, 'max_tools', 80) + max_total_tokens = getattr(args, 'max_total_tokens', 500000) + limits = {'maxTurns': args.max_turns, 'timeoutSeconds': args.timeout, + 'toolTimeoutSeconds': args.tool_timeout, 'maxOutputTokens': args.max_output_tokens, + 'maxAggregateRequests': args.max_turns, 'maxAggregateHostedWebCalls': args.max_turns, 'maxAggregateFunctionTools': max_tools, + 'maxObservedTotalTokens': max_total_tokens, 'maxRequestInputBytes': max_input_bytes, 'maxChildren': 8, 'maxChildDepth': 1} + observation = UsageObservation(args.max_turns, max_tools, max_total_tokens) + def fail(kind, message): + emit('error', error_type=kind, message=message, limits=limits, + usageObservation=observation.summary()) + return 1 + try: + key = dotenv_values(args.env_file).get('OPENAI_API_KEY') if args.env_file else os.environ.get('OPENAI_API_KEY') + if not isinstance(key, str) or not key.strip(): + return fail('SetupError', 'Set OPENAI_API_KEY to an OpenAI API key, or configure the OpenAI API-key credential profile. No model request was sent.') + cwd = str(Path(args.cwd).resolve(strict=True)) + if not Path(cwd).is_dir(): + return fail('SetupError', 'The working directory must exist.') + instructions = 'You are a helpful coding agent. Use available tools to complete the user request.' + if args.instructions: + with Path(args.instructions).open('rb') as source: + raw = source.read(max_input_bytes + 1) + if len(raw) > max_input_bytes: + return fail('SetupError', 'The instruction file exceeds the configured request input-byte budget.') + instructions += '\n\n' + raw.decode('utf-8') + instructions += '\nYour working directory is: ' + cwd + except (OSError, ValueError): + return fail('SetupError', 'Cannot read the working directory, instructions, or credential file. Check the supplied paths and permissions.') + deadline = start + args.timeout + children = 0 + child_usage = [] + child_lock = asyncio.Lock() + def remaining(): + return max(.001, deadline - time.monotonic()) + @function_tool(failure_error_function=None) async def execute_shell(command: str) -> str: - """Execute a bash command in the working directory. Read and edit files using ordinary shell tools.""" + """Execute bash in the working directory with host OS permissions; read/edit files. This is not a sandbox.""" + if len(command.encode()) > 30000: + return json.dumps({'error': 'shell command exceeds 30000 bytes'}) emit('tool_call', name='execute_shell', command=command) try: - result = await shell(command, cwd, args.tool_timeout, tool_env) + result = await shell(command, cwd, min(args.tool_timeout, remaining()), tool_env) except asyncio.TimeoutError: result = {'error': 'shell timeout; process group terminated'} + except OSError: + result = {'error': 'shell could not start; check working directory and bash availability'} emit('tool_result', name='execute_shell', result=result) return json.dumps(result) - - limits = {'maxTurns': args.max_turns, 'timeoutSeconds': args.timeout, - 'toolTimeoutSeconds': args.tool_timeout, 'maxOutputTokens': args.max_output_tokens} - emit('start', model=args.model, cwd=cwd, limits=limits) - observation = UsageObservation() - agent = Agent(name='Local coding agent', instructions=instructions, - model=args.model, tools=[execute_shell], - model_settings=ModelSettings(max_tokens=args.max_output_tokens, timeout=args.timeout, - preserve_raw_usage=True)) + @function_tool(failure_error_function=None) + async def retrieve_url(url: str) -> str: + """Retrieve text from a public HTTPS URL. Private addresses, credentials, non-443 ports and redirects are rejected.""" + emit('tool_call', name='retrieve_url', url=url[:4096]) + try: + result = await retrieve_public(url, min(args.tool_timeout, remaining())) + except (OSError, ValueError, asyncio.TimeoutError, asyncio.IncompleteReadError, asyncio.LimitOverrunError): + result = {'error': 'public retrieval failed; check the HTTPS URL, server availability and response format'} + emit('tool_result', name='retrieve_url', result=result) + return json.dumps(result) + client = AsyncOpenAI(api_key=key.strip(), max_retries=0, timeout=args.timeout) + client.responses = GuardedResponses(client.responses, max_input_bytes, observed_models) + config = RunConfig(tracing_disabled=True, model_provider=OpenAIProvider(openai_client=client)) + def make_agent(depth): + @function_tool(failure_error_function=None) + async def delegate(task: str) -> str: + """Run an independent fresh agent context. Supply all needed task inputs explicitly; no parent conversation is inherited. Files remain shared.""" + nonlocal children + if depth >= 1 or children >= 8: + raise BudgetExceeded() + if len(task.encode()) > 60000: + return json.dumps({'error': 'child input exceeds 60000 bytes'}) + # A single lock covers the complete child: sibling provider calls never overlap. + async with child_lock: + if children >= 8: + raise BudgetExceeded() + children += 1 + child_id = children + emit('tool_call', name='delegate', child_id=child_id, depth=depth + 1) + child = make_agent(depth + 1) + result = await Runner.run(child, task, max_turns=args.max_turns, + run_config=config, hooks=observation) + child_usage.append(result.context_wrapper.usage) + output = str(result.final_output) + if len(output.encode()) > 60000: + raise BudgetExceeded() + emit('tool_result', name='delegate', child_id=child_id, result=output) + return output + return Agent(name='Local coding agent', instructions=instructions, model=args.model, + tools=[execute_shell, retrieve_url, WebSearchTool(search_context_size='low')] + ([delegate] if depth == 0 else []), + model_settings=ModelSettings(max_tokens=args.max_output_tokens, timeout=args.timeout, + preserve_raw_usage=True, parallel_tool_calls=False, + retry=ModelRetrySettings(max_retries=0), + extra_args={'max_tool_calls': 1})) + emit('start', model=args.model, cwd=cwd, limits=limits, + permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, + 'networkSandbox': False, 'freshChildConversation': True}) try: - result = await asyncio.wait_for( - Runner.run(agent, args.prompt, max_turns=args.max_turns, - run_config=RunConfig(tracing_disabled=True), hooks=observation), args.timeout) - usage = result.context_wrapper.usage + result = await asyncio.wait_for(Runner.run(make_agent(0), args.prompt, + max_turns=args.max_turns, run_config=config, hooks=observation), remaining()) + normalized = [result.context_wrapper.usage] + child_usage + usage = {field: sum(getattr(value, field) for value in normalized) for field in ('requests', 'input_tokens', 'output_tokens', 'total_tokens')} emit('final', output=result.final_output, usage={ - 'requests': usage.requests, 'input_tokens': usage.input_tokens, - 'output_tokens': usage.output_tokens, 'total_tokens': usage.total_tokens}, + 'requests': usage['requests'], 'input_tokens': usage['input_tokens'], + 'output_tokens': usage['output_tokens'], 'total_tokens': usage['total_tokens']}, usageObservation=observation.summary()) return 0 + except asyncio.CancelledError: + return fail('CancelledError', 'Execution cancelled; active tool process groups terminated.') except Exception as error: - # Exception bodies may contain request/credential information; type suffices for public log. - emit('error', error_type=type(error).__name__ if type(error).__name__ in ('MaxTurnsExceeded', 'TimeoutError') else 'ExecutionError', limits=limits, - usageObservation=observation.summary()) - return 1 + name = type(error).__name__ + if name in ('AuthenticationError', 'PermissionDeniedError'): + return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.') + if name == 'NotFoundError': + return fail('SetupError', 'The configured OpenAI model is unavailable to this account. Check the model selection and account access; no fallback was selected.') + safe = name if name in ('MaxTurnsExceeded', 'TimeoutError', 'BudgetExceeded') else 'ExecutionError' + return fail(safe, 'Execution stopped. Check configured limits, tool availability and OpenAI account access.') + finally: + await client.close() def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument('--version', action='version', version='prose-agents-sdk 0.1.0') - parser.add_argument('--model', required=True) + parser.add_argument('--model', default='gpt-6.1-sol') parser.add_argument('--cwd', required=True) parser.add_argument('--instructions', help='Opaque text appended to generic instructions') parser.add_argument('--prompt', required=True) @@ -155,12 +394,27 @@ def main(): parser.add_argument('--tool-timeout', type=float, default=30) parser.add_argument('--max-turns', type=int, default=20) parser.add_argument('--max-output-tokens', type=int, default=12000) + parser.add_argument('--max-input-bytes', type=int, default=256000) + parser.add_argument('--max-tools', type=int, default=80) + parser.add_argument('--max-total-tokens', type=int, default=500000) args = parser.parse_args() if not 0 < args.max_turns <= 9007199254740991: parser.error('--max-turns must be a positive safe integer') if not math.isfinite(args.timeout) or not 0 < args.timeout <= 9007199254740.991: parser.error('--timeout must be positive and finite within the supported range') - raise SystemExit(asyncio.run(run(args))) + for label, value in (('--tool-timeout', args.tool_timeout),): + if not math.isfinite(value) or not 0 < value <= 9007199254740.991: + parser.error(label + ' must be positive and finite within the supported range') + for label, value in (('--max-input-bytes', args.max_input_bytes), ('--max-output-tokens', args.max_output_tokens), ('--max-tools', args.max_tools), ('--max-total-tokens', args.max_total_tokens)): + if not 0 < value <= 9007199254740991: + parser.error(label + ' must be a positive safe integer') + async def supervised(): + task = asyncio.create_task(run(args)) + loop = asyncio.get_running_loop() + for sig in (signal.SIGINT, signal.SIGTERM): + loop.add_signal_handler(sig, task.cancel) + return await task + raise SystemExit(asyncio.run(supervised())) if __name__ == '__main__': main() diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index c01cda95..fe0b4fff 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -10,6 +10,7 @@ import json from unittest.mock import AsyncMock, patch from types import SimpleNamespace +from agents.tool_context import ToolContext spec = importlib.util.spec_from_file_location('harness', Path(__file__).with_name('run.py')) harness = importlib.util.module_from_spec(spec) @@ -31,7 +32,29 @@ async def test_timeout_stops_delayed_effect(self): await asyncio.sleep(0.6) self.assertFalse(Path(directory, 'late').exists()) + async def test_large_output_is_bounded_while_pipe_drains(self): + with tempfile.TemporaryDirectory() as directory: + result = await harness.shell("head -c 2000000 /dev/zero", directory, 2, {'PATH': os.environ['PATH']}, output_limit=4096) + self.assertEqual(len(result['stdout']), 4096) + self.assertTrue(result['stdout_truncated']) + self.assertEqual(result['exit_code'], 0) + + async def test_cancellation_stops_delayed_descendant_effect(self): + with tempfile.TemporaryDirectory() as directory: + task = asyncio.create_task(harness.shell('(sleep 0.4; touch late) & wait', directory, 2, {'PATH': os.environ['PATH']})) + await asyncio.sleep(.05) + task.cancel() + with self.assertRaises(asyncio.CancelledError): + await task + await asyncio.sleep(.5) + self.assertFalse(Path(directory, 'late').exists()) + class BudgetTest(unittest.IsolatedAsyncioTestCase): + def setUp(self): + self.env = patch.dict(os.environ, {'OPENAI_API_KEY': 'fixture-no-provider-access'}) + self.env.start() + self.addCleanup(self.env.stop) + def args(self, directory): return argparse.Namespace(cwd=directory,model='fixture',instructions=None,env_file=None, prompt='opaque request',max_turns=40,timeout=300,tool_timeout=30,max_output_tokens=12000) @@ -44,7 +67,7 @@ async def test_forwarding_and_success_limits(self): self.assertEqual(code,0) self.assertEqual(run.call_args.kwargs['max_turns'],40) records=[json.loads(line) for line in output.getvalue().splitlines()] - self.assertEqual(records[0]['limits'],dict(maxTurns=40,timeoutSeconds=300,toolTimeoutSeconds=30,maxOutputTokens=12000)) + self.assertEqual(records[0]['limits']['maxAggregateRequests'],40) self.assertEqual(records[-1]['type'],'final') self.assertEqual(records[-1]['usage'], dict(requests=1,input_tokens=1,output_tokens=1,total_tokens=2)) self.assertEqual(records[-1]['usageObservation']['observedTokenTotals'], {}) @@ -75,7 +98,7 @@ async def runner(agent, prompt, **kwargs): hooks = kwargs['hooks'] self.assertTrue(agent.model_settings.preserve_raw_usage) self.assertEqual(agent.instructions, 'You are a helpful coding agent. Use available tools to complete the user request.\nYour working directory is: ' + str(Path(directory).resolve())) - self.assertEqual([tool.name for tool in agent.tools], ['execute_shell']) + self.assertEqual([tool.name for tool in agent.tools], ['execute_shell', 'retrieve_url', 'web_search', 'delegate']) self.assertEqual(agent.tools[0].params_json_schema['properties'], {'command': {'title': 'Command', 'type': 'string'}}) for index in range(2): await hooks.on_llm_start(None, agent, 'secret prompt', []) @@ -113,6 +136,276 @@ async def test_unobserved_error_has_no_invented_zero_tokens(self): self.assertEqual(stats['fieldResponseCounts'], {}) self.assertFalse(stats['totalRunUsageKnown']) + async def test_missing_key_fails_before_runner(self): + with tempfile.TemporaryDirectory() as directory, patch.dict(os.environ, {}, clear=True), patch.object(harness.Runner, 'run', AsyncMock()) as runner: + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 1) + runner.assert_not_called() + self.assertIn('OPENAI_API_KEY', output.getvalue()) + self.assertEqual(json.loads(output.getvalue())['error_type'], 'SetupError') + + async def test_children_are_fresh_and_aggregate_usage(self): + calls = [] + result = SimpleNamespace(final_output='done', context_wrapper=SimpleNamespace(usage=SimpleNamespace(requests=1,input_tokens=3,output_tokens=2,total_tokens=5))) + async def runner(agent, prompt, **kwargs): + calls.append((agent, prompt, kwargs)) + hooks = kwargs['hooks'] + await hooks.on_llm_start(None, agent, None, []) + await hooks.on_llm_end(None, agent, SimpleNamespace(response_id=str(len(calls)), raw_usage={'input_tokens':3, 'output_tokens':2, 'total_tokens':5})) + if len(calls) == 1: + delegate = next(tool for tool in agent.tools if tool.name == 'delegate') + await delegate.on_invoke_tool(ToolContext(context=None, tool_name='delegate', tool_call_id='fixture', tool_arguments='{}'), '{"task":"explicit child input"}') + return result + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 0) + self.assertEqual(calls[1][1], 'explicit child input') + self.assertIsNot(calls[0][0], calls[1][0]) + self.assertEqual(calls[0][0].instructions, calls[1][0].instructions) + self.assertNotIn('delegate', [tool.name for tool in calls[1][0].tools]) + self.assertIs(calls[0][2]['hooks'], calls[1][2]['hooks']) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['usage']['requests'], 2) + self.assertEqual(final['usage']['total_tokens'], 10) + + async def test_real_sdk_loop_delegates_without_parent_history_or_retries(self): + from openai.types.responses import Response + def response(index, output): + return Response.model_validate(dict(id=str(index), object='response', created_at=1, + model='fixture', output=output, parallel_tool_calls=False, tool_choice='auto', tools=[], + status='completed', usage=dict(input_tokens=3, output_tokens=2, total_tokens=5, + input_tokens_details=dict(cached_tokens=0,cache_write_tokens=0), output_tokens_details=dict(reasoning_tokens=0)))) + def text(value): + return dict(type='message', id=value, role='assistant', status='completed', content=[dict(type='output_text',text=value,annotations=[])]) + create = AsyncMock(side_effect=[response(1,[dict(type='function_call',id='call',call_id='child',name='delegate',arguments='{"task":"isolated input"}',status='completed')]), response(2,[text('child result')]), response(3,[text('parent result')])]) + client = SimpleNamespace(responses=SimpleNamespace(create=create), base_url='https://api.openai.com/v1/', close=AsyncMock()) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client) as factory: + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 0) + self.assertEqual(factory.call_args.kwargs['max_retries'], 0) + requests = [call.kwargs for call in create.call_args_list] + child_input = json.dumps(requests[1]['input']) + self.assertIn('isolated input', child_input) + self.assertNotIn('opaque request', child_input) + self.assertNotIn('parent result', child_input) + self.assertTrue(all(request['max_tool_calls'] == 1 for request in requests)) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['usage']['requests'], 3) + self.assertEqual(final['usage']['total_tokens'], 15) + self.assertEqual(final['usageObservation']['completedResponseCount'], 3) + + async def test_real_sdk_conversation_lock_error_is_not_retried(self): + import httpx2 as httpx + from openai import BadRequestError + error = BadRequestError('private-provider-message', response=httpx.Response(400, + request=httpx.Request('POST', 'https://example.invalid/v1/responses')), + body={'code': 'conversation_locked', 'message': 'private-provider-message'}) + create = AsyncMock(side_effect=error) + client = SimpleNamespace(responses=SimpleNamespace(create=create), base_url='https://api.openai.com/v1/', close=AsyncMock()) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client): + output = io.StringIO() + args = self.args(directory) + args.max_turns = 1 + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(args), 1) + self.assertEqual(create.await_count, 1) + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['usageObservation']['startedCallCount'], 1) + self.assertNotIn('private-provider-message', output.getvalue()) + client.close.assert_awaited_once() + + async def test_real_sdk_active_child_shell_cancel_preserves_usage_and_kills_effect(self): + from openai.types.responses import Response + def response(number,name,arguments): + return Response.model_validate(dict(id=str(number),object='response',created_at=1, + model='fixture',output=[dict(type='function_call',id='call'+str(number),call_id='call'+str(number),name=name,arguments=json.dumps(arguments),status='completed')],parallel_tool_calls=False,tool_choice='auto',tools=[],status='completed', + usage=dict(input_tokens=3,output_tokens=2,total_tokens=5,input_tokens_details=dict(cached_tokens=0,cache_write_tokens=0),output_tokens_details=dict(reasoning_tokens=0)))) + create=AsyncMock(side_effect=[response(1,'delegate',{'task':'explicit child only'}),response(2,'execute_shell',{'command':'(sleep 0.4; touch late) & wait'})]) + client=SimpleNamespace(responses=SimpleNamespace(create=create),base_url='https://api.openai.com/v1/',close=AsyncMock()) + started=asyncio.Event() + actual_shell=harness.shell + async def tracked_shell(*args,**kwargs): + # Set after process creation has had a chance to settle; cancellation remains in actual shell. + task=asyncio.create_task(actual_shell(*args,**kwargs)) + await asyncio.sleep(.03) + started.set() + return await task + with tempfile.TemporaryDirectory() as directory, patch.object(harness,'AsyncOpenAI',return_value=client), patch.object(harness,'shell',tracked_shell): + output=io.StringIO() + with contextlib.redirect_stdout(output): + task=asyncio.create_task(harness.run(self.args(directory))) + await asyncio.wait_for(started.wait(),1) + task.cancel() + self.assertEqual(await asyncio.wait_for(task,1),1) + await asyncio.sleep(.5) + self.assertFalse(Path(directory,'late').exists()) + records=[json.loads(line) for line in output.getvalue().splitlines()] + self.assertEqual(records[-1]['error_type'],'CancelledError') + self.assertEqual(records[-1]['usageObservation']['completedResponseCount'],2) + self.assertEqual(records[-1]['usageObservation']['observedTokenTotals']['total_tokens'],10) + self.assertFalse(any(record['type']=='final' for record in records)) + self.assertEqual(create.await_count,2) + client.close.assert_awaited_once() + + async def test_actual_request_input_guard_includes_tools_and_history(self): + resource = SimpleNamespace(create=AsyncMock()) + observed = set() + guard = harness.GuardedResponses(resource, 80, observed) + for field in ('instructions', 'input', 'tools'): + with self.assertRaises(harness.BudgetExceeded): + await guard.create(**{field: 'a' * 100}) + resource.create.assert_not_called() + resource.create.return_value = SimpleNamespace(model='gpt-fixture') + await guard.create(instructions='opaque', input=[], tools=[]) + self.assertEqual(observed, {'gpt-fixture'}) + + async def test_aggregate_request_and_tool_budgets(self): + observation = harness.UsageObservation(max_requests=1, max_tools=1) + await observation.on_llm_start(None, None, None, []) + with self.assertRaises(harness.BudgetExceeded): + await observation.on_llm_start(None, None, None, []) + await observation.on_tool_start(None, None, None) + with self.assertRaises(harness.BudgetExceeded): + await observation.on_tool_start(None, None, None) + self.assertEqual(observation.started, 1) + self.assertEqual(observation.tool_calls, 1) + + async def test_observed_token_limit_blocks_next_request(self): + observation = harness.UsageObservation(max_total_tokens=5) + await observation.on_llm_end(None, None, SimpleNamespace(response_id='one', raw_usage={'total_tokens': 5})) + with self.assertRaises(harness.BudgetExceeded): + await observation.on_llm_start(None, None, None, []) + + async def test_public_retrieval_rejects_private_and_unsafe_urls(self): + for url in ('http://example.com', 'https://user:pass@example.com', 'https://example.com:444/', 'https://example.com/\r\nInjected'): + with self.assertRaises(ValueError): + await harness.retrieve_public(url, .1) + with patch.object(asyncio.get_running_loop(), 'getaddrinfo', AsyncMock(return_value=[(None,None,None,None,('127.0.0.1',443))])), patch.object(asyncio, 'open_connection', AsyncMock()) as connect: + with self.assertRaises(ValueError): + await harness.retrieve_public('https://example.com', .1) + connect.assert_not_called() + + async def test_cancelled_parent_has_safe_terminal_error(self): + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', AsyncMock(side_effect=asyncio.CancelledError())): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 1) + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'CancelledError') + + async def test_auth_and_model_failures_do_not_leak_bodies(self): + for name in ('AuthenticationError', 'PermissionDeniedError', 'NotFoundError'): + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', AsyncMock(side_effect=type(name, (Exception,), {})('credential-value'))): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 1) + self.assertNotIn('credential-value', output.getvalue()) + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'SetupError') + + +class RetrievalTest(unittest.IsolatedAsyncioTestCase): + def connection(self, payload=None): + reader = asyncio.StreamReader(limit=16384) + if payload is not None: + reader.feed_data(payload) + reader.feed_eof() + writer = SimpleNamespace(write=unittest.mock.Mock(), drain=AsyncMock(), + close=unittest.mock.Mock(), wait_closed=AsyncMock()) + return reader, writer + + async def request(self, payload, output_limit=60000): + reader, writer = self.connection(payload) + lookup = AsyncMock(return_value=[(None,None,None,None,('93.184.216.34',443))]) + with patch.object(asyncio.get_running_loop(), 'getaddrinfo', lookup), patch.object(asyncio, 'open_connection', AsyncMock(return_value=(reader, writer))) as connect: + result = await harness.retrieve_public('https://example.com/path?q=fixture', .2, output_limit) + self.assertEqual(connect.call_args.args, ('93.184.216.34',443)) + self.assertEqual(connect.call_args.kwargs['server_hostname'], 'example.com') + self.assertTrue(connect.call_args.kwargs['ssl'].check_hostname) + self.assertEqual(connect.call_args.kwargs['ssl'].verify_mode, __import__('ssl').CERT_REQUIRED) + self.assertIn(b'GET /path?q=fixture HTTP/1.1\r\nHost: example.com\r\n', writer.write.call_args.args[0]) + writer.close.assert_called_once() + writer.wait_closed.assert_awaited_once() + return result + + async def test_successful_content_length_chunked_and_eof_framing(self): + for payload in (b'HTTP/1.1 200 OK\r\nContent-Length: 7\r\n\r\nfixture', + b'HTTP/1.0 200 OK\r\n\r\nfixture', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n3\r\nfix\r\n4;ext=1\r\nture\r\n0\r\nX-Test: yes\r\n\r\n'): + with self.subTest(payload=payload): + self.assertEqual(await self.request(payload), {'status':200,'content':'fixture','truncated':False}) + + async def test_truncation_is_explicit_for_each_framing(self): + for payload in (b'HTTP/1.1 200 OK\r\nContent-Length: 7\r\n\r\nfixture', + b'HTTP/1.0 200 OK\r\n\r\nfixture', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n7\r\nfixture\r\n0\r\n\r\n'): + with self.subTest(payload=payload): + self.assertEqual(await self.request(payload,3), {'status':200,'content':'fix','truncated':True}) + + async def test_incomplete_and_malformed_framing_cannot_claim_complete_content(self): + for payload in (b'HTTP/1.1 200 OK\r\nContent-Length: 8\r\n\r\nshort', + b'HTTP/1.1 200 OK\r\nContent-Length: -1\r\n\r\n', + b'HTTP/1.1 200 OK\r\nContent-Length: 1\r\nContent-Length: 1\r\n\r\nx', + b'HTTP/1.1 200 OK\r\nContent-Length: 1\r\nTransfer-Encoding: chunked\r\n\r\nx', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: gzip\r\n\r\nx', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n7\r\nshort', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n1\r\nxNO', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\nz\r\n', + b'HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n0\r\n', + b'garbage\r\n\r\n', b'HTTP/1.1 200 OK\r\nmalformed\r\n\r\n'): + with self.subTest(payload=payload): + with self.assertRaises((ValueError, asyncio.IncompleteReadError)): + await self.request(payload) + + async def test_redirects_and_compressed_content_do_not_return_source_content(self): + for payload in (b'HTTP/1.1 302 Found\r\nLocation: https://other.example\r\n\r\n', + b'HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\n\r\n'): + result = await self.request(payload) + self.assertIn('error', result) + self.assertNotIn('content', result) + + async def test_mixed_private_dns_rejected_without_connection(self): + addresses = [(None,None,None,None,('93.184.216.34',443)),(None,None,None,None,('192.168.1.1',443))] + with patch.object(asyncio.get_running_loop(),'getaddrinfo',AsyncMock(return_value=addresses)), patch.object(asyncio,'open_connection',AsyncMock()) as connect: + with self.assertRaises(ValueError): + await harness.retrieve_public('https://example.com',.2) + connect.assert_not_called() + + async def test_timeout_and_cancellation_close_established_connection(self): + for cancel in (False, True): + reader, writer = self.connection() + lookup = AsyncMock(return_value=[(None,None,None,None,('93.184.216.34',443))]) + with self.subTest(cancel=cancel), patch.object(asyncio.get_running_loop(),'getaddrinfo',lookup), patch.object(asyncio,'open_connection',AsyncMock(return_value=(reader,writer))): + task = asyncio.create_task(harness.retrieve_public('https://example.com',.05 if not cancel else 2)) + if cancel: + await asyncio.sleep(.01) + task.cancel() + with self.assertRaises(asyncio.CancelledError if cancel else asyncio.TimeoutError): + await task + writer.close.assert_called_once() + writer.wait_closed.assert_awaited_once() + + async def test_malformed_response_is_a_safe_tool_error_with_real_sdk_loop(self): + from openai.types.responses import Response + def response(number,output): + return Response.model_validate(dict(id=str(number),object='response',created_at=1, + model='fixture',output=output,parallel_tool_calls=False,tool_choice='auto',tools=[],status='completed', + usage=dict(input_tokens=3,output_tokens=2,total_tokens=5,input_tokens_details=dict(cached_tokens=0,cache_write_tokens=0),output_tokens_details=dict(reasoning_tokens=0)))) + create = AsyncMock(side_effect=[response(1,[dict(type='function_call',id='call',call_id='retrieve',name='retrieve_url',arguments='{"url":"https://example.com"}',status='completed')]),response(2,[dict(type='message',id='final',role='assistant',status='completed',content=[dict(type='output_text',text='reported safe retrieval error',annotations=[])])])]) + client = SimpleNamespace(responses=SimpleNamespace(create=create),base_url='https://api.openai.com/v1/',close=AsyncMock()) + reader, writer = self.connection(b'bad-status\r\n\r\n') + args = argparse.Namespace(cwd=None,model='fixture',instructions=None,env_file=None,prompt='opaque request',max_turns=3,timeout=2,tool_timeout=.2,max_output_tokens=100) + with tempfile.TemporaryDirectory() as directory, patch.dict(os.environ,{'OPENAI_API_KEY':'fixture-no-provider-access'}), patch.object(harness,'AsyncOpenAI',return_value=client), patch.object(asyncio.get_running_loop(),'getaddrinfo',AsyncMock(return_value=[(None,None,None,None,('93.184.216.34',443))])), patch.object(asyncio,'open_connection',AsyncMock(return_value=(reader,writer))): + args.cwd=directory + output=io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(args),0) + records=[json.loads(line) for line in output.getvalue().splitlines()] + tool_result=next(record for record in records if record['type']=='tool_result') + self.assertIn('error',tool_result['result']) + self.assertNotIn('bad-status',output.getvalue()) + + class UsageTest(unittest.IsolatedAsyncioTestCase): async def test_per_response_not_context_duplicates_and_missing_fields(self): observation = harness.UsageObservation() @@ -128,7 +421,7 @@ async def test_per_response_not_context_duplicates_and_missing_fields(self): self.assertEqual(stats['duplicateResponseCallbackCount'], 1) self.assertEqual(stats['observedTokenTotals'], {'input_tokens': 14, 'output_tokens': 0}) self.assertEqual(stats['fieldResponseCounts'], {'input_tokens': 2, 'output_tokens': 1}) - self.assertEqual(stats['aggregationScope'], 'unique_completed_responses_in_this_runner_run') + self.assertEqual(stats['aggregationScope'], 'unique_completed_responses_in_parent_and_children') async def test_response_without_provider_identity_deduplicated_by_object(self): observation = harness.UsageObservation() From dc384ea17003ae54fd468e7cae90803d3352646a Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 19:12:58 -0400 Subject: [PATCH 04/64] Preserve SDK request guards across client clones and pin standard tier --- cli/protocol/OWNERSHIP.md | 2 + harnesses/agents-sdk/README.md | 6 +-- harnesses/agents-sdk/run.py | 32 +++++++++++-- harnesses/agents-sdk/test_run.py | 82 ++++++++++++++++++++++++++++++++ 4 files changed, 115 insertions(+), 7 deletions(-) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index d084ea2a..b2e19b52 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -961,3 +961,5 @@ IMP-097 Bun measured compiled-identity fixture extension: `/root/configuration_c IMP-097 root fixture consistency extension: `cli/shared/fixtures/operations/doctor-report.json` configuration subobject only, to match the same resolver explanation oracle required by existing shared contract. IMP-097 Rust measured fixture reconciliation: `/root/sdk_final_review` owns `cli/rust/crates/prose-cli/tests/cli.rs` in place of root, limited to observed existing expectation changes for canonical paths, full explanation and earlier safe semantic validation. Preserve independent safety/lifecycle checks and report actual source failures. Root retains all production Rust. + +IMP-098 predictable provider tier wave: `/root/sdk_runtime` reclaims exact `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` for explicitly requested standard Responses service tier and safe requested/observed tier diagnostics, with pinned SDK tests. This supersedes earlier completed runtime repair leases. No new budget/currency settings, schema/product/Git edits or paid calls. Root freezes outer observation contract after this source change. diff --git a/harnesses/agents-sdk/README.md b/harnesses/agents-sdk/README.md index 94d252a5..43494bc8 100644 --- a/harnesses/agents-sdk/README.md +++ b/harnesses/agents-sdk/README.md @@ -2,7 +2,7 @@ This generic `Agent` + `Runner` loop transports the caller's instructions as opaque text. It does not interpret OpenProse contracts, discover instruction files, assess fulfillment or supply company procedures. The CLI release packages a native `prose-agents-sdk` sibling executable with Python and its pinned dependencies; normal users do not provision a virtual environment. Development from source uses the release-pinned Python 3.10.20 and `pip install -r requirements.txt`. -Supply an OpenAI API key through `OPENAI_API_KEY` or `--env-file FILE`. Only `OPENAI_API_KEY` is read from that file, without changing the parent environment. Missing credentials fail before a model request. Rejected credentials and unavailable models produce actionable setup errors, with no credential/model fallback or exception bodies. The standalone harness model default is `gpt-6.1-sol`; the outer CLI passes its resolved model explicitly. Tracing export is disabled, and the OpenAI client and SDK runner have retries explicitly disabled, including SDK conversation-lock compatibility retries. +Supply an OpenAI API key through `OPENAI_API_KEY` or `--env-file FILE`. Only `OPENAI_API_KEY` is read from that file, without changing the parent environment. Missing credentials fail before a model request. Rejected credentials and unavailable models produce actionable setup errors, with no credential/model fallback or exception bodies. The standalone harness model default is `gpt-6.1-sol`; the outer CLI passes its resolved model explicitly. Tracing export is disabled, and the OpenAI client and SDK runner have retries explicitly disabled, including SDK conversation-lock compatibility retries. Every parent and child Responses request explicitly sets `service_tier=default` through the SDK's `ModelSettings.extra_args`. This selects OpenAI's standard pricing/performance rather than inheriting a project's `auto` tier. The choice uses the existing [Responses service-tier option](https://developers.openai.com/api/reference/python/resources/responses/methods/create), rather than a custom operational convention. ```sh prose-agents-sdk --cwd /path/to/workspace --prompt 'Read README.md and perform the requested task.' @@ -29,7 +29,7 @@ SIGINT/SIGTERM cancel the running invocation, including active children and tool ## Usage and validation -Successful `final.usage` sums the normalized usage from the parent and each independently completed child once. It overlaps `usageObservation`; do not add them together. Observations sum only allowlisted nonnegative integer counters present in raw usage on unique completed model responses across parent and children: input/output/total, cached/cache-write and reasoning tokens. Missing/invalid fields remain absent. Event `modelIdentity` records the requested model and allowlisted observed model IDs from completed provider Responses without exposing response contents. `fieldResponseCounts` shows coverage. Provider IDs stay internal to deduplication and never appear in observation records. +Successful `final.usage` sums the normalized usage from the parent and each independently completed child once. It overlaps `usageObservation`; do not add them together. Observations sum only allowlisted nonnegative integer counters present in raw usage on unique completed model responses across parent and children: input/output/total, cached/cache-write and reasoning tokens. Missing/invalid fields remain absent. Event `modelIdentity` records the requested model and allowlisted observed model IDs from completed provider Responses without exposing response contents. Nested `modelIdentity.serviceTier` records `requested: default` and an `observed` array containing only documented response-tier enum values actually returned by the provider (`auto`, `default`, `flex`, `scale`, `priority`, `fast`, `ultrafast`). Missing or invalid response tiers yield an empty observation; the request tier is never used to invent the observed tier. A public client `with_options` wrapper preserves both the request-byte guard and observations on SDK-created no-retry client clones. `fieldResponseCounts` shows coverage. Provider IDs stay internal to deduplication and never appear in observation records. Logical hook starts/completions do not establish provider transport billing. `outstandingProviderRequestCount` is null and `totalRunUsageKnown` remains false, including success. Preserve failed/cancelled spending reservations unless independent accounting closes them. @@ -37,6 +37,6 @@ Logical hook starts/completions do not establish provider transport billing. `ou python test_run.py ``` -Provider-free tests exercise actual local file effects, byte-bounded output capture, process-group timeout/cancellation, missing credentials before inference, safe auth/model failures, aggregate request/tool/observed-token limits, the actual transmitted request byte guard including tools and history, private URL rejection, raw-usage deduplication and genuine SDK-loop child isolation. Mocked Responses transport verifies one hosted-tool allowance per request and no transport retries. Live artifact fulfillment and installed release readiness are separate qualification gates, retained by the owning task. +Provider-free tests exercise actual local file effects, byte-bounded output capture, process-group timeout/cancellation, missing credentials before inference, safe auth/model failures, aggregate request/tool/observed-token limits, the actual transmitted request byte guard including tools and history, actual OpenAI HTTP request standard-tier selection and clone-preserved guards, safe observed-tier reporting, HTTP 409 single-call failure, private URL rejection, raw-usage deduplication and genuine SDK-loop child isolation. Mocked Responses transport verifies one hosted-tool allowance per request and no transport retries. Live artifact fulfillment and installed release readiness are separate qualification gates, retained by the owning task. The integration reuses the pinned SDK 0.22.2 public Agent/Runner/RunHooks/ModelSettings/WebSearchTool APIs. OpenAI Python and dotenv versions are pinned in [requirements.txt](requirements.txt); packaging retains transitive dependency identities. See [SDK budgets](../../docs/sdk-budgets.md) for the outer-runner budgets and historical [evidence](EVIDENCE.md) for earlier source-only observations. diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index 3ef64447..64a21731 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -230,10 +230,11 @@ async def exchange(): class GuardedResponses: """Guard the actual SDK request after instruction/history/tool conversion.""" - def __init__(self, resource, max_input_bytes, observed_models): + def __init__(self, resource, max_input_bytes, observed_models, observed_tiers=None): self.resource = resource self.max_input_bytes = max_input_bytes self.observed_models = observed_models + self.observed_tiers = observed_tiers if observed_tiers is not None else set() async def create(self, **kwargs): payload = {field: kwargs[field] for field in ('instructions', 'input', 'tools', 'text') @@ -244,9 +245,30 @@ async def create(self, **kwargs): model = getattr(response, 'model', None) if isinstance(model, str) and 0 < len(model) <= 128 and all(c.isalnum() or c in '._-/' for c in model): self.observed_models.add(model) + tier = getattr(response, 'service_tier', None) + if isinstance(tier, str) and tier in ('auto', 'default', 'flex', 'scale', 'priority', 'fast', 'ultrafast'): + self.observed_tiers.add(tier) return response +class GuardedClient: + """Retain request guards when the SDK uses public client.with_options().""" + def __init__(self, client, max_input_bytes, observed_models, observed_tiers): + self.client = client + self.max_input_bytes = max_input_bytes + self.observed_models = observed_models + self.observed_tiers = observed_tiers + self.responses = GuardedResponses(client.responses, max_input_bytes, observed_models, observed_tiers) + + def __getattr__(self, name): + return getattr(self.client, name) + + def with_options(self, **kwargs): + clone = getattr(self.client, 'with_options', None) + client = clone(**kwargs) if callable(clone) else self.client + return GuardedClient(client, self.max_input_bytes, self.observed_models, self.observed_tiers) + + async def run(args): # This filters environment inheritance, not filesystem/network permissions. tool_env = {k: v for k, v in os.environ.items() @@ -259,8 +281,10 @@ async def run(args): tool_env[name] = original start = time.monotonic() observed_models = set() + observed_tiers = set() def emit(kind, **data): - data['modelIdentity'] = {'requested': args.model, 'observed': sorted(observed_models)} + data['modelIdentity'] = {'requested': args.model, 'observed': sorted(observed_models), + 'serviceTier': {'requested': 'default', 'observed': sorted(observed_tiers)}} print(json.dumps({'type': kind, 'event': kind, 'elapsed_seconds': round(time.monotonic()-start, 3), **data}), flush=True) max_input_bytes = getattr(args, 'max_input_bytes', 256000) max_tools = getattr(args, 'max_tools', 80) @@ -322,7 +346,7 @@ async def retrieve_url(url: str) -> str: emit('tool_result', name='retrieve_url', result=result) return json.dumps(result) client = AsyncOpenAI(api_key=key.strip(), max_retries=0, timeout=args.timeout) - client.responses = GuardedResponses(client.responses, max_input_bytes, observed_models) + client = GuardedClient(client, max_input_bytes, observed_models, observed_tiers) config = RunConfig(tracing_disabled=True, model_provider=OpenAIProvider(openai_client=client)) def make_agent(depth): @function_tool(failure_error_function=None) @@ -354,7 +378,7 @@ async def delegate(task: str) -> str: model_settings=ModelSettings(max_tokens=args.max_output_tokens, timeout=args.timeout, preserve_raw_usage=True, parallel_tool_calls=False, retry=ModelRetrySettings(max_retries=0), - extra_args={'max_tool_calls': 1})) + extra_args={'max_tool_calls': 1, 'service_tier': 'default'})) emit('start', model=args.model, cwd=cwd, limits=limits, permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, 'networkSandbox': False, 'freshChildConversation': True}) diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index fe0b4fff..db26bb86 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -192,6 +192,8 @@ def text(value): self.assertNotIn('opaque request', child_input) self.assertNotIn('parent result', child_input) self.assertTrue(all(request['max_tool_calls'] == 1 for request in requests)) + self.assertTrue(all(request['service_tier'] == 'default' for request in requests)) + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['modelIdentity']['serviceTier'], {'requested': 'default', 'observed': []}) final = json.loads(output.getvalue().splitlines()[-1]) self.assertEqual(final['usage']['requests'], 3) self.assertEqual(final['usage']['total_tokens'], 15) @@ -249,6 +251,86 @@ async def tracked_shell(*args,**kwargs): self.assertEqual(create.await_count,2) client.close.assert_awaited_once() + async def test_service_tier_observations_are_allowlisted_and_never_inferred(self): + observed_models, observed_tiers = set(), set() + resource = SimpleNamespace(create=AsyncMock()) + guard = harness.GuardedResponses(resource, 1024, observed_models, observed_tiers) + for invalid in (None, '', 'private-provider-value', {}, [], True): + resource.create.return_value = SimpleNamespace(model='fixture', service_tier=invalid) + await guard.create(input=[]) + self.assertEqual(observed_tiers, set()) + for tier in ('auto', 'default', 'flex', 'scale', 'priority', 'fast', 'ultrafast'): + resource.create.return_value = SimpleNamespace(model='fixture', service_tier=tier) + await guard.create(input=[]) + self.assertEqual(observed_tiers, {'auto', 'default', 'flex', 'scale', 'priority', 'fast', 'ultrafast'}) + + async def test_real_openai_http_transport_pins_standard_tier_and_observes_response(self): + import httpx2 as httpx + transmitted = [] + def respond(request): + transmitted.append(json.loads(request.content)) + return httpx.Response(200, json=dict(id='response', object='response', created_at=1, + model='fixture-observed', service_tier='default', parallel_tool_calls=False, + tool_choice='auto', tools=[], status='completed', + output=[dict(type='message', id='message', role='assistant', status='completed', + content=[dict(type='output_text', text='done', annotations=[])])], + usage=dict(input_tokens=3,output_tokens=2,total_tokens=5, + input_tokens_details=dict(cached_tokens=0,cache_write_tokens=0), + output_tokens_details=dict(reasoning_tokens=0)))) + client = harness.AsyncOpenAI(api_key='fixture-key', max_retries=0, + http_client=httpx.AsyncClient(transport=httpx.MockTransport(respond))) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 0) + self.assertEqual(len(transmitted), 1) + self.assertEqual(transmitted[0]['service_tier'], 'default') + self.assertEqual(transmitted[0]['max_tool_calls'], 1) + self.assertEqual(transmitted[0]['max_output_tokens'], 12000) + records = [json.loads(line) for line in output.getvalue().splitlines()] + self.assertEqual(records[0]['modelIdentity']['serviceTier'], {'requested': 'default', 'observed': []}) + self.assertEqual(records[-1]['modelIdentity']['serviceTier'], {'requested': 'default', 'observed': ['default']}) + self.assertEqual(records[-1]['modelIdentity']['observed'], ['fixture-observed']) + + async def test_real_openai_sdk_client_clone_cannot_bypass_input_guard(self): + import httpx2 as httpx + transmitted = [] + def respond(request): + transmitted.append(request) + raise AssertionError('oversized request reached transport') + client = harness.AsyncOpenAI(api_key='fixture-key', max_retries=0, + http_client=httpx.AsyncClient(transport=httpx.MockTransport(respond))) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client): + args = self.args(directory) + args.max_input_bytes = 80 + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(args), 1) + self.assertEqual(transmitted, []) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['error_type'], 'BudgetExceeded') + self.assertEqual(final['modelIdentity']['observed'], []) + self.assertEqual(final['modelIdentity']['serviceTier']['observed'], []) + + async def test_real_openai_http_409_is_not_retried_or_assumed_standard(self): + import httpx2 as httpx + transmitted = [] + def respond(request): + transmitted.append(json.loads(request.content)) + return httpx.Response(409, json={'error': {'message': 'private-provider-body', + 'code': 'conversation_locked', 'type': 'invalid_request_error'}}) + client = harness.AsyncOpenAI(api_key='fixture-key', max_retries=0, + http_client=httpx.AsyncClient(transport=httpx.MockTransport(respond))) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 1) + self.assertEqual(len(transmitted), 1) + self.assertEqual(transmitted[0]['service_tier'], 'default') + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['modelIdentity']['serviceTier'], {'requested': 'default', 'observed': []}) + self.assertNotIn('private-provider-body', output.getvalue()) + async def test_actual_request_input_guard_includes_tools_and_history(self): resource = SimpleNamespace(create=AsyncMock()) observed = set() From d54b3a8c4b6127301a7ba42d87875fb5ce58c97c Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 19:44:57 -0400 Subject: [PATCH 05/64] Integrate packaged SDK defaults and closed runtime observations --- .github/workflows/cli-distribution-check.yml | 34 +- .github/workflows/cli-publish.yml | 4 + README.md | 14 +- cli/README.md | 9 +- cli/bun/src/adapters/environment.ts | 10 +- cli/bun/src/adapters/executable.ts | 16 +- cli/bun/src/adapters/protocols.ts | 14 +- cli/bun/src/adapters/recipes.ts | 2 +- cli/bun/src/adapters/runner.ts | 3 +- cli/bun/src/adapters/sdk-limits.ts | 7 +- cli/bun/src/adapters/sdk-observation.ts | 26 + cli/bun/src/adapters/types.ts | 1 + cli/bun/src/cli.ts | 25 +- cli/bun/src/core/config.ts | 12 +- cli/bun/src/supervision/types.ts | 1 + cli/bun/test/cli.test.ts | 27 +- cli/bun/test/config-production.test.ts | 2 +- cli/bun/test/dx-parity.test.ts | 10 +- cli/bun/test/reporting-parity.test.ts | 2 +- cli/bun/test/sdk-native-limits.test.ts | 13 +- cli/bun/test/sdk-production.test.ts | 82 +++ cli/bun/test/shared-contract.test.ts | 5 +- cli/ci/check_architecture.py | 4 + cli/ci/check_workflows.py | 50 +- cli/ci/test_check_architecture.py | 20 + cli/ci/test_check_workflows.py | 48 ++ .../adapter-products/fake_live_harness.py | 59 +- .../cases/adapters/sdk-production-01.json | 105 +++ .../cases/adapters/sdk-production-02.json | 59 ++ .../cases/adapters/sdk-production-03.json | 60 ++ .../cases/adapters/sdk-production-04.json | 127 ++++ .../cases/adapters/sdk-production-05.json | 165 +++++ .../cases/adapters/sdk-production-06.json | 116 ++++ .../cases/adapters/sdk-production-07.json | 68 ++ .../cases/adapters/sdk-production-08.json | 127 ++++ .../cases/adapters/sdk-production-09.json | 152 +++++ .../cases/adapters/sdk-production-10.json | 152 +++++ .../cases/case-manifest.schema.json | 425 ++++++++++-- .../operations/config-production-01.json | 20 +- cli/conformance/runner/run.py | 120 +++- cli/conformance/runner/test_runner.py | 145 ++++- cli/protocol/OWNERSHIP.md | 38 ++ cli/rust/crates/prose-cli/src/main.rs | 129 ++-- cli/rust/crates/prose-cli/tests/cli.rs | 208 ++++-- .../prose-cli/tests/config_production.rs | 44 +- .../prose-cli/tests/service_programs.rs | 4 +- .../crates/prose-runner-core/src/config.rs | 66 +- .../src/installed_adapters.rs | 99 ++- cli/rust/crates/prose-runner-core/src/lib.rs | 1 + .../crates/prose-runner-core/src/runner.rs | 123 +++- .../prose-runner-core/src/sdk_observation.rs | 162 +++++ .../adapters/recipes/agents-sdk-jsonl.v1.json | 8 +- .../scenarios/agents-sdk-jsonl.v1.json | 36 +- .../fixtures/adapters/sdk-native-limits.json | 26 +- .../fixtures/adapters/sdk-production.json | 389 +++++++++++ cli/shared/fixtures/config/values-v1.json | 3 +- .../operations/configuration-explanation.json | 40 +- .../fixtures/operations/doctor-report.json | 14 +- cli/shared/schemas/native-limits.schema.json | 53 +- cli/shared/schemas/runner-error.schema.json | 2 + cli/shared/schemas/runner-result.schema.json | 615 ++++++++++++++++-- .../schemas/sdk-observation.schema.json | 197 ++++++ cli/shared/tests/test_contracts.py | 122 +++- docs/hosted-service-client.md | 2 +- docs/packaged-sdk-design.md | 71 ++ docs/sdk-budgets.md | 20 +- docs/user-configuration.md | 2 +- harnesses/agents-sdk/README.md | 2 + harnesses/agents-sdk/run.py | 1 + harnesses/agents-sdk/test_run.py | 6 + 70 files changed, 4439 insertions(+), 385 deletions(-) create mode 100644 cli/bun/src/adapters/sdk-observation.ts create mode 100644 cli/bun/test/sdk-production.test.ts create mode 100644 cli/conformance/cases/adapters/sdk-production-01.json create mode 100644 cli/conformance/cases/adapters/sdk-production-02.json create mode 100644 cli/conformance/cases/adapters/sdk-production-03.json create mode 100644 cli/conformance/cases/adapters/sdk-production-04.json create mode 100644 cli/conformance/cases/adapters/sdk-production-05.json create mode 100644 cli/conformance/cases/adapters/sdk-production-06.json create mode 100644 cli/conformance/cases/adapters/sdk-production-07.json create mode 100644 cli/conformance/cases/adapters/sdk-production-08.json create mode 100644 cli/conformance/cases/adapters/sdk-production-09.json create mode 100644 cli/conformance/cases/adapters/sdk-production-10.json create mode 100644 cli/rust/crates/prose-runner-core/src/sdk_observation.rs create mode 100644 cli/shared/fixtures/adapters/sdk-production.json create mode 100644 cli/shared/schemas/sdk-observation.schema.json diff --git a/.github/workflows/cli-distribution-check.yml b/.github/workflows/cli-distribution-check.yml index 7bd1eb6d..01f4007a 100644 --- a/.github/workflows/cli-distribution-check.yml +++ b/.github/workflows/cli-distribution-check.yml @@ -16,7 +16,7 @@ jobs: matrix: os: [ubuntu-22.04, ubuntu-22.04-arm, macos-15, macos-15-intel] runs-on: ${{ matrix.os }} - timeout-minutes: 45 + timeout-minutes: 60 steps: - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 with: @@ -52,6 +52,7 @@ jobs: # resolver checks this exact cwd filename before attempting a fetch. if [ "$RUNNER_OS" = macOS ] && [ "$RUNNER_ARCH" = X64 ]; then cp "$(command -v bun)" bun-darwin-x64-baseline-v1.3.5 + cp "$(command -v bun)" cli/bun/bun-darwin-x64-baseline-v1.3.5 fi rustup toolchain install 1.87.0 --profile minimal rustup default 1.87.0 @@ -59,6 +60,12 @@ jobs: python3 -m pip install --require-hashes --only-binary=:all: -r cli/ci/requirements-test.txt cd cli/bun bun install --frozen-lockfile --ignore-scripts + - name: Prepare the separately locked Agents SDK build interpreter + run: | + uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python" + "$RUNNER_TEMP/agents-sdk-python/bin/python3" -m pip install --require-hashes --only-binary=:all: -r harnesses/agents-sdk/requirements-build.txt + - name: Test the actual pinned Agents SDK runtime without provider credentials + run: '"$RUNNER_TEMP/agents-sdk-python/bin/python3" -m unittest discover -s harnesses/agents-sdk -p test_run.py' - name: Check the public surface # prose-cli is a public user client: no internal service detail or # developer-only surface in tracked files, release help or binaries. @@ -100,9 +107,20 @@ jobs: python3 -m unittest discover -s cli/ci -p test_assemble_kernel_rc.py python3 -m unittest discover -s cli/ci -p test_kernel_rc_evidence.py python3 -m unittest discover -s cli/ci -p test_sign_macos.py + python3 -m unittest discover -s cli/ci -p test_build_agents_sdk.py + python3 -m unittest discover -s cli/ci -p test_build_kernel_rc.py + python3 -m unittest cli.ci.test_package_local.PackagedSdkArtifactTests + python3 -m unittest discover -s cli/ci -p test_dependency_evidence.py + python3 -m unittest discover -s cli/ci -p test_check_workflows.py python3 -m unittest discover -s cli/ci -p test_rehearse_npm_identity.py - name: Build, package and exercise fresh standalone and npm installations run: python3 cli/ci/rehearse_release.py --output "$RUNNER_TEMP/cli-rehearsal" --trials 1 + - name: Build and verify production SDK standalone and npm installations + # This RC version is an unpublished construction fixture. Live model + # qualification and reviewed publication remain separate gates. + env: + RC_VERSION: '0.15.0-rc.1' + run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" - name: Test Homebrew candidate archive admission run: python3 -m unittest discover -s cli/ci -p test_homebrew_rehearsal.py - uses: Homebrew/actions/setup-homebrew@dc7099b3e807f1e2ecc61f3ecabc840eedd5586a @@ -111,6 +129,15 @@ jobs: "$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py --rehearsal "$RUNNER_TEMP/cli-rehearsal" --output "$RUNNER_TEMP/cli-homebrew-rehearsal" + - name: Rehearse Homebrew against the production SDK archives + env: + RC_VERSION: '0.15.0-rc.1' + EXPECTED_SOURCE: ${{ github.sha }} + run: | + "$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py \ + --kernel-rc "$RUNNER_TEMP/kernel-rc" \ + --expected-source "$EXPECTED_SOURCE" --expected-version "$RC_VERSION" \ + --output "$RUNNER_TEMP/kernel-rc/homebrew" - name: Retain rehearsal evidence if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 @@ -119,5 +146,10 @@ jobs: path: | ${{ runner.temp }}/cli-rehearsal ${{ runner.temp }}/cli-homebrew-rehearsal + ${{ runner.temp }}/kernel-rc/agents-sdk + ${{ runner.temp }}/kernel-rc/package + ${{ runner.temp }}/kernel-rc/logs + ${{ runner.temp }}/kernel-rc/build-report.json + ${{ runner.temp }}/kernel-rc/homebrew retention-days: 14 if-no-files-found: warn diff --git a/.github/workflows/cli-publish.yml b/.github/workflows/cli-publish.yml index aec0106e..6677d303 100644 --- a/.github/workflows/cli-publish.yml +++ b/.github/workflows/cli-publish.yml @@ -63,6 +63,10 @@ jobs: env: GH_TOKEN: ${{ github.token }} run: python3 cli/ci/publication.py fetch --plan "$RELEASE_PLAN" --artifacts "$RUNNER_TEMP/prose-publication" + - name: Verify retained production SDK source and sibling custody + # The reviewed manifests bind the helper, build receipt and notices + # across standalone/npm payloads. Never rebuild fetched candidates. + run: python3 cli/ci/publication.py verify --plan "$RELEASE_PLAN" --artifacts "$RUNNER_TEMP/prose-publication" - name: Verify required platform trust, sign artifact digests, and publish exact npm bytes env: GH_TOKEN: ${{ github.token }} diff --git a/README.md b/README.md index 84debcb5..532f993d 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,15 @@ Both runners support: - `--output human|json|jsonl`: rendering, independent of those completion rules. - `--native-log /absolute/new/file.jsonl`: optional private, bounded native-event capture for that same run. It does not prove fulfillment; see [capture limits](docs/native-capture.md). -For example, after an ordinary build and supplying the provider credential in the process environment: +The packaged local default is the Agents SDK with `gpt-6.1-sol`. Supply `OPENAI_API_KEY` in the process environment, then run your program without harness, model or authentication flags: + +```sh +prose run program.md +``` + +Use `prose cli config explain --json -- run program.md` to inspect the effective configuration without inference. Save explicit alternatives in `~/.prose/cli.toml` or with `prose cli harness use`; see [user configuration](docs/user-configuration.md). + +For an explicitly selected Claude route: ```sh /path/to/prose --harness claude --auth-profile anthropic-api-key \ @@ -30,7 +38,7 @@ For example, after an ordinary build and supplying the provider credential in th --output jsonl --native-log /absolute/new-run/native.jsonl run program.md ``` -The native-log parent directory must already exist. The image determines how it loads the requested program. Default harness selection is still `openprose`, which reports `HOSTED_UNAVAILABLE`; select an installed harness explicitly. No fallback occurs. +The native-log parent directory must already exist. The image determines how it loads the requested program. Missing SDK credentials give setup instructions; an incomplete SDK installation requires reinstalling the CLI. No harness, model or credential fallback occurs. ## Harnesses and environment profiles @@ -39,7 +47,7 @@ The native-log parent directory must already exist. The image determines how it | Claude | Installed login or explicit `anthropic-api-key`. The API profile adds native `--bare`, which in 2.1.243 restricts tools to Bash/Edit/Read even with an Agent tool request. Native `acceptEdits` is an explicit permission choice, not implied by API auth. | | Codex | Installed login or `openai-api-key` through a native custom Responses provider. Explicit `workspace-write` or `read-only` maps to native sandbox selection. Context/skill discovery is a separate concern. | | Prime / OMP | Explicit `provider/model` and credential profile. Provider-key routes use fresh private configuration; separate harness-login routes preserve native stores. Native tool use is supported. OMP validates its discovered tool inventory rather than requiring it empty. | -| Agents SDK | Optional generic `prose-agents-sdk` executable, explicit model and `openai-api-key`, ordinary shell tool, no built-in delegation or cached-login route. See [installation and limits](docs/agents-sdk-adapter.md). | +| Agents SDK | Packaged default `prose-agents-sdk` sibling, `gpt-6.1-sol` and `openai-api-key`. Public search/retrieval, shell files and fresh serial child contexts share aggregate limits; host OS permissions apply. See [installation and limits](docs/agents-sdk-adapter.md). | Exact admitted versions and platforms are checked at readiness; see the product docs and `cli harness list`. Selected auth is not proof of successful authentication, billing identity, or sufficient capabilities. See [credential routes](docs/api-credentials.md), [permissions](docs/permissions-and-native-notices.md), and [isolated evaluation](docs/isolated-evaluation.md). diff --git a/cli/README.md b/cli/README.md index c3937cf7..da9844c6 100644 --- a/cli/README.md +++ b/cli/README.md @@ -250,9 +250,12 @@ requirements. They currently carry execution evidence for Ubuntu 22.04 only; other Linux environments remain unverified, and the npm launcher refuses a detected glibc below that floor before spawning the packaged executable. -The default OpenProse-billed route remains unavailable and fails closed. The -four BYO adapters execute only when explicitly selected and never silently -change harness, wrapper-selected credential group, or billing owner. Harness- +The local built-in selects the packaged Agents SDK and `gpt-6.1-sol`, using +`OPENAI_API_KEY` from the process environment. A complete installation needs +no separately provisioned Python or PATH helper. Other harnesses remain +explicit alternatives, saved in `~/.prose/cli.toml` or selected for one command. +The explicitly selected OpenProse-billed local route remains unavailable and +fails closed. No adapter silently changes harness, credential group or billing owner. Harness- internal account/provider routing remains harness-managed. See [`protocol/STATUS.md`](protocol/STATUS.md) for the evidence and open gates. diff --git a/cli/bun/src/adapters/environment.ts b/cli/bun/src/adapters/environment.ts index 1e590ceb..416b93ec 100644 --- a/cli/bun/src/adapters/environment.ts +++ b/cli/bun/src/adapters/environment.ts @@ -1,4 +1,5 @@ import { failure } from "../core/errors"; +import { RunnerFailure } from "../core/types"; import type { InstalledAdapterDefinition } from "./types"; import { adapterAlwaysStrip, adapterBaseEnvironmentAllowlist, adapterOwnedEnvironmentControls } from "./recipes"; @@ -27,12 +28,17 @@ export function buildInstalledAdapterEnvironment(input: AdapterEnvironmentInput) throw failure("INTERNAL_ERROR", { adapterId: input.definition.id, reason: "Credential requirement is missing from the shared oracle." }); } const present = (expected: string): boolean => Object.entries(input.ambient).some(([name, value]) => - normalize(name) === normalize(expected) && value !== undefined && value.length > 0); + normalize(name) === normalize(expected) && value !== undefined && (input.definition.id==="agents-sdk/jsonl" ? value.trim().length>0 : value.length>0)); if (requirement.kind !== "probe-owned" && !requirement.alternatives.some((alternative) => alternative.every(present))) { - throw failure("HARNESS_NEEDS_AUTH", { + const error=failure("HARNESS_NEEDS_AUTH", { adapterId: input.definition.id, authProfile: input.credentialGroup, }); + if(input.definition.id==="agents-sdk/jsonl") { + const {schema:_,...shape}=error.toJSON(); + throw new RunnerFailure({...shape,action:"Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent."}); + } + throw error; } const alwaysStrip = new Set([...adapterAlwaysStrip, "OPENPROSE_API_KEY"].map(normalize)); const selected = new Set([ diff --git a/cli/bun/src/adapters/executable.ts b/cli/bun/src/adapters/executable.ts index 3615e9d1..25129e22 100644 --- a/cli/bun/src/adapters/executable.ts +++ b/cli/bun/src/adapters/executable.ts @@ -1,8 +1,8 @@ import codexCompatibilityJson from "../../../shared/capabilities/adapters/codex-compatibility.v1.json" with { type: "json" }; import { buildVersionProbeEnvironment } from "../supervision/environment"; -import { access } from "node:fs/promises"; +import { access, lstat, realpath } from "node:fs/promises"; import { constants } from "node:fs"; -import { delimiter, isAbsolute, join } from "node:path"; +import { delimiter, dirname, isAbsolute, join } from "node:path"; import { failure } from "../core/errors"; import { probeExecutableCommand, probeExecutableVersion, resolveExecutable } from "../supervision/process"; import { installedAdapterDefinition } from "./recipes"; @@ -18,12 +18,24 @@ export interface ResolveInstalledExecutableInput { wrapperExecutable?: string; platform?: NodeJS.Platform; arch?: NodeJS.Architecture; + /** Explicit source/test discovery only; production never falls back to PATH. */ + allowPathDiscovery?: boolean; } export async function resolveInstalledExecutable(input: ResolveInstalledExecutableInput): Promise { assertInstalledAdapterPlatform(input.adapterId, { platform: input.platform, arch: input.arch }); const definition = installedAdapterDefinition(input.adapterId); if (input.explicitPath !== undefined) return resolveExecutable(input.explicitPath, input.wrapperExecutable); + if(input.adapterId==="agents-sdk/jsonl" && input.allowPathDiscovery!==true) { + try { + const wrapper=await realpath(input.wrapperExecutable ?? process.execPath); + const sibling=join(dirname(wrapper),"prose-agents-sdk"),info=await lstat(sibling); + if(!info.isFile()||info.isSymbolicLink())throw new Error("SDK sibling is not a regular file."); + return await resolveExecutable(sibling,wrapper); + }catch { + throw failure("HARNESS_UNAVAILABLE",{adapterId:input.adapterId,executableNames:definition.recipe.identity.executableNames,admittedVersions:definition.recipe.support.admittedVersions,repairCommand:definition.recipe.support.repairCommand,fallbackAttempted:false}); + } + } const path = input.ambient.PATH; if (path === undefined || path.length === 0) { throw failure("HARNESS_UNAVAILABLE", { diff --git a/cli/bun/src/adapters/protocols.ts b/cli/bun/src/adapters/protocols.ts index e994ceea..f74a2eae 100644 --- a/cli/bun/src/adapters/protocols.ts +++ b/cli/bun/src/adapters/protocols.ts @@ -1,5 +1,6 @@ import {PrimeDrain} from "./prime-drain"; import {sdkNativeFailure} from "./sdk-limits"; +import {sdkObservations,type SdkObservations} from "./sdk-observation"; import {hasFreshClaudeResult} from "./claude-shutdown"; import { isDeepStrictEqual } from "node:util"; import { NativeToolLifecycle, hasNativeTools, ompTaskDefaults } from "./native-tool-lifecycle"; @@ -125,20 +126,26 @@ class CodexProtocol extends InstalledProtocol { } class AgentsSdkProtocol extends InstalledProtocol { + sdkObservations:SdkObservations={}; + constructor(version:string|null,private readonly requestedModel:string|null){super(version);} accept(value: unknown): RawTransportEvent | null { const record = this.record(value); if (record.type === "start") { if (typeof record.model !== "string" || typeof record.cwd !== "string") malformed("SDK start identity missing."); - return this.start(); + const started=this.start(); + this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)}; + return started; } if (!this.started) malformed("SDK event before start."); - if (record.type === "error") throw failure("HARNESS_FAILED", {nativeFailure:sdkNativeFailure(record)}); + if (record.type === "error") {this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)};throw failure("HARNESS_FAILED", {nativeFailure:sdkNativeFailure(record),...this.sdkObservations});} if (record.type === "tool_call" || record.type === "tool_result") { if (typeof record.name !== "string") malformed("SDK tool identity missing."); + this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)}; return null; } if (record.type === "final") { if (typeof record.output !== "string") malformed("SDK final output must be text."); + this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)}; this.complete(); return this.message(record.output); } @@ -769,9 +776,10 @@ export function installedProtocol( invocationId: string, ompPromptBytes: Uint8Array | null = null, nativeMode=false, + requestedModel:string|null=null, ): StructuredProtocolState { if (adapterId === "codex/exec-json") return new CodexProtocol(harnessVersion); - if (adapterId === "agents-sdk/jsonl") return new AgentsSdkProtocol(harnessVersion); + if (adapterId === "agents-sdk/jsonl") return new AgentsSdkProtocol(harnessVersion,requestedModel); if (adapterId === "claude/print-stream-json") return new ClaudeProtocol(harnessVersion,nativeMode); if (adapterId === "omp/rpc") { if (ompPromptBytes === null) malformed("OMP staged prompt bytes are unavailable."); diff --git a/cli/bun/src/adapters/recipes.ts b/cli/bun/src/adapters/recipes.ts index 1753b62a..0ddb0f0e 100644 --- a/cli/bun/src/adapters/recipes.ts +++ b/cli/bun/src/adapters/recipes.ts @@ -19,7 +19,7 @@ import type { const recipeValues = [codexJson, claudeJson, primeJson, ompJson, sdkJson] as unknown as InstalledAdapterRecipe[]; const recipeDigests: Record = { - "agents-sdk/jsonl": "1141c7ed0040d8f0783e5f8870f174c1b66a0eb3299d12e4edebfa98c29228e6", + "agents-sdk/jsonl": createHash("sha256").update(JSON.stringify(sdkJson,null,2)+"\n").digest("hex"), "codex/exec-json": "41c1fd72796defe256a338f92f5ebf850e6522d2eb265c8746512a40a2a03f82", "claude/print-stream-json": "25440e90c81774619f937d36a79786b9fd4bcb85b8ecfbc486f825307429213c", "prime/rpc": "7a9f69497424e56ca214d6cbcd7946b74dbbc03a61f4c110cde8d28a3ad562e1", diff --git a/cli/bun/src/adapters/runner.ts b/cli/bun/src/adapters/runner.ts index f17851ab..d4c432e1 100644 --- a/cli/bun/src/adapters/runner.ts +++ b/cli/bun/src/adapters/runner.ts @@ -170,7 +170,7 @@ export async function runInstalledAdapter(options: InstalledAdapterOptions): Pro ])].filter(Boolean).sort((left, right) => right.length - left.length); const capture = new NativeCapture(options.nativeLog, captureSecrets, nativeOutputBytes(options)); let observed: NativeObservation | null = null; - const protocol=installedProtocol(options.adapterId, options.harnessVersion ?? null, options.invocation.invocationId, (options.adapterId === "omp/rpc" || options.adapterId === "prime/rpc") ? plan.stdinBytes : null, options.outputContract === "native"); + const protocol=installedProtocol(options.adapterId, options.harnessVersion ?? null, options.invocation.invocationId, (options.adapterId === "omp/rpc" || options.adapterId === "prime/rpc") ? plan.stdinBytes : null, options.outputContract === "native",options.model ?? null); if(options.nativeProfile === "claude-workspace-tools") { const accept=protocol.accept.bind(protocol); protocol.accept=(record)=>{ observed=observeNativeInit(record,options.credentialGroup) ?? observed; return accept(record); }; @@ -204,6 +204,7 @@ export async function runInstalledAdapter(options: InstalledAdapterOptions): Pro return { plan, process, + ...(protocol.sdkObservations===undefined?{}:{sdkObservations:protocol.sdkObservations}), publicEvents: redactInstalledAdapterEvents(process.events, protectedValues), publicStderr: redactDiagnostic(process.stderr, protectedValues), nativeConfiguration:nativeConfiguration({...options,authProfile:options.credentialGroup},observed), diff --git a/cli/bun/src/adapters/sdk-limits.ts b/cli/bun/src/adapters/sdk-limits.ts index 19db92f6..e4d6be1e 100644 --- a/cli/bun/src/adapters/sdk-limits.ts +++ b/cli/bun/src/adapters/sdk-limits.ts @@ -7,7 +7,7 @@ export function nativeLimits(input:Selection):Record|undefined { if(input.nativeMaxTurns!==undefined){turns=Number(input.nativeMaxTurns);if(!/^[1-9][0-9]*$/.test(input.nativeMaxTurns)||!Number.isSafeInteger(turns))throw failure("CONFIG_INVALID",{reason:"Native max turns must be a positive safe integer."});} if(input.nativeTimeout!==undefined){const match=/^([1-9][0-9]*)(ms|s|m|h)$/.exec(input.nativeTimeout);if(!match)throw failure("CONFIG_INVALID",{reason:"Native timeout must be a positive duration."});timeoutMs=Number(match[1])*({ms:1,s:1000,m:60000,h:3600000}[match[2]!]!);if(!Number.isSafeInteger(timeoutMs))throw failure("CONFIG_INVALID",{reason:"Native timeout is outside the supported range."});} if(input.nativeToolTimeout!==undefined){const match=/^([1-9][0-9]*)(ms|s|m|h)$/.exec(input.nativeToolTimeout);if(!match)throw failure("CONFIG_INVALID",{reason:"Native timeout must be a positive duration."});toolTimeoutMs=Number(match[1])*({ms:1,s:1000,m:60000,h:3600000}[match[2]!]!);if(!Number.isSafeInteger(toolTimeoutMs))throw failure("CONFIG_INVALID",{reason:"Native timeout is outside the supported range."});} - return {maxTurns:turns,timeoutSeconds:timeoutMs/1000,toolTimeoutSeconds:toolTimeoutMs/1000,maxOutputTokens:12000}; + return {maxTurns:turns,timeoutSeconds:timeoutMs/1000,toolTimeoutSeconds:toolTimeoutMs/1000,maxOutputTokens:12000,maxAggregateRequests:turns,maxAggregateHostedWebCalls:turns,maxAggregateFunctionTools:80,maxObservedTotalTokens:500000,maxRequestInputBytes:256000,maxChildren:8,maxChildDepth:1}; } export function nativeLimitsArgv(input:Selection):string[]{const limits=nativeLimits(input);if(!limits)return [];return [...(input.nativeMaxTurns===undefined?[]:["--max-turns",String(limits.maxTurns)]),...(input.nativeTimeout===undefined?[]:["--timeout",String(limits.timeoutSeconds)]),...(input.nativeToolTimeout===undefined?[]:["--tool-timeout",String(limits.toolTimeoutSeconds)])];} export function sdkNativeFailure(record:Record):Record{ @@ -15,8 +15,9 @@ export function sdkNativeFailure(record:Record):Record=0)output.elapsedSeconds=elapsed; const limits=record.limits; if(limits!==null&&typeof limits==="object"&&!Array.isArray(limits)){ - const v=limits as Record,keys=["maxTurns","timeoutSeconds","toolTimeoutSeconds","maxOutputTokens"]; - if(Object.keys(v).length===4&&keys.every(k=>typeof v[k]==="number"&&Number.isFinite(v[k])&&(v[k] as number)>0)&&Number.isSafeInteger(v.maxTurns)&&Number.isSafeInteger(v.maxOutputTokens)&&(v.timeoutSeconds as number)<=Number.MAX_SAFE_INTEGER/1000&&(v.toolTimeoutSeconds as number)<=Number.MAX_SAFE_INTEGER/1000)output.limits=Object.fromEntries(keys.map(k=>[k,v[k]])); + const v=limits as Record,base=["maxTurns","timeoutSeconds","toolTimeoutSeconds","maxOutputTokens"],expanded=[...base,"maxAggregateRequests","maxAggregateHostedWebCalls","maxAggregateFunctionTools","maxObservedTotalTokens","maxRequestInputBytes","maxChildren","maxChildDepth"]; + const keys=Object.keys(v).length===4?base:expanded; + if(Object.keys(v).length===keys.length&&keys.every(k=>typeof v[k]==="number"&&Number.isFinite(v[k])&&(v[k] as number)>0)&&keys.filter(k=>!["timeoutSeconds","toolTimeoutSeconds"].includes(k)).every(k=>Number.isSafeInteger(v[k]))&&(v.timeoutSeconds as number)<=Number.MAX_SAFE_INTEGER/1000&&(v.toolTimeoutSeconds as number)<=Number.MAX_SAFE_INTEGER/1000&&(keys===base||(v.maxChildren===8&&v.maxChildDepth===1)))output.limits=Object.fromEntries(keys.map(k=>[k,v[k]])); } return output; } diff --git a/cli/bun/src/adapters/sdk-observation.ts b/cli/bun/src/adapters/sdk-observation.ts new file mode 100644 index 00000000..8e0bf62b --- /dev/null +++ b/cli/bun/src/adapters/sdk-observation.ts @@ -0,0 +1,26 @@ +/** Closed, bounded public projections; provider records and identifiers stay private. */ +export interface SdkObservations { + usageObservation?: Record; + modelIdentity?: Record; +} +const tokenKeys=["input_tokens","output_tokens","total_tokens","input_tokens_details.cached_tokens","input_tokens_details.cache_write_tokens","output_tokens_details.reasoning_tokens"]; +const countKeys=["startedCallCount","completedResponseCount","duplicateResponseCallbackCount","outstandingCallCount"]; +const tiers=new Set(["auto","default","flex","scale","priority","fast","ultrafast"]); +const record=(value:unknown):value is Record=>value!==null&&typeof value==="object"&&!Array.isArray(value); +const counter=(value:unknown):value is number=>Number.isSafeInteger(value)&&typeof value==="number"&&value>=0; +const identifier=(value:unknown):value is string=>typeof value==="string"&&value.length<=128&&/^[A-Za-z0-9][A-Za-z0-9._/-]*$/.test(value); +function counters(value:Record):Record{return Object.fromEntries(tokenKeys.filter(key=>counter(value[key])).map(key=>[key,value[key] as number]));} +export function sdkObservations(value:Record,requestedModel:string|null):SdkObservations { + const result:SdkObservations={}; + const usage=value.usageObservation; + if(record(usage)&&usage.source==="sdk_completed_response_raw_usage"&&usage.aggregationScope==="unique_completed_responses_in_parent_and_children"&&countKeys.every(key=>counter(usage[key]))&&usage.outstandingProviderRequestCount===null&&usage.totalRunUsageKnown===false&&record(usage.observedTokenTotals)&&record(usage.fieldResponseCounts)) { + result.usageObservation={source:usage.source,aggregationScope:usage.aggregationScope,...Object.fromEntries(countKeys.map(key=>[key,usage[key]])),outstandingProviderRequestCount:null,observedTokenTotals:counters(usage.observedTokenTotals),fieldResponseCounts:counters(usage.fieldResponseCounts),totalRunUsageKnown:false}; + } + const model=value.modelIdentity; + if(record(model)&&identifier(requestedModel)&&Array.isArray(model.observed)&&record(model.serviceTier)&&model.serviceTier.requested==="default"&&Array.isArray(model.serviceTier.observed)) { + const observed=[...new Set(model.observed.filter(identifier))].sort().slice(0,128); + const observedTiers=[...new Set(model.serviceTier.observed.filter((tier):tier is string=>typeof tier==="string"&&tiers.has(tier)))].sort(); + result.modelIdentity={requested:requestedModel,observed,serviceTier:{requested:"default",observed:observedTiers}}; + } + return result; +} diff --git a/cli/bun/src/adapters/types.ts b/cli/bun/src/adapters/types.ts index 62e4d5e3..16c7faf0 100644 --- a/cli/bun/src/adapters/types.ts +++ b/cli/bun/src/adapters/types.ts @@ -195,6 +195,7 @@ export interface InstalledAdapterOptions { } export interface InstalledAdapterResult { + sdkObservations?: import("./sdk-observation").SdkObservations; nativeConfiguration?: Record | undefined; plan: InstalledLaunchPlan; process: ProcessSupervisionResult; diff --git a/cli/bun/src/cli.ts b/cli/bun/src/cli.ts index 6f0c1ebd..9993d97a 100644 --- a/cli/bun/src/cli.ts +++ b/cli/bun/src/cli.ts @@ -173,7 +173,9 @@ export async function runCli(args: readonly string[], dependencies: CliDependenc catch(caught) {if(caught instanceof RunnerFailure && caught.code === "CONFIG_INVALID")throw failure("CONFIG_INVALID",{...(caught.details ?? {}),mutation});throw caught;} config.mutation=mutation; } else { - config=await resolveConfiguration(parsed.global,{...dependencies,...(parsed.kind === "operation"&&parsed.targetArgv!==undefined ? {targetArgv:parsed.targetArgv}:{})}); + const resolutionFlags={...parsed.global}; + if(parsed.kind==="operation"&&parsed.operation==="harness-use"){delete resolutionFlags.model;delete resolutionFlags.authProfile;} + config=await resolveConfiguration(resolutionFlags,{...dependencies,...(parsed.kind === "operation"&&parsed.targetArgv!==undefined ? {targetArgv:parsed.targetArgv}:{}),...(parsed.kind==="operation"&&parsed.operation==="harness-use"?{validateSelection:()=>validateUserHarnessSelection(parsed.value!,parsed.global)}:{})}); } // The default hosted harness runs no language command, so a language // command word that also names a service command is that rejection. @@ -184,6 +186,10 @@ export async function runCli(args: readonly string[], dependencies: CliDependenc if (parsed.kind === "operation") { return await runOperation(parsed.operation, parsed.value, parsed.global, config, mode, dependencies); } + if(config.values.harness==="agents-sdk" && parsed.global.dryRun!==true) { + negotiateTransport(selectHarness(config),config.values.transport); + buildInstalledAdapterEnvironment({definition:installedAdapterDefinition("agents-sdk/jsonl"),ambient:dependencies.env,credentialGroup:selectedCredentialGroup("agents-sdk/jsonl",config.values.authProfile),...(dependencies.platform===undefined?{}:{platform:dependencies.platform})}); + } const image = PUBLISHED_KERNEL_STARTUP && dependencies.imageBundle === undefined && ["codex", "claude", "prime", "omp", "agents-sdk"].includes(config.values.harness) ? await publishedKernel(undefined, dependencies.cancellationSignal) @@ -475,7 +481,8 @@ function validateUserHarnessSelection( if (authProfile !== null && definition.credentialGroups[authProfile] === undefined) { throw failure("CONFIG_INVALID", { adapterId, - reason: `Unknown auth_profile for ${adapterId}: ${authProfile}.`, + source:"--auth-profile", + reason: "Authentication profile is incompatible with the selected harness.", supportedAuthProfiles: Object.keys(definition.credentialGroups), }); } @@ -909,6 +916,7 @@ async function runInstalledProbeInvocation( }); const options: ProviderFreeAdapterOptions = { adapterId, + ...(adapterId==="agents-sdk/jsonl"?{model:config.values.model,outputContract:config.values.outputContract??"image-envelope",...(config.values.nativeMaxTurns===undefined?{}:{nativeMaxTurns:config.values.nativeMaxTurns}),...(config.values.nativeTimeout===undefined?{}:{nativeTimeout:config.values.nativeTimeout}),...(config.values.nativeToolTimeout===undefined?{}:{nativeToolTimeout:config.values.nativeToolTimeout})}:{}), executable: probe.executable, observationPath: probe.observationPath, credentialGroup: probe.credentialGroup, @@ -940,7 +948,7 @@ async function runInstalledProbeInvocation( if (outcome.publicStderr.length > 0) { dependencies.writeStderr(mode === "human" ? humanSafeMultiline(outcome.publicStderr) : outcome.publicStderr); } - const attemptedError = outcome.process.error === null + let attemptedError = outcome.process.error === null ? failure("SEMANTIC_STATUS_UNKNOWN", { adapterId, admissionStatus: "blocked", @@ -951,6 +959,10 @@ async function runInstalledProbeInvocation( outcome.process.error.code, installedProcessFailureDetails(outcome.process, adapterId), ); + if(outcome.sdkObservations!==undefined) { + const {schema:_,...shape}=attemptedError.toJSON(); + attemptedError=new RunnerFailure({...shape,details:{...attemptedError.details,...outcome.sdkObservations}}); + } const terminalAt = dependencies.clock.now(); const baseEvents = processEvents( invocation.invocationId, @@ -982,6 +994,7 @@ async function runInstalledProbeInvocation( deliveredImageSha256: outcome.plan.imageSha256, renderedPayloadSha256: outcome.plan.renderedPayloadSha256, }); + if(outcome.sdkObservations!==undefined)Object.assign(result,outcome.sdkObservations); emitAttemptFailure(result, attemptedError, mode, invocation.invocationId, baseEvents, dependencies); return attemptedError.exitCode; } @@ -1026,6 +1039,7 @@ async function prepareInstalledAdapter( } } const credentialGroup = selectedCredentialGroup(adapterId, config.values.authProfile); + if(adapterId==="agents-sdk/jsonl")buildInstalledAdapterEnvironment({definition,ambient:dependencies.env,credentialGroup,...(dependencies.platform===undefined?{}:{platform:dependencies.platform})}); const executable = await resolveInstalledExecutable({ adapterId, ambient: dependencies.env, @@ -1196,6 +1210,10 @@ async function runInstalledInvocation( attemptedError = normalizeFailure(caught); } } + if(attemptedError!==null && outcome.sdkObservations!==undefined) { + const {schema:_,...shape}=attemptedError.toJSON(); + attemptedError=new RunnerFailure({...shape,details:{...attemptedError.details,...outcome.sdkObservations}}); + } const terminalAt = dependencies.clock.now(); const baseEvents = processEvents( invocation.invocationId, @@ -1229,6 +1247,7 @@ async function runInstalledInvocation( renderedPayloadSha256: outcome.plan.renderedPayloadSha256, }); if(outcome.nativeConfiguration) result.nativeConfiguration=outcome.nativeConfiguration; + if(outcome.sdkObservations!==undefined)Object.assign(result,outcome.sdkObservations); if (attemptedError !== null) { humanOutput.stream?.abort(); emitAttemptFailure(result, attemptedError, mode, invocation.invocationId, baseEvents, dependencies); diff --git a/cli/bun/src/core/config.ts b/cli/bun/src/core/config.ts index cefaee16..449bc36b 100644 --- a/cli/bun/src/core/config.ts +++ b/cli/bun/src/core/config.ts @@ -20,6 +20,8 @@ export interface ConfigDependencies { platform?: NodeJS.Platform; homeDir?: string; targetArgv?: string[]; + /** Pure target-bundle validation after source parsing, before execution-route checks. */ + validateSelection?: () => void; } type ConfigKey = keyof EffectiveValues; @@ -92,7 +94,7 @@ export function timeoutMs(value: string): number | string { } const defaults: EffectiveValues = { - harness: "openprose", + harness: "agents-sdk", transport: "auto", model: null, timeout: "10m", @@ -119,7 +121,12 @@ export async function resolveConfiguration( dependencies: ConfigDependencies, ): Promise { try {return await resolveConfigured(flags,dependencies);} - catch(caught) {throw earlyConfigurationFailure(caught,dependencies);} + catch(caught) {if(caught instanceof SelectionValidationFailure)throw caught.original;throw earlyConfigurationFailure(caught,dependencies);} +} + +/** Selection errors contain only the independently validated target, never unresolved source values. */ +class SelectionValidationFailure extends Error { + constructor(readonly original:unknown){super("Target harness selection is invalid.");} } function earlyConfigurationFailure(caught:unknown,dependencies:ConfigDependencies):unknown { @@ -207,6 +214,7 @@ async function resolveConfigured(flags: GlobalFlags,dependencies: ConfigDependen if (projectConfigPath !== null) overlay(await readConfig(projectConfigPath),"project-config",2); overlay(parseEnvironment(dependencies.env),"environment",3); overlay(parseFlags(flags),"flag",4); + try {dependencies.validateSelection?.();}catch(caught){throw new SelectionValidationFailure(caught);} candidates.authProfile=candidates.authProfile!.filter(candidate=>{ if(candidate.selected || candidate.value===null)return true; const owner=candidateHarnesses.get(candidate); diff --git a/cli/bun/src/supervision/types.ts b/cli/bun/src/supervision/types.ts index b2b4d89e..0825bae3 100644 --- a/cli/bun/src/supervision/types.ts +++ b/cli/bun/src/supervision/types.ts @@ -107,6 +107,7 @@ export interface AdapterDiagnostic { } export interface StructuredProtocolState { + readonly sdkObservations?: import("../adapters/sdk-observation").SdkObservations; /** Optional final validation after natural process/pipe settlement. */ settleProcess?(exitCode:number|null):RawTransportEvent|null; readonly terminalEventObserved: boolean; diff --git a/cli/bun/test/cli.test.ts b/cli/bun/test/cli.test.ts index ca9706c3..cf532a4c 100644 --- a/cli/bun/test/cli.test.ts +++ b/cli/bun/test/cli.test.ts @@ -2,6 +2,7 @@ import { describe, expect, test } from "bun:test"; import { chmod, lstat, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join, resolve } from "node:path"; +import doctorFixture from "../../shared/fixtures/operations/doctor-report.json" with { type: "json" }; import configurationFixture from "../../shared/fixtures/operations/configuration-explanation.json" with { type: "json" }; import { humanHarnessList, runCli, type CliDependencies } from "../src/cli"; import { harnesses } from "../src/core/harnesses"; @@ -83,14 +84,14 @@ function operationFixture() { return { ...io, cwd, userConfigPath }; } -function expectedConfiguration(cwd: string, userConfigPath: string) { +function expectedConfiguration(cwd: string, userConfigPath: string, frozen:unknown=configurationFixture) { const replace = (value: unknown): unknown => { if(Array.isArray(value))return value.map(replace); if(value!==null && typeof value === "object")return Object.fromEntries(Object.entries(value).map(([key,item])=>[key,replace(item)])); if(typeof value === "string")return value.replaceAll("/workspace/config/openprose/cli.toml",userConfigPath).replaceAll("/workspace",cwd); return value; }; - return replace(configurationFixture); + return replace(frozen); } describe("CLI behavior", () => { @@ -498,9 +499,9 @@ describe("CLI behavior", () => { expect(observed).toMatchObject({ aborted: true, reason: "SIGINT" }); }); - test("the default is OpenProse billed and fails closed without fallback", async () => { + test("explicit OpenProse selection is billed by OpenProse and fails closed without fallback", async () => { const io = fixture(); - const exit = await runCli(["run", "example.prose.md"], io.deps); + const exit = await runCli(["--harness", "openprose", "run", "example.prose.md"], io.deps); expect(exit).toBe(10); expect(io.invocations).toHaveLength(0); @@ -511,7 +512,7 @@ describe("CLI behavior", () => { test("JSON failure emits one object on stdout and no diagnostic contamination", async () => { const io = fixture(); - const exit = await runCli(["--output", "json", "run", "example.prose.md"], io.deps); + const exit = await runCli(["--harness", "openprose", "--output", "json", "run", "example.prose.md"], io.deps); expect(exit).toBe(10); const parsed = JSON.parse(io.stdout()); expect(parsed).toMatchObject({ @@ -532,7 +533,7 @@ describe("CLI behavior", () => { test("JSONL failure has exactly one terminal event", async () => { const io = fixture(); - const exit = await runCli(["--output=jsonl", "run", "example.prose.md"], io.deps); + const exit = await runCli(["--harness", "openprose", "--output=jsonl", "run", "example.prose.md"], io.deps); expect(exit).toBe(10); const records = io.stdout().trimEnd().split("\n").map((line) => JSON.parse(line)); expect(records.at(-1)).toMatchObject({ @@ -933,14 +934,14 @@ describe("CLI behavior", () => { expect(await runCli(["cli", "harness", "list"], io.deps)).toBe(0); expect(io.stdout()).toBe([ "Harnesses:", - "* openprose availability=not-implemented transport=hosted (selected)", + " openprose availability=not-implemented transport=hosted", " prime availability=missing transport=rpc", " omp availability=missing transport=rpc", " runtime prerequisite: bun; availability: missing; detected version: missing; required version: 1.3.14 or newer", " runtime repair: npm install --global bun@1.3.14 @oh-my-pi/pi-coding-agent@18.0.9", " codex availability=missing transport=exec-json", " claude availability=missing transport=print-stream-json", - " agents-sdk availability=missing transport=jsonl", + "* agents-sdk availability=missing transport=jsonl (selected)", "", "Test-only harnesses:", " mock availability=available transport=deterministic,fake-process version=1.0.0", @@ -1007,9 +1008,9 @@ describe("CLI behavior", () => { expect(inventory.find((item) => item.id === "codex")?.detectedVersion).toBe(detectedVersion); }); - test("doctor emits the complete default report and returns the selected hosted problem exit", async () => { + test("doctor emits the complete explicitly hosted report and returns its problem exit", async () => { const io = operationFixture(); - expect(await runCli(["--output", "json", "cli", "doctor"], io.deps)).toBe(10); + expect(await runCli(["--harness", "openprose", "--output", "json", "cli", "doctor"], io.deps)).toBe(10); expect(JSON.parse(io.stdout())).toMatchObject({ schema: "openprose.doctor-report/1", ready: false, @@ -1020,7 +1021,7 @@ describe("CLI behavior", () => { runner: { name: "bun", version: "0.1.0", commit: "development" }, build: { profile: "development", testSeamsEnabled: true }, cwd: io.cwd, - configuration: expectedConfiguration(io.cwd, io.userConfigPath), + configuration: expectedConfiguration(io.cwd, io.userConfigPath,doctorFixture.configuration), }); expect(io.stderr()).toBe(""); expect(io.invocations).toHaveLength(0); @@ -1028,8 +1029,8 @@ describe("CLI behavior", () => { }); test.each([ - ["doctor", ["--transport", "unsupported", "--output", "json", "cli", "doctor"]], - ["run", ["--transport", "unsupported", "--output", "json", "run", "fixture.prose.md"]], + ["doctor", ["--harness", "openprose", "--transport", "unsupported", "--output", "json", "cli", "doctor"]], + ["run", ["--harness", "openprose", "--transport", "unsupported", "--output", "json", "run", "fixture.prose.md"]], ] as const)("%s rejects an unsupported hosted transport before hosted availability", async (_name, args) => { const io = operationFixture(); expect(await runCli(args, io.deps)).toBe(20); diff --git a/cli/bun/test/config-production.test.ts b/cli/bun/test/config-production.test.ts index 0ef83f2f..803032e6 100644 --- a/cli/bun/test/config-production.test.ts +++ b/cli/bun/test/config-production.test.ts @@ -146,7 +146,7 @@ test("early cwd failure preserves the exact target and makes no runtime claims", const root=await freshConfigurationRoot();let stdout=""; expect(await runCli(["cli","config","explain","--json","--","--cwd","absent","--harness","agents-sdk","run","x"],{processCwd:root,env:{HOME:join(root,"home")},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:()=>{}})).toBe(2); const partial=JSON.parse(stdout).details.configurationExplanation; - expect(partial).toMatchObject({cwd:{value:root,source:{kind:"default",location:"process cwd"}},target:{argv:["prose","run","x"]},values:{harness:{value:"openprose"}},locations:[],runtime:{transport:null}}); + expect(partial).toMatchObject({cwd:{value:root,source:{kind:"default",location:"process cwd"}},target:{argv:["prose","run","x"]},values:{harness:{value:"agents-sdk"}},locations:[],runtime:{transport:null}}); expect(validateExplanation(partial),JSON.stringify(validateExplanation.errors)).toBe(true);expect(await treeFiles(root)).toEqual([]); }); for(const args of [["cli","config","migrate","--json"],["cli","config","unset","model","--json"]])test(`mutation preflight ${args[2]} root failure has safe partial defaults`,async()=>{ diff --git a/cli/bun/test/dx-parity.test.ts b/cli/bun/test/dx-parity.test.ts index f4175d0f..681ce649 100644 --- a/cli/bun/test/dx-parity.test.ts +++ b/cli/bun/test/dx-parity.test.ts @@ -37,8 +37,8 @@ function withWorkspace(frozen: unknown): unknown { describe("frozen Phase-7 developer-experience parity", () => { test.each([ { - name: "default hosted", - argv: ["--output", "json", "--dry-run", "write", "fixture request"], + name: "explicit hosted", + argv: ["--harness", "openprose", "--output", "json", "--dry-run", "write", "fixture request"], exitCode: 10, expected: hostedDryRunFixture, }, @@ -51,14 +51,16 @@ describe("frozen Phase-7 developer-experience parity", () => { ])("matches the exact $name dry-run authority without starting a harness", async ({ argv, exitCode, expected }) => { const io = fixture(); expect(await runCli(argv, io.dependencies)).toBe(exitCode); - expect(JSON.parse(io.stdout())).toEqual(withWorkspace(expected)); + const frozen=structuredClone(expected); + if(frozen.selection.harness === "openprose") {const source=frozen.configuration.find(item=>item.key==="harness")!;source.source="flag";source.location="--harness";} + expect(JSON.parse(io.stdout())).toEqual(withWorkspace(frozen)); expect(io.stderr()).toBe(""); expect(io.invocations).toHaveLength(0); }); test("hosted JSONL refusal emits only the canonical failed terminal", async () => { const io = fixture(); - expect(await runCli(["--output", "jsonl", "run", "fixture.prose.md"], io.dependencies)).toBe(10); + expect(await runCli(["--harness", "openprose", "--output", "jsonl", "run", "fixture.prose.md"], io.dependencies)).toBe(10); const records = io.stdout().trimEnd().split("\n").map((line) => JSON.parse(line)); expect(records.map((record) => record.type)).toEqual(["runner.failed"]); expect(records[0].payload).toMatchObject({ diff --git a/cli/bun/test/reporting-parity.test.ts b/cli/bun/test/reporting-parity.test.ts index 2ce67656..de98ad02 100644 --- a/cli/bun/test/reporting-parity.test.ts +++ b/cli/bun/test/reporting-parity.test.ts @@ -10,6 +10,6 @@ test("explanation reports inherited defaults while concise execution summaries o const deps={processCwd:dir,userConfigPath:join(dir,"absent"),env:{}}; const base=await resolveConfiguration({},deps);expect(base.values.outputContract).toBe("image-envelope");expect(base.values.permissionMode).toBeNull(); const values=(configurationExplanation(base).values as Record);for(const [key,value] of Object.entries(f.explanationDefaults))expect(values[key]).toMatchObject({value,source:{kind:"default",location:"built-in"}});for(const key of f.defaultsOmitted){expect(values[key]).toMatchObject({source:{kind:"default",location:"built-in"}});expect(reportedConfigurationKeys(base)).not.toContain(key);} - for(const c of f.cases){const config=await resolveConfiguration(c,deps),report=configurationExplanation(config).values as any;for(const key of f.defaultsOmitted)expect(report[key]).toMatchObject({value:c[key as 'outputContract'|'permissionMode'],source:{kind:"flag"}});expect(report.outputContract.source.location).toBe("--output-contract");expect(report.permissionMode.source.location).toBe("--permission-mode");expect(config.values.outputContract).toBe(c.outputContract);expect(config.values.permissionMode).toBe(c.permissionMode);} + for(const c of f.cases){const config=await resolveConfiguration({...c,harness:"claude"},deps),report=configurationExplanation(config).values as any;for(const key of f.defaultsOmitted)expect(report[key]).toMatchObject({value:c[key as 'outputContract'|'permissionMode'],source:{kind:"flag"}});expect(report.outputContract.source.location).toBe("--output-contract");expect(report.permissionMode.source.location).toBe("--permission-mode");expect(config.values.outputContract).toBe(c.outputContract);expect(config.values.permissionMode).toBe(c.permissionMode);} }finally{await rm(dir,{recursive:true,force:true});} }); diff --git a/cli/bun/test/sdk-native-limits.test.ts b/cli/bun/test/sdk-native-limits.test.ts index bb6b3aaa..60b97a5e 100644 --- a/cli/bun/test/sdk-native-limits.test.ts +++ b/cli/bun/test/sdk-native-limits.test.ts @@ -3,6 +3,7 @@ import {mkdtemp,writeFile,rm} from "node:fs/promises"; import {tmpdir} from "node:os"; import {join} from "node:path"; import fixture from "../../shared/fixtures/adapters/sdk-native-limits.json"; +import production from "../../shared/fixtures/adapters/sdk-production.json"; import {nativeLimits,nativeLimitsArgv,sdkNativeFailure} from "../src/adapters/sdk-limits"; import {parseEntrypoint} from "../src/core/args"; import {resolveConfiguration} from "../src/core/config"; @@ -13,8 +14,8 @@ import {sentinelFixtureImage} from "./sentinel-fixture"; import {encodeRuntimeImage} from "../src/supervision/files"; import type {RunnerInvocation} from "../src/core/types"; test("shared SDK budgets preserve defaults, validate explicit inputs and reject unsupported hosts",()=>{ - expect(nativeLimits({harness:"agents-sdk"})).toEqual(fixture.defaults);expect(nativeLimitsArgv({harness:"agents-sdk"})).toEqual([]); - expect(nativeLimits({harness:"agents-sdk",...fixture.override})).toEqual(fixture.override.limits);expect(nativeLimitsArgv({harness:"agents-sdk",...fixture.override})).toEqual(fixture.override.argv); + expect(nativeLimits({harness:"agents-sdk"})).toEqual(production.nativeLimits);expect(nativeLimitsArgv({harness:"agents-sdk"})).toEqual([]); + expect(nativeLimits({harness:"agents-sdk",...fixture.override})).toEqual({...production.nativeLimits,...fixture.override.limits,maxAggregateRequests:40,maxAggregateHostedWebCalls:40});expect(nativeLimitsArgv({harness:"agents-sdk",...fixture.override})).toEqual(fixture.override.argv); for(const nativeMaxTurns of fixture.invalidTurns)expect(()=>nativeLimits({harness:"agents-sdk",nativeMaxTurns})).toThrow(); for(const nativeTimeout of fixture.invalidTimeouts)expect(()=>nativeLimits({harness:"agents-sdk",nativeTimeout})).toThrow(); for(const harness of ["claude","codex","prime","omp","mock","openprose"]){expect(nativeLimits({harness})).toBeUndefined();expect(()=>nativeLimits({harness,nativeMaxTurns:"2"})).toThrow();expect(()=>nativeLimits({harness,nativeTimeout:"1s"})).toThrow();} @@ -26,7 +27,7 @@ test("SDK configuration flags override environment and quoted TOML, leaving oute const userConfigPath=join(root,"user.toml");await writeFile(userConfigPath,'harness="agents-sdk"\nnative_max_turns="25"\nnative_timeout="2m"\n'); const parsed=parseEntrypoint(["--native-max-turns","40","--native-timeout","5m","--timeout","10m","task"]); const c=await resolveConfiguration(parsed.global,{processCwd:root,userConfigPath,env:{PROSE_NATIVE_MAX_TURNS:"30",PROSE_NATIVE_TIMEOUT:"3m"}}); - expect(nativeLimits(c.values)).toEqual(fixture.override.limits);expect(c.values.timeout).toBe("10m");expect(c.sources.nativeMaxTurns?.kind).toBe("flag"); + expect(nativeLimits(c.values)).toEqual({...production.nativeLimits,...fixture.override.limits,maxAggregateRequests:40,maxAggregateHostedWebCalls:40});expect(c.values.timeout).toBe("10m");expect(c.sources.nativeMaxTurns?.kind).toBe("flag"); await expect(resolveConfiguration({harness:"claude",nativeMaxTurns:"2"},{processCwd:root,userConfigPath:join(root,"none"),env:{}})).rejects.toThrow(); }finally{await rm(root,{recursive:true,force:true});} }); @@ -38,14 +39,14 @@ test("SDK actual launch forwards only explicit native budgets without changing t expect(base.argv).not.toContain("--max-turns");expect(base.argv).not.toContain("--timeout");expect(selected.argv.slice(1,5)).toEqual(fixture.override.argv);expect(selected.argv.slice(5)).toEqual(base.argv.slice(1)); }); test("SDK native errors preserve only closed diagnostic data and never settle",()=>{ - for(const sample of fixture.errorCases){const p=installedProtocol("agents-sdk/jsonl","0.1.0","fixture");p.accept({type:"start",model:"fixture",cwd:"/tmp"});let caught:any;try{p.accept({type:"error",error_type:sample.error_type,elapsed_seconds:1.5,limits:fixture.defaults});}catch(e){caught=e;}expect(caught.details.nativeFailure).toEqual({kind:sample.kind,elapsedSeconds:1.5,limits:fixture.defaults});expect(p.terminalEventObserved).toBe(false);} + for(const sample of fixture.errorCases){const p=installedProtocol("agents-sdk/jsonl","0.1.0","fixture");p.accept({type:"start",model:"fixture",cwd:"/tmp"});let caught:any;try{p.accept({type:"error",error_type:sample.error_type,elapsed_seconds:1.5,limits:fixture.historicalLimits});}catch(e){caught=e;}expect(caught.details.nativeFailure).toEqual({kind:sample.kind,elapsedSeconds:1.5,limits:fixture.historicalLimits});expect(p.terminalEventObserved).toBe(false);} expect(sdkNativeFailure({error_type:"secret",elapsed_seconds:Infinity,limits:{...fixture.defaults,secret:"value"}})).toEqual({kind:"execution"}); }); test("optional SDK tool timeout keeps parent and outer limits independent",async()=>{ for(const sample of fixture.toolOverrides){ const selected={harness:"agents-sdk",nativeToolTimeout:sample.value}; - expect(nativeLimits(selected)).toEqual({...fixture.defaults,toolTimeoutSeconds:sample.seconds}); + expect(nativeLimits(selected)).toEqual({...production.nativeLimits,toolTimeoutSeconds:sample.seconds}); expect(nativeLimitsArgv(selected)).toEqual(["--tool-timeout",String(sample.seconds)]); } for(const nativeToolTimeout of fixture.invalidTimeouts)expect(()=>nativeLimits({harness:"agents-sdk",nativeToolTimeout})).toThrow(); @@ -55,7 +56,7 @@ test("optional SDK tool timeout keeps parent and outer limits independent",async const userConfigPath=join(root,"user.toml");await writeFile(userConfigPath,'harness="agents-sdk"\nnative_tool_timeout="1s"\n'); const parsed=parseEntrypoint(["--native-tool-timeout","180s","--timeout","10m","task"]); const c=await resolveConfiguration(parsed.global,{processCwd:root,userConfigPath,env:{PROSE_NATIVE_TOOL_TIMEOUT:"2s"}}); - expect(nativeLimits(c.values)).toEqual({...fixture.defaults,toolTimeoutSeconds:180}); + expect(nativeLimits(c.values)).toEqual({...production.nativeLimits,toolTimeoutSeconds:180}); expect(c.sources.nativeToolTimeout?.kind).toBe("flag");expect(c.values.timeout).toBe("10m"); }finally{await rm(root,{recursive:true,force:true});} }); diff --git a/cli/bun/test/sdk-production.test.ts b/cli/bun/test/sdk-production.test.ts new file mode 100644 index 00000000..81aa7cbf --- /dev/null +++ b/cli/bun/test/sdk-production.test.ts @@ -0,0 +1,82 @@ +import {afterEach,expect,test} from "bun:test"; +import Ajv2020 from "ajv/dist/2020"; +import observationSchema from "../../shared/schemas/sdk-observation.schema.json"; +import {mkdtemp,mkdir,writeFile,readFile,realpath,rm,symlink} from "node:fs/promises"; +import {join} from "node:path"; +import {tmpdir} from "node:os"; +import oracle from "../../shared/fixtures/adapters/sdk-production.json"; +import {resolveInstalledExecutable} from "../src/adapters/executable"; +import {sdkObservations} from "../src/adapters/sdk-observation"; +import {installedProtocol} from "../src/adapters/protocols"; +import {nativeLimits,sdkNativeFailure} from "../src/adapters/sdk-limits"; +import {runCli} from "../src/cli"; +import {buildInstalledAdapterEnvironment} from "../src/adapters/environment"; +import {installedAdapterDefinition} from "../src/adapters/recipes"; +import {assertInstalledAdapterPlatform} from "../src/adapters/admission"; +import {runInstalledAdapter} from "../src/adapters/runner"; +import {verifyRuntimeImage,canonicalJson,sha256} from "../src/core/image"; +import {sentinelFixtureImage} from "./sentinel-fixture"; +import type {RunnerInvocation} from "../src/core/types"; +const roots:string[]=[]; +const validateObservation=new Ajv2020({strict:true,allErrors:true}).compile(observationSchema); +async function workspace(){const root=await realpath(await mkdtemp(join(tmpdir(),"prose-sdk-production-")));roots.push(root);await mkdir(join(root,".git"));return root;} +afterEach(async()=>{for(const root of roots.splice(0))await rm(root,{recursive:true,force:true});}); +test("packaged SDK native support admits both GNU Linux architectures and rejects musl",()=>{ + for(const arch of ["arm64","x64"] as const){expect(assertInstalledAdapterPlatform("agents-sdk/jsonl",{platform:"linux",arch,libc:"gnu"})).toBe(`linux-${arch}-gnu`);expect(()=>assertInstalledAdapterPlatform("agents-sdk/jsonl",{platform:"linux",arch,libc:"musl"})).toThrow();} +}); +test("packaged SDK discovery follows the CLI symlink, relocates together and refuses PATH fallback",async()=>{ + const root=await workspace(),payload=join(root,"payload");await mkdir(payload);await writeFile(join(payload,"prose"),"fixture",{mode:0o755});await writeFile(join(payload,"prose-agents-sdk"),"fixture",{mode:0o755});await symlink(join(payload,"prose"),join(root,"prose-link")); + const input={adapterId:"agents-sdk/jsonl" as const,ambient:{PATH:"/wrong"},wrapperExecutable:join(root,"prose-link"),platform:"darwin" as const,arch:"arm64" as const}; + expect(await resolveInstalledExecutable(input)).toBe(join(payload,"prose-agents-sdk")); + await rm(join(payload,"prose-agents-sdk"));await writeFile(join(root,"prose-agents-sdk"),"wrong install",{mode:0o755}); + await expect(resolveInstalledExecutable({...input,ambient:{PATH:root}})).rejects.toMatchObject({code:"HARNESS_UNAVAILABLE",details:{fallbackAttempted:false,repairCommand:oracle.discovery.repairCommand}}); + expect(await resolveInstalledExecutable({...input,ambient:{PATH:root},allowPathDiscovery:true})).toBe(join(root,"prose-agents-sdk")); + await symlink(join(root,"prose-agents-sdk"),join(payload,"prose-agents-sdk")); + await expect(resolveInstalledExecutable(input)).rejects.toMatchObject({code:"HARNESS_UNAVAILABLE"}); + const relocated=join(root,"relocated");await mkdir(relocated);await writeFile(join(relocated,"prose"),"fixture",{mode:0o755});await writeFile(join(relocated,"prose-agents-sdk"),"fixture",{mode:0o755}); + expect(await resolveInstalledExecutable({...input,wrapperExecutable:join(relocated,"prose")})).toBe(join(relocated,"prose-agents-sdk")); +}); +for(const key of [undefined,""," "])test(`SDK missing/blank key ${JSON.stringify(key)} fails before helper resolution`,async()=>{ + const root=await workspace();let stdout="",stderr=""; + const exit=await runCli(["--output","json","run","input.prose.md"],{processCwd:root,env:{HOME:join(root,"home"),...(key===undefined?{}:{OPENAI_API_KEY:key})},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:text=>{stderr+=text;}}); + expect(exit).toBe(10);expect(stderr).toBe("");expect(JSON.parse(stdout)).toMatchObject({code:"HARNESS_NEEDS_AUTH",details:{adapterId:"agents-sdk/jsonl",authProfile:"openai-api-key"}});expect(JSON.parse(stdout).action).toContain("OPENAI_API_KEY"); +}); +test("pure exact SDK defaults resolve without credentials or executable discovery",async()=>{ + const root=await workspace();let stdout=""; + expect(await runCli(["cli","config","explain","--json","--","run","input.prose.md"],{processCwd:root,env:{HOME:join(root,"home")},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:()=>{}})).toBe(0); + expect(JSON.parse(stdout)).toMatchObject({values:{harness:{value:oracle.defaults.harness},model:{value:oracle.defaults.model},authProfile:{value:oracle.defaults.authProfile}},runtime:{transport:oracle.defaults.transport,billingOwner:oracle.defaults.billingOwner,nativeLimits:oracle.nativeLimits}}); +}); +test("target bundle errors never expose unresolved inherited credential-profile values",async()=>{ + const root=await workspace();let stdout=""; + expect(await runCli(["cli","harness","use","codex","--auth-profile","target-profile-secret-sentinel","--json"],{processCwd:root,env:{HOME:join(root,"home"),PROSE_AUTH_PROFILE:"inherited-profile-secret-sentinel"},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:()=>{}})).toBe(2); + expect(stdout).not.toContain("target-profile-secret-sentinel");expect(stdout).not.toContain("inherited-profile-secret-sentinel");expect(JSON.parse(stdout)).toMatchObject({code:"CONFIG_INVALID",details:{source:"--auth-profile",reason:"Authentication profile is incompatible with the selected harness."}});expect(JSON.parse(stdout).details).not.toHaveProperty("configurationExplanation"); +}); +test("SDK native observation groups are bounded, invocation owned and omit invalid counters",()=>{ + const tainted={usageObservation:{...oracle.observation.completedUsage,raw_usage:"secret",observedTokenTotals:{...oracle.observation.completedUsage.observedTokenTotals,secret:"sdk-unrecognized-secret-sentinel",output_tokens:-1}},modelIdentity:{requested:"untrusted-request",observed:["z","a","a","sdk-unrecognized-secret-sentinel\n"],serviceTier:{requested:"default",observed:["priority","default","priority","sdk-unrecognized-secret-sentinel"]}}}; + const result=sdkObservations(tainted,"gpt-6.1-sol");expect(result).toMatchObject({usageObservation:{observedTokenTotals:{input_tokens:9,total_tokens:15}},modelIdentity:{requested:"gpt-6.1-sol",observed:["a","z"],serviceTier:{requested:"default",observed:["default","priority"]}}});expect(JSON.stringify(result)).not.toContain("sdk-unrecognized-secret-sentinel");expect(result.usageObservation!.observedTokenTotals).not.toHaveProperty("output_tokens"); + expect(validateObservation(result),JSON.stringify(validateObservation.errors)).toBe(true); + expect(sdkObservations({usageObservation:{...oracle.observation.completedUsage,startedCallCount:-1}},"gpt-6.1-sol")).toEqual({}); + expect(sdkObservations({},"gpt-6.1-sol")).toEqual({}); + for(const modelIdentity of [{observed:[]},{observed:[],serviceTier:{requested:"flex",observed:[]}},{observed:"invalid",serviceTier:{requested:"default",observed:[]}}])expect(sdkObservations({modelIdentity},"gpt-6.1-sol")).toEqual({}); + const bounded=sdkObservations({modelIdentity:{observed:Array.from({length:256},(_,index)=>`gpt-${index}`),serviceTier:{requested:"default",observed:[]}}},"gpt-6.1-sol");expect((bounded.modelIdentity!.observed as unknown[]).length).toBe(128);expect(validateObservation(bounded),JSON.stringify(validateObservation.errors)).toBe(true); +}); +test("SDK final and failure retain observations while usage remains a separate unavailable concept",()=>{ + for(const type of ["final","error"]){const protocol=installedProtocol("agents-sdk/jsonl","0.1.0","fixture",null,true,"gpt-6.1-sol");protocol.accept({type:"start",model:"untrusted",cwd:"/tmp"});const terminal={type,output:"done",error_type:"ExecutionError",usageObservation:oracle.observation.completedUsage,modelIdentity:oracle.observation.modelIdentity,limits:oracle.nativeLimits};if(type==="final")protocol.accept(terminal);else expect(()=>protocol.accept(terminal)).toThrow();expect(protocol.sdkObservations).toEqual({usageObservation:oracle.observation.completedUsage,modelIdentity:oracle.observation.modelIdentity});} + expect(nativeLimits({harness:"agents-sdk"})).toEqual(oracle.nativeLimits);expect(sdkNativeFailure({error_type:"ExecutionError",limits:oracle.nativeLimits})).toMatchObject({limits:oracle.nativeLimits});expect(sdkNativeFailure({limits:{...oracle.nativeLimits,maxChildDepth:2}})).not.toHaveProperty("limits"); + expect(()=>buildInstalledAdapterEnvironment({definition:installedAdapterDefinition("agents-sdk/jsonl"),ambient:{OPENAI_API_KEY:" "},credentialGroup:"openai-api-key"})).toThrow(); +}); +for(const scenario of ["success","failure","cancel"])test(`actual supervised SDK ${scenario} preserves completed observations`,async()=>{ + const root=await workspace(),helper=join(root,"prose-agents-sdk"),image=await verifyRuntimeImage(sentinelFixtureImage); + const observation=scenario==="success"?oracle.observation.completedUsage:oracle.observation.failureUsage; + const terminal={type:scenario==="success"?"final":"error",output:"native result",error_type:scenario==="cancel"?"CancelledError":"ExecutionError",limits:oracle.nativeLimits,usageObservation:observation,modelIdentity:oracle.observation.modelIdentity}; + const source=`#!/Users/mm/openprose/.scratch/imp-086-tools/bun-darwin-aarch64/bun\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));\nconst terminal=${JSON.stringify(terminal)};\n${scenario==="cancel"?"console.log(JSON.stringify({...terminal,type:'tool_result',name:'execute_shell'}));process.on('SIGTERM',()=>{console.log(JSON.stringify(terminal));process.exit(1)});setInterval(()=>{},1000);":"console.log(JSON.stringify(terminal));process.exit("+(scenario==="success"?0:1)+");"}`; + await writeFile(helper,source,{mode:0o755});const task={schema:image.manifest.taskEnvelope.schemaId,argv:["prose","run","input.prose.md"],interactionMode:"non-interactive" as const}; + const invocation:RunnerInvocation={schema:"openprose.runner-invocation/1",invocationId:"fixture-invocation-0001",cwd:root,languageImage:{formatVersion:image.manifest.imageFormatVersion,version:image.manifest.imageVersion,sha256:image.aggregateSha256},runner:{name:"bun",version:"test",commit:"test"},harness:"agents-sdk",transport:"jsonl",recursionToken:"fixture-recursion",task,taskDigestSha256:await sha256(canonicalJson(task))}; + const controller=new AbortController(),capture=join(root,"native.jsonl"); + const timer=scenario==="cancel"?setInterval(async()=>{if((await readFile(capture,"utf8").catch(()=>"")).includes('"tool_result"'))controller.abort();},25):undefined; + try { + const outcome=await runInstalledAdapter({adapterId:"agents-sdk/jsonl",executable:helper,harnessVersion:"0.1.0",credentialGroup:"openai-api-key",invocation,image,ambient:{OPENAI_API_KEY:"fixture-key"},model:"gpt-6.1-sol",outputContract:"native",timeoutMs:2000,temporaryRoot:root,cancelSignal:controller.signal,...(scenario==="cancel"?{nativeLog:capture}:{})}); + expect(outcome.sdkObservations).toEqual({usageObservation:observation,modelIdentity:oracle.observation.modelIdentity});expect(outcome.process.exitCode).toBe(scenario==="success"?0:1); + if(scenario==="cancel")expect(outcome.process.cancellationReason).toBe("caller");else expect(outcome.process.error===null).toBe(scenario==="success"); + }finally{if(timer!==undefined)clearInterval(timer);} +}); diff --git a/cli/bun/test/shared-contract.test.ts b/cli/bun/test/shared-contract.test.ts index 002e5875..0cecc0df 100644 --- a/cli/bun/test/shared-contract.test.ts +++ b/cli/bun/test/shared-contract.test.ts @@ -26,6 +26,7 @@ for (const name of [ "native-limits", "native-output-limits", "native-failure", + "sdk-observation", "runner-error", // runner-error's `details.planned` is `service-operation.schema.json#/$defs/plannedRequest` //; without it Ajv throws "can't resolve reference" and no test in this file runs. @@ -150,7 +151,7 @@ describe("shared runner contracts", () => { test("hosted-unavailable is a schema-valid attempted-run result with exact taxonomy", async () => { const io = fixture(); - expect(await runCli(["--output", "json", "run", "fixture.prose.md"], io.deps)).toBe(10); + expect(await runCli(["--harness", "openprose", "--output", "json", "run", "fixture.prose.md"], io.deps)).toBe(10); const result = JSON.parse(io.stdout()); expectValid(validateResult, result); expect(result).toMatchObject({ @@ -159,7 +160,7 @@ describe("shared runner contracts", () => { error: { code: "HOSTED_UNAVAILABLE", action: "To use the hosted service, run `cli run submit FILE --preview`; running programs on this machine needs a local harness (`cli harness list`).", - details: { suggestedArgv: ["--output", "json", "cli", "run", "submit", "fixture.prose.md", "--preview"] }, + details: { suggestedArgv: ["--harness", "openprose", "--output", "json", "cli", "run", "submit", "fixture.prose.md", "--preview"] }, }, }); expect(io.invocations).toHaveLength(0); diff --git a/cli/ci/check_architecture.py b/cli/ci/check_architecture.py index 509a704d..0ef2a425 100644 --- a/cli/ci/check_architecture.py +++ b/cli/ci/check_architecture.py @@ -1221,6 +1221,10 @@ def _tainted_names(code: str, *, rust: bool) -> set[str]: while changed: changed = False for name, expression in assignments: + # Rust's `_` is a discard pattern, never a bound local. Treating + # it as an alias also taints unrelated closure parameters `|_|`. + if rust and name == "_": + continue if name not in tainted and ( seed in expression or ".argv" in expression diff --git a/cli/ci/check_workflows.py b/cli/ci/check_workflows.py index ae112ab3..9e4bd0df 100644 --- a/cli/ci/check_workflows.py +++ b/cli/ci/check_workflows.py @@ -213,11 +213,20 @@ def require(condition: bool, message: str) -> None: ) require(inputs["plan"]["required"] == "true", "reviewed plan is required") fetch = by_name["Fetch exact draft artifacts without executing them"] + sdk_custody = by_name["Verify retained production SDK source and sibling custody"] + require(sdk_custody.get("if") is None and sdk_custody.get("env") is None, + "SDK custody must be unconditional and credential-free") + require(sdk_custody.get("run") == + 'python3 cli/ci/publication.py verify --plan "$RELEASE_PLAN" --artifacts "$RUNNER_TEMP/prose-publication"', + "SDK custody must verify reviewed fetched bytes") + require(not any(fragment in step.get("run", "") for step in steps + for fragment in ("build_agents_sdk.py", "sign_macos.py", "package_local.py", "build_kernel_rc.py")), + "publisher must not reconstruct reviewed candidates") publish = by_name[ "Verify required platform trust, sign artifact digests, and publish exact npm bytes" ] require( - steps.index(fetch) < steps.index(publish), + steps.index(fetch) < steps.index(sdk_custody) < steps.index(publish), "fetch exact bytes before publication", ) require( @@ -271,6 +280,20 @@ def require(condition: bool, message: str) -> None: ) python = by_name["Prepare pinned Python on every architecture"] require("uv python install 3.10.20" in python["run"], "Python pin drift") + if name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): + sdk_prepare = by_name["Prepare the separately locked Agents SDK build interpreter"] + sdk_test = by_name["Test the actual pinned Agents SDK runtime without provider credentials"] + require(sdk_prepare.get("if") is None and sdk_test.get("if") is None, + "locked SDK preparation and actual runtime tests must be unconditional") + require(sdk_prepare.get("run") == + 'uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python"\n' + '"$RUNNER_TEMP/agents-sdk-python/bin/python3" -m pip install --require-hashes --only-binary=:all: -r harnesses/agents-sdk/requirements-build.txt\n', + "SDK must use a separate pinned hash-locked interpreter") + require(sdk_test.get("run") == + '"$RUNNER_TEMP/agents-sdk-python/bin/python3" -m unittest discover -s harnesses/agents-sdk -p test_run.py', + "test actual SDK graph under pinned build Python") + require(steps.index(python) < steps.index(sdk_prepare) < steps.index(sdk_test), + "prepare pinned SDK interpreter before runtime qualification") if name == "cli-ci.yml": require( "rustup target add x86_64-pc-windows-msvc --toolchain 1.87.0" in prepare["run"], @@ -305,6 +328,31 @@ def require(condition: bool, message: str) -> None: by_name[required_step]["run"] == command, "required qualification command drift", ) + if name == "cli-distribution-check.yml": + sdk_build = by_name["Build and verify production SDK standalone and npm installations"] + sdk_homebrew = by_name["Rehearse Homebrew against the production SDK archives"] + require(sdk_build.get("if") is None and sdk_homebrew.get("if") is None, + "production SDK installations must be qualified unconditionally") + require(sdk_build.get("env") == {"RC_VERSION": "0.15.0-rc.1"} + and sdk_build.get("run") == + 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3"', + "production SDK must use the native source-bound RC builder") + require(sdk_homebrew.get("env") == {"RC_VERSION": "0.15.0-rc.1", "EXPECTED_SOURCE": "${{ github.sha }}"} + and sdk_homebrew.get("run") == + '"$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py \\\n' + ' --kernel-rc "$RUNNER_TEMP/kernel-rc" \\\n' + ' --expected-source "$EXPECTED_SOURCE" --expected-version "$RC_VERSION" \\\n' + ' --output "$RUNNER_TEMP/kernel-rc/homebrew"\n', + "production Homebrew must bind current source and exact SDK archives") + retention = next(step for step in steps if step.get("uses", "").startswith("actions/upload-artifact@")) + retained = retention.get("with", {}).get("path", "").splitlines() + require(all("${{ runner.temp }}/kernel-rc/" + path in retained for path in + ("agents-sdk", "package", "logs", "build-report.json", "homebrew")), + "retain SDK trio, release manifest, native probes and Homebrew custody") + setup = next(step for step in steps if step.get("uses", "").startswith("Homebrew/actions/setup-homebrew@")) + require(steps.index(sdk_test) < steps.index(sdk_build) < steps.index(setup) + < steps.index(sdk_homebrew) < steps.index(retention), + "freeze/install SDK before exact archive Homebrew checks and retention") if name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): kernel_rc = name == "cli-kernel-rc.yml" label = ("Rehearse Homebrew against these exact release archives" if kernel_rc diff --git a/cli/ci/test_check_architecture.py b/cli/ci/test_check_architecture.py index 7c6c78a8..a7ad9c7a 100644 --- a/cli/ci/test_check_architecture.py +++ b/cli/ci/test_check_architecture.py @@ -718,6 +718,26 @@ def test_node_net_remains_rejected_for_every_other_bun_source_path(self) -> None [violation.rule for violation in violations], ) + def test_rust_discard_does_not_taint_unrelated_paths(self) -> None: + temporary, root = self.fixture() + self.addCleanup(temporary.cleanup) + source = root / "cli/rust/crates/runner/src/lib.rs" + source.parent.mkdir(parents=True, exist_ok=True) + source.write_text( + 'fn read_config(keys: &[String]) {\n' + ' let _ = keys.len();\n' + ' let source = config_path().map_err(|_| "unavailable");\n' + ' let _ = std::fs::read(source);\n' + '}\n', 'utf-8') + self.assertNotIn("opaque-program-boundary", {v.rule for v in check_repository(root)}) + source.write_text( + 'fn forbidden(argv: &[String]) {\n' + ' let _ = argv.len();\n' + ' let source = &argv[0];\n' + ' let _ = std::fs::read(source);\n' + '}\n', 'utf-8') + self.assertIn("opaque-program-boundary", {v.rule for v in check_repository(root)}) + @unittest.skipUnless(hasattr(Path, "symlink_to"), "requires symlink support") def test_symlinked_stable_source_is_never_followed_as_product_code(self) -> None: temporary, root = self.fixture() diff --git a/cli/ci/test_check_workflows.py b/cli/ci/test_check_workflows.py index 93c488cf..e099cbce 100644 --- a/cli/ci/test_check_workflows.py +++ b/cli/ci/test_check_workflows.py @@ -242,6 +242,54 @@ def test_locked_dependency_guards_and_required_commands(self): name, lambda w, j, i=index: j["steps"][i].update({"if": "false"}) ) + def test_production_sdk_setup_and_runtime_cannot_be_bypassed(self): + for name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): + for label in ("Prepare the separately locked Agents SDK build interpreter", + "Test the actual pinned Agents SDK runtime without provider credentials"): + index = next(i for i, step in enumerate(self.workflows[name]["jobs"][JOBS[name]]["steps"]) + if step.get("name") == label) + self.changed(name, lambda w, j, i=index: j["steps"][i].update({"if": "false"})) + self.changed(name, lambda w, j, i=index: j["steps"][i].update(run="echo skipped")) + index = next(i for i, step in enumerate(self.workflows[name]["jobs"][JOBS[name]]["steps"]) + if step.get("name") == "Prepare the separately locked Agents SDK build interpreter") + for before, after in (("--require-hashes", ""), ("3.10.20", "3.12"), + ("agents-sdk-python", "distribution-python"), + ("requirements-build.txt", "requirements.txt")): + self.changed(name, lambda w, j, i=index, a=before, b=after: j["steps"][i].update( + run=j["steps"][i]["run"].replace(a, b))) + + def test_distribution_production_sdk_custody_is_required(self): + name = "cli-distribution-check.yml" + steps = self.workflows[name]["jobs"][JOBS[name]]["steps"] + for label in ("Build and verify production SDK standalone and npm installations", + "Rehearse Homebrew against the production SDK archives"): + index = next(i for i, step in enumerate(steps) if step.get("name") == label) + self.changed(name, lambda w, j, i=index: j["steps"].pop(i)) + self.changed(name, lambda w, j, i=index: j["steps"][i].update({"if": "false"})) + self.changed(name, lambda w, j, i=index: j["steps"][i].update(run="echo skipped")) + self.changed(name, lambda w, j, i=index: j["steps"][i]["env"].update(RC_VERSION="unreviewed")) + retention = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("actions/upload-artifact@")) + for path in ("agents-sdk", "package", "logs", "build-report.json", "homebrew"): + self.changed(name, lambda w, j, i=retention, p=path: j["steps"][i]["with"].update( + path=j["steps"][i]["with"]["path"].replace("${{ runner.temp }}/kernel-rc/" + p, "omitted"))) + build = next(i for i, step in enumerate(steps) + if step.get("name") == "Build and verify production SDK standalone and npm installations") + self.changed(name, lambda w, j, i=build: j["steps"].insert(0, j["steps"].pop(i))) + + def test_publisher_sdk_custody_cannot_mutate_reviewed_candidates(self): + name = "cli-publish.yml" + steps = self.workflows[name]["jobs"][JOBS[name]]["steps"] + index = next(i for i, step in enumerate(steps) + if step.get("name") == "Verify retained production SDK source and sibling custody") + self.changed(name, lambda w, j, i=index: j["steps"].pop(i)) + self.changed(name, lambda w, j, i=index: j["steps"][i].update({"if": "false"})) + self.changed(name, lambda w, j, i=index: j["steps"].insert(0, j["steps"].pop(i))) + self.changed(name, lambda w, j, i=index: j["steps"][i].update(run="echo verified")) + for script in ("build_agents_sdk.py", "sign_macos.py", "package_local.py", "build_kernel_rc.py"): + self.changed(name, lambda w, j, p=script: j["steps"].append( + {"name": "Mutate candidate", "run": "python3 cli/ci/" + p})) + def test_admission_pipeline_cannot_hide_failure(self): self.changed( "cli-ci.yml", diff --git a/cli/conformance/adversarial/adapter-products/fake_live_harness.py b/cli/conformance/adversarial/adapter-products/fake_live_harness.py index 67abeddd..b05ccab6 100644 --- a/cli/conformance/adversarial/adapter-products/fake_live_harness.py +++ b/cli/conformance/adversarial/adapter-products/fake_live_harness.py @@ -17,12 +17,14 @@ VERSIONS = { + "prose-agents-sdk": "prose-agents-sdk 0.1.0", "codex": "codex-cli 0.149.0-alpha.4.1", "claude": "2.1.243 (Claude Code)", "prime-agent": "prime-agent 0.7.0", "omp": "omp/18.0.9", } ADAPTERS = { + "prose-agents-sdk": "agents-sdk/jsonl", "codex": "codex/exec-json", "claude": "claude/print-stream-json", "prime-agent": "prime/rpc", @@ -99,7 +101,7 @@ def prompt_id(stdin: bytes) -> str | None: def prompt_files(argv: list[str]) -> list[dict[str, Any]]: result: list[dict[str, Any]] = [] - for flag in ("--append-system-prompt-file", "--append-system-prompt", "--config"): + for flag in ("--append-system-prompt-file", "--append-system-prompt", "--config", "--instructions"): if flag not in argv: continue index = argv.index(flag) @@ -272,11 +274,66 @@ def terminal_for_image( ) +def sdk_emit(record: dict[str, Any], lexemes: dict[str, str]) -> None: + # Closed provider-free fixtures exercise equivalent JSON numeric encodings. + encoded = compact(record) + for key, lexeme in lexemes.items(): + value = json.loads(lexeme) + if isinstance(value, bool) or not isinstance(value, (int, float)): + raise ValueError('SDK fixture numeric lexeme must be a JSON number') + token = compact(key) + ':' + start = encoded.find(token) + if start < 0: + continue + start += len(token) + end = start + while end < len(encoded) and encoded[end] not in ',}': + end += 1 + if json.loads(encoded[start:end]) != value: + raise ValueError('SDK fixture numeric lexeme changed its value') + encoded = encoded[:start] + lexeme + encoded[end:] + if json.loads(encoded) != record: + raise ValueError('SDK fixture wire encoding changed its record') + sys.stdout.write(encoded + '\n') + sys.stdout.flush() + + +def sdk_main(argv: list[str]) -> int: + """No provider dependencies: closed records supplied by the shared SDK oracle.""" + workspace = Path.cwd() + control = json.loads((workspace / ".sdk-compatibility-fixture.json").read_text("utf-8")) + if Path(sys.argv[0]).resolve() != Path(control["expectedHelper"]): + (workspace / ".sdk-wrong-helper-used").write_text("PATH helper used", encoding="utf-8") + return 88 + if argv == ["--version"]: + (workspace / ".sdk-harness-probed").write_text("version", encoding="utf-8") + print(VERSIONS['prose-agents-sdk']) + return 0 + (workspace / ".sdk-harness-started").write_text("started", encoding="utf-8") + if control["scenario"] == "trap": + return 88 + task = task_from_launch(argv, b'') + files = prompt_files(argv) + model = argv[argv.index('--model') + 1] + if model != control["model"] or '--cwd' not in argv or Path(argv[argv.index('--cwd') + 1]).resolve() != workspace.resolve(): + return 89 + terminal = terminal_for_image(argv, b'', files, task) + common = dict(usageObservation=control['usageObservation'], modelIdentity=control['modelIdentity']) + sdk_emit(dict(type='start', model=model, cwd=str(workspace), limits=control['limits']), control['wireNumberLexemes']) + if control['scenario'] == 'failure': + sdk_emit(dict(type='error', error_type='ExecutionError', message='safe fixture failure', limits=control['limits'], **common), control['wireNumberLexemes']) + return 1 + sdk_emit(dict(type='final', output=terminal, usage={'requests':3,'input_tokens':9,'output_tokens':6,'total_tokens':15}, **common), control['wireNumberLexemes']) + return 0 + + def main() -> int: executable = Path(sys.argv[0]).name argv = sys.argv[1:] if executable not in ADAPTERS: raise ValueError(f"unexpected executable identity: {executable}") + if executable == "prose-agents-sdk": + return sdk_main(argv) compatibility_path = Path.cwd() / ".claude-compatibility-fixture.json" compatibility = json.loads(compatibility_path.read_text("utf-8")) if executable == "claude" and compatibility_path.is_file() else {} if executable == "codex": diff --git a/cli/conformance/cases/adapters/sdk-production-01.json b/cli/conformance/cases/adapters/sdk-production-01.json new file mode 100644 index 00000000..bfa809e8 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-01.json @@ -0,0 +1,105 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-01", + "summary": "Pure exact explanation resolves shipped SDK defaults without probing a helper or credential.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "trap", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "cli", + "config", + "explain", + "--json", + "--", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config" + } + }, + "expected": { + "exitCode": 0, + "stdout": { + "kind": "json", + "schema": "openprose.configuration-explanation/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "target": { + "argv": [ + "prose", + "run", + "input.prose.md" + ] + }, + "values": { + "harness": { + "value": "agents-sdk", + "source": { + "kind": "default", + "location": "built-in" + } + }, + "model": { + "value": "gpt-6.1-sol", + "source": { + "kind": "default", + "location": "built-in:agents-sdk" + } + }, + "authProfile": { + "value": "openai-api-key", + "source": { + "kind": "default", + "location": "built-in:agents-sdk" + } + } + }, + "runtime": { + "transport": "jsonl", + "authProfile": "openai-api-key", + "billingOwner": "user-provider", + "permissionMode": null, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + } + } + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-02.json b/cli/conformance/cases/adapters/sdk-production-02.json new file mode 100644 index 00000000..3b0e676f --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-02.json @@ -0,0 +1,59 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-02", + "summary": "Missing API key fails actionably before harness execution or inference.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "trap", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config" + } + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-error/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "errorCode": "HARNESS_NEEDS_AUTH", + "errorAction": "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.", + "resultMatches": { + "code": "HARNESS_NEEDS_AUTH", + "details": { + "authProfile": "openai-api-key" + } + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-03.json b/cli/conformance/cases/adapters/sdk-production-03.json new file mode 100644 index 00000000..ad997ec8 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-03.json @@ -0,0 +1,60 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-03", + "summary": "Blank API key fails actionably before harness execution or inference.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "trap", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": " " + } + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-error/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "errorCode": "HARNESS_NEEDS_AUTH", + "errorAction": "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.", + "resultMatches": { + "code": "HARNESS_NEEDS_AUTH", + "details": { + "authProfile": "openai-api-key" + } + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-04.json b/cli/conformance/cases/adapters/sdk-production-04.json new file mode 100644 index 00000000..cb3499d5 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-04.json @@ -0,0 +1,127 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-04", + "summary": "Default invocation uses the sibling helper and retains safe completed observations without cost authority.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "success", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 0, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 3, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 0, + "terminal": { + "classification": "success", + "transportCompleted": true, + "terminalEventObserved": true + }, + "semantic": { + "status": "not-applicable" + } + }, + "forwardedTask": { + "argv": [ + "prose", + "run", + "input.prose.md" + ], + "interactionMode": "non-interactive" + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-05.json b/cli/conformance/cases/adapters/sdk-production-05.json new file mode 100644 index 00000000..005ad252 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-05.json @@ -0,0 +1,165 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-05", + "summary": "Integral decimal and exponent JSON native limits survive SDK failure with safe observations.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "execution", + "limits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + } + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Inspect the harness diagnostic on stderr and invoke the `cli doctor` runner operation." + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-06.json b/cli/conformance/cases/adapters/sdk-production-06.json new file mode 100644 index 00000000..348f61ba --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-06.json @@ -0,0 +1,116 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-06", + "summary": "Unknown fields and malformed raw observation strings cannot reach normalized output.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "tainted", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 0, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 3, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15, + "input_tokens_details.cached_tokens": 0 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 0, + "terminal": { + "classification": "success", + "transportCompleted": true, + "terminalEventObserved": true + }, + "semantic": { + "status": "not-applicable" + } + }, + "forwardedTask": { + "argv": [ + "prose", + "run", + "input.prose.md" + ], + "interactionMode": "non-interactive" + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-07.json b/cli/conformance/cases/adapters/sdk-production-07.json new file mode 100644 index 00000000..90167600 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-07.json @@ -0,0 +1,68 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-07", + "summary": "Saved alternative harness survives the SDK-default upgrade with its contextual authentication route.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "trap", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "cli", + "config", + "explain", + "--json" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config" + } + }, + "expected": { + "exitCode": 0, + "stdout": { + "kind": "json", + "schema": "openprose.configuration-explanation/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "values": { + "harness": { + "value": "claude" + }, + "model": { + "value": null + }, + "authProfile": { + "value": "claude-subscription" + } + }, + "runtime": { + "authProfile": "claude-subscription", + "nativeLimits": null + } + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-08.json b/cli/conformance/cases/adapters/sdk-production-08.json new file mode 100644 index 00000000..e54d25a9 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-08.json @@ -0,0 +1,127 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-08", + "summary": "Relocated CLI reached through a symlink resolves its own canonical sibling, never a PATH helper.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "success", + "sdkInstallation": "symlink" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 0, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 3, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 0, + "terminal": { + "classification": "success", + "transportCompleted": true, + "terminalEventObserved": true + }, + "semantic": { + "status": "not-applicable" + } + }, + "forwardedTask": { + "argv": [ + "prose", + "run", + "input.prose.md" + ], + "interactionMode": "non-interactive" + } + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-09.json b/cli/conformance/cases/adapters/sdk-production-09.json new file mode 100644 index 00000000..37b94aba --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-09.json @@ -0,0 +1,152 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-09", + "summary": "Invalid SDK maxChildren is omitted from failure limits without losing safe observations.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "execution" + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Inspect the harness diagnostic on stderr and invoke the `cli doctor` runner operation." + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-10.json b/cli/conformance/cases/adapters/sdk-production-10.json new file mode 100644 index 00000000..d6023a2d --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-10.json @@ -0,0 +1,152 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-10", + "summary": "Invalid SDK maxChildDepth is omitted from failure limits without losing safe observations.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "execution" + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Inspect the harness diagnostic on stderr and invoke the `cli doctor` runner operation." + } +} diff --git a/cli/conformance/cases/case-manifest.schema.json b/cli/conformance/cases/case-manifest.schema.json index 613eb249..8d5ff940 100644 --- a/cli/conformance/cases/case-manifest.schema.json +++ b/cli/conformance/cases/case-manifest.schema.json @@ -4,104 +4,320 @@ "title": "OpenProse runner black-box case v1", "type": "object", "additionalProperties": false, - "required": ["schema", "id", "summary", "phase", "tags", "controls", "invocation", "expected"], + "required": [ + "schema", + "id", + "summary", + "phase", + "tags", + "controls", + "invocation", + "expected" + ], "properties": { - "schema": { "const": "openprose.runner-case/1" }, - "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9._-]+$" }, - "summary": { "type": "string", "minLength": 1 }, - "phase": { "type": "integer", "minimum": 0, "maximum": 7 }, - "tags": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "type": "string", "pattern": "^[a-z0-9][a-z0-9._-]+$" } }, - "platforms": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["all", "unix", "windows"] } }, + "schema": { + "const": "openprose.runner-case/1" + }, + "id": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9._-]+$" + }, + "summary": { + "type": "string", + "minLength": 1 + }, + "phase": { + "type": "integer", + "minimum": 0, + "maximum": 7 + }, + "tags": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "type": "string", + "pattern": "^[a-z0-9][a-z0-9._-]+$" + } + }, + "platforms": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "enum": [ + "all", + "unix", + "windows" + ] + } + }, "controls": { - "type": "object", "additionalProperties": false, - "required": ["fixedInvocationId", "fixedTimestamp", "fixedMonotonicMs", "network"], + "type": "object", + "additionalProperties": false, + "required": [ + "fixedInvocationId", + "fixedTimestamp", + "fixedMonotonicMs", + "network" + ], "properties": { - "fixedInvocationId": { "type": "string", "minLength": 1 }, - "fixedTimestamp": { "type": "string", "format": "date-time" }, - "fixedMonotonicMs": { "type": "integer", "minimum": 0 }, - "network": { "const": "denied" }, + "fixedInvocationId": { + "type": "string", + "minLength": 1 + }, + "fixedTimestamp": { + "type": "string", + "format": "date-time" + }, + "fixedMonotonicMs": { + "type": "integer", + "minimum": 0 + }, + "network": { + "const": "denied" + }, "configurationFixture": { - "enum": ["operations.config-production-01", "operations.config-production-02", "operations.config-production-03", "operations.config-production-04", "operations.config-production-05", "operations.config-production-06", "operations.config-production-07", "operations.config-production-08", "operations.config-production-09", "operations.config-production-10", "operations.config-production-11", "operations.config-production-12"] + "enum": [ + "operations.config-production-01", + "operations.config-production-02", + "operations.config-production-03", + "operations.config-production-04", + "operations.config-production-05", + "operations.config-production-06", + "operations.config-production-07", + "operations.config-production-08", + "operations.config-production-09", + "operations.config-production-10", + "operations.config-production-11", + "operations.config-production-12" + ] }, "omitEnvironment": { "type": "array", "uniqueItems": true, - "items": { "enum": ["HOME", "XDG_CONFIG_HOME", "APPDATA"] } + "items": { + "enum": [ + "HOME", + "XDG_CONFIG_HOME", + "APPDATA" + ] + } }, "fakeHarness": { - "type": "object", "additionalProperties": false, - "required": ["scenario"], + "type": "object", + "additionalProperties": false, + "required": [ + "scenario" + ], "properties": { - "scenario": { "enum": ["success", "malformed", "truncated", "eof-without-terminal", "nonzero", "terminal-nonzero", "delay", "stderr", "fragmented", "crlf", "duplicate-terminal", "reordered", "descendant"] }, - "delayMs": { "type": "integer", "minimum": 0 }, - "cancelAfterMs": { "type": "integer", "minimum": 0 } + "scenario": { + "enum": [ + "success", + "malformed", + "truncated", + "eof-without-terminal", + "nonzero", + "terminal-nonzero", + "delay", + "stderr", + "fragmented", + "crlf", + "duplicate-terminal", + "reordered", + "descendant" + ] + }, + "delayMs": { + "type": "integer", + "minimum": 0 + }, + "cancelAfterMs": { + "type": "integer", + "minimum": 0 + } } }, "installedAdapter": { - "type": "object", "additionalProperties": false, - "required": ["adapterId"], + "type": "object", + "additionalProperties": false, + "required": [ + "adapterId" + ], "properties": { "adapterId": { "enum": [ "codex/exec-json", "claude/print-stream-json", "prime/rpc", - "omp/rpc" + "omp/rpc", + "agents-sdk/jsonl" + ] + }, + "version": { + "type": "string", + "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+(?:-[0-9A-Za-z.-]+)?$" + }, + "capabilityScenario": { + "enum": [ + "missing-json", + "missing-sandbox", + "prefix-json", + "descriptive-json", + "wrong-stream", + "nonzero", + "oversized" + ] + }, + "protocolScenario": { + "enum": [ + "unknown-type", + "missing-terminal" + ] + }, + "telemetryScenario": { + "enum": [ + "unknown-type", + "unknown-subtype" ] }, - "version": { "type": "string", "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+(?:-[0-9A-Za-z.-]+)?$" }, - "capabilityScenario": { "enum": ["missing-json", "missing-sandbox", "prefix-json", "descriptive-json", "wrong-stream", "nonzero", "oversized"] }, - "protocolScenario": { "enum": ["unknown-type", "missing-terminal"] }, - "telemetryScenario": { "enum": ["unknown-type", "unknown-subtype"] }, "versionProbeScenario": { - "enum": ["wrong-stream-success"] + "enum": [ + "wrong-stream-success" + ] + }, + "sdkScenario": { + "enum": [ + "trap", + "success", + "failure", + "tainted" + ] + }, + "sdkInstallation": { + "enum": [ + "sibling", + "symlink" + ] } } } } }, "invocation": { - "type": "object", "additionalProperties": false, - "required": ["argv", "cwd", "environment"], + "type": "object", + "additionalProperties": false, + "required": [ + "argv", + "cwd", + "environment" + ], "properties": { - "argv": { "type": "array", "items": { "type": "string" } }, - "cwd": { "type": "string", "minLength": 1 }, - "environment": { "type": "object", "additionalProperties": { "type": "string" } } + "argv": { + "type": "array", + "items": { + "type": "string" + } + }, + "cwd": { + "type": "string", + "minLength": 1 + }, + "environment": { + "type": "object", + "additionalProperties": { + "type": "string" + } + } } }, "expected": { - "type": "object", "additionalProperties": false, - "required": ["exitCode", "stdout", "stderr", "startedHarness"], + "type": "object", + "additionalProperties": false, + "required": [ + "exitCode", + "stdout", + "stderr", + "startedHarness" + ], "dependentSchemas": { "resultFixture": { "properties": { "stdout": { - "properties": { "kind": { "const": "json" } } + "properties": { + "kind": { + "const": "json" + } + } } } } }, "properties": { - "exitCode": { "type": "integer", "minimum": 0, "maximum": 255 }, - "stdout": { "$ref": "#/$defs/outputExpectation" }, - "stderr": { "$ref": "#/$defs/outputExpectation" }, - "startedHarness": { "type": "boolean" }, - "errorCode": { "type": "string", "minLength": 1 }, - "errorAction": { "type": "string", "minLength": 1 }, - "resultMatches": { "type": "object" }, + "exitCode": { + "type": "integer", + "minimum": 0, + "maximum": 255 + }, + "stdout": { + "$ref": "#/$defs/outputExpectation" + }, + "stderr": { + "$ref": "#/$defs/outputExpectation" + }, + "startedHarness": { + "type": "boolean" + }, + "errorCode": { + "type": "string", + "minLength": 1 + }, + "errorAction": { + "type": "string", + "minLength": 1 + }, + "resultMatches": { + "type": "object" + }, "resultFixture": { "type": "string", "pattern": "^cli/shared/fixtures/dx/[a-z0-9][a-z0-9._-]+\\.json$" }, "forwardedTask": { - "type": "object", "additionalProperties": false, - "required": ["argv", "interactionMode"], - "properties": { "argv": { "type": "array", "minItems": 1, "items": { "type": "string" } }, "interactionMode": { "const": "non-interactive" } } + "type": "object", + "additionalProperties": false, + "required": [ + "argv", + "interactionMode" + ], + "properties": { + "argv": { + "type": "array", + "minItems": 1, + "items": { + "type": "string" + } + }, + "interactionMode": { + "const": "non-interactive" + } + } }, "fakeObservation": { - "type": "object", "additionalProperties": false, - "required": ["exactImageBytes", "exactTaskBytes"], - "properties": { "exactImageBytes": { "const": true }, "exactTaskBytes": { "const": true } } + "type": "object", + "additionalProperties": false, + "required": [ + "exactImageBytes", + "exactTaskBytes" + ], + "properties": { + "exactImageBytes": { + "const": true + }, + "exactTaskBytes": { + "const": true + } + } }, "fileEffects": { "type": "array", @@ -109,10 +325,18 @@ "items": { "type": "object", "additionalProperties": false, - "required": ["path", "utf8"], + "required": [ + "path", + "utf8" + ], "properties": { - "path": { "type": "string", "minLength": 1 }, - "utf8": { "type": "string" } + "path": { + "type": "string", + "minLength": 1 + }, + "utf8": { + "type": "string" + } } } } @@ -122,11 +346,98 @@ "$defs": { "outputExpectation": { "oneOf": [ - { "type": "object", "additionalProperties": false, "required": ["kind"], "properties": { "kind": { "const": "empty" } } }, - { "type": "object", "additionalProperties": false, "required": ["kind", "fixture"], "properties": { "kind": { "const": "exact-fixture" }, "fixture": { "type": "string", "minLength": 1 } } }, - { "type": "object", "additionalProperties": false, "required": ["kind", "contains"], "properties": { "kind": { "const": "contains" }, "contains": { "type": "array", "minItems": 1, "items": { "type": "string" } } } }, - { "type": "object", "additionalProperties": false, "required": ["kind", "schema"], "properties": { "kind": { "const": "json" }, "schema": { "type": "string", "minLength": 1 } } }, - { "type": "object", "additionalProperties": false, "required": ["kind", "eventTypes", "terminalType"], "properties": { "kind": { "const": "jsonl" }, "eventTypes": { "type": "array", "items": { "type": "string" } }, "terminalType": { "enum": ["runner.completed", "runner.failed"] } } } + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind" + ], + "properties": { + "kind": { + "const": "empty" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "fixture" + ], + "properties": { + "kind": { + "const": "exact-fixture" + }, + "fixture": { + "type": "string", + "minLength": 1 + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "contains" + ], + "properties": { + "kind": { + "const": "contains" + }, + "contains": { + "type": "array", + "minItems": 1, + "items": { + "type": "string" + } + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "schema" + ], + "properties": { + "kind": { + "const": "json" + }, + "schema": { + "type": "string", + "minLength": 1 + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "kind", + "eventTypes", + "terminalType" + ], + "properties": { + "kind": { + "const": "jsonl" + }, + "eventTypes": { + "type": "array", + "items": { + "type": "string" + } + }, + "terminalType": { + "enum": [ + "runner.completed", + "runner.failed" + ] + } + } + } ] } } diff --git a/cli/conformance/cases/operations/config-production-01.json b/cli/conformance/cases/operations/config-production-01.json index d06d8530..04477fc6 100644 --- a/cli/conformance/cases/operations/config-production-01.json +++ b/cli/conformance/cases/operations/config-production-01.json @@ -44,13 +44,13 @@ "diagnostics": [], "values": { "harness": { - "value": "openprose" + "value": "agents-sdk" }, "model": { - "value": null + "value": "gpt-6.1-sol" }, "authProfile": { - "value": null + "value": "openai-api-key" }, "nativeMaxTurns": { "value": null @@ -60,7 +60,19 @@ } }, "runtime": { - "nativeLimits": null + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + } } } } diff --git a/cli/conformance/runner/run.py b/cli/conformance/runner/run.py index a4f8a13e..ed6bb35e 100755 --- a/cli/conformance/runner/run.py +++ b/cli/conformance/runner/run.py @@ -50,6 +50,7 @@ b"raise SystemExit(64)\n" ) INSTALLED_ADAPTER_EXECUTABLES = { + "agents-sdk/jsonl": "prose-agents-sdk", "codex/exec-json": "codex", "claude/print-stream-json": "claude", "prime/rpc": "prime-agent", @@ -155,6 +156,7 @@ class Observation: stdout_truncated: bool = False stderr_truncated: bool = False configuration_before: dict[str, tuple[str, str]] | None = None + sdk_fixture: dict[str, Any] | None = None @dataclass(frozen=True) @@ -1621,6 +1623,107 @@ def isolate_workspace(workspace: Path) -> None: raise ValueError("mechanical workspace Git boundary must be a directory or file") +def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: Path, + environment_root: Path) -> tuple[Product, dict[str, Any], Path]: + """Exercise real sibling discovery using an exact native candidate byte copy.""" + workspace = workspace.resolve() + environment_root = environment_root.resolve() + if product.interpreter is not None or product.execution_interpreter is not None: + raise ValueError("SDK installation cases require a native compiled candidate") + oracle = json.loads((CLI / 'shared/fixtures/adapters/sdk-production.json').read_text('utf-8')) + entries = [entry for entry in oracle['cases'] if entry['id'] == case['id']] + if len(entries) != 1: + raise ValueError("SDK installation case must reference its closed shared oracle") + fixture = entries[0] + control = case['controls']['installedAdapter'] + if control.get('sdkScenario') != fixture['scenario'] or control.get('sdkInstallation') != fixture['installation']: + raise ValueError("SDK installation controls differ from the shared oracle") + install = environment_root / 'relocated installation' / 'real bin' + install.mkdir(parents=True, exist_ok=False) + source = product.execution_executable or product.executable + native = install / 'prose' + shutil.copyfile(source, native) + native.chmod(0o700) + helper = install / 'prose-agents-sdk' + shutil.copyfile(INSTALLED_ADAPTER_HARNESS, helper) + helper.chmod(0o700) + poison = environment_root / 'foreign-path' + poison.mkdir() + shutil.copyfile(INSTALLED_ADAPTER_HARNESS, poison / 'prose-agents-sdk') + (poison / 'prose-agents-sdk').chmod(0o700) + (poison / 'python3').symlink_to(Path(sys.executable).resolve()) + if fixture['installation'] == 'symlink': + shim = environment_root / 'user bin' + shim.mkdir() + target = shim / 'prose' + target.symlink_to(native) + else: + target = native + if fixture['userConfig'] is not None: + user = workspace / 'home/.prose/cli.toml' + user.parent.mkdir(parents=True) + user.write_bytes(fixture['userConfig'].encode('utf-8')) + usage = deepcopy(oracle['observation']['failureUsage' if fixture['scenario']=='failure' else 'completedUsage']) + identity = deepcopy(oracle['observation']['modelIdentity']) + if fixture['scenario'] == 'tainted': + usage['unknown'] = 'sdk-unrecognized-secret-sentinel' + usage['observedTokenTotals'].update(unknown='sdk-unrecognized-secret-sentinel', **{'input_tokens_details.cached_tokens': -0.0, 'input_tokens_details.cache_write_tokens': -1, 'output_tokens_details.reasoning_tokens': True}) + identity['requested'] = 'sdk-unrecognized-secret-sentinel' + identity['observed'].append('sdk-unrecognized-secret-sentinel\nunsafe') + identity['serviceTier']['requested'] = 'sdk-unrecognized-secret-sentinel' + identity['serviceTier']['observed'].append('sdk-unrecognized-secret-sentinel') + limits = {**oracle['nativeLimits'], **fixture.get('rawLimitsOverrides', {})} + setup = dict(scenario=fixture['scenario'],expectedHelper=str(helper.resolve()),model=oracle['defaults']['model'],limits=limits,usageObservation=usage,modelIdentity=identity,wireNumberLexemes=fixture.get('wireNumberLexemes', {})) + (workspace / '.sdk-compatibility-fixture.json').write_text(json.dumps(setup), encoding='utf-8') + execution = Product(product.name, product.executable, product.runner_name, + execution_executable=target) + fixture = {**fixture, 'nativePath': str(native), 'nativeSha256': sha256(source.read_bytes())} + return execution, fixture, poison + + +def validate_sdk_effects(observation: Observation) -> list[str]: + fixture = observation.sdk_fixture + if fixture is None: + return [] + workspace = observation.workspace + failures = [] + if workspace is None: + return ['SDK fixture lacks its isolated workspace'] + try: + if sha256(Path(fixture['nativePath']).read_bytes()) != fixture['nativeSha256']: + failures.append('SDK relocated candidate bytes changed during execution') + if (workspace / '.sdk-wrong-helper-used').exists(): + failures.append('SDK discovery used a foreign PATH helper') + if fixture['noProbe'] and (workspace / '.sdk-harness-probed').exists(): + failures.append('Pure SDK configuration operation probed a helper') + if fixture['noStart'] and (workspace / '.sdk-harness-started').exists(): + failures.append('SDK helper started for a provider-free setup/configuration failure') + if fixture['userConfig'] is not None and (workspace / 'home/.prose/cli.toml').read_bytes() != fixture['userConfig'].encode('utf-8'): + failures.append('SDK default upgrade changed an explicit alternative harness preference') + except OSError: + failures.append('SDK fixture effect bytes unavailable') + for group in fixture.get('absentObservationGroups', []): + parsed = observation.parsed + error = parsed.get('error') if isinstance(parsed, dict) else None + details = error.get('details') if isinstance(error, dict) else None + if isinstance(parsed, dict) and (group in parsed or (isinstance(details, dict) and group in details)): + failures.append('SDK malformed observation group was retained: ' + group) + for path in fixture.get('absentResultPaths', []): + current = observation.parsed + present = True + for key in path: + if not isinstance(current, dict) or key not in current: + present = False + break + current = current[key] + if present: + failures.append('SDK malformed result field was retained: $.' + '.'.join(path)) + for sentinel in fixture['outputAbsent']: + if any(sentinel.encode('utf-8') in stream for stream in (observation.stdout, observation.stderr)): + failures.append('SDK output exposed a forbidden raw-observation sentinel') + return failures + + def execute( product: Product, case: dict[str, Any], @@ -1649,6 +1752,8 @@ def execute( if fake is not None and installed_adapter is not None: raise ValueError("a case cannot select both fakeHarness and installedAdapter") observation_path: Path | None = None + execution_product = product + sdk_fixture = None if fake is not None: observation_path = environment_root / "fake-observation.json" descendant_path = environment_root / "descendants.json" @@ -1666,7 +1771,11 @@ def execute( additions["OPENPROSE_CONFORMANCE_CANCEL_AFTER_MS"] = str( fake["cancelAfterMs"] ) - if installed_adapter is not None: + if installed_adapter is not None and installed_adapter["adapterId"] == "agents-sdk/jsonl": + execution_product, sdk_fixture, poison = prepare_sdk_installation(product, case, canonical_workspace, environment_root) + additions["PATH"] = str(poison) + observation_path = canonical_workspace / ".sdk-harness-started" + elif installed_adapter is not None: adapter_id = installed_adapter["adapterId"] executable_name = INSTALLED_ADAPTER_EXECUTABLES[adapter_id] harness_bin = environment_root / "installed-adapter-bin" @@ -1696,7 +1805,7 @@ def execute( for name in case["controls"].get("omitEnvironment", []): environment.pop(name, None) completed = run_owned_process( - product.execution_argv(case["invocation"]["argv"]), + execution_product.execution_argv(case["invocation"]["argv"]), cwd=cwd, environment=environment, timeout_seconds=15, @@ -1713,6 +1822,7 @@ def execute( stdout_truncated=completed.stdout_truncated, stderr_truncated=completed.stderr_truncated, configuration_before=configuration_before, + sdk_fixture=sdk_fixture, ) @@ -1986,7 +2096,10 @@ def validate_output(observation: Observation, contracts: ContractRegistry) -> li f"exact JSON fixture {expected['resultFixture']}: {difference}" ) if "errorCode" in expected: - error_record = result.get("error") + error_record = ( + result if result.get("schema") == "openprose.runner-error/1" + else result.get("error") + ) if ( error_record is None and result.get("schema") == "openprose.runner-dry-run-report/1" @@ -2077,6 +2190,7 @@ def validate_output(observation: Observation, contracts: ContractRegistry) -> li f"got {sha256(actual.encode('utf-8'))}" ) failures.extend(validate_configuration_effects(observation)) + failures.extend(validate_sdk_effects(observation)) return failures diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index db5c59a2..8acedfc5 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -109,7 +109,122 @@ def test_host_oracle_rejects_unsupported_host_without_skipping_case(self): wanted = runner.expected_for_host(case, "linux", "aarch64") self.assertEqual("arm64", wanted["resultMatches"]["error"]["details"]["hostArchitecture"]) self.assertTrue(runner.deep_subset({"terminal": {"classification": "success"}}, wanted["resultMatches"])) - self.assertEqual(76, len(list(runner.case_paths(7, set())))) + self.assertEqual(86, len(list(runner.case_paths(7, set())))) + + def test_sdk_installation_fixture_executes_canonical_clone_and_keeps_helper_off_path(self): + for number in range(1,11): + case = json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + environment, workspace = runner.product_roots(root, 'fixture') + runner.isolate_workspace(workspace) + product = runner.Product('fixture', Path('/usr/bin/true')) + execution, fixture, poison = runner.prepare_sdk_installation(product,case,workspace,environment) + native = Path(fixture['nativePath']) + self.assertEqual(product.executable.read_bytes(), native.read_bytes()) + argv = execution.execution_argv(case['invocation']['argv']) + self.assertEqual(Path(argv[0]).resolve(), native) + self.assertEqual(case['controls']['installedAdapter']['sdkInstallation']=='symlink', Path(argv[0]).is_symlink()) + self.assertTrue((native.parent/'prose-agents-sdk').is_file()) + self.assertNotEqual(native.parent,poison) + self.assertTrue((poison/'prose-agents-sdk').is_file()) + observation = runner.Observation(product,case,0,b'',b'',workspace=workspace,sdk_fixture=fixture) + self.assertEqual([],runner.validate_sdk_effects(observation)) + (workspace/'.sdk-wrong-helper-used').touch() + self.assertIn('SDK discovery used a foreign PATH helper',runner.validate_sdk_effects(observation)) + + def test_sdk_fixture_refuses_unfrozen_controls_and_interpreted_candidates(self): + case = json.loads((runner.CASES/'adapters/sdk-production-04.json').read_text()) + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + with self.assertRaisesRegex(ValueError,'native compiled'): + runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true'),interpreter=Path(sys.executable)),case,root,root/'environment') + case['controls']['installedAdapter']['sdkScenario']='trap' + with self.assertRaisesRegex(ValueError,'differ from'): + runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,root,root/'environment') + + def test_sdk_fake_wire_is_provider_free_and_reuses_frozen_observations(self): + oracle = json.loads((runner.CLI/'shared/fixtures/adapters/sdk-production.json').read_text()) + for number in (4,5,6,9,10): + case = json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: + root=Path(temporary); environment,workspace=runner.product_roots(root,'fixture') + runner.isolate_workspace(workspace) + _,fixture,poison=runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,workspace,environment) + helper=Path(fixture['nativePath']).parent/'prose-agents-sdk' + image=workspace/'instructions';image.write_bytes(b'OPENPROSE_SENTINEL_IMAGE_V1') + argv=[sys.executable,str(helper),'--cwd',str(workspace),'--instructions',str(image),'--model','gpt-6.1-sol','--prompt',json.dumps({'argv':['prose','run','input.prose.md']})] + result=runner.run_owned_process(argv,cwd=workspace,environment=runner.hermetic_environment(environment,{'PATH':str(poison)}),timeout_seconds=3) + self.assertEqual(1 if number in (5,9,10) else 0,result.exit_code,result.stderr) + records=[json.loads(line) for line in result.stdout.splitlines()] + self.assertEqual('start',records[0]['type']) + self.assertEqual('error' if number in (5,9,10) else 'final',records[-1]['type']) + if number!=6: + self.assertEqual(oracle['observation']['failureUsage' if number in (5,9,10) else 'completedUsage'],records[-1]['usageObservation']) + self.assertEqual(oracle['observation']['modelIdentity'],records[-1]['modelIdentity']) + else: + self.assertEqual('sdk-unrecognized-secret-sentinel',records[-1]['usageObservation']['unknown']) + if number == 5: + self.assertIn(b'"maxTurns":20.0', result.stdout) + self.assertIn(b'"maxOutputTokens":12000e0', result.stdout) + self.assertIn(b'"maxChildren":8.0', result.stdout) + self.assertIn(b'"maxChildDepth":1e0', result.stdout) + if number == 6: + self.assertIn(b'"completedResponseCount":3e0', result.stdout) + self.assertIn(b'"duplicateResponseCallbackCount":-0.0', result.stdout) + self.assertIn(b'"input_tokens_details.cached_tokens":-0.0', result.stdout) + if number in (9,10): + key = 'maxChildren' if number == 9 else 'maxChildDepth' + self.assertEqual(9 if number == 9 else 2, records[-1]['limits'][key]) + other = 'maxChildDepth' if number == 9 else 'maxChildren' + self.assertEqual(oracle['nativeLimits'][other], records[-1]['limits'][other]) + self.assertTrue((workspace/'.sdk-harness-started').is_file()) + if number not in (5,9,10): + terminal=json.loads(records[-1]['output']) + self.assertEqual('OPENPROSE_SENTINEL_TERMINAL_V1',terminal['marker']) + + def test_sdk_malformed_groups_must_be_absent_without_relaxing_usage_checks(self): + case=json.loads((runner.CASES/'adapters/sdk-production-06.json').read_text()) + with tempfile.TemporaryDirectory() as temporary: + root=Path(temporary);environment,workspace=runner.product_roots(root,'fixture');runner.isolate_workspace(workspace) + _,fixture,_=runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,workspace,environment) + observation=runner.Observation(runner.Product('fixture',Path('/usr/bin/true')),case,0,b'',b'',workspace=workspace,sdk_fixture=fixture) + for parsed in ({'modelIdentity':{}},{'error':{'details':{'modelIdentity':{}}}}): + observation.parsed=parsed + self.assertIn('SDK malformed observation group was retained: modelIdentity',runner.validate_sdk_effects(observation)) + for parsed in ({'error':None},{'error':'malformed'},{}): + observation.parsed=parsed + self.assertEqual([],runner.validate_sdk_effects(observation)) + observation.stdout=b'sdk-unrecognized-secret-sentinel' + self.assertEqual(['SDK output exposed a forbidden raw-observation sentinel'],runner.validate_sdk_effects(observation)) + + def test_sdk_invalid_child_limits_require_nested_absence_even_if_null(self): + for number in (9,10): + case=json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: + root=Path(temporary);environment,workspace=runner.product_roots(root,'fixture');runner.isolate_workspace(workspace) + _,fixture,_=runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,workspace,environment) + observation=runner.Observation(runner.Product('fixture',Path('/usr/bin/true')),case,22,b'',b'',workspace=workspace,sdk_fixture=fixture) + for retained in (None, {}, {'maxChildren':9}): + observation.parsed={'error':{'details':{'nativeFailure':{'kind':'execution','limits':retained}}}} + self.assertIn('SDK malformed result field was retained: $.error.details.nativeFailure.limits',runner.validate_sdk_effects(observation)) + observation.parsed={'error':{'details':{'nativeFailure':{'kind':'execution'}}}} + self.assertEqual([],runner.validate_sdk_effects(observation)) + self.assertEqual([],runner.deep_subset(observation.parsed,{'error':{'details':{'nativeFailure':{'kind':'execution'}}}})) + self.assertTrue(runner.deep_subset(observation.parsed,{'error':{'details':{'nativeFailure':{'kind':'timeout'}}}})) + + def test_sdk_setup_errors_and_pure_operations_have_independent_marker_guards(self): + for number,marker,message in ((1,'.sdk-harness-probed','probed'),(2,'.sdk-harness-started','started'),(7,'.sdk-harness-started','started')): + case=json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: + root=Path(temporary);environment,workspace=runner.product_roots(root,'fixture');runner.isolate_workspace(workspace) + _,fixture,_=runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,workspace,environment) + observation=runner.Observation(runner.Product('fixture',Path('/usr/bin/true')),case,0,b'',b'',workspace=workspace,sdk_fixture=fixture) + self.assertEqual([],runner.validate_sdk_effects(observation)) + (workspace/marker).touch() + self.assertTrue(any(message in failure for failure in runner.validate_sdk_effects(observation))) + Path(fixture['nativePath']).write_bytes(b'changed candidate') + self.assertIn('SDK relocated candidate bytes changed during execution',runner.validate_sdk_effects(observation)) def test_configuration_corpus_prepares_each_product_independently_and_checks_effects(self): for identifier in runner.CONFIGURATION_CASE_IDS: @@ -659,6 +774,34 @@ def test_exact_json_fixture_freezes_order_redaction_and_workspace(self) -> None: any("exact JSON fixture" in failure for failure in failures) ) + def test_bare_runner_error_checks_exact_code_and_action(self) -> None: + oracle = json.loads((runner.CLI / "shared/fixtures/adapters/sdk-production.json").read_text()) + action = oracle["credentialAbsence"]["action"] + error = { + "schema": "openprose.runner-error/1", "code": "HARNESS_NEEDS_AUTH", + "boundary": "authentication", "exitCode": 10, "retryable": False, + "message": "The Agents SDK requires OPENAI_API_KEY.", "action": action, + "details": {"authProfile": "openai-api-key"}, + } + observation = runner.Observation( + runner.Product("fixture", Path(sys.executable)), + {"expected": { + "exitCode": 10, + "stdout": {"kind": "json", "schema": "openprose.runner-error/1"}, + "stderr": {"kind": "empty"}, + "errorCode": "HARNESS_NEEDS_AUTH", "errorAction": action, + }}, 10, json.dumps(error).encode(), b"", + ) + contracts = runner.ContractRegistry() + self.assertEqual([], runner.validate_output(observation, contracts)) + error["code"] = "HARNESS_FAILED" + observation.stdout = json.dumps(error).encode() + self.assertTrue(any("HARNESS_NEEDS_AUTH" in failure for failure in runner.validate_output(observation, contracts))) + error["code"] = "HARNESS_NEEDS_AUTH" + error["action"] = "wrong" + observation.stdout = json.dumps(error).encode() + self.assertTrue(any(action in failure for failure in runner.validate_output(observation, contracts))) + def test_dry_run_blocking_error_is_the_error_validation_surface(self) -> None: fixture = runner.load_exact_result_fixture( "cli/shared/fixtures/dx/dry-run-default-hosted.json" diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index bcdd153a..5aba7563 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -964,3 +964,41 @@ IMP-097 Rust measured fixture reconciliation: `/root/sdk_final_review` owns `cli IMP-098 predictable provider tier wave: `/root/sdk_runtime` reclaims exact `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` for explicitly requested standard Responses service tier and safe requested/observed tier diagnostics, with pinned SDK tests. This supersedes earlier completed runtime repair leases. No new budget/currency settings, schema/product/Git edits or paid calls. Root freezes outer observation contract after this source change. IMP-097 actual differential repair wave: `/root/conformance_plumbing` owns `cli/conformance/cases/operations/harness-use-user.json`, `harness-use-prime-bundle.json`, `harness-use-omp-bundle.json`, `config-production-03.json`, `config-production-04.json`, `config-production-05.json`, `config-production-09.json`, `config-production-10.json` in that same directory, plus `cli/conformance/runner/run.py` and `test_runner.py`. Scope canonical path controls, complete frozen expected source/diagnostic array objects and truthful isolated Git discovery boundary before filesystem baselines. Do not relax normalization/deep comparisons; preserve within-workspace ancestor discovery. + +IMP-098 shared default and observation contract wave: `/root/conformance_plumbing` owns `docs/packaged-sdk-design.md`, `cli/shared/schemas/native-limits.schema.json`, `cli/shared/schemas/runner-result.schema.json`, `cli/shared/schemas/runner-error.schema.json`, new `cli/shared/schemas/sdk-observation.schema.json`, `cli/shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json`, `cli/shared/fixtures/adapters/scenarios/agents-sdk-jsonl.v1.json`, `cli/shared/tests/test_contracts.py`, new `cli/shared/fixtures/adapters/sdk-production.json` and new `cli/conformance/cases/adapters/sdk-production-01.json` through `sdk-production-08.json`. Freeze public observable defaults, sibling discovery/relocation, actionable credential absence and safe optional observations before independent product implementation. No product/runtime/Git/paid calls. Request runner or further fixture leases before edits; do not weaken existing exact arrays or assertions. + +IMP-098 signing workflow integration wave: `/root/operational_design` owns `.github/workflows/cli-distribution-check.yml`, `.github/workflows/cli-publish.yml`, `cli/ci/check_workflows.py`, `cli/ci/test_check_workflows.py`. Root releases those paths. Integrate packaged helper native building/signing/retained identity with existing custody validators; preserve historical alpha and closed input checks. No dispatch, credentials, paid calls, Git or product/shared contract edits. + +IMP-098 SDK black-box plumbing extension: `/root/conformance_plumbing` owns `cli/conformance/cases/case-manifest.schema.json`, `cli/conformance/runner/run.py`, `cli/conformance/runner/test_runner.py`, `cli/conformance/adversarial/adapter-products/fake_live_harness.py` for closed SDK sibling/relocation fixtures and safe raw success/failure observation controls. Preserve credential-free hermetic isolation and exact existing comparisons; no product changes. + +IMP-098 independent Rust production integration: root owns `cli/rust/crates/prose-runner-core/src/config.rs`, `cli/rust/crates/prose-runner-core/src/runner.rs`, `cli/rust/crates/prose-runner-core/src/installed_adapters.rs`, `cli/rust/crates/prose-cli/src/main.rs`, `cli/rust/crates/prose-runner-core/src/lib.rs` and new `cli/rust/crates/prose-runner-core/src/sdk_observation.rs`. Implement only frozen shared SDK default/discovery/observation/limits contract; no Bun source edits. Product-specific test reconciliation must be evidence-backed and scoped separately. No paid calls. + +IMP-098 independent Bun production integration wave: `/root/configuration_contract` owns `cli/bun/src/core/config.ts`, `cli/bun/src/adapters/executable.ts`, `cli/bun/src/cli.ts`, `cli/bun/src/adapters/environment.ts`, `cli/bun/src/adapters/protocols.ts`, `cli/bun/src/adapters/sdk-limits.ts`, new `cli/bun/src/adapters/sdk-observation.ts`, `cli/bun/src/adapters/types.ts`, `cli/bun/src/supervision/types.ts`, `cli/bun/src/core/types.ts`, `cli/bun/src/adapters/recipes.ts`, `cli/bun/src/core/harnesses.ts`, `cli/bun/test/agents-sdk.test.ts`, `cli/bun/test/sdk-native-limits.test.ts`, new `cli/bun/test/sdk-production.test.ts`, `cli/bun/test/config.test.ts`, `cli/bun/test/config-production.test.ts`, `cli/bun/test/adapters-installed.test.ts`. Frozen SDK production shared contract precedes independent implementation; root retains all Rust. Request exact additional test paths after measured failures. No Git/model calls/dependencies/formatting outside lease. + +IMP-098 Bun observation caller extension: `/root/configuration_contract` owns `cli/bun/src/adapters/runner.ts` for retaining the safe protocol observations in installed results across success/failure/cancel and exposing them to CLI normalization. No supervision safety or lifecycle weakening. + +IMP-098 shared budget reconciliation extension: `/root/conformance_plumbing` owns `cli/shared/fixtures/adapters/sdk-native-limits.json` and `docs/sdk-budgets.md` for eleven production limit fields and accurate fresh child aggregation/cancellation documentation. Preserve explicit historical four-field error controls, use measured failures rather than weakening assertions. + +IMP-098 shared default reconciliation extension: `/root/conformance_plumbing` owns `cli/shared/fixtures/config/values-v1.json`, `cli/shared/fixtures/config/production-v2.json`, `cli/shared/fixtures/config/optional-reporting.json`, `cli/conformance/cases/operations/config-production-01.json` solely for measured built-in SDK default changes on this IMP-098 branch. Where an old fixture specifically tests hosted-only or non-SDK behavior, explicitly select its harness rather than weaken the failure. Keep IMP-097 branch's pre-switch oracle unchanged. Additional cases require exact lease after measured comparison. + +IMP-098 periodic usage runtime follow-up: `/root/sdk_runtime` owns `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, `harnesses/agents-sdk/README.md` only for including current safe usage observations on existing start/tool_call/tool_result events, preserving SDK native event types and completed response accounting. This replaces root's released runtime integration lane. Provider-free tests must show completed response observations arrive before an external cancellation; never claim total run billing known or weaken outer supervision cleanup. No new tool/budget parameter or paid calls. + +IMP-098 periodic usage lane integration: no runtime agent could be started within the current tool thread limit. Root reclaims the exact run.py/test_run.py/README.md follow-up lease above, with the same narrow scope and provider-free requirements. + +IMP-098 root IMP-097 admission carry-forward lease: `cli/ci/check_architecture.py`, `cli/ci/test_check_architecture.py`, `cli/rust/crates/prose-cli/tests/cli.rs`, `cli/rust/crates/prose-cli/tests/config_production.rs`, `cli/rust/crates/prose-cli/tests/service_programs.rs` for exact already-committed IMP-097 discard-taint, formatting and feature-scoped fixture repairs on the stacked source. Existing root config/main leases cover their exact behavior-preserving refactor. No unrelated test changes. + +IMP-098 root consumer documentation lease: `README.md`, `cli/README.md`, `docs/hosted-service-client.md`, `docs/user-configuration.md` for SDK built-in selection, only-key setup, saved compatible alternatives and explicit hosted route. Preserve historical evidence and distinguish implementation from qualification; no kernel/program edits. + +IMP-098 Bun measured default reconciliation extension: `/root/configuration_contract` owns `cli/bun/test/cli.test.ts`, `cli/bun/test/reporting-parity.test.ts`, `cli/bun/test/dx-parity.test.ts` for the 17 observed default-switch failures. Explicitly select hosted harness where the test is about hosted behavior, and use the shared frozen SDK expectation for actual default behavior. Preserve real auth/permission/effect assertions; no skips or relaxed comparisons. + +IMP-098 shared reporting default extension: `/root/conformance_plumbing` owns `cli/shared/fixtures/operations/configuration-explanation.json` and `cli/shared/fixtures/operations/doctor-report.json` only for configuration values/sources/runtime reflecting the frozen SDK built-in. Preserve independent selected mock/native report fields and default-omission summary. Root retains product implementations. + +IMP-098 root schema registration extension: `cli/bun/test/shared-contract.test.ts` solely to register sdk-observation.schema.json before the existing referenced runner-error/result schemas in strict AJV. No relaxed schema validation. + +IMP-098 root measured shared Bun test reconciliation: `cli/bun/test/shared-contract.test.ts` hosted-unavailable case explicitly selects --harness openprose, preserving its exact hosted error, schema, suggested command and no-invocation checks after the built-in SDK switch. The broader source suite's single remaining failure establishes this scope. + +## IMP-098 numeric acceptance follow-up + +Lead leases `cli/shared/fixtures/adapters/sdk-production.json`, `cli/conformance/cases/adapters/sdk-production-05.json`, `cli/conformance/cases/adapters/sdk-production-06.json`, `cli/conformance/cases/adapters/sdk-production-09.json`, `cli/conformance/cases/adapters/sdk-production-10.json`, `cli/conformance/runner/run.py`, and `cli/conformance/runner/test_runner.py` to `conformance_plumbing` for integral JSON numeric representations and independently invalid child/depth omission controls. Shared schema constants remain authoritative. No product writes, builds or Git; lead owns integration. + +`conformance_plumbing` additionally owns `cli/conformance/adversarial/adapter-products/fake_live_harness.py` and `cli/shared/tests/test_contracts.py` for the approved numeric SDK controls. `config_assessment` owns `cli/rust/crates/prose-cli/tests/cli.rs` for the eight obsolete default/explicit hosted expectation failures observed in sdk-rust-workspace-first.log; preserve exact receipt assertions and 097 byte order. Agents do not build or use Git; root controls serial admission. diff --git a/cli/rust/crates/prose-cli/src/main.rs b/cli/rust/crates/prose-cli/src/main.rs index 841cec9c..405780e3 100644 --- a/cli/rust/crates/prose-cli/src/main.rs +++ b/cli/rust/crates/prose-cli/src/main.rs @@ -9,7 +9,7 @@ use prose_runner_core::runner::{ use prose_runner_core::service::ServiceCommand; use prose_runner_core::{ CancellationToken, OutputMode, SignalCancellationGuard, SystemClock, SystemContext, - SystemIdSource, parse_invocation, resolve_config, + SystemIdSource, parse_invocation, }; use std::collections::BTreeSet; use std::io; @@ -459,58 +459,19 @@ fn account_command_outcome( })) } -fn prepare( - args: &[String], - cancellation: &CancellationToken, - human_stdout: &mut dyn io::Write, -) -> CommandOutcome { +fn prepare_configuration( + parsed: &ParsedInvocation, + system: &SystemContext, +) -> Result { let clock = SystemClock; let ids = SystemIdSource; - let parsed = match parse(args) { - Ok(parsed) => parsed, - Err(outcome) => return outcome, - }; - - if let Some(outcome) = identity_outcome(&parsed) { - return outcome; - } - - let system = match SystemContext::capture() { - Ok(system) => system, - Err(error) => { - return error_outcome( - error, - parsed.globals.output.unwrap_or_default(), - &clock, - &ids, - ); - } - }; - if let Action::Runner { - command: RunnerCommand::Service(ref service), - .. - } = parsed.action - { - let mut stderr = io::stderr(); - return prose_runner_core::service::execute( - service, - &parsed.globals, - &system, - cancellation, - human_stdout, - &mut stderr, - ); - } - if let Some(outcome) = account_command_outcome(&parsed, args, &system, cancellation) { - return outcome; - } let mutation = if let Action::Runner { command, json } = &parsed.action { let result = match command { RunnerCommand::ConfigMigrate => Some( - prose_runner_core::config::migrate_user_configuration(&system), + prose_runner_core::config::migrate_user_configuration(system), ), RunnerCommand::ConfigUnset(keys) => Some( - prose_runner_core::config::unset_user_configuration(&system, keys), + prose_runner_core::config::unset_user_configuration(system, keys), ), _ => None, }; @@ -520,9 +481,9 @@ fn prepare( let error = prose_runner_core::config::mutation_error_context( error, &parsed.globals, - &system, + system, ); - return error_outcome( + return Err(error_outcome( error, if *json { OutputMode::Json @@ -531,7 +492,7 @@ fn prepare( }, &clock, &ids, - ); + )); } None => None, } @@ -550,7 +511,21 @@ fn prepare( resolution_flags.model = None; resolution_flags.auth_profile = None; } - let mut config = match resolve_config(&resolution_flags, &system) { + let mut config = match prose_runner_core::config::resolve_config_with_selection_validation( + &resolution_flags, + system, + || { + if let Action::Runner { + command: RunnerCommand::HarnessUse(harness), + .. + } = &parsed.action + { + prose_runner_core::runner::validate_harness_selection(harness, &parsed.globals) + } else { + Ok(()) + } + }, + ) { Ok(config) => config, Err(error) => { let mut error = error; @@ -574,7 +549,7 @@ fn prepare( } else { error }; - return error_outcome(error, mode, &clock, &ids); + return Err(error_outcome(error, mode, &clock, &ids)); } }; config.mutation = mutation; @@ -585,6 +560,58 @@ fn prepare( { config.explanation_target = Some(argv.clone()); } + Ok(config) +} + +fn prepare( + args: &[String], + cancellation: &CancellationToken, + human_stdout: &mut dyn io::Write, +) -> CommandOutcome { + let clock = SystemClock; + let ids = SystemIdSource; + let parsed = match parse(args) { + Ok(parsed) => parsed, + Err(outcome) => return outcome, + }; + + if let Some(outcome) = identity_outcome(&parsed) { + return outcome; + } + + let system = match SystemContext::capture() { + Ok(system) => system, + Err(error) => { + return error_outcome( + error, + parsed.globals.output.unwrap_or_default(), + &clock, + &ids, + ); + } + }; + if let Action::Runner { + command: RunnerCommand::Service(ref service), + .. + } = parsed.action + { + let mut stderr = io::stderr(); + return prose_runner_core::service::execute( + service, + &parsed.globals, + &system, + cancellation, + human_stdout, + &mut stderr, + ); + } + if let Some(outcome) = account_command_outcome(&parsed, args, &system, cancellation) { + return outcome; + } + let config = match prepare_configuration(&parsed, &system) { + Ok(config) => config, + Err(outcome) => return outcome, + }; // The default hosted harness runs no language command, so a language // command word that also names a service command is that rejection. if let Action::Forward { diff --git a/cli/rust/crates/prose-cli/tests/cli.rs b/cli/rust/crates/prose-cli/tests/cli.rs index 739454c5..82728129 100644 --- a/cli/rust/crates/prose-cli/tests/cli.rs +++ b/cli/rust/crates/prose-cli/tests/cli.rs @@ -1,6 +1,6 @@ #[cfg(feature = "test-seams")] use prose_runner_core::image::sha256_hex; -use serde_json::{json, Value}; +use serde_json::{Value, json}; #[cfg(any(feature = "test-seams", not(debug_assertions)))] use std::ffi::OsString; use std::fmt::Write as _; @@ -65,8 +65,8 @@ impl PublishedFixtureCleanup { fn cleanup_published_group(&self) { use rustix::process::{ - getpgid, getpgrp, kill_process_group, test_kill_process, test_kill_process_group, - Signal, + Signal, getpgid, getpgrp, kill_process_group, test_kill_process, + test_kill_process_group, }; let Some(observed) = validated_fixture_identities(&self.identities_path) else { @@ -428,6 +428,7 @@ fn assert_release_mock_rpc_is_rejected(binary: &Path, root: &Path) { } } +#[cfg(feature = "test-seams")] fn fake_harness() -> PathBuf { Path::new(env!("CARGO_MANIFEST_DIR")) .join("../../../conformance/fake-harness/fake_harness.py") @@ -554,7 +555,7 @@ os._exit(0) #[cfg(unix)] fn kill_escaped_identity(identity: &Path) { - use rustix::process::{kill_process, Pid, Signal}; + use rustix::process::{Pid, Signal, kill_process}; let Ok(bytes) = fs::read(identity) else { return; @@ -1192,7 +1193,7 @@ fn operation_fixture(name: &str) -> Value { fn harness_supported(harness: &str) -> bool { use prose_runner_core::installed_adapters::{ - assert_platform_supported, for_harness, HostPlatform, + HostPlatform, assert_platform_supported, for_harness, }; assert_platform_supported(for_harness(harness).unwrap(), HostPlatform::current()).is_ok() } @@ -1296,10 +1297,12 @@ fn account_verb_help_prints_its_manifest_topic_and_starts_nothing() { } } // Help persisted nothing. - assert!(fs::read_dir(temp.path().join("home").join("xdg")) - .unwrap() - .next() - .is_none()); + assert!( + fs::read_dir(temp.path().join("home").join("xdg")) + .unwrap() + .next() + .is_none() + ); } #[test] @@ -1370,11 +1373,18 @@ fn prime_cleanup_uses_the_invoked_binary_and_never_an_ambient_path_prose() { } #[test] -fn default_is_openprose_billed_and_never_falls_back() { +fn explicitly_selected_openprose_billed_and_never_falls_back() { let temp = TempDir::new().unwrap(); let output = prose( temp.path(), - &["--output", "json", "run", "fixture.prose.md"], + &[ + "--harness", + "openprose", + "--output", + "json", + "run", + "fixture.prose.md", + ], ); assert_eq!(output.status.code(), Some(10)); let result = json_stdout(&output); @@ -1561,6 +1571,8 @@ fn doctor_rejects_an_installed_transport_before_inventory_or_version_probes() { fn hosted_transport_mismatch_precedes_availability_for_doctor_and_run() { for args in [ vec![ + "--harness", + "openprose", "--transport", "unsupported", "--output", @@ -1569,6 +1581,8 @@ fn hosted_transport_mismatch_precedes_availability_for_doctor_and_run() { "doctor", ], vec![ + "--harness", + "openprose", "--transport", "unsupported", "--output", @@ -1634,9 +1648,11 @@ fn output_parse_errors_retain_only_an_earlier_recognized_machine_channel() { let human_output = prose(temp.path(), &["--output=invalid", "cli", "doctor"]); assert_eq!(human_output.status.code(), Some(2)); assert!(human_output.stdout.is_empty()); - assert!(String::from_utf8(human_output.stderr) - .unwrap() - .contains("INVOCATION_INVALID at invocation")); + assert!( + String::from_utf8(human_output.stderr) + .unwrap() + .contains("INVOCATION_INVALID at invocation") + ); let opaque = prose( temp.path(), @@ -1879,8 +1895,8 @@ fn provider_free_installed_adapters_preserve_bytes_and_settle_the_echo_placehold #[cfg(all(feature = "test-seams", unix))] #[test] -fn private_file_cleanup_failure_dominates_installed_success_and_child_failure_without_leaking_authority( -) { +fn private_file_cleanup_failure_dominates_installed_success_and_child_failure_without_leaking_authority() + { use std::os::unix::fs::PermissionsExt as _; for (harness, transport, adapter_id) in [ @@ -2289,9 +2305,11 @@ fn prime_parser_failures_expose_only_the_closed_framing_diagnostic() { ); assert!(!serialized.contains("candidateSecret"), "{fault}"); assert!(result["error"]["details"].get("reason").is_none()); - assert!(result["error"]["details"] - .get("admittedRecordCount") - .is_none()); + assert!( + result["error"]["details"] + .get("admittedRecordCount") + .is_none() + ); } } @@ -2396,10 +2414,12 @@ fn omp_runtime_prerequisite_blocks_before_omp_auth_and_never_falls_through_path( assert!(!rendered_error.contains(forbidden)); } assert_eq!(fs::read_to_string(&bun_observation).unwrap(), ""); - assert!(!temp - .path() - .join("omp-version-probe-observation.json") - .exists()); + assert!( + !temp + .path() + .join("omp-version-probe-observation.json") + .exists() + ); let mut doctor = base.to_vec(); doctor.extend(["--output", "json", "cli", "doctor"]); @@ -2433,10 +2453,12 @@ fn omp_runtime_prerequisite_blocks_before_omp_auth_and_never_falls_through_path( assert!(text.contains(line), "missing {line:?}: {text}"); } assert!(!text.contains(temp.path().to_str().unwrap())); - assert!(!temp - .path() - .join("omp-version-probe-observation.json") - .exists()); + assert!( + !temp + .path() + .join("omp-version-probe-observation.json") + .exists() + ); fs::remove_file(path[1].join("bun")).unwrap(); let output = prose_with_live_adapter_path( @@ -2455,10 +2477,11 @@ fn omp_runtime_prerequisite_blocks_before_omp_auth_and_never_falls_through_path( assert_eq!(omp["availability"], "available"); assert_eq!(omp["detectedVersion"], "omp/18.0.9"); assert_eq!(omp["runtimePrerequisites"][0]["detectedVersion"], "1.3.14"); - assert!(temp - .path() - .join("omp-version-probe-observation.json") - .exists()); + assert!( + temp.path() + .join("omp-version-probe-observation.json") + .exists() + ); } #[cfg(unix)] @@ -2763,8 +2786,7 @@ fn successful_wrong_stream_versions_report_exact_safe_repair_details() { executable_name: "prime-agent", version: "prime-agent 0.7.0", admitted: json!(["0.7.0", "0.8.1"]), - repair: - "curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh -s -- 0.8.1", + repair: "curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh -s -- 0.8.1", auth_profile: Some("prime-harness-login"), }, WrongStreamVersionCase { @@ -2845,9 +2867,11 @@ fn prime_and_omp_harness_login_routes_require_explicit_profiles_and_qualified_mo let report = json_stdout(&missing_profile); assert_eq!(report["problems"][0]["code"], "CONFIG_INVALID"); assert_eq!(report["problems"][0]["details"]["adapterId"], adapter_id); - assert!(serde_json::to_string(&report) - .unwrap() - .contains(auth_profile)); + assert!( + serde_json::to_string(&report) + .unwrap() + .contains(auth_profile) + ); for invalid in [ "unqualified", @@ -2882,9 +2906,11 @@ fn prime_and_omp_harness_login_routes_require_explicit_profiles_and_qualified_mo let report = json_stdout(&output); assert_eq!(report["code"], "CONFIG_INVALID"); assert_eq!(report["boundary"], "configuration"); - assert!(serde_json::to_string(&report) - .unwrap() - .contains("provider/model")); + assert!( + serde_json::to_string(&report) + .unwrap() + .contains("provider/model") + ); } } } @@ -3329,9 +3355,11 @@ fn fake_process_protocol_failures_and_stderr_keep_stream_contracts() { assert_eq!(result["terminal"]["exitCode"], 17); assert_eq!(result["terminal"]["signal"], Value::Null); assert_eq!(result["semantic"]["status"], "unknown"); - assert!(result["semantic"]["terminalEnvelopeDigestSha256"] - .as_str() - .is_some()); + assert!( + result["semantic"]["terminalEnvelopeDigestSha256"] + .as_str() + .is_some() + ); assert_eq!( result["error"]["details"], json!({ @@ -3501,7 +3529,7 @@ fn fake_process_cancellation_reports_native_signal_and_cleans_descendants() { #[cfg(all(feature = "test-seams", unix))] #[test] fn actual_sigint_emits_one_cancelled_terminal_and_cleans_controlled_in_group_descendants() { - use rustix::process::{kill_process, test_kill_process, Pid, Signal}; + use rustix::process::{Pid, Signal, kill_process, test_kill_process}; for output_mode in ["json", "jsonl"] { let temp = TempDir::new().unwrap(); @@ -3913,7 +3941,7 @@ fn harness_use_persists_an_explicit_prime_bundle_from_suffix_or_prefix_without_s let path = home.join(".prose/cli.toml"); assert_eq!( fs::read_to_string(&path).unwrap(), - "harness = \"prime\"\nmodel = \"openai/gpt-5.4\"\nauth_profile = \"prime-harness-login\"\n" + "auth_profile = \"prime-harness-login\"\nharness = \"prime\"\nmodel = \"openai/gpt-5.4\"\n" ); assert!(!observation.exists()); @@ -4002,7 +4030,9 @@ fn missing_prime_selection_options_have_exact_human_invocation_repair() { assert_eq!(output.status.code(), Some(2)); assert!(output.stdout.is_empty()); let stderr = String::from_utf8(output.stderr).unwrap(); - assert!(stderr.starts_with("INVOCATION_INVALID at invocation: Runner invocation is invalid.\n")); + assert!( + stderr.starts_with("INVOCATION_INVALID at invocation: Runner invocation is invalid.\n") + ); assert!(stderr.contains( "Detail: Prime and OMP selection requires explicit CLI --model and --auth-profile options; inherited configuration does not select a credential route.\n" )); @@ -4106,7 +4136,7 @@ fn codex_claude_and_openprose_switches_clear_stale_bundle_values() { assert!(codex.status.success()); assert_eq!( fs::read_to_string(&path).unwrap(), - "timeout = \"30s\"\nharness = \"codex\"\nmodel = \"gpt-5.4\"\nauth_profile = \"cached-chatgpt-login\"\n" + "timeout = \"30s\"\nauth_profile = \"cached-chatgpt-login\"\nharness = \"codex\"\nmodel = \"gpt-5.4\"\n" ); let openprose = prose( @@ -4270,21 +4300,31 @@ fn dry_run_is_machine_readable_and_starts_no_run() { assert_eq!(report["configuration"][0]["redacted"], false); assert_eq!(report["configuration"][8]["redacted"], false); assert_eq!(report["configuration"][8]["value"], Value::Null); - assert!(report["configuration"] - .as_array() - .unwrap() - .iter() - .take(8) - .all(|entry| entry["redacted"] == false)); + assert!( + report["configuration"] + .as_array() + .unwrap() + .iter() + .take(8) + .all(|entry| entry["redacted"] == false) + ); } } #[test] -fn default_dry_run_identifies_the_hosted_adapter_without_fallback() { +fn explicit_hosted_dry_run_identifies_the_hosted_adapter_without_fallback() { let temp = TempDir::new().unwrap(); let output = prose( temp.path(), - &["--dry-run", "--output", "json", "write", "fixture.prose.md"], + &[ + "--harness", + "openprose", + "--dry-run", + "--output", + "json", + "write", + "fixture.prose.md", + ], ); assert_eq!(output.status.code(), Some(10)); let report = json_stdout(&output); @@ -4761,8 +4801,15 @@ fn runner_diagnostics_and_identity_stay_local() { assert_eq!(doctor.status.code(), Some(10)); let doctor_report = json_stdout(&doctor); assert_eq!(doctor_report["schema"], "openprose.doctor-report/1"); - assert_eq!(doctor_report["selectedHarness"], "openprose"); + assert_eq!(doctor_report["selectedHarness"], "agents-sdk"); assert_eq!(doctor_report["ready"], false); + assert_eq!(doctor_report["selectedAdapterId"], "agents-sdk/jsonl"); + assert_eq!(doctor_report["billingOwner"], "user-provider"); + assert_eq!(doctor_report["problems"][0]["code"], "HARNESS_UNAVAILABLE"); + assert_eq!( + doctor_report["problems"][0]["details"]["fallbackAttempted"], + false + ); let list = prose(temp.path(), &["cli", "harness", "list", "--json"]); assert!(list.status.success()); @@ -4874,6 +4921,7 @@ fn harness_list_matches_the_closed_ordered_inventory() { assert!(output.status.success()); let mut expected = operation_fixture("harnesses"); expected["harnesses"] = expected_harness_inventory(); + expected["selected"] = json!("agents-sdk"); assert_eq!(json_stdout(&output), expected); } @@ -4884,9 +4932,15 @@ fn human_harness_list_marks_the_default_and_gives_ordered_next_actions() { assert!(output.status.success()); assert!(output.stderr.is_empty()); let stdout = String::from_utf8(output.stdout).unwrap(); - assert!( - stdout.contains("* openprose availability=not-implemented transport=hosted (selected)\n") - ); + assert!(stdout.contains(" openprose availability=not-implemented transport=hosted\n")); + let sdk_availability = if harness_supported("agents-sdk") { + "missing" + } else { + "incompatible" + }; + assert!(stdout.contains(&format!( + "* agents-sdk availability={sdk_availability} transport=jsonl (selected)\n" + ))); for (harness, transport) in [ ("prime", "rpc"), ("omp", "rpc"), @@ -4992,7 +5046,7 @@ fn assert_guarded_model_choice( assert_eq!( fs::read_to_string(valid_root.join("home/.prose/cli.toml")).unwrap(), format!( - "harness = \"{harness}\"\nmodel = \"openai/gpt-5.4\"\nauth_profile = \"{auth_profile}\"\n" + "auth_profile = \"{auth_profile}\"\nharness = \"{harness}\"\nmodel = \"openai/gpt-5.4\"\n" ) ); } @@ -5033,11 +5087,13 @@ fn exact_runner_inventory_guidance_never_resolves_a_hostile_path_prose() { for line in stdout.lines().filter(|line| line.starts_with("Choose ")) { let (_, command) = line.rsplit_once(": ").expect("choice has a command"); assert!(!command.contains(['<', '>', '|'])); - assert!(Command::new("/bin/sh") - .args(["-n", "-c", command]) - .status() - .unwrap() - .success()); + assert!( + Command::new("/bin/sh") + .args(["-n", "-c", command]) + .status() + .unwrap() + .success() + ); } for (label, harness, profile) in [ ("Prime", "prime", "prime-harness-login"), @@ -5077,11 +5133,24 @@ fn human_harness_list_includes_a_safely_detected_compatible_version() { #[test] fn doctor_matches_the_closed_full_report_and_selected_problem_exit() { let temp = TempDir::new().unwrap(); - let output = prose(temp.path(), &["--output", "json", "cli", "doctor"]); + let output = prose( + temp.path(), + &[ + "--harness", + "openprose", + "--output", + "json", + "cli", + "doctor", + ], + ); assert_eq!(output.status.code(), Some(10)); let mut expected = operation_fixture("doctor"); expected["cwd"] = Value::String(fs::canonicalize(temp.path()).unwrap().display().to_string()); - expected["configuration"] = expected_configuration(temp.path()); + let configuration_paths = expected_configuration(temp.path()); + for field in ["cwd", "userConfigPath", "locations"] { + expected["configuration"][field] = configuration_paths[field].clone(); + } expected["runner"] = json!({ "name": "rust", "version": env!("CARGO_PKG_VERSION"), @@ -5424,7 +5493,10 @@ fn dev_endpoint_builds_name_the_invoked_executable_in_copyable_commands() { #[test] fn human_failures_keep_result_stdout_clean() { let temp = TempDir::new().unwrap(); - let output = prose(temp.path(), &["run", "fixture.prose.md"]); + let output = prose( + temp.path(), + &["--harness", "openprose", "run", "fixture.prose.md"], + ); assert_eq!(output.status.code(), Some(10)); assert!(output.stdout.is_empty()); let diagnostic = String::from_utf8(output.stderr).unwrap(); @@ -5552,7 +5624,7 @@ fn human_doctor_confines_a_hostile_diagnostic_to_one_physical_detail_line() { let rendered = String::from_utf8(human.stderr).unwrap(); assert!(rendered.contains(&format!( "Detail: {}\n", - expected_human_safe_scalar(&expected_reason) + expected_human_safe_scalar(expected_reason) ))); assert_eq!( rendered diff --git a/cli/rust/crates/prose-cli/tests/config_production.rs b/cli/rust/crates/prose-cli/tests/config_production.rs index aefe4131..c1cfce56 100644 --- a/cli/rust/crates/prose-cli/tests/config_production.rs +++ b/cli/rust/crates/prose-cli/tests/config_production.rs @@ -390,10 +390,12 @@ fn selected_file_candidates_identify_the_assignment_line() { .iter() .find(|candidate| candidate["selected"] == true) .unwrap(); - assert!(chosen["source"]["location"] - .as_str() - .unwrap() - .ends_with("cli.toml:2")); + assert!( + chosen["source"]["location"] + .as_str() + .unwrap() + .ends_with("cli.toml:2") + ); assert_eq!(chosen["value"], "2m"); } @@ -465,11 +467,13 @@ fn invalid_ignored_legacy_root_has_safe_diagnostic_without_blocking_canonical() .unwrap(); let output = report(&raw, 0); assert_eq!(output["values"]["timeout"]["value"], "2m"); - assert!(output["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["source"] == "XDG_CONFIG_HOME")); + assert!( + output["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["source"] == "XDG_CONFIG_HOME") + ); assert!(!String::from_utf8_lossy(&raw.stdout).contains("relative-legacy-root-sentinel")); } @@ -499,21 +503,25 @@ fn readonly_physical_alias_loads_once_but_mutations_refuse_symlink() { .count(), 1 ); - assert!(!output["diagnostics"] - .as_array() - .unwrap() - .iter() - .any(|diagnostic| diagnostic["code"] == "LEGACY_CONFIG_IGNORED")); + assert!( + !output["diagnostics"] + .as_array() + .unwrap() + .iter() + .any(|diagnostic| diagnostic["code"] == "LEGACY_CONFIG_IGNORED") + ); for args in [ vec!["cli", "config", "migrate", "--json"], vec!["cli", "config", "unset", "timeout", "--json"], ] { let failed = report(&fixture.invoke(&args, None), 2); assert_eq!(failed["code"], "CONFIG_INVALID"); - assert!(fs::symlink_metadata(fixture.path("home/.prose/cli.toml")) - .unwrap() - .file_type() - .is_symlink()); + assert!( + fs::symlink_metadata(fixture.path("home/.prose/cli.toml")) + .unwrap() + .file_type() + .is_symlink() + ); assert_eq!( fs::read(fixture.path("legacy/openprose/cli.toml")).unwrap(), original diff --git a/cli/rust/crates/prose-cli/tests/service_programs.rs b/cli/rust/crates/prose-cli/tests/service_programs.rs index 855aa13a..0f7a59a5 100644 --- a/cli/rust/crates/prose-cli/tests/service_programs.rs +++ b/cli/rust/crates/prose-cli/tests/service_programs.rs @@ -2,7 +2,9 @@ //! express compactly. Behavior is pinned by //! `cli/conformance/cases/service/programs/`; the Bun twin is //! `cli/bun/test/service-programs.test.ts`. -use serde_json::{Value, json}; +use serde_json::Value; +#[cfg(feature = "test-seams")] +use serde_json::json; use std::process::Command; use tempfile::TempDir; diff --git a/cli/rust/crates/prose-runner-core/src/config.rs b/cli/rust/crates/prose-runner-core/src/config.rs index 7eaacb9f..e298dcd9 100644 --- a/cli/rust/crates/prose-runner-core/src/config.rs +++ b/cli/rust/crates/prose-runner-core/src/config.rs @@ -4,6 +4,7 @@ use serde::Serialize; use serde_json::{Value, json}; use std::collections::BTreeMap; use std::env; +use std::fmt::Write as _; use std::fs; use std::fs::OpenOptions; use std::io::Write as _; @@ -113,6 +114,10 @@ impl SystemContext { } /// Historical discovery is retained only as a visible migration candidate. + /// + /// # Errors + /// Returns `CONFIG_INVALID` for an explicit canonical root or an invalid + /// or absent historical configuration root. pub fn legacy_user_config_path(&self) -> Result { if self.environment.contains_key("PROSE_CONFIG_DIR") { return Err(RunnerError::config( @@ -289,14 +294,14 @@ pub fn write_user_harness( .ok_or_else(|| RunnerError::config("user configuration has no parent directory"))?; prepare_private_config_parent(parent)?; refuse_symlinked_config_destination(&path)?; - let source_path = if !path.exists() { + let source_path = if path.exists() { + &path + } else { config .legacy_user_config .as_ref() .filter(|legacy| legacy.is_file()) .unwrap_or(&path) - } else { - &path }; refuse_symlinked_config_destination(source_path)?; let existing_bytes = match fs::read(source_path) { @@ -375,7 +380,7 @@ pub fn write_user_harness( } for key in ["auth_profile", "harness", "model"] { if let Some(value) = table.get(key) { - retained.push_str(&format!("{key} = {value}\n")); + writeln!(retained, "{key} = {value}").expect("writing to String cannot fail"); } } let loaded = parse_file(&retained, &path)?; @@ -671,7 +676,7 @@ impl EffectiveConfig { project_config, user_config, harness: Sourced { - value: "openprose".to_owned(), + value: "agents-sdk".to_owned(), source: ConfigSource::default(), }, transport: Sourced { @@ -1034,7 +1039,29 @@ pub fn resolve_config( flags: &GlobalFlags, system: &SystemContext, ) -> Result { - resolve_config_inner(flags, system).map_err(|error| { + resolve_config_with_selection_validation(flags, system, || Ok(())) +} + +/// Parse all configuration sources before validating an explicit saved selection. +/// Ambient harness semantics are checked after this pure validation stage. +/// +/// # Errors +/// Returns configuration parse/semantic failures or the selection validator's error. +pub fn resolve_config_with_selection_validation( + flags: &GlobalFlags, + system: &SystemContext, + validate_selection: impl FnOnce() -> Result<(), RunnerError>, +) -> Result { + let selection_failed = std::cell::Cell::new(false); + resolve_config_inner(flags, system, || { + let result = validate_selection(); + selection_failed.set(result.is_err()); + result + }) + .map_err(|error| { + if selection_failed.get() { + return error; + } if error .details .as_deref() @@ -1056,6 +1083,7 @@ pub fn resolve_config( fn resolve_config_inner( flags: &GlobalFlags, system: &SystemContext, + validate_selection: impl FnOnce() -> Result<(), RunnerError>, ) -> Result { let requested_cwd = flags.cwd.as_ref().map_or_else( || system.current_dir.clone(), @@ -1122,7 +1150,7 @@ fn resolve_config_inner( project_config.clone(), Some(user_config_candidate.clone()), ); - config.legacy_user_config = legacy.clone(); + config.legacy_user_config.clone_from(&legacy); if legacy_result.is_err() && !system.environment.contains_key("PROSE_CONFIG_DIR") && system.xdg_config_home.is_some() @@ -1163,10 +1191,12 @@ fn resolve_config_inner( }) .collect(); if !differing.is_empty() { - reason.push_str(&format!( + write!( + reason, " Differing explicit keys: {}.", differing.join(", ") - )); + ) + .expect("writing to String cannot fail"); } } Err(_) => reason.push_str(" Ignored legacy configuration is invalid."), @@ -1210,6 +1240,7 @@ fn resolve_config_inner( record_candidates(&mut config); apply_flags(&mut config, flags).map_err(|error| explanation_error(error, &config))?; record_candidates(&mut config); + validate_selection()?; contextual_defaults(&mut config).map_err(|error| explanation_error(error, &config))?; native_checks(&mut config).map_err(|error| explanation_error(error, &config))?; sanitize_overridden_profiles(&mut config); @@ -1447,9 +1478,7 @@ fn contextual_defaults(config: &mut EffectiveConfig) -> Result<(), RunnerError> } fn resolved_runtime(config: &EffectiveConfig) -> Value { - let transport = if config.transport.value != "auto" { - config.transport.value.as_str() - } else { + let transport = if config.transport.value == "auto" { crate::installed_adapters::for_harness(&config.harness.value).map_or_else( || { if config.harness.value == "openprose" { @@ -1460,6 +1489,8 @@ fn resolved_runtime(config: &EffectiveConfig) -> Value { }, crate::installed_adapters::InstalledAdapter::transport, ) + } else { + config.transport.value.as_str() }; json!({"transport":transport,"permissionMode":config.permission_mode.value, "authProfile":config.auth_profile.value,"billingOwner":match config.harness.value.as_str() {"openprose" => "openprose", "mock" => "test-fixture", _ => "user-provider"}, @@ -2419,7 +2450,7 @@ mod tests { &context(&child, &parent.join("home")), ) .unwrap(); - assert_eq!(config.harness.value, "openprose"); + assert_eq!(config.harness.value, "agents-sdk"); } #[test] @@ -2432,7 +2463,7 @@ mod tests { home_dir: Some(absolute_home.clone()), xdg_config_home: Some(PathBuf::new()), appdata: None, - environment: BTreeMap::from([("PROSE_CONFIG_DIR".into(), "".into())]), + environment: BTreeMap::from([("PROSE_CONFIG_DIR".into(), String::new())]), platform: Platform::Unix, }, SystemContext { @@ -2962,8 +2993,13 @@ pub(crate) fn native_limits(config: &EffectiveConfig) -> Option, ) -> Option { + if adapter == InstalledAdapter::AgentsSdkJsonl { + let cli = std::env::current_exe().ok()?.canonicalize().ok()?; + let sibling = cli.parent()?.join("prose-agents-sdk"); + let metadata = fs::symlink_metadata(&sibling).ok()?; + return (metadata.is_file() + && !metadata.file_type().is_symlink() + && is_executable(&metadata)) + .then_some(sibling); + } resolve_named_executable(adapter.executable_names(), search_path) } @@ -3454,7 +3463,12 @@ pub fn auth_readiness( } let nonempty = |expected: &str| { ambient.iter().any(|(name, value)| { - environment_name_matches(name, expected) && !value.as_os_str().is_empty() + environment_name_matches(name, expected) + && if adapter == InstalledAdapter::AgentsSdkJsonl { + !value.to_string_lossy().trim().is_empty() + } else { + !value.as_os_str().is_empty() + } }) }; let present = match (adapter, auth_group) { @@ -3472,9 +3486,13 @@ pub fn auth_readiness( if present { Ok("unknown") } else { - Err(RunnerError::catalog(ErrorCode::HarnessNeedsAuth) + let mut error = RunnerError::catalog(ErrorCode::HarnessNeedsAuth) .with_detail("adapterId", adapter.id()) - .with_detail("authProfile", auth_group)) + .with_detail("authProfile", auth_group); + if adapter == InstalledAdapter::AgentsSdkJsonl { + "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.".clone_into(&mut error.action); + } + Err(error) } } @@ -4679,6 +4697,48 @@ mod tests { ); } + #[test] + fn packaged_sdk_admission_rejects_unbuilt_linux_abis() { + for arch in ["aarch64", "x86_64"] { + let sdk = InstalledAdapter::AgentsSdkJsonl; + assert!( + assert_platform_supported( + sdk, + HostPlatform { + os: "macos", + arch, + libc: None + } + ) + .is_ok() + ); + assert!( + assert_platform_supported( + sdk, + HostPlatform { + os: "linux", + arch, + libc: Some("gnu") + } + ) + .is_ok() + ); + for libc in [None, Some("musl")] { + let error = assert_platform_supported( + sdk, + HostPlatform { + os: "linux", + arch, + libc, + }, + ) + .unwrap_err(); + assert_eq!(error.code, ErrorCode::HarnessIncompatible); + assert_eq!(error.details.unwrap()["fallbackAttempted"], false); + } + } + } + #[test] fn rejected_host_diagnostics_use_portable_platform_names() { for (os, arch, wanted_os, wanted_arch) in [ @@ -7482,7 +7542,7 @@ pub(crate) fn sdk_native_failure(record: &Value) -> Value { } if let Some(l) = record .get("limits") - .filter(|l| l.as_object().is_some_and(|m| m.len() == 4)) + .filter(|l| l.as_object().is_some_and(|m| m.len() == 4 || m.len() == 11)) { let ints = ["maxTurns", "maxOutputTokens"].iter().all(|k| { l.get(k).and_then(Value::as_f64).is_some_and(|n| { @@ -7494,7 +7554,26 @@ pub(crate) fn sdk_native_failure(record: &Value) -> Value { .and_then(Value::as_f64) .is_some_and(|n| n.is_finite() && n > 0.0 && n <= 9_007_199_254_740.99) }); - if ints && times { + let expanded = l.as_object().is_some_and(|m| m.len() == 4) + || [ + "maxAggregateRequests", + "maxAggregateHostedWebCalls", + "maxAggregateFunctionTools", + "maxObservedTotalTokens", + "maxRequestInputBytes", + "maxChildren", + "maxChildDepth", + ] + .iter() + .all(|key| { + l.get(key).and_then(Value::as_f64).is_some_and(|n| { + n.is_finite() && n > 0.0 && n.fract() == 0.0 && n <= 9_007_199_254_740_991.0 + }) + }); + let child_constants = l.as_object().is_some_and(|m| m.len() == 4) + || (l.get("maxChildren").and_then(Value::as_f64) == Some(8.0) + && l.get("maxChildDepth").and_then(Value::as_f64) == Some(1.0)); + if ints && times && expanded && child_constants { result["limits"] = l.clone(); } } @@ -7505,6 +7584,16 @@ pub(crate) fn sdk_native_failure(record: &Value) -> Value { mod sdk_budget_diagnostic_tests { use super::*; #[test] + fn integral_decimal_native_limits_are_admitted() { + let record: Value = serde_json::from_str(r#"{"limits":{"maxTurns":2e1,"timeoutSeconds":180.0,"toolTimeoutSeconds":3e1,"maxOutputTokens":12000.0,"maxAggregateRequests":20.0,"maxAggregateHostedWebCalls":2e1,"maxAggregateFunctionTools":80.0,"maxObservedTotalTokens":5e5,"maxRequestInputBytes":256000.0,"maxChildren":8.0,"maxChildDepth":1e0}}"#).unwrap(); + assert_eq!(sdk_native_failure(&record)["limits"], record["limits"]); + for (key, invalid) in [("maxChildren", 9), ("maxChildDepth", 2)] { + let mut invalid_record = record.clone(); + invalid_record["limits"][key] = json!(invalid); + assert!(sdk_native_failure(&invalid_record).get("limits").is_none()); + } + } + #[test] fn native_failure_is_closed_and_validated() { let f: Value = serde_json::from_str(include_str!( "../../../../shared/fixtures/adapters/sdk-native-limits.json" diff --git a/cli/rust/crates/prose-runner-core/src/lib.rs b/cli/rust/crates/prose-runner-core/src/lib.rs index a9df76a9..843a700a 100644 --- a/cli/rust/crates/prose-runner-core/src/lib.rs +++ b/cli/rust/crates/prose-runner-core/src/lib.rs @@ -30,5 +30,6 @@ pub mod kernel_startup; mod credential_store; pub mod registry; +mod sdk_observation; pub mod service; pub mod service_account; diff --git a/cli/rust/crates/prose-runner-core/src/runner.rs b/cli/rust/crates/prose-runner-core/src/runner.rs index 2428e312..c6802b3f 100644 --- a/cli/rust/crates/prose-runner-core/src/runner.rs +++ b/cli/rust/crates/prose-runner-core/src/runner.rs @@ -298,7 +298,11 @@ pub fn action_output_mode(parsed: &ParsedInvocation, config: &EffectiveConfig) - } } -fn validate_harness_selection(harness: &str, flags: &GlobalFlags) -> Result<(), RunnerError> { +/// Validate the explicitly requested saved bundle without ambient execution configuration. +/// +/// # Errors +/// Returns invocation/configuration errors for incomplete or incompatible selections. +pub fn validate_harness_selection(harness: &str, flags: &GlobalFlags) -> Result<(), RunnerError> { if harness == "openprose" { if flags.model.is_some() || flags.auth_profile.is_some() { return Err(RunnerError::catalog(ErrorCode::ConfigInvalid).with_detail( @@ -1236,6 +1240,8 @@ struct InstalledRunObserver<'a> { auth_source_failed: bool, sdk: bool, native_failure: Option, + sdk_observations: Value, + sdk_requested_model: Option, capture: Option, human: Option>, omp: Option, @@ -1254,6 +1260,13 @@ impl std::fmt::Debug for InstalledRunObserver<'_> { impl RecordObserver for InstalledRunObserver<'_> { fn observe_parsed(&mut self, record: &Value) -> Result<(), SupervisorFailure> { + if self.sdk { + crate::sdk_observation::update( + &mut self.sdk_observations, + record, + self.sdk_requested_model.as_deref(), + ); + } if self.sdk && record.get("type").and_then(Value::as_str) == Some("error") { self.native_failure = Some(installed_adapters::sdk_native_failure(record)); } @@ -1735,6 +1748,19 @@ fn inspect_installed_adapter( } }; let ambient = env::vars_os().collect::>(); + if adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl { + if let Err(problem) = installed_adapters::auth_readiness(adapter, &auth_group, &ambient) { + return InstalledAdapterDiscovery { + executable: None, + version: None, + runtime_prerequisites: Vec::new(), + environment: None, + auth_group: Some(auth_group), + auth_readiness: "missing", + problem: Some(problem.with_detail("fallbackAttempted", false)), + }; + } + } let search_path = ambient .iter() .find(|(name, _)| name == std::ffi::OsStr::new("PATH")) @@ -2278,6 +2304,8 @@ fn execute_installed_adapter( auth_source_failed: false, sdk: adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl, native_failure: None, + sdk_observations: json!({}), + sdk_requested_model: config.model.value.clone(), capture, human: human_stream, omp: omp_controller, @@ -2439,6 +2467,7 @@ fn execute_installed_adapter( &invocation_id, failure, run_observer.native_failure.clone(), + &run_observer.sdk_observations, detected_version.as_deref(), rendered_payload_digest.as_deref(), config, @@ -2706,6 +2735,11 @@ fn installed_adapter_success_result( if let Some(native) = native_configuration(config, Some(&outcome.records)) { result["nativeConfiguration"] = native; } + if adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl { + let observations = + crate::sdk_observation::from_records(&outcome.records, config.model.value.as_deref()); + crate::sdk_observation::apply(&mut result, &observations); + } match mode { OutputMode::Human => { let remaining = &terminal.visible_text[human_stream_settlement.emitted_prefix_bytes..]; @@ -2846,6 +2880,7 @@ fn installed_adapter_failure_result( invocation_id: &str, failure: SupervisorFailure, native_failure: Option, + sdk_observations: &Value, detected_version: Option<&str>, rendered_payload_digest: Option<&str>, config: &EffectiveConfig, @@ -2898,6 +2933,11 @@ fn installed_adapter_failure_result( if let Some(diagnostic) = native_failure { error = error.with_detail("nativeFailure", diagnostic); } + for key in ["usageObservation", "modelIdentity"] { + if let Some(value) = sdk_observations.get(key) { + error = error.with_detail(key, value.clone()); + } + } render_installed_failure( adapter, task, @@ -2943,6 +2983,25 @@ fn render_installed_failure( clock: &dyn Clock, native_records: Option<&[Value]>, ) -> CommandOutcome { + let mut error = error; + if adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl { + let observations = crate::sdk_observation::from_records( + native_records.unwrap_or(&[]), + config.model.value.as_deref(), + ); + for key in ["usageObservation", "modelIdentity"] { + if error + .details + .as_ref() + .and_then(|details| details.get(key)) + .is_none() + { + if let Some(value) = observations.get(key) { + error = error.with_detail(key, value.clone()); + } + } + } + } let timestamp = clock.now_rfc3339(); let invocation = installed_invocation(adapter, task, task_digest, invocation_id, config, image); let invocation_digest = sha256_hex(&serde_json::to_vec(&invocation).expect("invocation JSON")); @@ -3004,6 +3063,10 @@ fn render_installed_failure( if let Some(native) = native_configuration(config, native_records) { result["nativeConfiguration"] = native; } + if adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl { + let observations = result["error"]["details"].clone(); + crate::sdk_observation::apply(&mut result, &observations); + } match mode { OutputMode::Human => CommandOutcome::human( "", @@ -4735,6 +4798,8 @@ mod tests { auth_source_failed: false, sdk: true, native_failure: None, + sdk_observations: json!({}), + sdk_requested_model: Some("gpt-6.1-sol".to_owned()), capture: None, human: None, omp: None, @@ -4753,6 +4818,60 @@ mod tests { assert!(observer.native_failure.is_none()); } + #[test] + fn sdk_failure_preserves_observer_identity_newer_than_retained_records() { + struct FixedClock; + impl Clock for FixedClock { + fn now_rfc3339(&self) -> String { + "2026-10-06T12:00:00.000Z".to_owned() + } + } + let temp = TempDir::new().unwrap(); + let config = installed_config( + temp.path(), + "agents-sdk", + "jsonl", + Some("gpt-6.1-sol"), + "openai-api-key", + ); + let earlier = json!({"type":"tool_result","modelIdentity":{"requested":"gpt-6.1-sol","observed":["model-a"],"serviceTier":{"requested":"default","observed":["default"]}}}); + let newer = json!({"type":"error","modelIdentity":{"requested":"gpt-6.1-sol","observed":["model-a","model-b"],"serviceTier":{"requested":"default","observed":["default","priority"]}}}); + let observed = crate::sdk_observation::from_records( + &[earlier.clone(), newer], + config.model.value.as_deref(), + ); + let error = RunnerError::catalog(ErrorCode::HarnessFailed) + .with_detail("modelIdentity", observed["modelIdentity"].clone()); + let outcome = render_installed_failure( + installed_adapters::InstalledAdapter::AgentsSdkJsonl, + &json!({}), + "task", + "invocation", + error, + true, + false, + Some(1), + None, + Some("0.1.0"), + None, + String::new(), + &config, + &sentinel_image(), + OutputMode::Json, + &FixedClock, + Some(&[earlier]), + ); + let crate::output::Payload::JsonWithDiagnostic { value, .. } = outcome.payload else { + panic!("expected JSON failure"); + }; + assert_eq!(value["modelIdentity"], observed["modelIdentity"]); + assert_eq!( + value["error"]["details"]["modelIdentity"], + observed["modelIdentity"] + ); + assert_eq!(value["usage"], json!({"status":"unavailable"})); + } + #[test] fn optional_reporting_preserves_defaults_and_explicit_selections() { let fixture: Value = serde_json::from_str(include_str!( @@ -4893,6 +5012,8 @@ mod tests { auth_source_failed: false, sdk: false, native_failure: None, + sdk_observations: json!({}), + sdk_requested_model: Some("gpt-6.1-sol".to_owned()), capture: None, human: None, omp: None, diff --git a/cli/rust/crates/prose-runner-core/src/sdk_observation.rs b/cli/rust/crates/prose-runner-core/src/sdk_observation.rs new file mode 100644 index 00000000..d53a8ca8 --- /dev/null +++ b/cli/rust/crates/prose-runner-core/src/sdk_observation.rs @@ -0,0 +1,162 @@ +//! Closed, bounded public projections of SDK observations. These are not bills. +use serde_json::{Map, Value, json}; + +const SAFE_INTEGER: u64 = 9_007_199_254_740_991; +const COUNTERS: [&str; 6] = [ + "input_tokens", + "output_tokens", + "total_tokens", + "input_tokens_details.cached_tokens", + "input_tokens_details.cache_write_tokens", + "output_tokens_details.reasoning_tokens", +]; +const TIERS: [&str; 7] = [ + "auto", + "default", + "flex", + "scale", + "priority", + "fast", + "ultrafast", +]; +fn count(value: &Value) -> Option { + if let Some(n) = value.as_u64() { + return (n <= SAFE_INTEGER).then_some(n); + } + let n = value.as_f64()?; + if !n.is_finite() || n < 0.0 || n.fract() != 0.0 || n > 9_007_199_254_740_991.0 { + return None; + } + if n == 0.0 { + return Some(0); + } + // JSON Schema integers include 3.0 and 3e0. Normalize their representation + // without an unchecked numeric cast, matching the independent JS product. + format!("{n:.0}").parse().ok() +} +fn public_model(value: &str) -> bool { + !value.is_empty() + && value.len() <= 128 + && value.as_bytes()[0].is_ascii_alphanumeric() + && value + .bytes() + .all(|c| c.is_ascii_alphanumeric() || b"._/-".contains(&c)) +} +fn tokens(value: &Value) -> Option { + let source = value.as_object()?; + let mut output = Map::new(); + for key in COUNTERS { + if let Some(value) = source.get(key).and_then(count) { + output.insert(key.to_owned(), json!(value)); + } + } + Some(Value::Object(output)) +} +fn usage(record: &Value) -> Option { + let source = record.get("usageObservation")?; + if source.get("source")? != "sdk_completed_response_raw_usage" + || source.get("aggregationScope")? != "unique_completed_responses_in_parent_and_children" + || !source.get("outstandingProviderRequestCount")?.is_null() + || source.get("totalRunUsageKnown")? != false + { + return None; + } + let mut output = json!({"source":"sdk_completed_response_raw_usage", "aggregationScope":"unique_completed_responses_in_parent_and_children", "outstandingProviderRequestCount":null, "totalRunUsageKnown":false}); + for key in [ + "startedCallCount", + "completedResponseCount", + "duplicateResponseCallbackCount", + "outstandingCallCount", + ] { + output[key] = json!(count(source.get(key)?)?); + } + for key in ["observedTokenTotals", "fieldResponseCounts"] { + output[key] = tokens(source.get(key)?)?; + } + Some(output) +} +fn model(record: &Value, requested: Option<&str>) -> Option { + let requested = requested.filter(|v| public_model(v))?; + let source = record.get("modelIdentity")?.as_object()?; + let observed = source.get("observed")?.as_array()?; + let mut identities: std::collections::BTreeSet<&str> = std::collections::BTreeSet::new(); + // Work is bounded by the already admitted native record byte limit. + for value in observed { + if let Some(value) = value.as_str().filter(|v| public_model(v)) { + identities.insert(value); + } + } + let tier = source.get("serviceTier")?.as_object()?; + if tier.get("requested")? != "default" { + return None; + } + let observed_tiers: std::collections::BTreeSet<&str> = tier + .get("observed")? + .as_array()? + .iter() + .filter_map(Value::as_str) + .filter(|v| TIERS.contains(v)) + .collect(); + Some( + json!({"requested":requested, "observed":identities.into_iter().take(128).collect::>(), "serviceTier":{"requested":"default", "observed":observed_tiers}}), + ) +} +/// Update independent valid groups; absent or invalid groups do not erase prior observations. +pub(crate) fn update(output: &mut Value, record: &Value, requested: Option<&str>) { + if let Some(value) = usage(record) { + output["usageObservation"] = value; + } + if let Some(value) = model(record, requested) { + output["modelIdentity"] = value; + } +} +pub(crate) fn from_records(records: &[Value], requested: Option<&str>) -> Value { + let mut output = json!({}); + for record in records { + update(&mut output, record, requested); + } + output +} +pub(crate) fn apply(output: &mut Value, observations: &Value) { + for key in ["usageObservation", "modelIdentity"] { + if let Some(value) = observations.get(key) { + output[key] = value.clone(); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn observations_are_closed_and_invocation_owned() { + let mut record = json!({"usageObservation":{"source":"sdk_completed_response_raw_usage", "aggregationScope":"unique_completed_responses_in_parent_and_children", "startedCallCount":4,"completedResponseCount":3,"duplicateResponseCallbackCount":0,"outstandingCallCount":1,"outstandingProviderRequestCount":null,"totalRunUsageKnown":false,"observedTokenTotals":{"input_tokens":9,"unknown":77,"output_tokens":-1},"fieldResponseCounts":{"input_tokens":3}},"modelIdentity":{"requested":"untrusted","observed":["gpt-6.1-sol","gpt-6.1-sol","secret\ntext","é"],"serviceTier":{"requested":"default","observed":["default","unknown"]}}}); + let value = from_records(&[record.clone()], Some("gpt-6.1-sol")); + assert_eq!(value["modelIdentity"]["requested"], "gpt-6.1-sol"); + assert_eq!(value["modelIdentity"]["observed"], json!(["gpt-6.1-sol"])); + assert_eq!( + value["usageObservation"]["observedTokenTotals"], + json!({"input_tokens":9}) + ); + record["usageObservation"]["startedCallCount"] = json!(4.0); + assert_eq!( + from_records(&[record.clone()], Some("gpt-6.1-sol"))["usageObservation"]["startedCallCount"], + json!(4) + ); + for raw in ["-0.0", "0e0", "3.0"] { + let numeric: Value = serde_json::from_str(raw).unwrap(); + record["usageObservation"]["startedCallCount"] = numeric; + assert!( + from_records(&[record.clone()], Some("gpt-6.1-sol")) + .get("usageObservation") + .is_some() + ); + } + record["usageObservation"]["startedCallCount"] = json!(-1); + assert!( + from_records(&[record], Some("gpt-6.1-sol")) + .get("usageObservation") + .is_none() + ); + } +} diff --git a/cli/shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json b/cli/shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json index 8d0a9e7c..7077dc93 100644 --- a/cli/shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json +++ b/cli/shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json @@ -16,9 +16,12 @@ "admittedVersions": [ "0.1.0" ], - "repairCommand": "Install harnesses/agents-sdk requirements and expose prose-agents-sdk on PATH", + "repairCommand": "Reinstall or upgrade the CLI through its existing npm, standalone or Homebrew installation route to restore the packaged prose-agents-sdk sibling; do not provision Python or add a PATH harness.", "platforms": [ - "darwin-arm64" + "darwin-arm64", + "darwin-x64", + "linux-arm64-gnu", + "linux-x64-gnu" ] }, "probe": { @@ -106,7 +109,6 @@ "preserved": [ "native-safety-prompt", "managed-policy", - "selected-permissions", "harness-auth" ] }, diff --git a/cli/shared/fixtures/adapters/scenarios/agents-sdk-jsonl.v1.json b/cli/shared/fixtures/adapters/scenarios/agents-sdk-jsonl.v1.json index 31c27d73..ae5d1918 100644 --- a/cli/shared/fixtures/adapters/scenarios/agents-sdk-jsonl.v1.json +++ b/cli/shared/fixtures/adapters/scenarios/agents-sdk-jsonl.v1.json @@ -71,7 +71,39 @@ }, { "type": "final", - "output": "Echoed task argv: [\"prose\",\"run\",\"path with spaces/example.prose.md\",\"--model\",\"雪\",\"\",\"line\\nbreak\",\";$(touch nope)\"]\n{\"schema\":\"openprose.echo-terminal/1\",\"semanticStatus\":\"not-applicable\",\"placeholder\":true,\"marker\":\"OPENPROSE_ECHO_TERMINAL_V0\",\"task\":{\"argv\":[\"prose\",\"run\",\"path with spaces/example.prose.md\",\"--model\",\"雪\",\"\",\"line\\nbreak\",\";$(touch nope)\"]}}" + "output": "Echoed task argv: [\"prose\",\"run\",\"path with spaces/example.prose.md\",\"--model\",\"\u96ea\",\"\",\"line\\nbreak\",\";$(touch nope)\"]\n{\"schema\":\"openprose.echo-terminal/1\",\"semanticStatus\":\"not-applicable\",\"placeholder\":true,\"marker\":\"OPENPROSE_ECHO_TERMINAL_V0\",\"task\":{\"argv\":[\"prose\",\"run\",\"path with spaces/example.prose.md\",\"--model\",\"\u96ea\",\"\",\"line\\nbreak\",\";$(touch nope)\"]}}", + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 3, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } } ], "expectedTerminal": "final", @@ -87,7 +119,7 @@ "run", "path with spaces/example.prose.md", "--model", - "雪", + "\u96ea", "", "line\nbreak", ";$(touch nope)" diff --git a/cli/shared/fixtures/adapters/sdk-native-limits.json b/cli/shared/fixtures/adapters/sdk-native-limits.json index 7bb3c3e6..d32c70c5 100644 --- a/cli/shared/fixtures/adapters/sdk-native-limits.json +++ b/cli/shared/fixtures/adapters/sdk-native-limits.json @@ -3,7 +3,14 @@ "maxTurns": 20, "timeoutSeconds": 180, "toolTimeoutSeconds": 30, - "maxOutputTokens": 12000 + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 }, "override": { "nativeMaxTurns": "40", @@ -12,7 +19,14 @@ "maxTurns": 40, "timeoutSeconds": 300, "toolTimeoutSeconds": 30, - "maxOutputTokens": 12000 + "maxOutputTokens": 12000, + "maxAggregateRequests": 40, + "maxAggregateHostedWebCalls": 40, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 }, "argv": [ "--max-turns", @@ -64,5 +78,11 @@ "value": "1ms", "seconds": 0.001 } - ] + ], + "historicalLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000 + } } diff --git a/cli/shared/fixtures/adapters/sdk-production.json b/cli/shared/fixtures/adapters/sdk-production.json new file mode 100644 index 00000000..854c5705 --- /dev/null +++ b/cli/shared/fixtures/adapters/sdk-production.json @@ -0,0 +1,389 @@ +{ + "schema": "openprose.sdk-production-oracle/1", + "defaults": { + "harness": "agents-sdk", + "model": "gpt-6.1-sol", + "authProfile": "openai-api-key", + "transport": "jsonl", + "billingOwner": "user-provider", + "permissionMode": null + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "discovery": { + "mode": "canonical-cli-sibling", + "helperName": "prose-agents-sdk", + "pathFallback": false, + "platforms": [ + "darwin-arm64", + "darwin-x64", + "linux-arm64", + "linux-x64" + ], + "repairCommand": "Reinstall or upgrade the CLI through its existing npm, standalone or Homebrew installation route to restore the packaged prose-agents-sdk sibling; do not provision Python or add a PATH harness.", + "helperSymlinkAllowed": false, + "admissionPlatforms": [ + "darwin-arm64", + "darwin-x64", + "linux-arm64-gnu", + "linux-x64-gnu" + ], + "linuxMinimumGlibc": "2.34", + "platformControls": [ + { + "platform": "darwin-arm64", + "supported": true + }, + { + "platform": "darwin-x64", + "supported": true + }, + { + "platform": "linux-arm64-gnu", + "supported": true + }, + { + "platform": "linux-x64-gnu", + "supported": true + }, + { + "platform": "linux-arm64-musl", + "supported": false + }, + { + "platform": "linux-x64-musl", + "supported": false + }, + { + "platform": "linux-arm64", + "supported": false + }, + { + "platform": "linux-x64", + "supported": false + } + ] + }, + "observation": { + "usageUnavailable": { + "status": "unavailable" + }, + "emptyUsage": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 0, + "completedResponseCount": 0, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": {}, + "fieldResponseCounts": {}, + "totalRunUsageKnown": false + }, + "completedUsage": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 3, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 0, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "allowedTokenCounters": [ + "input_tokens", + "output_tokens", + "total_tokens", + "input_tokens_details.cached_tokens", + "input_tokens_details.cache_write_tokens", + "output_tokens_details.reasoning_tokens" + ], + "allowedServiceTiers": [ + "auto", + "default", + "flex", + "scale", + "priority", + "fast", + "ultrafast" + ], + "failureUsage": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + } + }, + "permissions": { + "shell": "host_os_permissions", + "filesystemSandbox": false, + "networkSandbox": false, + "freshChildConversation": true + }, + "credentialAbsence": { + "variable": "OPENAI_API_KEY", + "blankIsMissing": true, + "errorCode": "HARNESS_NEEDS_AUTH", + "modelRequests": 0, + "fallback": false, + "action": "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent." + }, + "cases": [ + { + "id": "adapters.sdk-production-01", + "scenario": "trap", + "installation": "sibling", + "userConfig": null, + "noProbe": true, + "noStart": true, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-02", + "scenario": "trap", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": true, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-03", + "scenario": "trap", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": true, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-04", + "scenario": "success", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-05", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [], + "wireNumberLexemes": { + "maxTurns": "20.0", + "maxOutputTokens": "12000e0", + "maxAggregateRequests": "20.0", + "maxAggregateHostedWebCalls": "2e1", + "maxAggregateFunctionTools": "80.0", + "maxObservedTotalTokens": "5e5", + "maxRequestInputBytes": "256000.0", + "maxChildren": "8.0", + "maxChildDepth": "1e0" + } + }, + { + "id": "adapters.sdk-production-06", + "scenario": "tainted", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-unrecognized-secret-sentinel" + ], + "absentObservationGroups": [ + "modelIdentity" + ], + "wireNumberLexemes": { + "startedCallCount": "3.0", + "completedResponseCount": "3e0", + "duplicateResponseCallbackCount": "-0.0", + "outstandingCallCount": "-0.0", + "input_tokens_details.cached_tokens": "-0.0" + } + }, + { + "id": "adapters.sdk-production-07", + "scenario": "trap", + "installation": "sibling", + "userConfig": "harness = \"claude\"\n", + "noProbe": true, + "noStart": true, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-08", + "scenario": "success", + "installation": "symlink", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [] + }, + { + "id": "adapters.sdk-production-09", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [], + "wireNumberLexemes": {}, + "rawLimitsOverrides": { + "maxChildren": 9 + }, + "absentResultPaths": [ + [ + "error", + "details", + "nativeFailure", + "limits" + ] + ] + }, + { + "id": "adapters.sdk-production-10", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [], + "wireNumberLexemes": {}, + "rawLimitsOverrides": { + "maxChildDepth": 2 + }, + "absentResultPaths": [ + [ + "error", + "details", + "nativeFailure", + "limits" + ] + ] + } + ], + "rejectedModelIdentities": [ + { + "id": "observed-not-array", + "raw": { + "requested": "gpt-6.1-sol", + "observed": "sdk-unrecognized-secret-sentinel", + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "expected": "omit-modelIdentity" + }, + { + "id": "missing-tier", + "raw": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ] + }, + "expected": "omit-modelIdentity" + }, + { + "id": "tier-not-object", + "raw": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": "sdk-unrecognized-secret-sentinel" + }, + "expected": "omit-modelIdentity" + }, + { + "id": "tier-observed-not-array", + "raw": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": "sdk-unrecognized-secret-sentinel" + } + }, + "expected": "omit-modelIdentity" + }, + { + "id": "wrong-requested-tier", + "raw": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "priority", + "observed": [ + "default" + ] + } + }, + "expected": "omit-modelIdentity" + } + ], + "harnessUseValidationOrder": [ + "parse-file-environment-flag-layers", + "validate-explicit-selection-target", + "validate-contextual-model-and-auth" + ] +} diff --git a/cli/shared/fixtures/config/values-v1.json b/cli/shared/fixtures/config/values-v1.json index 947d96e5..e691d369 100644 --- a/cli/shared/fixtures/config/values-v1.json +++ b/cli/shared/fixtures/config/values-v1.json @@ -301,7 +301,8 @@ { "id": "env-native-budget-needs-agents-sdk", "environment": { - "PROSE_NATIVE_MAX_TURNS": "3" + "PROSE_NATIVE_MAX_TURNS": "3", + "PROSE_HARNESS": "openprose" }, "expected": { "error": { diff --git a/cli/shared/fixtures/operations/configuration-explanation.json b/cli/shared/fixtures/operations/configuration-explanation.json index 7647d413..b51f9f02 100644 --- a/cli/shared/fixtures/operations/configuration-explanation.json +++ b/cli/shared/fixtures/operations/configuration-explanation.json @@ -11,7 +11,7 @@ "userConfigPath": "/workspace/config/openprose/cli.toml", "values": { "harness": { - "value": "openprose", + "value": "agents-sdk", "source": { "kind": "default", "location": "built-in" @@ -25,10 +25,10 @@ } }, "model": { - "value": null, + "value": "gpt-6.1-sol", "source": { "kind": "default", - "location": "built-in" + "location": "built-in:agents-sdk" } }, "timeout": { @@ -137,10 +137,10 @@ } }, "authProfile": { - "value": null, + "value": "openai-api-key", "source": { "kind": "default", - "location": "built-in" + "location": "built-in:agents-sdk" } } }, @@ -162,7 +162,7 @@ "candidates": { "harness": [ { - "value": "openprose", + "value": "agents-sdk", "source": { "kind": "default", "location": "built-in" @@ -182,10 +182,10 @@ ], "model": [ { - "value": null, + "value": "gpt-6.1-sol", "source": { "kind": "default", - "location": "built-in" + "location": "built-in:agents-sdk" }, "selected": true } @@ -350,10 +350,10 @@ ], "authProfile": [ { - "value": null, + "value": "openai-api-key", "source": { "kind": "default", - "location": "built-in" + "location": "built-in:agents-sdk" }, "selected": true } @@ -361,11 +361,23 @@ }, "diagnostics": [], "runtime": { - "transport": "hosted", + "transport": "jsonl", "permissionMode": null, - "authProfile": null, - "billingOwner": "openprose", - "nativeLimits": null, + "authProfile": "openai-api-key", + "billingOwner": "user-provider", + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, "nativeOutputLimits": null } } diff --git a/cli/shared/fixtures/operations/doctor-report.json b/cli/shared/fixtures/operations/doctor-report.json index 3074b7b3..aa50d110 100644 --- a/cli/shared/fixtures/operations/doctor-report.json +++ b/cli/shared/fixtures/operations/doctor-report.json @@ -41,8 +41,8 @@ "harness": { "value": "openprose", "source": { - "kind": "default", - "location": "built-in" + "kind": "flag", + "location": "--harness" } }, "transport": { @@ -190,11 +190,19 @@ "candidates": { "harness": [ { - "value": "openprose", + "value": "agents-sdk", "source": { "kind": "default", "location": "built-in" }, + "selected": false + }, + { + "value": "openprose", + "source": { + "kind": "flag", + "location": "--harness" + }, "selected": true } ], diff --git a/cli/shared/schemas/native-limits.schema.json b/cli/shared/schemas/native-limits.schema.json index 32fc86d2..da52c9d0 100644 --- a/cli/shared/schemas/native-limits.schema.json +++ b/cli/shared/schemas/native-limits.schema.json @@ -29,6 +29,57 @@ "type": "integer", "exclusiveMinimum": 0, "maximum": 9007199254740991 + }, + "maxAggregateRequests": { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 9007199254740991 + }, + "maxAggregateHostedWebCalls": { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 9007199254740991 + }, + "maxAggregateFunctionTools": { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 9007199254740991 + }, + "maxObservedTotalTokens": { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 9007199254740991 + }, + "maxRequestInputBytes": { + "type": "integer", + "exclusiveMinimum": 0, + "maximum": 9007199254740991 + }, + "maxChildren": { + "const": 8 + }, + "maxChildDepth": { + "const": 1 + } + }, + "oneOf": [ + { + "maxProperties": 4 + }, + { + "required": [ + "maxTurns", + "timeoutSeconds", + "toolTimeoutSeconds", + "maxOutputTokens", + "maxAggregateRequests", + "maxAggregateHostedWebCalls", + "maxAggregateFunctionTools", + "maxObservedTotalTokens", + "maxRequestInputBytes", + "maxChildren", + "maxChildDepth" + ] } - } + ] } diff --git a/cli/shared/schemas/runner-error.schema.json b/cli/shared/schemas/runner-error.schema.json index 467a14fb..ff9cce16 100644 --- a/cli/shared/schemas/runner-error.schema.json +++ b/cli/shared/schemas/runner-error.schema.json @@ -111,6 +111,8 @@ "adapterDiagnostic": { "$ref": "adapter-diagnostic.schema.json" }, + "usageObservation": { "$ref": "sdk-observation.schema.json#/$defs/usageObservation" }, + "modelIdentity": { "$ref": "sdk-observation.schema.json#/$defs/modelIdentity" }, "nativeFailure": { "$ref": "native-failure.schema.json" }, diff --git a/cli/shared/schemas/runner-result.schema.json b/cli/shared/schemas/runner-result.schema.json index 6afa3299..da9b6281 100644 --- a/cli/shared/schemas/runner-result.schema.json +++ b/cli/shared/schemas/runner-result.schema.json @@ -5,121 +5,598 @@ "type": "object", "additionalProperties": false, "required": [ - "schema", "invocationId", "runner", "adapter", "transport", - "negotiatedCapabilities", "languageImage", "digests", "cwd", "timing", - "terminal", "semantic", "usage", "billing", "diagnosticRefs", + "schema", + "invocationId", + "runner", + "adapter", + "transport", + "negotiatedCapabilities", + "languageImage", + "digests", + "cwd", + "timing", + "terminal", + "semantic", + "usage", + "billing", + "diagnosticRefs", "runnerExitCode" ], "properties": { - "codexCompatibility": { "$ref": "codex-compatibility.schema.json" }, - "nativeLimits": { "$ref": "native-limits.schema.json" }, - "nativeOutputLimits": { "$ref": "native-output-limits.schema.json" }, - "nativeConfiguration": { "$ref": "native-configuration.schema.json" }, - "schema": { "const": "openprose.runner-result/1" }, - "invocationId": { "$ref": "common.schema.json#/$defs/nonEmptyString", "x-openprose-volatile": true }, + "codexCompatibility": { + "$ref": "codex-compatibility.schema.json" + }, + "usageObservation": { + "$ref": "sdk-observation.schema.json#/$defs/usageObservation" + }, + "modelIdentity": { + "$ref": "sdk-observation.schema.json#/$defs/modelIdentity" + }, + "nativeLimits": { + "$ref": "native-limits.schema.json" + }, + "nativeOutputLimits": { + "$ref": "native-output-limits.schema.json" + }, + "nativeConfiguration": { + "$ref": "native-configuration.schema.json" + }, + "schema": { + "const": "openprose.runner-result/1" + }, + "invocationId": { + "$ref": "common.schema.json#/$defs/nonEmptyString", + "x-openprose-volatile": true + }, "runner": { - "type": "object", "additionalProperties": false, - "required": ["name", "version", "commit"], - "properties": { "name": { "enum": ["rust", "bun"] }, "version": { "type": "string", "minLength": 1 }, "commit": { "type": "string", "minLength": 1 } } + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "version", + "commit" + ], + "properties": { + "name": { + "enum": [ + "rust", + "bun" + ] + }, + "version": { + "type": "string", + "minLength": 1 + }, + "commit": { + "type": "string", + "minLength": 1 + } + } }, "adapter": { - "type": "object", "additionalProperties": false, - "required": ["id", "harnessVersion", "descriptorDigestSha256"], - "properties": { "id": { "type": "string", "minLength": 1 }, "harnessVersion": { "type": ["string", "null"] }, "descriptorDigestSha256": { "$ref": "common.schema.json#/$defs/sha256" } } + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "harnessVersion", + "descriptorDigestSha256" + ], + "properties": { + "id": { + "type": "string", + "minLength": 1 + }, + "harnessVersion": { + "type": [ + "string", + "null" + ] + }, + "descriptorDigestSha256": { + "$ref": "common.schema.json#/$defs/sha256" + } + } + }, + "transport": { + "type": "string", + "minLength": 1 }, - "transport": { "type": "string", "minLength": 1 }, "negotiatedCapabilities": { - "type": "object", "additionalProperties": false, - "required": ["promptPlacement", "isolation", "streaming", "cancellation", "terminal"], + "type": "object", + "additionalProperties": false, + "required": [ + "promptPlacement", + "isolation", + "streaming", + "cancellation", + "terminal" + ], "properties": { - "promptPlacement": { "type": "string", "minLength": 1 }, - "isolation": { "type": "string", "minLength": 1 }, - "streaming": { "type": "string", "minLength": 1 }, - "cancellation": { "type": "string", "minLength": 1 }, - "terminal": { "type": "string", "minLength": 1 } + "promptPlacement": { + "type": "string", + "minLength": 1 + }, + "isolation": { + "type": "string", + "minLength": 1 + }, + "streaming": { + "type": "string", + "minLength": 1 + }, + "cancellation": { + "type": "string", + "minLength": 1 + }, + "terminal": { + "type": "string", + "minLength": 1 + } } }, "languageImage": { - "type": "object", "additionalProperties": false, - "required": ["formatVersion", "version", "sha256"], - "properties": { "formatVersion": { "const": "openprose.skill-runtime-image/1" }, "version": { "type": "string", "minLength": 1 }, "sha256": { "$ref": "common.schema.json#/$defs/sha256" } } + "type": "object", + "additionalProperties": false, + "required": [ + "formatVersion", + "version", + "sha256" + ], + "properties": { + "formatVersion": { + "const": "openprose.skill-runtime-image/1" + }, + "version": { + "type": "string", + "minLength": 1 + }, + "sha256": { + "$ref": "common.schema.json#/$defs/sha256" + } + } }, "digests": { - "type": "object", "additionalProperties": false, - "required": ["invocationSha256", "taskSha256", "normalizedEventsSha256"], - "properties": { "invocationSha256": { "$ref": "common.schema.json#/$defs/sha256" }, "taskSha256": { "$ref": "common.schema.json#/$defs/sha256" }, "normalizedEventsSha256": { "$ref": "common.schema.json#/$defs/sha256" }, "deliveredImageSha256": { "anyOf": [{ "$ref": "common.schema.json#/$defs/sha256" }, { "type": "null" }] }, "renderedPayloadSha256": { "anyOf": [{ "$ref": "common.schema.json#/$defs/sha256" }, { "type": "null" }] } } + "type": "object", + "additionalProperties": false, + "required": [ + "invocationSha256", + "taskSha256", + "normalizedEventsSha256" + ], + "properties": { + "invocationSha256": { + "$ref": "common.schema.json#/$defs/sha256" + }, + "taskSha256": { + "$ref": "common.schema.json#/$defs/sha256" + }, + "normalizedEventsSha256": { + "$ref": "common.schema.json#/$defs/sha256" + }, + "deliveredImageSha256": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/sha256" + }, + { + "type": "null" + } + ] + }, + "renderedPayloadSha256": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/sha256" + }, + { + "type": "null" + } + ] + } + } }, "cwd": { - "type": "object", "additionalProperties": false, - "required": ["path", "identitySha256"], - "properties": { "path": { "type": "string", "minLength": 1 }, "identitySha256": { "$ref": "common.schema.json#/$defs/sha256" } } + "type": "object", + "additionalProperties": false, + "required": [ + "path", + "identitySha256" + ], + "properties": { + "path": { + "type": "string", + "minLength": 1 + }, + "identitySha256": { + "$ref": "common.schema.json#/$defs/sha256" + } + } }, "timing": { - "type": "object", "additionalProperties": false, - "required": ["startedAt", "firstEventAt", "cancellationAt", "terminalAt", "durationMs"], + "type": "object", + "additionalProperties": false, + "required": [ + "startedAt", + "firstEventAt", + "cancellationAt", + "terminalAt", + "durationMs" + ], "properties": { - "startedAt": { "$ref": "common.schema.json#/$defs/rfc3339", "x-openprose-volatile": true }, - "firstEventAt": { "anyOf": [{ "$ref": "common.schema.json#/$defs/rfc3339" }, { "type": "null" }], "x-openprose-volatile": true }, - "cancellationAt": { "anyOf": [{ "$ref": "common.schema.json#/$defs/rfc3339" }, { "type": "null" }], "x-openprose-volatile": true }, - "terminalAt": { "$ref": "common.schema.json#/$defs/rfc3339", "x-openprose-volatile": true }, - "durationMs": { "type": "integer", "minimum": 0, "x-openprose-volatile": true } + "startedAt": { + "$ref": "common.schema.json#/$defs/rfc3339", + "x-openprose-volatile": true + }, + "firstEventAt": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/rfc3339" + }, + { + "type": "null" + } + ], + "x-openprose-volatile": true + }, + "cancellationAt": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/rfc3339" + }, + { + "type": "null" + } + ], + "x-openprose-volatile": true + }, + "terminalAt": { + "$ref": "common.schema.json#/$defs/rfc3339", + "x-openprose-volatile": true + }, + "durationMs": { + "type": "integer", + "minimum": 0, + "x-openprose-volatile": true + } } }, "terminal": { - "type": "object", "additionalProperties": false, - "required": ["classification", "transportCompleted", "terminalEventObserved", "exitCode", "signal"], + "type": "object", + "additionalProperties": false, + "required": [ + "classification", + "transportCompleted", + "terminalEventObserved", + "exitCode", + "signal" + ], "properties": { - "classification": { "enum": ["success", "semantic-failed", "cancelled", "terminated", "timeout", "killed", "exit-code", "runner-error"] }, - "transportCompleted": { "type": "boolean" }, - "terminalEventObserved": { "type": "boolean" }, - "exitCode": { "type": ["integer", "null"] }, - "signal": { "type": ["string", "null"] } + "classification": { + "enum": [ + "success", + "semantic-failed", + "cancelled", + "terminated", + "timeout", + "killed", + "exit-code", + "runner-error" + ] + }, + "transportCompleted": { + "type": "boolean" + }, + "terminalEventObserved": { + "type": "boolean" + }, + "exitCode": { + "type": [ + "integer", + "null" + ] + }, + "signal": { + "type": [ + "string", + "null" + ] + } } }, "semantic": { - "type": "object", "additionalProperties": false, - "required": ["status", "terminalSchemaSha256", "terminalEnvelopeDigestSha256"], + "type": "object", + "additionalProperties": false, + "required": [ + "status", + "terminalSchemaSha256", + "terminalEnvelopeDigestSha256" + ], "properties": { - "status": { "enum": ["success", "semantic-failed", "unknown", "not-applicable"] }, - "terminalSchemaSha256": { "anyOf": [{ "$ref": "common.schema.json#/$defs/sha256" }, { "type": "null" }] }, - "terminalEnvelopeDigestSha256": { "anyOf": [{ "$ref": "common.schema.json#/$defs/sha256" }, { "type": "null" }] } + "status": { + "enum": [ + "success", + "semantic-failed", + "unknown", + "not-applicable" + ] + }, + "terminalSchemaSha256": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/sha256" + }, + { + "type": "null" + } + ] + }, + "terminalEnvelopeDigestSha256": { + "anyOf": [ + { + "$ref": "common.schema.json#/$defs/sha256" + }, + { + "type": "null" + } + ] + } } }, "usage": { "oneOf": [ - { "type": "object", "additionalProperties": false, "required": ["status"], "properties": { "status": { "const": "unavailable" } } }, - { "type": "object", "additionalProperties": false, "required": ["status", "inputTokens", "outputTokens", "cost", "currency"], "properties": { "status": { "const": "authoritative" }, "inputTokens": { "type": "integer", "minimum": 0 }, "outputTokens": { "type": "integer", "minimum": 0 }, "cost": { "type": "number", "minimum": 0 }, "currency": { "type": "string", "pattern": "^[A-Z]{3}$" } } } + { + "type": "object", + "additionalProperties": false, + "required": [ + "status" + ], + "properties": { + "status": { + "const": "unavailable" + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "status", + "inputTokens", + "outputTokens", + "cost", + "currency" + ], + "properties": { + "status": { + "const": "authoritative" + }, + "inputTokens": { + "type": "integer", + "minimum": 0 + }, + "outputTokens": { + "type": "integer", + "minimum": 0 + }, + "cost": { + "type": "number", + "minimum": 0 + }, + "currency": { + "type": "string", + "pattern": "^[A-Z]{3}$" + } + } + } ] }, "billing": { - "type": "object", "additionalProperties": false, - "required": ["owner", "authCategory"], - "properties": { "owner": { "$ref": "common.schema.json#/$defs/billingOwner" }, "authCategory": { "$ref": "common.schema.json#/$defs/authCategory" } } + "type": "object", + "additionalProperties": false, + "required": [ + "owner", + "authCategory" + ], + "properties": { + "owner": { + "$ref": "common.schema.json#/$defs/billingOwner" + }, + "authCategory": { + "$ref": "common.schema.json#/$defs/authCategory" + } + } }, - "diagnosticRefs": { "type": "array", "uniqueItems": true, "items": { "type": "string", "minLength": 1 } }, - "runnerExitCode": { "type": "integer", "enum": [0, 2, 10, 20, 21, 22, 23, 24, 25, 30, 70] }, - "error": { "$ref": "runner-error.schema.json" } + "diagnosticRefs": { + "type": "array", + "uniqueItems": true, + "items": { + "type": "string", + "minLength": 1 + } + }, + "runnerExitCode": { + "type": "integer", + "enum": [ + 0, + 2, + 10, + 20, + 21, + 22, + 23, + 24, + 25, + 30, + 70 + ] + }, + "error": { + "$ref": "runner-error.schema.json" + } }, "allOf": [ { - "if": { "properties": { "semantic": { "properties": { "status": { "const": "success" } }, "required": ["status"] } } }, - "then": { "properties": { "terminal": { "properties": { "classification": { "const": "success" } } }, "runnerExitCode": { "const": 0 } } } + "if": { + "properties": { + "semantic": { + "properties": { + "status": { + "const": "success" + } + }, + "required": [ + "status" + ] + } + } + }, + "then": { + "properties": { + "terminal": { + "properties": { + "classification": { + "const": "success" + } + } + }, + "runnerExitCode": { + "const": 0 + } + } + } + }, + { + "if": { + "properties": { + "semantic": { + "properties": { + "status": { + "const": "semantic-failed" + } + }, + "required": [ + "status" + ] + } + } + }, + "then": { + "properties": { + "terminal": { + "properties": { + "classification": { + "const": "semantic-failed" + } + } + }, + "runnerExitCode": { + "const": 30 + } + } + } }, { - "if": { "properties": { "semantic": { "properties": { "status": { "const": "semantic-failed" } }, "required": ["status"] } } }, - "then": { "properties": { "terminal": { "properties": { "classification": { "const": "semantic-failed" } } }, "runnerExitCode": { "const": 30 } } } + "if": { + "properties": { + "semantic": { + "properties": { + "status": { + "const": "not-applicable" + } + }, + "required": [ + "status" + ] + } + } + }, + "then": { + "properties": { + "terminal": { + "properties": { + "classification": { + "const": "success" + } + } + }, + "runnerExitCode": { + "const": 0 + } + } + } }, { - "if": { "properties": { "semantic": { "properties": { "status": { "const": "not-applicable" } }, "required": ["status"] } } }, - "then": { "properties": { "terminal": { "properties": { "classification": { "const": "success" } } }, "runnerExitCode": { "const": 0 } } } + "if": { + "properties": { + "runnerExitCode": { + "enum": [ + 2, + 10, + 20, + 21, + 22, + 23, + 24, + 25, + 70 + ] + } + } + }, + "then": { + "required": [ + "error" + ] + } }, { - "if": { "properties": { "runnerExitCode": { "enum": [2, 10, 20, 21, 22, 23, 24, 25, 70] } } }, - "then": { "required": ["error"] } + "if": { + "properties": { + "runnerExitCode": { + "enum": [ + 0, + 30 + ] + } + } + }, + "then": { + "not": { + "required": [ + "error" + ] + } + } }, { - "if": { "properties": { "runnerExitCode": { "enum": [0, 30] } } }, - "then": { "not": { "required": ["error"] } } + "if": { + "properties": { + "adapter": { + "properties": { + "id": { + "const": "agents-sdk/jsonl" + } + }, + "required": [ + "id" + ] + } + }, + "required": [ + "adapter" + ] + }, + "then": { + "properties": { + "usage": { + "properties": { + "status": { + "const": "unavailable" + } + } + } + } + } } ] } diff --git a/cli/shared/schemas/sdk-observation.schema.json b/cli/shared/schemas/sdk-observation.schema.json new file mode 100644 index 00000000..601066c2 --- /dev/null +++ b/cli/shared/schemas/sdk-observation.schema.json @@ -0,0 +1,197 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schemas.openprose.org/cli/v1/sdk-observation.schema.json", + "title": "Safe optional SDK runtime observations", + "type": "object", + "additionalProperties": false, + "minProperties": 1, + "properties": { + "usageObservation": { + "$ref": "#/$defs/usageObservation" + }, + "modelIdentity": { + "$ref": "#/$defs/modelIdentity" + } + }, + "$defs": { + "usageObservation": { + "type": "object", + "additionalProperties": false, + "required": [ + "source", + "aggregationScope", + "startedCallCount", + "completedResponseCount", + "duplicateResponseCallbackCount", + "outstandingCallCount", + "outstandingProviderRequestCount", + "observedTokenTotals", + "fieldResponseCounts", + "totalRunUsageKnown" + ], + "properties": { + "source": { + "const": "sdk_completed_response_raw_usage" + }, + "aggregationScope": { + "const": "unique_completed_responses_in_parent_and_children" + }, + "startedCallCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "completedResponseCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "duplicateResponseCallbackCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "outstandingCallCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "outstandingProviderRequestCount": { + "type": "null" + }, + "observedTokenTotals": { + "type": "object", + "additionalProperties": false, + "properties": { + "input_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "output_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "total_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "input_tokens_details.cached_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "input_tokens_details.cache_write_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "output_tokens_details.reasoning_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + } + } + }, + "fieldResponseCounts": { + "type": "object", + "additionalProperties": false, + "properties": { + "input_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "output_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "total_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "input_tokens_details.cached_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "input_tokens_details.cache_write_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + }, + "output_tokens_details.reasoning_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991 + } + } + }, + "totalRunUsageKnown": { + "const": false + } + } + }, + "modelIdentity": { + "type": "object", + "additionalProperties": false, + "required": [ + "requested", + "observed", + "serviceTier" + ], + "properties": { + "requested": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "pattern": "^[A-Za-z0-9][A-Za-z0-9._/-]*$" + }, + "observed": { + "type": "array", + "uniqueItems": true, + "maxItems": 128, + "items": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "pattern": "^[A-Za-z0-9][A-Za-z0-9._/-]*$" + } + }, + "serviceTier": { + "type": "object", + "additionalProperties": false, + "required": [ + "requested", + "observed" + ], + "properties": { + "requested": { + "const": "default" + }, + "observed": { + "type": "array", + "uniqueItems": true, + "maxItems": 7, + "items": { + "enum": [ + "auto", + "default", + "flex", + "scale", + "priority", + "fast", + "ultrafast" + ] + } + } + } + } + } + } + } +} diff --git a/cli/shared/tests/test_contracts.py b/cli/shared/tests/test_contracts.py index 0c816127..77e62336 100755 --- a/cli/shared/tests/test_contracts.py +++ b/cli/shared/tests/test_contracts.py @@ -133,6 +133,88 @@ def test_native_sdk_limits_and_failures_are_closed(self): for bad in [{'kind':'arbitrary'}, {'kind':'execution','message':'secret'}, {'kind':'timeout','elapsedSeconds':-1}]: self.assertTrue(list(self.validator('native-failure.schema.json').iter_errors(bad))) + def test_sdk_production_limits_and_optional_observations_are_closed(self): + oracle = load_json(FIXTURES / 'adapters/sdk-production.json') + self.assertEqual('openprose.sdk-production-oracle/1', oracle['schema']) + self.assertEqual(dict(harness='agents-sdk', model='gpt-6.1-sol', authProfile='openai-api-key', transport='jsonl', billingOwner='user-provider', permissionMode=None), oracle['defaults']) + limits = oracle['nativeLimits'] + self.assertEqual(11, len(limits)) + self.assertEqual(limits['maxTurns'], limits['maxAggregateRequests']) + self.assertEqual(limits['maxTurns'], limits['maxAggregateHostedWebCalls']) + self.assertEqual((80,500000,256000,8,1), tuple(limits[field] for field in ('maxAggregateFunctionTools','maxObservedTotalTokens','maxRequestInputBytes','maxChildren','maxChildDepth'))) + self.assert_valid('native-limits.schema.json', limits) + self.assert_valid('native-limits.schema.json', {key: float(value) for key,value in limits.items()}) + for field, invalid in (('maxChildren',9),('maxChildDepth',2)): + self.assertTrue(list(self.validator('native-limits.schema.json').iter_errors({**limits,field:invalid}))) + historical = {field: limits[field] for field in ('maxTurns','timeoutSeconds','toolTimeoutSeconds','maxOutputTokens')} + self.assert_valid('native-limits.schema.json', historical) + for field in set(limits) - set(historical): + self.assertTrue(list(self.validator('native-limits.schema.json').iter_errors({**historical,field:limits[field]}))) + observation = oracle['observation'] + for group in ('emptyUsage','completedUsage','failureUsage'): + usage = observation[group] + self.assertEqual(10, len(usage)) + self.assertIsNone(usage['outstandingProviderRequestCount']) + self.assertFalse(usage['totalRunUsageKnown']) + self.assert_valid('sdk-observation.schema.json', dict(usageObservation=usage, modelIdentity=observation['modelIdentity'])) + numeric_usage = deepcopy(observation['completedUsage']) + numeric_usage.update(startedCallCount=3.0,completedResponseCount=3.0,duplicateResponseCallbackCount=-0.0,outstandingCallCount=-0.0) + numeric_usage['observedTokenTotals']['input_tokens_details.cached_tokens'] = -0.0 + self.assert_valid('sdk-observation.schema.json', dict(usageObservation=numeric_usage)) + for mutation in (lambda value: value['usageObservation'].update(extra='secret'), + lambda value: value['usageObservation'].update(totalRunUsageKnown=True), + lambda value: value['usageObservation'].update(outstandingProviderRequestCount=0), + lambda value: value['usageObservation'].update(startedCallCount=True), + lambda value: value['usageObservation']['observedTokenTotals'].update(unknown=3), + lambda value: value['usageObservation']['observedTokenTotals'].update(input_tokens=-1), + lambda value: value['modelIdentity']['observed'].append('secret\nmodel'), + lambda value: value['modelIdentity']['serviceTier']['observed'].append('secret-tier'), + lambda value: value['modelIdentity']['serviceTier'].update(requested='priority')): + value = dict(usageObservation=deepcopy(observation['completedUsage']), modelIdentity=deepcopy(observation['modelIdentity'])) + mutation(value) + self.assertTrue(list(self.validator('sdk-observation.schema.json').iter_errors(value))) + self.assertEqual(5, len(oracle['rejectedModelIdentities'])) + for control in oracle['rejectedModelIdentities']: + self.assertEqual('omit-modelIdentity', control['expected']) + self.assertTrue(list(self.validator('sdk-observation.schema.json').iter_errors(dict(modelIdentity=control['raw'])))) + valid_empty = deepcopy(observation['modelIdentity']) + valid_empty['observed'] = [] + valid_empty['serviceTier']['observed'] = [] + self.assert_valid('sdk-observation.schema.json', dict(modelIdentity=valid_empty)) + self.assert_valid('native-limits.schema.json', load_json(FIXTURES/'adapters/sdk-native-limits.json')['historicalLimits']) + result = load_json(FIXTURES / 'transport/runner-result-success.json') + result.update(nativeLimits=limits, usageObservation=observation['completedUsage'], modelIdentity=observation['modelIdentity']) + self.assert_valid('runner-result.schema.json', result) + error = next(row for row in load_json(SHARED/'errors/taxonomy.v1.json')['errors'] if row['code']=='HARNESS_FAILED') + error = dict(schema='openprose.runner-error/1', **error, details=dict(usageObservation=observation['failureUsage'],modelIdentity=observation['modelIdentity'])) + self.assert_valid('runner-error.schema.json', error) + error['details']['usageObservation']['totalRunUsageKnown'] = True + self.assertTrue(list(self.validator('runner-error.schema.json').iter_errors(error))) + + def test_sdk_production_cases_are_frozen_and_provider_free(self): + oracle = load_json(FIXTURES/'adapters/sdk-production.json') + identifiers = [f'adapters.sdk-production-{number:02}' for number in range(1,11)] + self.assertEqual(identifiers, [case['id'] for case in oracle['cases']]) + for control in oracle['cases']: + case = load_json(CASES/'adapters'/f"{control['id'].split('.')[-1]}.json") + self.assertEqual(case['id'], control['id']) + self.assertEqual('denied', case['controls']['network']) + self.assertEqual(dict(adapterId='agents-sdk/jsonl', sdkScenario=control['scenario'], sdkInstallation=control['installation']), case['controls']['installedAdapter']) + self.assertEqual(not control['noStart'], case['expected']['startedHarness']) + if 'OPENAI_API_KEY' in case['invocation']['environment']: + self.assertIn(case['invocation']['environment']['OPENAI_API_KEY'], (' ','fixture-provider-free-openai-key')) + recipe = load_json(SHARED/'capabilities/adapters/recipes/agents-sdk-jsonl.v1.json') + self.assertEqual(oracle['discovery']['admissionPlatforms'], recipe['support']['platforms']) + self.assertEqual(['darwin-arm64','darwin-x64','linux-arm64','linux-x64'], oracle['discovery']['platforms']) + self.assertEqual(['darwin-arm64','darwin-x64','linux-arm64-gnu','linux-x64-gnu'], recipe['support']['platforms']) + self.assertEqual('2.34', oracle['discovery']['linuxMinimumGlibc']) + for control in oracle['discovery']['platformControls']: + self.assertEqual(control['supported'], control['platform'] in recipe['support']['platforms']) + self.assertEqual(['parse-file-environment-flag-layers','validate-explicit-selection-target','validate-contextual-model-and-auth'], oracle['harnessUseValidationOrder']) + self.assertFalse(oracle['discovery']['pathFallback']) + self.assertFalse(oracle['discovery']['helperSymlinkAllowed']) + self.assertNotIn('selected-permissions', recipe['isolation']['preserved']) + def test_optional_reporting_fields_are_named_and_closed(self): fixture=json.loads((SHARED / 'fixtures/config/optional-reporting.json').read_text()) config=json.loads((SHARED / 'fixtures/operations/configuration-explanation.json').read_text()) @@ -314,10 +396,37 @@ def test_machine_operation_examples_validate_and_share_inventory(self) -> None: examples["doctor-report.json"]["runner"], {"name": "rust", "version": "0.1.0-fixture", "commit": "fixture-commit"}, ) - self.assertEqual( - examples["configuration-explanation.json"], - examples["doctor-report.json"]["configuration"], - ) + # The standalone explanation is the SDK default. The hosted doctor + # explicitly selects its route and retains the overridden SDK candidate. + default_config = examples["configuration-explanation.json"] + doctor_config = examples["doctor-report.json"]["configuration"] + self.assertEqual(default_config["values"]["harness"]["value"], "agents-sdk") + self.assertEqual(doctor_config["values"]["harness"], { + "value": "openprose", "source": {"kind": "flag", "location": "--harness"}, + }) + self.assertEqual(doctor_config["candidates"]["harness"], [ + {"value": "agents-sdk", "source": {"kind": "default", "location": "built-in"}, "selected": False}, + {"value": "openprose", "source": {"kind": "flag", "location": "--harness"}, "selected": True}, + ]) + for key in ("model", "authProfile"): + self.assertEqual(doctor_config["values"][key], { + "value": None, "source": {"kind": "default", "location": "built-in"}, + }) + self.assertEqual(doctor_config["candidates"][key], [ + {"value": None, "source": {"kind": "default", "location": "built-in"}, "selected": True}, + ]) + self.assertEqual(doctor_config["runtime"], { + "transport": "hosted", "permissionMode": None, "authProfile": None, + "billingOwner": "openprose", "nativeLimits": None, "nativeOutputLimits": None, + }) + self.assertEqual(doctor_config["values"]["harness"]["value"], examples["doctor-report.json"]["selectedHarness"]) + for key in default_config: + if key not in ("values", "candidates", "runtime"): + self.assertEqual(default_config[key], doctor_config[key]) + for key in default_config["values"]: + if key not in ("harness", "model", "authProfile"): + self.assertEqual(default_config["values"][key], doctor_config["values"][key]) + self.assertEqual(default_config["candidates"][key], doctor_config["candidates"][key]) self.assertEqual( examples["doctor-report.json"]["problems"][0]["exitCode"], 10 ) @@ -898,7 +1007,10 @@ def test_all_case_manifests_validate_and_error_actions_are_frozen(self) -> None: if "errorCode" in case["expected"]: frozen = taxonomy[case["expected"]["errorCode"]] self.assertEqual(case["expected"]["exitCode"], frozen["exitCode"]) - self.assertEqual(case["expected"]["errorAction"], frozen["action"]) + if case["id"] in {"adapters.sdk-production-02", "adapters.sdk-production-03"}: + self.assertEqual(case["expected"]["errorAction"], load_json(FIXTURES / "adapters/sdk-production.json")["credentialAbsence"]["action"]) + else: + self.assertEqual(case["expected"]["errorAction"], frozen["action"]) ids = [case["id"] for case in cases] self.assertEqual(len(ids), len(set(ids))) self.assertTrue({"core.initial-help", "core.opaque-argv", "core.mock-success", "core.openprose-hosted-unavailable"}.issubset(ids)) diff --git a/docs/hosted-service-client.md b/docs/hosted-service-client.md index 28490bcb..c6dbedde 100644 --- a/docs/hosted-service-client.md +++ b/docs/hosted-service-client.md @@ -1,6 +1,6 @@ # The OpenProse service from the CLI -`prose cli` reaches the OpenProse hosted service for pricing, hosted runs, run records, saved programs, published results, jobs, the wallet and organizations. These commands are service operations. They do not select a model harness or run anything locally. `prose ` with the default `openprose` harness still reports `HOSTED_UNAVAILABLE`; use `prose cli run submit` to run hosted. +`prose cli` reaches the OpenProse hosted service for pricing, hosted runs, run records, saved programs, published results, jobs, the wallet and organizations. These commands are service operations. They do not select a model harness or run anything locally. Local programs default to the packaged Agents SDK using `OPENAI_API_KEY`. Explicitly selecting the `openprose` harness for a local program reports `HOSTED_UNAVAILABLE`; use `prose cli run submit` to run hosted. ```sh prose cli service triage --json diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index f3cf7c4c..20ee6d3d 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -26,3 +26,74 @@ IMP-097 owns canonical discovery, migration, explicit override editing, exact-co The generic Python harness transports opaque instructions/tasks, offers bounded public search/retrieval, local tools and fresh child contexts without interpreting contracts. Shared aggregate bounds cover parent and child model calls, tools, usage and cancellation. Environment filtering is not filesystem isolation; permissions and effective authority must be explicit and tested. Configure API client retries as zero for qualification, so callers own retry allocation. The workspace assignment allocates $50 total and twelve top-level attempts, one at a time, including children/evaluators/failures. Freeze a matrix and conservative reservation before live calls; unknown terminal accounting retains reservation. No merge or publication is authorized. Release readiness requires independent Rust/Bun/shared/installed-route and live artifact evidence, not process completion alone. + +## Frozen production interface (October 6, 2026) + +The production built-in is `agents-sdk`, model `gpt-6.1-sol`, authentication +`openai-api-key`, JSONL transport and `user-provider` billing. Preserve explicit +alternative harness bundles under IMP-097 precedence. SDK permissions are host +OS permissions, with no filesystem or network sandbox; no selected permission +flag may imply enforcement the helper does not implement. + +Resolve `prose-agents-sdk` beside the canonical CLI executable, following the +CLI's symlink to the installed payload before selecting its sibling. The helper +must be a regular executable; reject a sibling helper symlink. Relocating +both regular executables together preserves discovery. Never search PATH as a +production fallback or accidentally pair a CLI with another installation's +helper. Existing explicit developer/test seams remain separate from production. +Repair a missing/incompatible sibling by reinstalling the CLI through its current +installation route. Missing or blank `OPENAI_API_KEY` fails with +`HARNESS_NEEDS_AUTH`, actionable setup guidance, no inference and no credential, +model or harness fallback. Explanation remains pure even without either key or +helper; it resolves static configuration and never probes executable/credentials. + +Every production SDK report emits all eleven `nativeLimits` fields frozen in +`cli/shared/fixtures/adapters/sdk-production.json`. `maxAggregateRequests` and +`maxAggregateHostedWebCalls` equal resolved `maxTurns`; fixed defaults are 80 +function tools, 500000 observed total tokens, 256000 serialized request input +bytes, eight children and one child depth. The observed token limit stops the +next request, not an in-flight response. Historical four-field records remain +valid; partial mixtures of the old and expanded limits are invalid. + +Optional `usageObservation` and `modelIdentity` appear at the normalized result +root and under `runner-error.details` on failure. `modelIdentity.serviceTier` +records requested `default` and an allowlisted observed tier array. Never promote +these observations into authoritative `usage`: SDK usage stays +`{status:"unavailable"}` because incomplete/cancelled provider work and currency +remain unknown. Preserve observations already received on failure or cancellation without +inventing zeros for absent token counters. If stdio has already closed before an +observation arrives, retain unknown usage rather than infer provider completion. +The helper includes current usage on existing start/tool_call/tool_result events; +this does not add event types or make total run usage known. Provider IDs, exception bodies and raw unknown +fields are never retained. + +The ten usage-observation fields and six flattened token counters are closed by +`sdk-observation.schema.json`. Counts are nonnegative safe integers; +`outstandingProviderRequestCount` is always null and `totalRunUsageKnown` always +false. Retain a usage group only when its mandatory counters/constants are valid; +drop unknown properties and unknown/invalid token-counter entries. No invalid +mandatory counter may be replaced with a zero. Requested model identity comes +from the resolved invocation, rather than trusting an arbitrary event's request +string. Observed models are unique sorted ASCII public identifiers, at most 128 +characters and 128 identities; unsupported values are dropped. Omit the whole model group when `observed` is not an array, `serviceTier` is absent +or not an object, its `observed` is not an array, or its `requested` is not +`default`. Valid empty observed arrays retain honest unknown identity/tier state. +Within a structurally valid group, observed tiers are unique sorted members of +the frozen enum; unsupported entries are dropped. +Absent groups stay absent. Result/error observations must be bounded and must +never forward raw usage, provider request IDs or unrecognized values. + +For `harness use`, parse file, environment and flag layers before validating the +explicit selection target, then perform contextual model/auth semantic checks. +Malformed configuration files retain `CONFIG_INVALID` precedence. Missing required +explicit Prime/OMP selection options yield `INVOCATION_INVALID` before an ambient +old route can fail against the SDK default. This ordering does not suppress layers +or skip parsing their contents. + +Recipe platform support covers the existing four native release payloads. Archive +IDs remain `linux-arm64` and `linux-x64`; runtime admission uses +`linux-arm64-gnu` and `linux-x64-gnu`, requiring GLIBC 2.34 or newer. Musl and bare +unspecified Linux platform identities are not admitted. This +frozen contract is acceptance intent; it does not itself establish native build, +installed-route, live fulfillment or release qualification. The shared black-box +cases and independent product controls must prove each behavior before release. diff --git a/docs/sdk-budgets.md b/docs/sdk-budgets.md index 2b57d878..5af596e2 100644 --- a/docs/sdk-budgets.md +++ b/docs/sdk-budgets.md @@ -1,18 +1,28 @@ # Agents SDK execution budgets -Both runners accept these optional environment controls for `--harness agents-sdk`: +Both runners accept these optional configuration controls for `--harness agents-sdk`: -- `--native-max-turns N`: positive safe integer. Default 20. An SDK turn is one model invocation, including any tool calls; it is not a tool-call count. +- `--native-max-turns N`: positive safe integer. Default 20. The allowance is aggregate across parent and fresh child contexts. Function-tool calls have a separate limit. - `--native-timeout TIME`: positive duration using `ms`, `s`, `m`, or `h`. Default 180s. This controls the SDK harness's inner whole-run deadline and model request timeout. -- `--native-tool-timeout TIME`: positive duration with the same units. Default 30s. This controls each SDK shell action; an explicit `180s` forwards the existing helper option `--tool-timeout 180`. +- `--native-tool-timeout TIME`: positive duration with the same units. Default 30s. This controls each SDK shell or public-retrieval action; an explicit `180s` forwards the existing helper option `--tool-timeout 180`. `--timeout` remains the independent outer runner deadline. The first applicable limit wins. Setting an outer deadline longer than 180s does not silently extend the SDK deadline. For example, `--timeout 10m --native-timeout 8m --native-max-turns 60` grants up to 60 SDK turns and 480 seconds inside the 600-second outer limit. These flags do not guarantee completion, add retries, or change the task. Other harnesses reject explicit native budget options instead of silently ignoring them. Quoted TOML keys are `native_max_turns="60"` and `native_timeout="8m"`, and `native_tool_timeout="180s"`; environment variables are `PROSE_NATIVE_MAX_TURNS` `PROSE_NATIVE_TIMEOUT`, and `PROSE_NATIVE_TOOL_TIMEOUT`. Normal flag > environment > project > user precedence applies. Zero, negative, malformed, infinite or overflowing values are rejected. Omitted values preserve the existing native launch defaults. -SDK invocation, dry-run and result records expose `nativeLimits`: `maxTurns`, `timeoutSeconds`, `toolTimeoutSeconds` (default 30), and `maxOutputTokens` (12000 per model request). The generic SDK harness also reports its configured limits in start and error records. These are configured allowances, not measured consumption. The harness's direct `--max-turns`, `--timeout`, and `--tool-timeout` arguments remain available without either outer runner. +SDK invocation, dry-run, explanation and result records expose eleven `nativeLimits` fields: `maxTurns`, `timeoutSeconds`, `toolTimeoutSeconds` (default 30), `maxOutputTokens` (12000 per model request), `maxAggregateRequests`, `maxAggregateHostedWebCalls`, `maxAggregateFunctionTools` (80), `maxObservedTotalTokens` (500000), `maxRequestInputBytes` (256000), `maxChildren` (8) and `maxChildDepth` (1). Aggregate requests and hosted calls inherit `maxTurns`; hosted tools are capped at one per model request. Historical four-field diagnostics remain readable. The generic SDK harness also reports its configured limits in start and error records. These are configured allowances, not measured consumption. The harness's direct `--max-turns`, `--timeout`, and `--tool-timeout` arguments remain available without either outer runner. Normalized native failures carry a closed `nativeFailure.kind`: `max-turns`, `timeout`, or `execution`, with validated numeric limits and elapsed seconds when supplied. Arbitrary exception names and bodies are not passed through. A shell-action timeout is a tool result the agent may handle; an SDK run timeout terminates that run; outer cancellation has its existing separate error code. Cleanup signals do not replace the recorded native cause. A file written before a turn limit is reached remains an artifact, not proof of a completed invocation. Success still requires a native final record and successful process settlement. -Increasing the tool allowance does not increase the SDK parent or outer deadline. Omission forwards no extra tool-timeout argument; explicit `30s` reports its selected configuration source while preserving the same effective allowance. A tool timeout can return a tool error for the agent to handle, not guaranteed cancellation of every external effect. No delegation facility or OpenProse-specific behavior is added. +Increasing the tool allowance does not increase the SDK parent or outer deadline. Omission forwards no extra tool-timeout argument; explicit `30s` reports its selected configuration source while preserving the same effective allowance. A tool timeout can return a tool error for the agent to handle, not guaranteed cancellation of every external effect. `delegate(task)` starts an independent SDK Agent/Runner conversation with only the explicit task, the same opaque instructions and shared working directory. It inherits no parent conversation or tool history. Children execute serially and cannot delegate; their calls, tools and observed usage consume the shared invocation allowances. Files remain shared. Whole-run timeout and cancellation cover active parent/child contexts and ordinary tool process groups; detached descendants escaping a group are outside that guarantee. No OpenProse-specific interpretation is introduced. + +The request input-byte guard measures actual serialized instructions, history, +tools and output schema before transmission. The observed total-token ceiling +stops the next request; it cannot interrupt or price an in-flight request. Safe +`usageObservation` and `modelIdentity.serviceTier` diagnostics preserve completed +observations across success and failure, while normalized `usage` remains +unavailable. Missing counters remain absent, provider outstanding requests remain +unknown and `totalRunUsageKnown` remains false. These are execution allowances and +observations, never an authoritative currency/billing ceiling. Explicit API-client +and SDK retry settings disable transport and conversation-lock retries. diff --git a/docs/user-configuration.md b/docs/user-configuration.md index c9fdc36e..997cb830 100644 --- a/docs/user-configuration.md +++ b/docs/user-configuration.md @@ -2,7 +2,7 @@ IMP-097 design, frozen October 6, 2026. This document specifies the candidate behavior; implementation and qualification are recorded separately in the -workspace task. IMP-097 retains the `openprose` default harness. IMP-098 changes +workspace task. IMP-097 established this configuration interface with the prior harness default. IMP-098 changes that built-in to the packaged `agents-sdk` harness after qualification. ## Peer comparison and selected convention diff --git a/harnesses/agents-sdk/README.md b/harnesses/agents-sdk/README.md index 43494bc8..e8ddd049 100644 --- a/harnesses/agents-sdk/README.md +++ b/harnesses/agents-sdk/README.md @@ -40,3 +40,5 @@ python test_run.py Provider-free tests exercise actual local file effects, byte-bounded output capture, process-group timeout/cancellation, missing credentials before inference, safe auth/model failures, aggregate request/tool/observed-token limits, the actual transmitted request byte guard including tools and history, actual OpenAI HTTP request standard-tier selection and clone-preserved guards, safe observed-tier reporting, HTTP 409 single-call failure, private URL rejection, raw-usage deduplication and genuine SDK-loop child isolation. Mocked Responses transport verifies one hosted-tool allowance per request and no transport retries. Live artifact fulfillment and installed release readiness are separate qualification gates, retained by the owning task. The integration reuses the pinned SDK 0.22.2 public Agent/Runner/RunHooks/ModelSettings/WebSearchTool APIs. OpenAI Python and dotenv versions are pinned in [requirements.txt](requirements.txt); packaging retains transitive dependency identities. See [SDK budgets](../../docs/sdk-budgets.md) for the outer-runner budgets and historical [evidence](EVIDENCE.md) for earlier source-only observations. + +Safe completed-response usage observations accompany start and tool events as well as terminal events. The outer runner preserves observations it received before cancellation; closing stdout may prevent receiving the final cancellation snapshot. Missing observations stay unknown, and total billing is never claimed complete. diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index 64a21731..d20ceba8 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -283,6 +283,7 @@ async def run(args): observed_models = set() observed_tiers = set() def emit(kind, **data): + data.setdefault('usageObservation', observation.summary()) data['modelIdentity'] = {'requested': args.model, 'observed': sorted(observed_models), 'serviceTier': {'requested': 'default', 'observed': sorted(observed_tiers)}} print(json.dumps({'type': kind, 'event': kind, 'elapsed_seconds': round(time.monotonic()-start, 3), **data}), flush=True) diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index db26bb86..06f9e51c 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -247,6 +247,12 @@ async def tracked_shell(*args,**kwargs): self.assertEqual(records[-1]['error_type'],'CancelledError') self.assertEqual(records[-1]['usageObservation']['completedResponseCount'],2) self.assertEqual(records[-1]['usageObservation']['observedTokenTotals']['total_tokens'],10) + # These records precede cancellation: outer cleanup may close stdout + # before the terminal error is received, so retain completed usage now. + shell_call = next(record for record in records if record['type'] == 'tool_call' and record['name'] == 'execute_shell') + self.assertEqual(shell_call['usageObservation']['completedResponseCount'], 2) + self.assertEqual(shell_call['usageObservation']['observedTokenTotals']['total_tokens'], 10) + self.assertFalse(shell_call['usageObservation']['totalRunUsageKnown']) self.assertFalse(any(record['type']=='final' for record in records)) self.assertEqual(create.await_count,2) client.close.assert_awaited_once() From cbfe843fc4d9798cedc728dedd79e37c4258cc8c Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 19:46:03 -0400 Subject: [PATCH 06/64] Expect credential setup before SDK helper readiness --- cli/rust/crates/prose-cli/tests/cli.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cli/rust/crates/prose-cli/tests/cli.rs b/cli/rust/crates/prose-cli/tests/cli.rs index 82728129..6e51884d 100644 --- a/cli/rust/crates/prose-cli/tests/cli.rs +++ b/cli/rust/crates/prose-cli/tests/cli.rs @@ -4805,7 +4805,7 @@ fn runner_diagnostics_and_identity_stay_local() { assert_eq!(doctor_report["ready"], false); assert_eq!(doctor_report["selectedAdapterId"], "agents-sdk/jsonl"); assert_eq!(doctor_report["billingOwner"], "user-provider"); - assert_eq!(doctor_report["problems"][0]["code"], "HARNESS_UNAVAILABLE"); + assert_eq!(doctor_report["problems"][0]["code"], "HARNESS_NEEDS_AUTH"); assert_eq!( doctor_report["problems"][0]["details"]["fallbackAttempted"], false From 59f7c23736fd2bdfb01167255e2b286034b8534f Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:26:05 -0400 Subject: [PATCH 07/64] Preserve safe SDK setup recovery and executable service corrections --- cli/bun/src/adapters/protocols.ts | 11 +- cli/bun/src/adapters/sdk-limits.ts | 9 + cli/bun/src/core/service/manifest.ts | 57 +- cli/bun/test/args.test.ts | 43 + cli/bun/test/sdk-native-limits.test.ts | 2 +- cli/bun/test/sdk-production.test.ts | 15 + cli/ci/check_architecture.py | 1 + .../adapter-products/fake_live_harness.py | 5 +- .../cases/adapters/sdk-production-11.json | 166 +++ .../cases/adapters/sdk-production-12.json | 166 +++ .../cases/adapters/sdk-production-13.json | 166 +++ .../cases/adapters/sdk-production-14.json | 53 + .../cases/adapters/sdk-production-15.json | 53 + .../cases/adapters/sdk-production-16.json | 53 + .../cases/adapters/sdk-production-17.json | 165 +++ .../core/service-word-correction-01.json | 57 + .../core/service-word-correction-02.json | 63 ++ .../core/service-word-correction-03.json | 59 ++ .../core/service-word-correction-04.json | 59 ++ .../core/service-word-correction-05.json | 63 ++ .../core/service-word-correction-06.json | 58 ++ .../core/service-word-correction-07.json | 80 ++ cli/conformance/runner/run.py | 2 +- cli/conformance/runner/test_runner.py | 43 +- cli/protocol/OWNERSHIP.md | 16 + .../src/installed_adapters.rs | 29 +- .../prose-runner-core/src/invocation.rs | 196 +++- .../crates/prose-runner-core/src/runner.rs | 15 + .../prose-runner-core/src/service/mod.rs | 120 ++- .../fixtures/adapters/sdk-native-limits.json | 32 + .../fixtures/adapters/sdk-production.json | 127 ++- .../fixtures/service-word-corrections.json | 986 ++++++++++++++++++ cli/shared/schemas/native-failure.schema.json | 35 +- cli/shared/tests/test_contracts.py | 122 ++- docs/packaged-sdk-design.md | 8 + docs/sdk-budgets.md | 4 +- harnesses/agents-sdk/run.py | 23 +- harnesses/agents-sdk/test_run.py | 24 +- 38 files changed, 3138 insertions(+), 48 deletions(-) create mode 100644 cli/conformance/cases/adapters/sdk-production-11.json create mode 100644 cli/conformance/cases/adapters/sdk-production-12.json create mode 100644 cli/conformance/cases/adapters/sdk-production-13.json create mode 100644 cli/conformance/cases/adapters/sdk-production-14.json create mode 100644 cli/conformance/cases/adapters/sdk-production-15.json create mode 100644 cli/conformance/cases/adapters/sdk-production-16.json create mode 100644 cli/conformance/cases/adapters/sdk-production-17.json create mode 100644 cli/conformance/cases/core/service-word-correction-01.json create mode 100644 cli/conformance/cases/core/service-word-correction-02.json create mode 100644 cli/conformance/cases/core/service-word-correction-03.json create mode 100644 cli/conformance/cases/core/service-word-correction-04.json create mode 100644 cli/conformance/cases/core/service-word-correction-05.json create mode 100644 cli/conformance/cases/core/service-word-correction-06.json create mode 100644 cli/conformance/cases/core/service-word-correction-07.json create mode 100644 cli/shared/fixtures/service-word-corrections.json diff --git a/cli/bun/src/adapters/protocols.ts b/cli/bun/src/adapters/protocols.ts index f74a2eae..f1e3b8a9 100644 --- a/cli/bun/src/adapters/protocols.ts +++ b/cli/bun/src/adapters/protocols.ts @@ -1,5 +1,5 @@ import {PrimeDrain} from "./prime-drain"; -import {sdkNativeFailure} from "./sdk-limits"; +import {sdkNativeFailure,sdkSetupFailureAction} from "./sdk-limits"; import {sdkObservations,type SdkObservations} from "./sdk-observation"; import {hasFreshClaudeResult} from "./claude-shutdown"; import { isDeepStrictEqual } from "node:util"; @@ -137,7 +137,14 @@ class AgentsSdkProtocol extends InstalledProtocol { return started; } if (!this.started) malformed("SDK event before start."); - if (record.type === "error") {this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)};throw failure("HARNESS_FAILED", {nativeFailure:sdkNativeFailure(record),...this.sdkObservations});} + if (record.type === "error") { + this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)}; + const nativeFailure=sdkNativeFailure(record),problem=failure("HARNESS_FAILED",{nativeFailure,...this.sdkObservations}); + const action=sdkSetupFailureAction(nativeFailure); + if(action===undefined)throw problem; + const {schema:_schema,...shape}=problem.toJSON(); + throw new RunnerFailure({...shape,action}); + } if (record.type === "tool_call" || record.type === "tool_result") { if (typeof record.name !== "string") malformed("SDK tool identity missing."); this.sdkObservations={...this.sdkObservations,...sdkObservations(record,this.requestedModel)}; diff --git a/cli/bun/src/adapters/sdk-limits.ts b/cli/bun/src/adapters/sdk-limits.ts index e4d6be1e..f2791446 100644 --- a/cli/bun/src/adapters/sdk-limits.ts +++ b/cli/bun/src/adapters/sdk-limits.ts @@ -10,8 +10,17 @@ export function nativeLimits(input:Selection):Record|undefined { return {maxTurns:turns,timeoutSeconds:timeoutMs/1000,toolTimeoutSeconds:toolTimeoutMs/1000,maxOutputTokens:12000,maxAggregateRequests:turns,maxAggregateHostedWebCalls:turns,maxAggregateFunctionTools:80,maxObservedTotalTokens:500000,maxRequestInputBytes:256000,maxChildren:8,maxChildDepth:1}; } export function nativeLimitsArgv(input:Selection):string[]{const limits=nativeLimits(input);if(!limits)return [];return [...(input.nativeMaxTurns===undefined?[]:["--max-turns",String(limits.maxTurns)]),...(input.nativeTimeout===undefined?[]:["--timeout",String(limits.timeoutSeconds)]),...(input.nativeToolTimeout===undefined?[]:["--tool-timeout",String(limits.toolTimeoutSeconds)])];} +const setupActions:Record={ + "credential-or-permission":"Check OPENAI_API_KEY and this account's permission to use the configured model, then retry. No fallback was selected.", + "model-unavailable":"Choose a model available to this OpenAI account or restore access to the configured model, then retry. No fallback was selected.", + "local-input":"Check the working directory, instruction and credential file paths, file permissions and configured input-byte limit, then retry.", +}; +export function sdkSetupFailureAction(nativeFailure:Record):string|undefined { + return nativeFailure.kind==="setup"&&typeof nativeFailure.setupReason==="string"&&Object.hasOwn(setupActions,nativeFailure.setupReason)?setupActions[nativeFailure.setupReason]:undefined; +} export function sdkNativeFailure(record:Record):Record{ const output:Record={kind:record.error_type==="MaxTurnsExceeded"?"max-turns":record.error_type==="TimeoutError"?"timeout":"execution"}; + if(record.error_type==="SetupError"&&typeof record.setup_reason==="string"&&Object.hasOwn(setupActions,record.setup_reason)){output.kind="setup";output.setupReason=record.setup_reason;} const elapsed=record.elapsed_seconds;if(typeof elapsed==="number"&&Number.isFinite(elapsed)&&elapsed>=0)output.elapsedSeconds=elapsed; const limits=record.limits; if(limits!==null&&typeof limits==="object"&&!Array.isArray(limits)){ diff --git a/cli/bun/src/core/service/manifest.ts b/cli/bun/src/core/service/manifest.ts index dbf00c5a..3416aff9 100644 --- a/cli/bun/src/core/service/manifest.ts +++ b/cli/bun/src/core/service/manifest.ts @@ -1238,7 +1238,48 @@ function loneCommand(noun: string, verb: string | undefined): [string[], number] * language commands: the caller forwards them unless the hosted harness * would refuse. */ -function serviceWord(prefix: readonly string[], rest: readonly string[]): CliRedirect | undefined { +function serviceDisplayPrefix(prefix:readonly string[]):string[]{ + const output:string[]=[]; + for(let at=0;attoken==="--help"||token==="-h"))return argv; + const operation=manifest.operations.find(item=>item.id==="run.submit"); + const fileIndex=firstPositional(operation,tail),cwd=prefixValue(nativePrefix,"--cwd"),model=prefixValue(nativePrefix,"--model"); + if(fileIndex!==undefined){ + const file=tail[fileIndex]!; + // Keep stdin and lexical path components intact; no filesystem lookup. + if(cwd!==undefined&&file!=="-"&&!file.startsWith("/")&&!/^[A-Za-z]:[\\/]/.test(file))tail[fileIndex]=cwd+(cwd.endsWith("/")||cwd.endsWith("\\")?"":"/")+file; + } + if(model!==undefined&&!tail.slice(0,stop).some(token=>token==="--model"||token.startsWith("--model="))){ + // A FILE beyond -- is positional-only; converted service options belong + // before that delimiter (and before an existing preview flag). + const preview=tail.slice(0,stop).indexOf("--preview"); + const modelAt=fileIndex===undefined?0:fileIndex>=stop?(preview<0?stop:preview):fileIndex+1; + tail.splice(modelAt,0,"--model",model); + } + if(nativePrefix.includes("--dry-run")){ + const separator=tail.includes("--")?tail.indexOf("--"):tail.length; + if(!tail.slice(0,separator).includes("--preview"))tail.splice(separator,0,"--preview"); + } + return [...argv.slice(0,start),...tail]; +} +function serviceWord(nativePrefix: readonly string[], rest: readonly string[]): CliRedirect | undefined { + const prefix=serviceDisplayPrefix(nativePrefix); const [noun, next] = rest; if (noun === undefined) return undefined; const positions = new Positions([...prefix, "cli", ...rest], rest); @@ -1254,11 +1295,11 @@ function serviceWord(prefix: readonly string[], rest: readonly string[]): CliRed const action = words.length === 0 ? "Show the runner help: `{command}`" : `Show the help of \`cli ${words.join(" ")}\`: \`{command}\``; meant = { rejected: reject("", { kind: "argv", action, argv }), typed: 1 + words.length }; operands = ["help"]; - } else if (noun === "run" && next !== undefined && !next.startsWith("-")) { + } else if (noun === "run" && next !== undefined && (!next.startsWith("-")||next==="-")) { const tail = rest.slice(2); const stop = tail.includes("--") ? tail.indexOf("--") : tail.length; const head = tail.slice(0, stop); - const argv = [...prefix, "cli", "run", "submit", next, ...head, ...(head.includes("--preview") ? [] : ["--preview"]), ...tail.slice(stop)]; + const argv = serviceRunCorrection(nativePrefix,[...prefix, "cli", "run", "submit", next, ...head, ...(head.includes("--preview") ? [] : ["--preview"]), ...tail.slice(stop)]); meant = { rejected: settle(reject("", { kind: "argv", action: "Use `cli run submit`, previewing the run first: `{command}`", argv })), typed: 2 }; operands = [next]; } else { @@ -1383,12 +1424,20 @@ export function cliRedirect(args: readonly string[], index: number, globalKind: // `prose list jobs`: a verb before its group is never a lone synonym. const verbBefore = verbFirst(rest, new Positions([...argv, "cli", ...rest], rest)) !== undefined; const found = verbBefore ? undefined : loneCommand(noun, verb); - if (found === undefined) return replaced.length === 0 ? serviceWord(argv, rest) : undefined; + if (found === undefined) { + if(replaced.length===0)return serviceWord(argv,rest); + const corrected=[...argv,...rest]; + const command=invalidCommand(reject(reasons.join(" "),{kind:"argv",action:`Replace ${replaced.join(", ")}: \`{command}\``,argv:corrected}),rest); + return {argv:corrected,operands:[],command,language:true,hintOnly:true}; + } language = isLanguageCommand(noun); [command, typed] = found; synonym = command.join(" "); } + const nativePrefix=[...argv]; + argv.splice(0,argv.length,...serviceDisplayPrefix(argv)); argv.push("cli", ...command, ...rest.slice(typed)); + argv.splice(0,argv.length,...serviceRunCorrection(nativePrefix,argv)); const kind = bareRun ? "alone runs nothing here: hosted runs are `prose cli run submit FILE`" : language ? LANGUAGE_WORD : synonym !== undefined ? "is not a command" : "is a service command"; diff --git a/cli/bun/test/args.test.ts b/cli/bun/test/args.test.ts index 79dc6293..c588ebdb 100644 --- a/cli/bun/test/args.test.ts +++ b/cli/bun/test/args.test.ts @@ -1,6 +1,49 @@ import { describe, expect, test } from "bun:test"; import { inferOutputMode, parseEntrypoint } from "../src/core/args"; import { readFileSync } from "node:fs"; +import corrections from "../../shared/fixtures/service-word-corrections.json"; +import {parseService} from "../src/core/service/manifest"; + +test("service word corrections follow through both pure parsers without changing opaque argv",()=>{ + for(const sample of corrections.cases){ + const original=[...sample.argv]; + if("rejectedOriginal" in sample&&sample.rejectedOriginal!==undefined){ + let caught:unknown;try{parseEntrypoint(original);}catch(error){caught=error;} + expect(caught).toMatchObject({code:sample.rejectedOriginal.code,details:{reason:sample.rejectedOriginal.reason}}); + expect(original).toEqual(sample.originalArgvPreserved);continue; + } + if(sample.follow===undefined||sample.suggestedArgv===undefined)throw new Error("complete follow oracle expected"); + const parsed=parseEntrypoint(original); + expect(original).toEqual(sample.originalArgvPreserved); + expect(parsed.kind).toBe("language"); + if(parsed.kind!=="language")throw new Error("language boundary expected"); + expect(parsed.redirect?.argv).toEqual(sample.suggestedArgv); + if("languageArgv" in sample)expect(parsed.argv).toEqual(sample.languageArgv); + const followed=parseEntrypoint(sample.suggestedArgv); + if(sample.follow.kind==="help"){ + expect(followed).toMatchObject({kind:"service",command:{kind:"help"}}); + const service=parseService(sample.suggestedArgv.slice(sample.suggestedArgv.indexOf("cli")+1),sample.suggestedArgv); + expect(service.kind).toBe("help"); + if(service.kind!=="help")throw new Error("specific service help expected"); + expect(service.text).toContain("cli run submit"); + expect({kind:"help",command:sample.suggestedArgv.slice(sample.suggestedArgv.indexOf("cli"),-1),globalDisplay:{output:followed.global.output??"human",noColor:followed.global.color===false,verbose:followed.global.verbose===true}} as unknown).toEqual(sample.follow); + } + else if(sample.follow.kind==="account")expect(followed).toMatchObject({kind:"operation",operation:"org-list"}); + else expect(String(followed.kind)).toBe(sample.follow.kind); + if(followed.kind==="language"){expect(followed.argv).toEqual((sample.follow as any).argv);expect(followed.global).toEqual((sample.follow as any).global);} + else if(sample.follow.kind==="account"&&followed.kind==="operation"){ + expect(parseService(["org","list"],sample.suggestedArgv).kind).toBe("invalid"); + expect({kind:"account",operation:followed.operation,manifestOperation:"org.list",arguments:{},options:{},flags:[],preview:false,globalDisplay:{output:followed.global.output??"human",noColor:followed.global.color===false,verbose:followed.global.verbose===true}} as unknown).toEqual(sample.follow); + } + else if(sample.follow.kind==="service"){ + const service=parseService(sample.suggestedArgv.slice(sample.suggestedArgv.indexOf("cli")+1),sample.suggestedArgv); + expect(service.kind).toBe("invoke"); + if(service.kind!=="invoke")throw new Error("executable service correction expected"); + expect(service.invocation.error).toBeUndefined(); + expect({kind:"service",operation:service.invocation.operation,arguments:Object.fromEntries(service.invocation.arguments),options:Object.fromEntries(service.invocation.options),flags:[...service.invocation.flags],preview:service.invocation.preview,globalDisplay:{output:followed.global.output??"human",noColor:followed.global.color===false,verbose:followed.global.verbose===true}} as unknown).toEqual(sample.follow); + } + } +}); const invocationAction = "Review the runner syntax with the --help option, place global options before cli, and retry the command."; diff --git a/cli/bun/test/sdk-native-limits.test.ts b/cli/bun/test/sdk-native-limits.test.ts index 60b97a5e..1eedb2c4 100644 --- a/cli/bun/test/sdk-native-limits.test.ts +++ b/cli/bun/test/sdk-native-limits.test.ts @@ -39,7 +39,7 @@ test("SDK actual launch forwards only explicit native budgets without changing t expect(base.argv).not.toContain("--max-turns");expect(base.argv).not.toContain("--timeout");expect(selected.argv.slice(1,5)).toEqual(fixture.override.argv);expect(selected.argv.slice(5)).toEqual(base.argv.slice(1)); }); test("SDK native errors preserve only closed diagnostic data and never settle",()=>{ - for(const sample of fixture.errorCases){const p=installedProtocol("agents-sdk/jsonl","0.1.0","fixture");p.accept({type:"start",model:"fixture",cwd:"/tmp"});let caught:any;try{p.accept({type:"error",error_type:sample.error_type,elapsed_seconds:1.5,limits:fixture.historicalLimits});}catch(e){caught=e;}expect(caught.details.nativeFailure).toEqual({kind:sample.kind,elapsedSeconds:1.5,limits:fixture.historicalLimits});expect(p.terminalEventObserved).toBe(false);} + for(const sample of fixture.errorCases){const p=installedProtocol("agents-sdk/jsonl","0.1.0","fixture");p.accept({type:"start",model:"fixture",cwd:"/tmp"});let caught:any;try{p.accept({type:"error",...sample,message:"native-secret-sentinel",elapsed_seconds:1.5,limits:fixture.historicalLimits});}catch(e){caught=e;}expect(caught.details.nativeFailure).toEqual({kind:sample.kind,...("setupReason" in sample?{setupReason:sample.setupReason}:{}),elapsedSeconds:1.5,limits:fixture.historicalLimits});expect(caught.code).toBe("HARNESS_FAILED");expect(caught.exitCode).toBe(22);if("setupReason" in sample)expect(caught.action).toBe(production.setupFailure.reasons[sample.setupReason as keyof typeof production.setupFailure.reasons]);expect(JSON.stringify(caught.toJSON())).not.toContain("native-secret-sentinel");expect(p.terminalEventObserved).toBe(false);} expect(sdkNativeFailure({error_type:"secret",elapsed_seconds:Infinity,limits:{...fixture.defaults,secret:"value"}})).toEqual({kind:"execution"}); }); diff --git a/cli/bun/test/sdk-production.test.ts b/cli/bun/test/sdk-production.test.ts index 81aa7cbf..d1b75e5d 100644 --- a/cli/bun/test/sdk-production.test.ts +++ b/cli/bun/test/sdk-production.test.ts @@ -65,6 +65,21 @@ test("SDK final and failure retain observations while usage remains a separate u expect(nativeLimits({harness:"agents-sdk"})).toEqual(oracle.nativeLimits);expect(sdkNativeFailure({error_type:"ExecutionError",limits:oracle.nativeLimits})).toMatchObject({limits:oracle.nativeLimits});expect(sdkNativeFailure({limits:{...oracle.nativeLimits,maxChildDepth:2}})).not.toHaveProperty("limits"); expect(()=>buildInstalledAdapterEnvironment({definition:installedAdapterDefinition("agents-sdk/jsonl"),ambient:{OPENAI_API_KEY:" "},credentialGroup:"openai-api-key"})).toThrow(); }); +test("SDK setup recovery admits only recognized native reasons and retains completed observations",()=>{ + for(const setupReason of [...Object.keys(oracle.setupFailure.reasons),"unknown-provider-body",null,{},"toString"]){ + const protocol=installedProtocol("agents-sdk/jsonl","0.1.0","fixture",null,true,"gpt-6.1-sol"); + protocol.accept({type:"start",model:"gpt-6.1-sol",cwd:"/tmp"}); + let caught:any; + try{protocol.accept({type:"error",error_type:"SetupError",setup_reason:setupReason,message:"sdk-provider-body-secret-sentinel",usageObservation:oracle.observation.failureUsage,modelIdentity:oracle.observation.modelIdentity,limits:oracle.nativeLimits});}catch(error){caught=error;} + expect(caught.code).toBe("HARNESS_FAILED");expect(caught.exitCode).toBe(22); + const recognized=typeof setupReason==="string"&&Object.hasOwn(oracle.setupFailure.reasons,setupReason); + expect(caught.details.nativeFailure).toEqual({kind:recognized?"setup":"execution",...(recognized?{setupReason}:{}),limits:oracle.nativeLimits}); + if(recognized)expect(caught.action).toBe(oracle.setupFailure.reasons[setupReason as keyof typeof oracle.setupFailure.reasons]); + expect(caught.details.usageObservation).toEqual(oracle.observation.failureUsage); + expect(caught.details.modelIdentity).toEqual(oracle.observation.modelIdentity); + expect(JSON.stringify(caught.toJSON())).not.toContain("sdk-provider-body-secret-sentinel"); + } +}); for(const scenario of ["success","failure","cancel"])test(`actual supervised SDK ${scenario} preserves completed observations`,async()=>{ const root=await workspace(),helper=join(root,"prose-agents-sdk"),image=await verifyRuntimeImage(sentinelFixtureImage); const observation=scenario==="success"?oracle.observation.completedUsage:oracle.observation.failureUsage; diff --git a/cli/ci/check_architecture.py b/cli/ci/check_architecture.py index 0ef2a425..a736eaaa 100644 --- a/cli/ci/check_architecture.py +++ b/cli/ci/check_architecture.py @@ -121,6 +121,7 @@ "cli/shared/fixtures/kernel-startup/release.json", "cli/shared/fixtures/native-output-budget.json", "cli/shared/fixtures/transport-diagnostics.json", + "cli/shared/fixtures/service-word-corrections.json", "cli/conformance/fake-harness/fake_harness.py", "cli/shared/capabilities/transport-limits.v1.json", "cli/shared/capabilities/adapters/oracle.v1.json", diff --git a/cli/conformance/adversarial/adapter-products/fake_live_harness.py b/cli/conformance/adversarial/adapter-products/fake_live_harness.py index b05ccab6..567faf66 100644 --- a/cli/conformance/adversarial/adapter-products/fake_live_harness.py +++ b/cli/conformance/adversarial/adapter-products/fake_live_harness.py @@ -321,7 +321,10 @@ def sdk_main(argv: list[str]) -> int: common = dict(usageObservation=control['usageObservation'], modelIdentity=control['modelIdentity']) sdk_emit(dict(type='start', model=model, cwd=str(workspace), limits=control['limits']), control['wireNumberLexemes']) if control['scenario'] == 'failure': - sdk_emit(dict(type='error', error_type='ExecutionError', message='safe fixture failure', limits=control['limits'], **common), control['wireNumberLexemes']) + record = dict(type='error', error_type=control['errorType'], message=control['errorMessage'], limits=control['limits'], **common) + if control['setupReason'] is not None: + record['setup_reason'] = control['setupReason'] + sdk_emit(record, control['wireNumberLexemes']) return 1 sdk_emit(dict(type='final', output=terminal, usage={'requests':3,'input_tokens':9,'output_tokens':6,'total_tokens':15}, **common), control['wireNumberLexemes']) return 0 diff --git a/cli/conformance/cases/adapters/sdk-production-11.json b/cli/conformance/cases/adapters/sdk-production-11.json new file mode 100644 index 00000000..abf42e0b --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-11.json @@ -0,0 +1,166 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-11", + "summary": "SDK setup recovery for credential-or-permission in JSON output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "setup", + "limits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "setupReason": "credential-or-permission" + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Check OPENAI_API_KEY and this account's permission to use the configured model, then retry. No fallback was selected." + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-12.json b/cli/conformance/cases/adapters/sdk-production-12.json new file mode 100644 index 00000000..e5f8f182 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-12.json @@ -0,0 +1,166 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-12", + "summary": "SDK setup recovery for model-unavailable in JSON output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "setup", + "limits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "setupReason": "model-unavailable" + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Choose a model available to this OpenAI account or restore access to the configured model, then retry. No fallback was selected." + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-13.json b/cli/conformance/cases/adapters/sdk-production-13.json new file mode 100644 index 00000000..fac101f9 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-13.json @@ -0,0 +1,166 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-13", + "summary": "SDK setup recovery for local-input in JSON output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "setup", + "limits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "setupReason": "local-input" + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Check the working directory, instruction and credential file paths, file permissions and configured input-byte limit, then retry." + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-14.json b/cli/conformance/cases/adapters/sdk-production-14.json new file mode 100644 index 00000000..f331742c --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-14.json @@ -0,0 +1,53 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-14", + "summary": "SDK setup recovery for credential-or-permission in human output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "empty" + }, + "stderr": { + "kind": "contains", + "contains": [ + "HARNESS_FAILED", + "Action: Check OPENAI_API_KEY and this account's permission to use the configured model, then retry. No fallback was selected." + ] + }, + "startedHarness": true + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-15.json b/cli/conformance/cases/adapters/sdk-production-15.json new file mode 100644 index 00000000..9a503fb9 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-15.json @@ -0,0 +1,53 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-15", + "summary": "SDK setup recovery for model-unavailable in human output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "empty" + }, + "stderr": { + "kind": "contains", + "contains": [ + "HARNESS_FAILED", + "Action: Choose a model available to this OpenAI account or restore access to the configured model, then retry. No fallback was selected." + ] + }, + "startedHarness": true + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-16.json b/cli/conformance/cases/adapters/sdk-production-16.json new file mode 100644 index 00000000..3e58dc15 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-16.json @@ -0,0 +1,53 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-16", + "summary": "SDK setup recovery for local-input in human output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "empty" + }, + "stderr": { + "kind": "contains", + "contains": [ + "HARNESS_FAILED", + "Action: Check the working directory, instruction and credential file paths, file permissions and configured input-byte limit, then retry." + ] + }, + "startedHarness": true + } +} diff --git a/cli/conformance/cases/adapters/sdk-production-17.json b/cli/conformance/cases/adapters/sdk-production-17.json new file mode 100644 index 00000000..141c3bf2 --- /dev/null +++ b/cli/conformance/cases/adapters/sdk-production-17.json @@ -0,0 +1,165 @@ +{ + "schema": "openprose.runner-case/1", + "id": "adapters.sdk-production-17", + "summary": "SDK setup recovery for unknown reason in JSON output suppresses raw provider bodies.", + "phase": 7, + "tags": [ + "adapter", + "agents-sdk", + "production", + "provider-free", + "differential" + ], + "platforms": [ + "unix" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied", + "installedAdapter": { + "adapterId": "agents-sdk/jsonl", + "sdkScenario": "failure", + "sdkInstallation": "sibling" + } + }, + "invocation": { + "argv": [ + "--output", + "json", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": { + "HOME": "{{WORKSPACE}}/home", + "XDG_CONFIG_HOME": "{{WORKSPACE}}/config", + "OPENAI_API_KEY": "fixture-provider-free-openai-key" + } + }, + "expected": { + "exitCode": 22, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": true, + "resultMatches": { + "adapter": { + "id": "agents-sdk/jsonl" + }, + "transport": "jsonl", + "usage": { + "status": "unavailable" + }, + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + }, + "billing": { + "owner": "user-provider", + "authCategory": "harness-managed" + }, + "runnerExitCode": 22, + "error": { + "details": { + "nativeFailure": { + "kind": "execution", + "limits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + } + }, + "usageObservation": { + "source": "sdk_completed_response_raw_usage", + "aggregationScope": "unique_completed_responses_in_parent_and_children", + "startedCallCount": 4, + "completedResponseCount": 3, + "duplicateResponseCallbackCount": 0, + "outstandingCallCount": 1, + "outstandingProviderRequestCount": null, + "observedTokenTotals": { + "input_tokens": 9, + "output_tokens": 6, + "total_tokens": 15 + }, + "fieldResponseCounts": { + "input_tokens": 3, + "output_tokens": 3, + "total_tokens": 3 + }, + "totalRunUsageKnown": false + }, + "modelIdentity": { + "requested": "gpt-6.1-sol", + "observed": [ + "gpt-6.1-sol" + ], + "serviceTier": { + "requested": "default", + "observed": [ + "default" + ] + } + } + } + } + }, + "errorCode": "HARNESS_FAILED", + "errorAction": "Inspect the harness diagnostic on stderr and invoke the `cli doctor` runner operation." + } +} diff --git a/cli/conformance/cases/core/service-word-correction-01.json b/cli/conformance/cases/core/service-word-correction-01.json new file mode 100644 index 00000000..ecea1816 --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-01.json @@ -0,0 +1,57 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-01", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-02.json b/cli/conformance/cases/core/service-word-correction-02.json new file mode 100644 index 00000000..7ea031b0 --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-02.json @@ -0,0 +1,63 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-02", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output=json", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "hosted", + "--timeout", + "3m", + "run", + "input.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output=json", + "--no-color", + "--verbose", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-03.json b/cli/conformance/cases/core/service-word-correction-03.json new file mode 100644 index 00000000..5a4a02de --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-03.json @@ -0,0 +1,59 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-03", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "run", + "space file.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "./space file.prose.md", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-04.json b/cli/conformance/cases/core/service-word-correction-04.json new file mode 100644 index 00000000..a9b7c311 --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-04.json @@ -0,0 +1,59 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-04", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "{{WORKSPACE}}", + "run", + "../space file.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "{{WORKSPACE}}/../space file.prose.md", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-05.json b/cli/conformance/cases/core/service-word-correction-05.json new file mode 100644 index 00000000..8b6ac939 --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-05.json @@ -0,0 +1,63 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-05", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "--model", + "service-model", + "run", + "/programs/absolute.prose.md" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "/programs/absolute.prose.md", + "--model", + "service-model", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-06.json b/cli/conformance/cases/core/service-word-correction-06.json new file mode 100644 index 00000000..1d6a5327 --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-06.json @@ -0,0 +1,58 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-06", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md", + "--preview" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 10, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "error": { + "code": "HOSTED_UNAVAILABLE", + "details": { + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ] + } + } + } + } +} diff --git a/cli/conformance/cases/core/service-word-correction-07.json b/cli/conformance/cases/core/service-word-correction-07.json new file mode 100644 index 00000000..9fe6c2bc --- /dev/null +++ b/cli/conformance/cases/core/service-word-correction-07.json @@ -0,0 +1,80 @@ +{ + "schema": "openprose.runner-case/1", + "id": "core.service-word-correction-07", + "summary": "Pure service-word correction preserves display and program identity without native option leakage.", + "phase": 7, + "tags": [ + "cli", + "correction", + "provider-free", + "differential" + ], + "controls": { + "fixedInvocationId": "fixture-invocation-0001", + "fixedTimestamp": "2025-01-01T00:00:00Z", + "fixedMonotonicMs": 0, + "network": "denied" + }, + "invocation": { + "argv": [ + "--output", + "json", + "--dry-run", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "rpc", + "--cwd", + "missing/../space dir", + "--model", + "ignored-model", + "--auth-profile", + "ignored-auth", + "--permission-mode", + "default", + "--native-max-turns", + "3", + "--native-timeout", + "2m", + "--native-tool-timeout", + "5s", + "--native-output-bytes", + "1048576", + "--native-log", + "private-log", + "org", + "list" + ], + "cwd": "{{WORKSPACE}}", + "environment": {} + }, + "expected": { + "exitCode": 2, + "stdout": { + "kind": "json", + "schema": "openprose.service-operation/1" + }, + "stderr": { + "kind": "empty" + }, + "startedHarness": false, + "resultMatches": { + "problem": { + "code": "INVOCATION_INVALID", + "details": { + "suggestedArgv": [ + "--output", + "json", + "--no-color", + "--verbose", + "cli", + "org", + "list" + ] + } + } + } + } +} diff --git a/cli/conformance/runner/run.py b/cli/conformance/runner/run.py index ed6bb35e..55acc6ea 100755 --- a/cli/conformance/runner/run.py +++ b/cli/conformance/runner/run.py @@ -1673,7 +1673,7 @@ def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: identity['serviceTier']['requested'] = 'sdk-unrecognized-secret-sentinel' identity['serviceTier']['observed'].append('sdk-unrecognized-secret-sentinel') limits = {**oracle['nativeLimits'], **fixture.get('rawLimitsOverrides', {})} - setup = dict(scenario=fixture['scenario'],expectedHelper=str(helper.resolve()),model=oracle['defaults']['model'],limits=limits,usageObservation=usage,modelIdentity=identity,wireNumberLexemes=fixture.get('wireNumberLexemes', {})) + setup = dict(scenario=fixture['scenario'],expectedHelper=str(helper.resolve()),model=oracle['defaults']['model'],limits=limits,usageObservation=usage,modelIdentity=identity,wireNumberLexemes=fixture.get('wireNumberLexemes', {}),errorType=fixture.get('rawErrorType','ExecutionError'),setupReason=fixture.get('rawSetupReason'),errorMessage=fixture.get('rawMessage','safe fixture failure')) (workspace / '.sdk-compatibility-fixture.json').write_text(json.dumps(setup), encoding='utf-8') execution = Product(product.name, product.executable, product.runner_name, execution_executable=target) diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index 8acedfc5..0dd80bf5 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -109,10 +109,10 @@ def test_host_oracle_rejects_unsupported_host_without_skipping_case(self): wanted = runner.expected_for_host(case, "linux", "aarch64") self.assertEqual("arm64", wanted["resultMatches"]["error"]["details"]["hostArchitecture"]) self.assertTrue(runner.deep_subset({"terminal": {"classification": "success"}}, wanted["resultMatches"])) - self.assertEqual(86, len(list(runner.case_paths(7, set())))) + self.assertEqual(100, len(list(runner.case_paths(7, set())))) def test_sdk_installation_fixture_executes_canonical_clone_and_keeps_helper_off_path(self): - for number in range(1,11): + for number in range(1,18): case = json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: root = Path(temporary) @@ -145,7 +145,7 @@ def test_sdk_fixture_refuses_unfrozen_controls_and_interpreted_candidates(self): def test_sdk_fake_wire_is_provider_free_and_reuses_frozen_observations(self): oracle = json.loads((runner.CLI/'shared/fixtures/adapters/sdk-production.json').read_text()) - for number in (4,5,6,9,10): + for number in (4,5,6,9,10,11,12,13,14,15,16,17): case = json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) with self.subTest(case=case['id']), tempfile.TemporaryDirectory() as temporary: root=Path(temporary); environment,workspace=runner.product_roots(root,'fixture') @@ -155,15 +155,21 @@ def test_sdk_fake_wire_is_provider_free_and_reuses_frozen_observations(self): image=workspace/'instructions';image.write_bytes(b'OPENPROSE_SENTINEL_IMAGE_V1') argv=[sys.executable,str(helper),'--cwd',str(workspace),'--instructions',str(image),'--model','gpt-6.1-sol','--prompt',json.dumps({'argv':['prose','run','input.prose.md']})] result=runner.run_owned_process(argv,cwd=workspace,environment=runner.hermetic_environment(environment,{'PATH':str(poison)}),timeout_seconds=3) - self.assertEqual(1 if number in (5,9,10) else 0,result.exit_code,result.stderr) + self.assertEqual(1 if number in (5,9,10,11,12,13,14,15,16,17) else 0,result.exit_code,result.stderr) records=[json.loads(line) for line in result.stdout.splitlines()] self.assertEqual('start',records[0]['type']) - self.assertEqual('error' if number in (5,9,10) else 'final',records[-1]['type']) + self.assertEqual('error' if number in (5,9,10,11,12,13,14,15,16,17) else 'final',records[-1]['type']) if number!=6: - self.assertEqual(oracle['observation']['failureUsage' if number in (5,9,10) else 'completedUsage'],records[-1]['usageObservation']) + self.assertEqual(oracle['observation']['failureUsage' if number in (5,9,10,11,12,13,14,15,16,17) else 'completedUsage'],records[-1]['usageObservation']) self.assertEqual(oracle['observation']['modelIdentity'],records[-1]['modelIdentity']) else: self.assertEqual('sdk-unrecognized-secret-sentinel',records[-1]['usageObservation']['unknown']) + if number >= 11: + control=next(row for row in oracle['cases'] if row['id']==case['id']) + self.assertEqual('SetupError',records[-1]['error_type']) + self.assertEqual(control['rawSetupReason'],records[-1]['setup_reason']) + self.assertIn(b'sdk-provider-body-secret-sentinel',result.stdout) + self.assertEqual([], result.stderr.splitlines()) if number == 5: self.assertIn(b'"maxTurns":20.0', result.stdout) self.assertIn(b'"maxOutputTokens":12000e0', result.stdout) @@ -179,7 +185,7 @@ def test_sdk_fake_wire_is_provider_free_and_reuses_frozen_observations(self): other = 'maxChildDepth' if number == 9 else 'maxChildren' self.assertEqual(oracle['nativeLimits'][other], records[-1]['limits'][other]) self.assertTrue((workspace/'.sdk-harness-started').is_file()) - if number not in (5,9,10): + if number not in (5,9,10,11,12,13,14,15,16,17): terminal=json.loads(records[-1]['output']) self.assertEqual('OPENPROSE_SENTINEL_TERMINAL_V1',terminal['marker']) @@ -198,6 +204,29 @@ def test_sdk_malformed_groups_must_be_absent_without_relaxing_usage_checks(self) observation.stdout=b'sdk-unrecognized-secret-sentinel' self.assertEqual(['SDK output exposed a forbidden raw-observation sentinel'],runner.validate_sdk_effects(observation)) + def test_sdk_setup_human_actions_and_body_suppression_are_independent_guards(self): + for number in range(11,18): + case=json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + with self.subTest(case=case['id']),tempfile.TemporaryDirectory() as temporary: + root=Path(temporary);environment,workspace=runner.product_roots(root,'fixture');runner.isolate_workspace(workspace) + _,fixture,_=runner.prepare_sdk_installation(runner.Product('fixture',Path('/usr/bin/true')),case,workspace,environment) + observation=runner.Observation(runner.Product('fixture',Path('/usr/bin/true')),case,22,b'',b'',workspace=workspace,sdk_fixture=fixture) + self.assertEqual([],runner.validate_sdk_effects(observation)) + for stream in ('stdout','stderr'): + setattr(observation,stream,b'sdk-provider-body-secret-sentinel') + self.assertIn('SDK output exposed a forbidden raw-observation sentinel',runner.validate_sdk_effects(observation)) + setattr(observation,stream,b'') + if 14<=number<=16: + (workspace/'.sdk-harness-started').touch() + observation.fake_observation=workspace/'.sdk-harness-started' + observation.stderr=('\n'.join(case['expected']['stderr']['contains'])+'\n').encode() + self.assertEqual([],runner.validate_output(observation,runner.ContractRegistry())) + observation.stderr=b'HARNESS_FAILED\nAction: wrong\n' + self.assertTrue(any('Action:' in error for error in runner.validate_output(observation,runner.ContractRegistry()))) + if number==17: + observation.parsed={'error':{'details':{'nativeFailure':{'kind':'execution','setupReason':'local-input'}}}} + self.assertIn('SDK malformed result field was retained: $.error.details.nativeFailure.setupReason',runner.validate_sdk_effects(observation)) + def test_sdk_invalid_child_limits_require_nested_absence_even_if_null(self): for number in (9,10): case=json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 20bdaa26..ead524d9 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1011,3 +1011,19 @@ IMP-097 root Clippy admission cleanup extends exact lease to `cli/rust/crates/pr IMP-097 root compiled test-seam reconciliation: `cli/rust/crates/prose-cli/tests/cli.rs` three saved-bundle byte expectations only, in `harness_use_persists_an_explicit_prime_bundle_from_suffix_or_prefix_without_spawning`, `codex_claude_and_openprose_switches_clear_stale_bundle_values`, and `assert_harness_guidance_executes_directly`. Full-admission failures show the frozen alphabetic auth_profile/harness/model publication order; preserve exact comparisons and no-spawn/unchanged checks. IMP-097 measured full-Bun admission repair: `configuration_contract` owns `cli/bun/test/cli.test.ts` expectedConfiguration location adaptation only. Shared complete report originates in a one-directory fixture; the real worktree fixture visits cwd, cli and repository root before its Git boundary. Retain independent complete comparisons and explicit expected filesystem context; no production/shared-schema changes or received-report copying. Root owns integration. + +## IMP-098 setup recovery contract follow-up + +`conformance_plumbing` owns `cli/shared/schemas/native-failure.schema.json`, `cli/shared/fixtures/adapters/sdk-production.json`, `cli/shared/fixtures/adapters/sdk-native-limits.json`, `cli/shared/tests/test_contracts.py`, new `cli/conformance/cases/adapters/sdk-production-11.json` through `sdk-production-17.json`, `cli/conformance/cases/case-manifest.schema.json`, `cli/conformance/runner/run.py`, `cli/conformance/runner/test_runner.py`, `cli/conformance/adversarial/adapter-products/fake_live_harness.py`, `docs/sdk-budgets.md`, and `docs/packaged-sdk-design.md` to freeze closed setup reason and fixed recovery actions before independent products. HARNESS_FAILED/22 remains; only allowlisted native reasons are projected, raw provider messages are excluded. Six JSON/human cases and independent unknown-reason control remain exact. No products/runtime/Git/provider calls; root owns later Rust/runtime integration. + +IMP-098 independent setup-recovery product wave: root retains `harnesses/agents-sdk/run.py`, `harnesses/agents-sdk/test_run.py`, Rust `installed_adapters.rs`, `runner.rs`, `sdk_observation.rs`, and exact SDK result tests for the frozen setup reason/actions. `configuration_contract` owns Bun `cli/bun/src/adapters/sdk-limits.ts`, `cli/bun/src/adapters/runner.ts`, `cli/bun/src/cli.ts`, `cli/bun/test/sdk-production.test.ts`, `cli/bun/test/sdk-native-limits.test.ts` for independent mapping/actions and source evidence. Preserve lifecycle, latest observations, fixed HARNESS_FAILED22 and arbitrary-message exclusion. No service correction changes in this wave, Git, heavy builds or paid calls. + +Setup action propagation additionally leases `cli/bun/src/adapters/protocols.ts` to `configuration_contract`; preserve primary cancellation/cleanup/protocol errors. Root owns `cli/rust/crates/prose-runner-core/src/installed_adapters.rs` and `runner.rs` plus runtime source/test explicitly for this same frozen wave. + +## IMP-098 executable service correction oracle + +`conformance_plumbing` owns new `cli/shared/fixtures/service-word-corrections.json`, `cli/shared/tests/test_contracts.py`, new `cli/conformance/cases/core/service-word-correction01.json` through `service-word-correction07.json`, and narrow `cli/conformance/cases/case-manifest.schema.json`, `cli/conformance/runner/run.py`, `cli/conformance/runner/test_runner.py` plumbing if required. Freeze exact service-supported display prefixes, supported model-option placement, lexical cwd/file identity, correct error boundary and parser-only follow validation before independent products. Original opaque language argv and alias-only canonicalization must remain intact. No content reads, execution, model calls, builds or Git. + +IMP-098 frozen service correction product wave: `configuration_contract` owns `cli/bun/src/core/service/manifest.ts`, `cli/bun/src/core/args.ts`, `cli/bun/test/service-framework.test.ts` and `cli/bun/test/args.test.ts` for independent eight-case pure parser-follow validation. Root owns Rust `cli/rust/crates/prose-runner-core/src/service/mod.rs` and `cli/rust/crates/prose-runner-core/src/invocation.rs` and exact new Rust parser tests for the same frozen oracle. No execution/client/network, opaque argv inspection, file reads or lexical path normalization. Preserve alias-only native globals. No Git/heavy builds/provider calls by agents. + +IMP-098 correction oracle registration: root owns `cli/ci/check_architecture.py` solely to admit the exact shared `service-word-corrections.json` fixture referenced by independent product tests. Retain all other import, dependency and opaque-input boundaries. diff --git a/cli/rust/crates/prose-runner-core/src/installed_adapters.rs b/cli/rust/crates/prose-runner-core/src/installed_adapters.rs index 0f347df6..7fdca053 100644 --- a/cli/rust/crates/prose-runner-core/src/installed_adapters.rs +++ b/cli/rust/crates/prose-runner-core/src/installed_adapters.rs @@ -7526,6 +7526,21 @@ fn claude_correlated_shutdown_requires_closed_known_native_tasks() { } } +pub(crate) fn sdk_setup_action(reason: &str) -> Option<&'static str> { + match reason { + "credential-or-permission" => Some( + "Check OPENAI_API_KEY and this account's permission to use the configured model, then retry. No fallback was selected.", + ), + "model-unavailable" => Some( + "Choose a model available to this OpenAI account or restore access to the configured model, then retry. No fallback was selected.", + ), + "local-input" => Some( + "Check the working directory, instruction and credential file paths, file permissions and configured input-byte limit, then retry.", + ), + _ => None, + } +} + pub(crate) fn sdk_native_failure(record: &Value) -> Value { let kind = match record.get("error_type").and_then(Value::as_str) { Some("MaxTurnsExceeded") => "max-turns", @@ -7533,6 +7548,16 @@ pub(crate) fn sdk_native_failure(record: &Value) -> Value { _ => "execution", }; let mut result = json!({"kind":kind}); + if record.get("error_type").and_then(Value::as_str) == Some("SetupError") { + if let Some(reason) = record + .get("setup_reason") + .and_then(Value::as_str) + .filter(|reason| sdk_setup_action(reason).is_some()) + { + result["kind"] = json!("setup"); + result["setupReason"] = json!(reason); + } + } if let Some(n) = record .get("elapsed_seconds") .and_then(Value::as_f64) @@ -7600,9 +7625,11 @@ mod sdk_budget_diagnostic_tests { )) .unwrap(); for case in f["errorCases"].as_array().unwrap() { - let record = json!({"error_type":case["error_type"],"limits":f["defaults"],"elapsed_seconds":1.5}); + let record = json!({"error_type":case["error_type"],"setup_reason":case["setup_reason"],"message":"sdk-provider-body-secret-sentinel","limits":f["defaults"],"elapsed_seconds":1.5}); let d = sdk_native_failure(&record); assert_eq!(d["kind"], case["kind"]); + assert_eq!(d.get("setupReason"), case.get("setupReason")); + assert!(!d.to_string().contains("sdk-provider-body-secret-sentinel")); assert_eq!(d["limits"], f["defaults"]); assert_eq!(d["elapsedSeconds"], 1.5); assert!(!d.to_string().contains("Untrusted")); diff --git a/cli/rust/crates/prose-runner-core/src/invocation.rs b/cli/rust/crates/prose-runner-core/src/invocation.rs index 2e5ceb03..e2eb868b 100644 --- a/cli/rust/crates/prose-runner-core/src/invocation.rs +++ b/cli/rust/crates/prose-runner-core/src/invocation.rs @@ -710,6 +710,188 @@ fn known_runner_help_path(args: &[String]) -> bool { mod tests { use super::*; + #[test] + fn service_word_corrections_follow_the_shared_parser_oracle() { + let oracle: serde_json::Value = serde_json::from_str(include_str!( + "../../../../shared/fixtures/service-word-corrections.json" + )) + .unwrap(); + for case in oracle["cases"].as_array().unwrap() { + let words = |value: &serde_json::Value| { + value + .as_array() + .unwrap() + .iter() + .map(|word| word.as_str().unwrap().to_owned()) + .collect::>() + }; + let original = words(&case["argv"]); + if !case["rejectedOriginal"].is_null() { + let error = parse_invocation(original.clone()).unwrap_err(); + assert_eq!(error.code, crate::ErrorCode::InvocationInvalid); + assert_eq!( + error.details.as_ref().unwrap()["reason"], + case["rejectedOriginal"]["reason"] + ); + assert!(case["suggestedArgv"].is_null()); + assert!(case["follow"].is_null()); + assert_eq!(original, words(&case["originalArgvPreserved"])); + continue; + } + let parsed = parse_invocation(original.clone()).unwrap(); + let suggested = match parsed.action { + Action::Forward { + argv, service_hint, .. + } => { + if case["boundary"] == "alias-only" { + assert_eq!( + argv, + strings(&["prose", "--format", "json", "opaque-word", "opaque task"]) + ); + } else if !case["languageArgv"].is_null() { + assert_eq!(argv, words(&case["languageArgv"]), "{}", case["id"]); + } else { + let at = original + .iter() + .position(|word| word == "run" || word == "help") + .unwrap(); + let mut expected = vec!["prose".to_owned()]; + expected.extend_from_slice(&original[at..]); + assert_eq!(argv, expected, "{}", case["id"]); + } + service_hint.unwrap() + } + Action::Runner { + command: + RunnerCommand::Service(crate::service::ServiceCommand::Invalid(invalid)), + .. + } => { + let Some(crate::service::Correction::Argv { argv, .. }) = invalid.correction + else { + panic!("missing correction: {}", case["id"]); + }; + argv + } + other => panic!("unexpected action: {other:?}"), + }; + assert_eq!(original, words(&case["originalArgvPreserved"])); + assert_eq!(suggested, words(&case["suggestedArgv"]), "{}", case["id"]); + let follow = parse_invocation(suggested.clone()).unwrap(); + if case["follow"]["kind"] == "language" { + let Action::Forward { argv, .. } = follow.action else { + panic!("expected language"); + }; + assert_eq!(argv, words(&case["follow"]["argv"])); + assert_eq!( + follow.globals.harness.as_deref(), + case["follow"]["global"]["harness"].as_str() + ); + } else if case["follow"]["kind"] == "help" { + let Action::Runner { + command: RunnerCommand::Service(crate::service::ServiceCommand::Help(text)), + .. + } = follow.action + else { + panic!("expected service help"); + }; + assert!(!text.is_empty()); + assert_eq!( + follow.globals.no_color, + case["follow"]["globalDisplay"]["noColor"] + .as_bool() + .unwrap() + ); + assert_eq!( + follow.globals.verbose, + case["follow"]["globalDisplay"]["verbose"] + .as_bool() + .unwrap() + ); + } else if case["follow"]["kind"] == "account" { + assert!(matches!( + follow.action, + Action::Runner { + command: RunnerCommand::OrgList, + .. + } + )); + assert_eq!(case["follow"]["operation"], "org-list"); + assert_eq!(case["follow"]["manifestOperation"], "org.list"); + assert_eq!( + follow.globals.no_color, + case["follow"]["globalDisplay"]["noColor"] + .as_bool() + .unwrap() + ); + assert_eq!( + follow.globals.verbose, + case["follow"]["globalDisplay"]["verbose"] + .as_bool() + .unwrap() + ); + assert!(follow.globals.harness.is_none()); + assert!(follow.globals.cwd.is_none()); + assert!(follow.globals.model.is_none()); + assert!(!follow.globals.dry_run); + } else { + let invocation = match follow.action { + Action::Runner { + command: + RunnerCommand::Service(crate::service::ServiceCommand::Invoke(invocation)), + .. + } => invocation, + other => panic!( + "correction did not parse as a service operation: {} {other:?}", + case["id"] + ), + }; + assert!( + invocation.error.is_none(), + "{}: {:?}", + case["id"], + invocation.error + ); + assert_eq!( + invocation.operation, + case["follow"]["operation"].as_str().unwrap() + ); + assert_eq!( + serde_json::to_value(invocation.arguments).unwrap(), + case["follow"]["arguments"] + ); + assert_eq!( + serde_json::to_value(invocation.options).unwrap(), + case["follow"]["options"] + ); + assert_eq!( + serde_json::to_value(invocation.flags).unwrap(), + case["follow"]["flags"] + ); + assert_eq!( + invocation.preview, + case["follow"]["preview"].as_bool().unwrap() + ); + assert_eq!( + follow.globals.no_color, + case["follow"]["globalDisplay"]["noColor"] + .as_bool() + .unwrap() + ); + assert_eq!( + follow.globals.verbose, + case["follow"]["globalDisplay"]["verbose"] + .as_bool() + .unwrap() + ); + assert!(follow.globals.harness.is_none()); + assert!(follow.globals.cwd.is_none()); + assert!(follow.globals.model.is_none()); + assert!(!follow.globals.dry_run); + } + assert_eq!(follow.globals.output, Some(OutputMode::Json)); + } + } + /// The retired service-selection option, spelled so the public-surface /// scan does not match this negative test. const RETIRED_OPTION: &str = concat!("--service-", "environment"); @@ -1058,14 +1240,12 @@ mod tests { parsed.action, Action::Forward { argv: strings(&["prose", "run", "submit", "hello.prose"]), - service_hint: Some(strings(&[ - "--cwd", - &cwd, - "cli", - "run", - "submit", - "hello.prose" - ])), + service_hint: Some(vec![ + "cli".to_owned(), + "run".to_owned(), + "submit".to_owned(), + format!("{cwd}/hello.prose") + ]), hosted_rejection: None, hint_in_details_only: false, } diff --git a/cli/rust/crates/prose-runner-core/src/runner.rs b/cli/rust/crates/prose-runner-core/src/runner.rs index c6802b3f..412eb7b4 100644 --- a/cli/rust/crates/prose-runner-core/src/runner.rs +++ b/cli/rust/crates/prose-runner-core/src/runner.rs @@ -3002,6 +3002,21 @@ fn render_installed_failure( } } } + if adapter == installed_adapters::InstalledAdapter::AgentsSdkJsonl + && error.code == ErrorCode::HarnessFailed + { + if let Some(action) = error + .details + .as_ref() + .and_then(|details| details.get("nativeFailure")) + .filter(|diagnostic| diagnostic.get("kind").and_then(Value::as_str) == Some("setup")) + .and_then(|diagnostic| diagnostic.get("setupReason")) + .and_then(Value::as_str) + .and_then(installed_adapters::sdk_setup_action) + { + action.clone_into(&mut error.action); + } + } let timestamp = clock.now_rfc3339(); let invocation = installed_invocation(adapter, task, task_digest, invocation_id, config, image); let invocation_digest = sha256_hex(&serde_json::to_vec(&invocation).expect("invocation JSON")); diff --git a/cli/rust/crates/prose-runner-core/src/service/mod.rs b/cli/rust/crates/prose-runner-core/src/service/mod.rs index f41b6227..d2c79b99 100644 --- a/cli/rust/crates/prose-runner-core/src/service/mod.rs +++ b/cli/rust/crates/prose-runner-core/src/service/mod.rs @@ -1625,7 +1625,98 @@ fn lone_command<'a>(noun: &str, verb: Option<&'a str>) -> Option<(Vec<&'a str>, /// which is `cli run submit FILE --preview`. `prefix` is the runner globals /// before the words. `help` and `run` are language commands: the caller /// forwards them unless the hosted harness would refuse. -fn service_word(prefix: &[String], rest: &[String]) -> Option { +// Service corrections retain only display globals. Native execution settings +// belong to the original language invocation, never the suggested service one. +fn service_correction( + prefix: &[String], + argv: &[String], + global_kind: &impl Fn(&str) -> Option, +) -> Vec { + let mut display = Vec::new(); + let mut cwd = None; + let mut model = None; + let mut dry_run = false; + let mut at = 0; + while let Some(token) = prefix.get(at) { + let (name, inline) = token + .split_once('=') + .map_or((token.as_str(), None), |(name, value)| (name, Some(value))); + let takes_value = global_kind(name) == Some(true); + let width = if takes_value && inline.is_none() { + 2 + } else { + 1 + }; + let value = inline.or_else(|| { + takes_value + .then(|| prefix.get(at + 1)) + .flatten() + .map(String::as_str) + }); + match name { + "--output" | "--no-color" | "--verbose" => { + display.extend_from_slice(&prefix[at..(at + width).min(prefix.len())]); + } + "--cwd" => cwd = value.map(str::to_owned), + "--model" => model = value.map(str::to_owned), + "--dry-run" => dry_run = true, + _ => {} + } + at += width; + } + let mut command = argv[prefix.len()..].to_vec(); + let Some(operation) = command_operation(&command) else { + display.extend(command); + return display; + }; + if operation["id"] == "run.submit" { + let file_at = first_positional(Some(operation), &command[3..]).map(|at| at + 3); + if let (Some(cwd), Some(file)) = (cwd, file_at.and_then(|at| command.get_mut(at))) { + if file != "-" && !std::path::Path::new(file.as_str()).is_absolute() { + *file = std::path::Path::new(&cwd) + .join(file.as_str()) + .to_string_lossy() + .into_owned(); + } + } + } + let stop = command + .iter() + .position(|word| word == "--") + .unwrap_or(command.len()); + let mut additions = Vec::new(); + if let Some(model) = model { + if operation["options"] + .as_array() + .is_some_and(|options| options.iter().any(|option| option["name"] == "--model")) + && !command[..stop] + .iter() + .any(|word| word == "--model" || word.starts_with("--model=")) + { + additions.extend(["--model".to_owned(), model]); + } + } + if dry_run + && operation["preview"] == true + && !command[..stop].iter().any(|word| word == "--preview") + { + additions.push("--preview".to_owned()); + } + // Keep a generated preview last, after any converted model option. + let insertion = command[..stop] + .iter() + .position(|word| word == "--preview") + .unwrap_or(stop); + command.splice(insertion..insertion, additions); + display.extend(command); + display +} + +fn service_word( + prefix: &[String], + rest: &[String], + global_kind: &impl Fn(&str) -> Option, +) -> Option { let words = rest.iter().map(String::as_str).collect::>(); let noun = *words.first()?; let mut original = prefix.to_vec(); @@ -1663,7 +1754,7 @@ fn service_word(prefix: &[String], rest: &[String]) -> Option { vec!["help".to_owned()], ) } - ["run", file, tail @ ..] if !file.starts_with('-') => { + ["run", file, tail @ ..] if *file == "-" || !file.starts_with('-') => { let stop = tail .iter() .position(|token| *token == "--") @@ -1715,6 +1806,7 @@ fn service_word(prefix: &[String], rest: &[String]) -> Option { } Correction::Text(_) => return None, }; + let argv = service_correction(prefix, &argv, global_kind); let language = is_language_command(noun); let typed = words[..meant.typed].join(" "); let why = meant @@ -1909,9 +2001,27 @@ pub fn cli_redirect( .flatten() else { return if replaced.is_empty() { - service_word(&corrected, &args[at..]) + service_word(&corrected, &args[at..], &global_kind) } else { - None + let mut fixed = corrected.clone(); + fixed.extend_from_slice(&args[at..]); + let rejection = reject( + reasons.join(" "), + Correction::Argv { + action: format!("Replace {}: `{{command}}`", replaced.join(", ")), + argv: fixed.clone(), + }, + ); + Some(CliRedirect { + argv: fixed, + operands: Vec::new(), + command: ServiceCommand::Invalid(Box::new(invalid_command( + rejection, + &args[at..], + ))), + language: true, + hint_only: true, + }) }; }; language = is_language_command(word); @@ -1920,9 +2030,11 @@ pub fn cli_redirect( } [] => return None, }; + let service_prefix = corrected.clone(); corrected.push("cli".to_owned()); corrected.extend(command.iter().map(|word| (*word).to_owned())); corrected.extend_from_slice(&rest[typed..]); + corrected = service_correction(&service_prefix, &corrected, &global_kind); let spelled = words[..typed].join(" "); let kind = if words.as_slice() == ["run"] { "alone runs nothing here: hosted runs are `prose cli run submit FILE`" diff --git a/cli/shared/fixtures/adapters/sdk-native-limits.json b/cli/shared/fixtures/adapters/sdk-native-limits.json index d32c70c5..4c4ead68 100644 --- a/cli/shared/fixtures/adapters/sdk-native-limits.json +++ b/cli/shared/fixtures/adapters/sdk-native-limits.json @@ -63,6 +63,38 @@ { "error_type": "Untrusted secret text", "kind": "execution" + }, + { + "error_type": "SetupError", + "setup_reason": "credential-or-permission", + "kind": "setup", + "setupReason": "credential-or-permission" + }, + { + "error_type": "SetupError", + "setup_reason": "model-unavailable", + "kind": "setup", + "setupReason": "model-unavailable" + }, + { + "error_type": "SetupError", + "setup_reason": "local-input", + "kind": "setup", + "setupReason": "local-input" + }, + { + "error_type": "SetupError", + "setup_reason": "unknown-provider-body", + "kind": "execution" + }, + { + "error_type": "SetupError", + "kind": "execution" + }, + { + "error_type": "ExecutionError", + "setup_reason": "local-input", + "kind": "execution" } ], "toolOverrides": [ diff --git a/cli/shared/fixtures/adapters/sdk-production.json b/cli/shared/fixtures/adapters/sdk-production.json index 854c5705..d2529f25 100644 --- a/cli/shared/fixtures/adapters/sdk-production.json +++ b/cli/shared/fixtures/adapters/sdk-production.json @@ -312,6 +312,119 @@ "limits" ] ] + }, + { + "id": "adapters.sdk-production-11", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "credential-or-permission", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-12", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "model-unavailable", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-13", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "local-input", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-14", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "credential-or-permission", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-15", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "model-unavailable", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-16", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "local-input", + "rawMessage": "sdk-provider-body-secret-sentinel" + }, + { + "id": "adapters.sdk-production-17", + "scenario": "failure", + "installation": "sibling", + "userConfig": null, + "noProbe": false, + "noStart": false, + "outputAbsent": [ + "sdk-provider-body-secret-sentinel" + ], + "wireNumberLexemes": {}, + "rawErrorType": "SetupError", + "rawSetupReason": "sdk-provider-body-secret-sentinel", + "rawMessage": "sdk-provider-body-secret-sentinel", + "absentResultPaths": [ + [ + "error", + "details", + "nativeFailure", + "setupReason" + ] + ] } ], "rejectedModelIdentities": [ @@ -385,5 +498,17 @@ "parse-file-environment-flag-layers", "validate-explicit-selection-target", "validate-contextual-model-and-auth" - ] + ], + "setupFailure": { + "nativeErrorType": "SetupError", + "reasons": { + "credential-or-permission": "Check OPENAI_API_KEY and this account's permission to use the configured model, then retry. No fallback was selected.", + "model-unavailable": "Choose a model available to this OpenAI account or restore access to the configured model, then retry. No fallback was selected.", + "local-input": "Check the working directory, instruction and credential file paths, file permissions and configured input-byte limit, then retry." + }, + "outerCode": "HARNESS_FAILED", + "exitCode": 22, + "unknownReasonKind": "execution", + "rawMessageForwarded": false + } } diff --git a/cli/shared/fixtures/service-word-corrections.json b/cli/shared/fixtures/service-word-corrections.json new file mode 100644 index 00000000..45903e4c --- /dev/null +++ b/cli/shared/fixtures/service-word-corrections.json @@ -0,0 +1,986 @@ +{ + "schema": "openprose.service-word-corrections/1", + "rules": { + "prefixRetained": [ + "--output", + "--no-color", + "--verbose" + ], + "otherNativePrefixDropped": true, + "cwdFileJoin": "lexical-only-no-stat-no-read-no-canonicalization", + "absoluteFileUnchanged": true, + "modelMovedToSupportedServiceOption": true, + "originalLanguageArgvOpaque": true, + "aliasOnlyPreservesNativeGlobals": true, + "followValidation": "entrypoint-and-account-or-service-parser-only-no-execution-no-client-no-request", + "fileOperandSelection": "parse-first-positional-after-command-options-never-fixed-index", + "stdinFileUnchanged": true, + "helpAndSavedProgramWithoutFileUnchanged": true, + "cwdPrefixLastWins": true, + "existingServiceTailModelWinsWithoutDuplication": true, + "duplicateGlobalModelRejected": true, + "convertedOptionsBeforeDelimiter": true + }, + "cases": [ + { + "id": "service-word-correction-01", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "input.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + }, + "languageArgv": [ + "prose", + "run", + "input.prose.md" + ] + }, + { + "id": "service-word-correction-02", + "argv": [ + "--output=json", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "hosted", + "--timeout", + "3m", + "run", + "input.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output=json", + "--no-color", + "--verbose", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output=json", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "hosted", + "--timeout", + "3m", + "run", + "input.prose.md" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "input.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": true, + "verbose": true + } + }, + "languageArgv": [ + "prose", + "run", + "input.prose.md" + ] + }, + { + "id": "service-word-correction-03", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "run", + "space file.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "./space file.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "run", + "space file.prose.md" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "./space file.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + }, + "languageArgv": [ + "prose", + "run", + "space file.prose.md" + ] + }, + { + "id": "service-word-correction-04", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "/work", + "run", + "../space file.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "/work/../space file.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "/work", + "run", + "../space file.prose.md" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "/work/../space file.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + }, + "languageArgv": [ + "prose", + "run", + "../space file.prose.md" + ] + }, + { + "id": "service-word-correction-05", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "--model", + "service-model", + "run", + "/programs/absolute.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "/programs/absolute.prose.md", + "--model", + "service-model", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "--model", + "service-model", + "run", + "/programs/absolute.prose.md" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "/programs/absolute.prose.md" + ] + }, + "options": { + "--model": [ + "service-model" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + }, + "languageArgv": [ + "prose", + "run", + "/programs/absolute.prose.md" + ] + }, + { + "id": "service-word-correction-06", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md", + "--preview" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "input.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "run", + "input.prose.md", + "--preview" + ], + "boundary": "hosted", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "input.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + }, + "languageArgv": [ + "prose", + "run", + "input.prose.md", + "--preview" + ] + }, + { + "id": "service-word-correction-07", + "argv": [ + "--output", + "json", + "--dry-run", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "rpc", + "--cwd", + "missing/../space dir", + "--model", + "ignored-model", + "--auth-profile", + "ignored-auth", + "--permission-mode", + "default", + "--native-max-turns", + "3", + "--native-timeout", + "2m", + "--native-tool-timeout", + "5s", + "--native-output-bytes", + "1048576", + "--native-log", + "private-log", + "org", + "list" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "--no-color", + "--verbose", + "cli", + "org", + "list" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--dry-run", + "--no-color", + "--verbose", + "--harness", + "openprose", + "--transport", + "rpc", + "--cwd", + "missing/../space dir", + "--model", + "ignored-model", + "--auth-profile", + "ignored-auth", + "--permission-mode", + "default", + "--native-max-turns", + "3", + "--native-timeout", + "2m", + "--native-tool-timeout", + "5s", + "--native-output-bytes", + "1048576", + "--native-log", + "private-log", + "org", + "list" + ], + "boundary": "syntax", + "follow": { + "kind": "account", + "operation": "org-list", + "arguments": {}, + "options": {}, + "flags": [], + "preview": false, + "globalDisplay": { + "output": "json", + "noColor": true, + "verbose": true + }, + "manifestOperation": "org.list" + } + }, + { + "id": "service-word-correction-08", + "argv": [ + "--harness", + "claude", + "--format", + "json", + "opaque-word", + "opaque task" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--harness", + "claude", + "--output", + "json", + "opaque-word", + "opaque task" + ], + "originalArgvPreserved": [ + "--harness", + "claude", + "--format", + "json", + "opaque-word", + "opaque task" + ], + "boundary": "alias-only", + "follow": { + "kind": "language", + "argv": [ + "prose", + "opaque-word", + "opaque task" + ], + "global": { + "harness": "claude", + "output": "json" + } + } + }, + { + "id": "service-word-correction-09", + "argv": [ + "--output", + "json", + "--cwd", + "/work", + "help", + "run", + "submit" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "--help" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/work", + "help", + "run", + "submit" + ], + "boundary": "pure-help", + "follow": { + "kind": "help", + "command": [ + "cli", + "run", + "submit" + ], + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-10", + "argv": [ + "--output", + "json", + "--cwd", + "/work", + "run", + "submit", + "--from", + "saved-program", + "--preview" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "--from", + "saved-program", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/work", + "run", + "submit", + "--from", + "saved-program", + "--preview" + ], + "boundary": "pure-saved-program", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": {}, + "options": { + "--from": [ + "saved-program" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-11", + "argv": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "--model", + "service-model", + "--dry-run", + "run", + "submit", + "space file.prose.md", + "--preview" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "./space file.prose.md", + "--model", + "service-model", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--harness", + "openprose", + "--cwd", + "./", + "--model", + "service-model", + "--dry-run", + "run", + "submit", + "space file.prose.md", + "--preview" + ], + "boundary": "pure-explicit-file", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "./space file.prose.md" + ] + }, + "options": { + "--model": [ + "service-model" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-12", + "argv": [ + "--output", + "json", + "--cwd", + "/work", + "run", + "-" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "-", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/work", + "run", + "-" + ], + "boundary": "pure-stdin", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "-" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-13", + "argv": [ + "--output", + "json", + "--cwd", + "/old", + "--cwd", + "/work", + "run", + "input.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "/work/input.prose.md", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/old", + "--cwd", + "/work", + "run", + "input.prose.md" + ], + "boundary": "pure-prefix-precedence", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "/work/input.prose.md" + ] + }, + "options": {}, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-14", + "argv": [ + "--output", + "json", + "--model", + "old-model", + "run", + "input.prose.md", + "--model", + "newer-model" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "input.prose.md", + "--model", + "newer-model", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--model", + "old-model", + "run", + "input.prose.md", + "--model", + "newer-model" + ], + "boundary": "pure-prefix-precedence", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "input.prose.md" + ] + }, + "options": { + "--model": [ + "newer-model" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-15", + "argv": [ + "--output", + "json", + "--cwd", + "/work", + "--model", + "chosen-model", + "--dry-run", + "run", + "submit", + "input.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "/work/input.prose.md", + "--model", + "chosen-model", + "--preview" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/work", + "--model", + "chosen-model", + "--dry-run", + "run", + "submit", + "input.prose.md" + ], + "boundary": "pure-prefix-precedence", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "/work/input.prose.md" + ] + }, + "options": { + "--model": [ + "chosen-model" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + }, + { + "id": "service-word-correction-16", + "argv": [ + "--output", + "json", + "--model", + "old-model", + "--model", + "newer-model", + "run", + "input.prose.md" + ], + "processCwd": "/work", + "originalArgvPreserved": [ + "--output", + "json", + "--model", + "old-model", + "--model", + "newer-model", + "run", + "input.prose.md" + ], + "boundary": "original-parse-rejection", + "rejectedOriginal": { + "code": "INVOCATION_INVALID", + "reason": "runner option --model was specified more than once" + } + }, + { + "id": "service-word-correction-17", + "argv": [ + "--output", + "json", + "--cwd", + "/work", + "--model", + "chosen-model", + "--dry-run", + "run", + "submit", + "--", + "input.prose.md" + ], + "processCwd": "/work", + "suggestedArgv": [ + "--output", + "json", + "cli", + "run", + "submit", + "--model", + "chosen-model", + "--preview", + "--", + "/work/input.prose.md" + ], + "originalArgvPreserved": [ + "--output", + "json", + "--cwd", + "/work", + "--model", + "chosen-model", + "--dry-run", + "run", + "submit", + "--", + "input.prose.md" + ], + "languageArgv": [ + "prose", + "run", + "submit", + "--", + "input.prose.md" + ], + "boundary": "pure-delimiter", + "follow": { + "kind": "service", + "operation": "run.submit", + "arguments": { + "FILE": [ + "/work/input.prose.md" + ] + }, + "options": { + "--model": [ + "chosen-model" + ] + }, + "flags": [], + "preview": true, + "globalDisplay": { + "output": "json", + "noColor": false, + "verbose": false + } + } + } + ] +} diff --git a/cli/shared/schemas/native-failure.schema.json b/cli/shared/schemas/native-failure.schema.json index 0daf6d01..74c11ce0 100644 --- a/cli/shared/schemas/native-failure.schema.json +++ b/cli/shared/schemas/native-failure.schema.json @@ -11,7 +11,8 @@ "enum": [ "max-turns", "timeout", - "execution" + "execution", + "setup" ] }, "elapsedSeconds": { @@ -20,6 +21,36 @@ }, "limits": { "$ref": "native-limits.schema.json" + }, + "setupReason": { + "enum": [ + "credential-or-permission", + "model-unavailable", + "local-input" + ] + } + }, + "allOf": [ + { + "if": { + "properties": { + "kind": { + "const": "setup" + } + } + }, + "then": { + "required": [ + "setupReason" + ] + }, + "else": { + "not": { + "required": [ + "setupReason" + ] + } + } } - } + ] } diff --git a/cli/shared/tests/test_contracts.py b/cli/shared/tests/test_contracts.py index 77e62336..25970bdc 100755 --- a/cli/shared/tests/test_contracts.py +++ b/cli/shared/tests/test_contracts.py @@ -129,8 +129,11 @@ def test_native_sdk_limits_and_failures_are_closed(self): self.assert_valid('native-limits.schema.json',fixture['defaults']) self.assert_valid('native-limits.schema.json',fixture['override']['limits']) for case in fixture['errorCases']: - self.assert_valid('native-failure.schema.json',{'kind':case['kind'],'limits':fixture['defaults'],'elapsedSeconds':1.5}) - for bad in [{'kind':'arbitrary'}, {'kind':'execution','message':'secret'}, {'kind':'timeout','elapsedSeconds':-1}]: + value={'kind':case['kind'],'limits':fixture['defaults'],'elapsedSeconds':1.5} + if 'setupReason' in case: + value['setupReason']=case['setupReason'] + self.assert_valid('native-failure.schema.json',value) + for bad in [{'kind':'arbitrary'}, {'kind':'execution','message':'secret'}, {'kind':'timeout','elapsedSeconds':-1}, {'kind':'setup'}, {'kind':'setup','setupReason':'raw secret'}, {'kind':'execution','setupReason':'local-input'}]: self.assertTrue(list(self.validator('native-failure.schema.json').iter_errors(bad))) def test_sdk_production_limits_and_optional_observations_are_closed(self): @@ -193,7 +196,7 @@ def test_sdk_production_limits_and_optional_observations_are_closed(self): def test_sdk_production_cases_are_frozen_and_provider_free(self): oracle = load_json(FIXTURES/'adapters/sdk-production.json') - identifiers = [f'adapters.sdk-production-{number:02}' for number in range(1,11)] + identifiers = [f'adapters.sdk-production-{number:02}' for number in range(1,18)] self.assertEqual(identifiers, [case['id'] for case in oracle['cases']]) for control in oracle['cases']: case = load_json(CASES/'adapters'/f"{control['id'].split('.')[-1]}.json") @@ -203,6 +206,26 @@ def test_sdk_production_cases_are_frozen_and_provider_free(self): self.assertEqual(not control['noStart'], case['expected']['startedHarness']) if 'OPENAI_API_KEY' in case['invocation']['environment']: self.assertIn(case['invocation']['environment']['OPENAI_API_KEY'], (' ','fixture-provider-free-openai-key')) + setup=oracle['setupFailure'] + self.assertEqual('HARNESS_FAILED',setup['outerCode']) + self.assertEqual(22,setup['exitCode']) + self.assertFalse(setup['rawMessageForwarded']) + for number,reason in enumerate(setup['reasons'],11): + action=setup['reasons'][reason] + case=load_json(CASES/'adapters'/f'sdk-production-{number:02}.json') + self.assertEqual(action,case['expected']['errorAction']) + details=case['expected']['resultMatches']['error']['details'] + self.assertEqual('setup',details['nativeFailure']['kind']) + self.assertEqual(reason,details['nativeFailure']['setupReason']) + for group in ('usageObservation','modelIdentity'): + self.assertEqual(case['expected']['resultMatches'][group],details[group]) + human=load_json(CASES/'adapters'/f'sdk-production-{number+3:02}.json') + self.assertEqual(['run','input.prose.md'],human['invocation']['argv']) + self.assertEqual('empty',human['expected']['stdout']['kind']) + self.assertEqual(['HARNESS_FAILED','Action: '+action],human['expected']['stderr']['contains']) + unknown=load_json(CASES/'adapters/sdk-production-17.json') + self.assertEqual('execution',unknown['expected']['resultMatches']['error']['details']['nativeFailure']['kind']) + self.assertNotIn('setupReason',unknown['expected']['resultMatches']['error']['details']['nativeFailure']) recipe = load_json(SHARED/'capabilities/adapters/recipes/agents-sdk-jsonl.v1.json') self.assertEqual(oracle['discovery']['admissionPlatforms'], recipe['support']['platforms']) self.assertEqual(['darwin-arm64','darwin-x64','linux-arm64','linux-x64'], oracle['discovery']['platforms']) @@ -225,6 +248,96 @@ def test_optional_reporting_fields_are_named_and_closed(self): config['values']['permissionMode']['value']='arbitrary' self.assertTrue(list(self.validator('configuration-explanation.schema.json').iter_errors(config))) + def test_service_word_corrections_preserve_exact_route_and_pure_follow_contract(self): + oracle=load_json(FIXTURES/'service-word-corrections.json') + self.assertEqual({'schema','rules','cases'},set(oracle)) + self.assertEqual('openprose.service-word-corrections/1',oracle['schema']) + self.assertEqual(['--output','--no-color','--verbose'],oracle['rules']['prefixRetained']) + self.assertEqual('entrypoint-and-account-or-service-parser-only-no-execution-no-client-no-request',oracle['rules']['followValidation']) + self.assertEqual('lexical-only-no-stat-no-read-no-canonicalization',oracle['rules']['cwdFileJoin']) + self.assertEqual(17,len(oracle['cases'])) + operations={record['id']:record for record in load_json(SHARED/'service/operations.v1.json')['operations']} + def workspace_paths(value): + return [word.replace('/work','{{WORKSPACE}}') for word in value] + for number,control in enumerate(oracle['cases'],1): + self.assertEqual(f'service-word-correction-{number:02}',control['id']) + self.assertEqual(control['argv'],control['originalArgvPreserved']) + if number==16: + self.assertEqual('original-parse-rejection',control['boundary']) + self.assertEqual({'code':'INVOCATION_INVALID','reason':'runner option --model was specified more than once'},control['rejectedOriginal']) + self.assertNotIn('follow',control) + self.assertNotIn('suggestedArgv',control) + continue + if number==8: + self.assertEqual('alias-only',control['boundary']) + self.assertEqual(['--harness','claude','--output','json','opaque-word','opaque task'],control['suggestedArgv']) + self.assertEqual('language',control['follow']['kind']) + continue + if number>8: + command=control['suggestedArgv'][control['suggestedArgv'].index('cli'):] + self.assertEqual(['cli','run','submit'],command[:3]) + if number==9: + self.assertEqual(['--output','json','cli','run','submit','--help'],control['suggestedArgv']) + self.assertEqual('help',control['follow']['kind']) + self.assertEqual(['cli','run','submit'],control['follow']['command']) + else: + self.assertEqual('service',control['follow']['kind']) + self.assertEqual('run.submit',control['follow']['operation']) + self.assertEqual(1,command.count('--preview')) + if number==10: + self.assertEqual({},control['follow']['arguments']) + self.assertEqual({'--from':['saved-program']},control['follow']['options']) + self.assertEqual(['cli','run','submit','--from','saved-program','--preview'],command) + elif number==11: + self.assertEqual(['cli','run','submit','./space file.prose.md','--model','service-model','--preview'],command) + self.assertEqual({'FILE':['./space file.prose.md']},control['follow']['arguments']) + self.assertEqual({'--model':['service-model']},control['follow']['options']) + elif number in (13,14,15): + file='/work/input.prose.md' if number in (13,15) else 'input.prose.md' + model={14:'newer-model',15:'chosen-model'}.get(number) + self.assertEqual({'FILE':[file]},control['follow']['arguments']) + self.assertEqual({'--model':[model]} if model else {},control['follow']['options']) + self.assertEqual(['cli','run','submit',file]+(['--model',model] if model else [])+['--preview'],command) + self.assertEqual(1 if model else 0,command.count('--model')) + elif number==17: + self.assertEqual(['cli','run','submit','--model','chosen-model','--preview','--','/work/input.prose.md'],command) + self.assertEqual({'FILE':['/work/input.prose.md']},control['follow']['arguments']) + self.assertEqual({'--model':['chosen-model']},control['follow']['options']) + self.assertEqual(['prose','run','submit','--','input.prose.md'],control['languageArgv']) + self.assertLess(command.index('--model'),command.index('--')) + self.assertLess(command.index('--preview'),command.index('--')) + elif number==12: + self.assertEqual(['cli','run','submit','-','--preview'],command) + self.assertEqual({'FILE':['-']},control['follow']['arguments']) + continue + case=load_json(CASES/'core'/f"{control['id']}.json") + self.assertEqual(workspace_paths(control['argv']),case['invocation']['argv']) + surface='problem' if control['boundary']=='syntax' else 'error' + expected=case['expected']['resultMatches'][surface] + self.assertEqual(workspace_paths(control['suggestedArgv']),expected['details']['suggestedArgv']) + self.assertEqual('INVOCATION_INVALID' if surface=='problem' else 'HOSTED_UNAVAILABLE',expected['code']) + self.assertEqual(2 if surface=='problem' else 10,case['expected']['exitCode']) + follow=control['follow'] + if number==7: + # Organization listing belongs to the account entrypoint; + # parseService covers the hosted service cases above only. + self.assertEqual('account',follow['kind']) + self.assertEqual('org-list',follow['operation']) + self.assertEqual('org.list',follow['manifestOperation']) + else: + self.assertEqual('service',follow['kind']) + operation=operations[follow.get('manifestOperation',follow['operation'])] + command=['cli',*control['suggestedArgv'][control['suggestedArgv'].index('cli')+1:]] + self.assertEqual(operation['command'],command[:len(operation['command'])]) + self.assertTrue(set(follow['arguments']).issubset({entry['name'] for entry in operation['arguments']})) + self.assertTrue(set(follow['options']).issubset({entry['name'] for entry in operation['options']})) + self.assertEqual([],follow['flags']) + if follow['preview']: + self.assertTrue(operation['preview']) + self.assertEqual(1,control['suggestedArgv'].count('--preview')) + if surface=='error': + self.assertEqual(['prose',*control['argv'][control['argv'].index('run'):]],control['languageArgv']) + def test_every_schema_is_valid_draft_2020_12_and_has_unique_id(self) -> None: self.assertGreaterEqual(len(self.schemas), 10) ids = [] @@ -1009,6 +1122,9 @@ def test_all_case_manifests_validate_and_error_actions_are_frozen(self) -> None: self.assertEqual(case["expected"]["exitCode"], frozen["exitCode"]) if case["id"] in {"adapters.sdk-production-02", "adapters.sdk-production-03"}: self.assertEqual(case["expected"]["errorAction"], load_json(FIXTURES / "adapters/sdk-production.json")["credentialAbsence"]["action"]) + elif case["id"] in {"adapters.sdk-production-11", "adapters.sdk-production-12", "adapters.sdk-production-13"}: + reason=case["expected"]["resultMatches"]["error"]["details"]["nativeFailure"]["setupReason"] + self.assertEqual(case["expected"]["errorAction"], load_json(FIXTURES / "adapters/sdk-production.json")["setupFailure"]["reasons"][reason]) else: self.assertEqual(case["expected"]["errorAction"], frozen["action"]) ids = [case["id"] for case in cases] diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 20ee6d3d..5ed8c46a 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -97,3 +97,11 @@ unspecified Linux platform identities are not admitted. This frozen contract is acceptance intent; it does not itself establish native build, installed-route, live fulfillment or release qualification. The shared black-box cases and independent product controls must prove each behavior before release. + +Native `SetupError` reports carry only a closed `setup_reason`: credential-or-permission, +model-unavailable or local-input. Outer failures retain `HARNESS_FAILED` and exit 22, +with `nativeFailure.kind=setup` and `setupReason` only for a recognized reason. +The runner owns fixed recovery actions from the shared SDK oracle, for both human +and JSON output. Unknown/missing reasons stay execution failures. Never forward +arbitrary native messages or infer request completion, usage or billing from a +setup classification; preserve valid observations already received. diff --git a/docs/sdk-budgets.md b/docs/sdk-budgets.md index 5af596e2..5ffd091d 100644 --- a/docs/sdk-budgets.md +++ b/docs/sdk-budgets.md @@ -13,7 +13,7 @@ Quoted TOML keys are `native_max_turns="60"` and `native_timeout="8m"`, and `nat SDK invocation, dry-run, explanation and result records expose eleven `nativeLimits` fields: `maxTurns`, `timeoutSeconds`, `toolTimeoutSeconds` (default 30), `maxOutputTokens` (12000 per model request), `maxAggregateRequests`, `maxAggregateHostedWebCalls`, `maxAggregateFunctionTools` (80), `maxObservedTotalTokens` (500000), `maxRequestInputBytes` (256000), `maxChildren` (8) and `maxChildDepth` (1). Aggregate requests and hosted calls inherit `maxTurns`; hosted tools are capped at one per model request. Historical four-field diagnostics remain readable. The generic SDK harness also reports its configured limits in start and error records. These are configured allowances, not measured consumption. The harness's direct `--max-turns`, `--timeout`, and `--tool-timeout` arguments remain available without either outer runner. -Normalized native failures carry a closed `nativeFailure.kind`: `max-turns`, `timeout`, or `execution`, with validated numeric limits and elapsed seconds when supplied. Arbitrary exception names and bodies are not passed through. A shell-action timeout is a tool result the agent may handle; an SDK run timeout terminates that run; outer cancellation has its existing separate error code. Cleanup signals do not replace the recorded native cause. A file written before a turn limit is reached remains an artifact, not proof of a completed invocation. Success still requires a native final record and successful process settlement. +Normalized native failures carry a closed `nativeFailure.kind`: `max-turns`, `timeout`, `execution`, or `setup`, with validated numeric limits and elapsed seconds when supplied. Arbitrary exception names and bodies are not passed through. A shell-action timeout is a tool result the agent may handle; an SDK run timeout terminates that run; outer cancellation has its existing separate error code. Cleanup signals do not replace the recorded native cause. A file written before a turn limit is reached remains an artifact, not proof of a completed invocation. Success still requires a native final record and successful process settlement. Increasing the tool allowance does not increase the SDK parent or outer deadline. Omission forwards no extra tool-timeout argument; explicit `30s` reports its selected configuration source while preserving the same effective allowance. A tool timeout can return a tool error for the agent to handle, not guaranteed cancellation of every external effect. `delegate(task)` starts an independent SDK Agent/Runner conversation with only the explicit task, the same opaque instructions and shared working directory. It inherits no parent conversation or tool history. Children execute serially and cannot delegate; their calls, tools and observed usage consume the shared invocation allowances. Files remain shared. Whole-run timeout and cancellation cover active parent/child contexts and ordinary tool process groups; detached descendants escaping a group are outside that guarantee. No OpenProse-specific interpretation is introduced. @@ -26,3 +26,5 @@ unavailable. Missing counters remain absent, provider outstanding requests remai unknown and `totalRunUsageKnown` remains false. These are execution allowances and observations, never an authoritative currency/billing ceiling. Explicit API-client and SDK retry settings disable transport and conversation-lock retries. + +Setup failures use only the closed `setupReason` values credential-or-permission, model-unavailable and local-input. A recognized native SetupError receives a fixed runner-owned recovery action in human and JSON output while retaining HARNESS_FAILED/22. Unknown reasons remain execution failures and arbitrary native messages are suppressed. Safe observations remain available without proving request completion or total billing. diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index d20ceba8..c7037987 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -295,27 +295,31 @@ def emit(kind, **data): 'maxAggregateRequests': args.max_turns, 'maxAggregateHostedWebCalls': args.max_turns, 'maxAggregateFunctionTools': max_tools, 'maxObservedTotalTokens': max_total_tokens, 'maxRequestInputBytes': max_input_bytes, 'maxChildren': 8, 'maxChildDepth': 1} observation = UsageObservation(args.max_turns, max_tools, max_total_tokens) - def fail(kind, message): + def fail(kind, message, setup_reason=None): + extra = {"setup_reason": setup_reason} if kind == "SetupError" and setup_reason in ("credential-or-permission", "model-unavailable", "local-input") else {} emit('error', error_type=kind, message=message, limits=limits, - usageObservation=observation.summary()) + usageObservation=observation.summary(), **extra) return 1 + emit('start', model=args.model, cwd=args.cwd, limits=limits, + permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, + 'networkSandbox': False, 'freshChildConversation': True}) try: key = dotenv_values(args.env_file).get('OPENAI_API_KEY') if args.env_file else os.environ.get('OPENAI_API_KEY') if not isinstance(key, str) or not key.strip(): - return fail('SetupError', 'Set OPENAI_API_KEY to an OpenAI API key, or configure the OpenAI API-key credential profile. No model request was sent.') + return fail('SetupError', 'Set OPENAI_API_KEY to an OpenAI API key, or configure the OpenAI API-key credential profile. No model request was sent.', setup_reason='credential-or-permission') cwd = str(Path(args.cwd).resolve(strict=True)) if not Path(cwd).is_dir(): - return fail('SetupError', 'The working directory must exist.') + return fail('SetupError', 'The working directory must exist.', setup_reason='local-input') instructions = 'You are a helpful coding agent. Use available tools to complete the user request.' if args.instructions: with Path(args.instructions).open('rb') as source: raw = source.read(max_input_bytes + 1) if len(raw) > max_input_bytes: - return fail('SetupError', 'The instruction file exceeds the configured request input-byte budget.') + return fail('SetupError', 'The instruction file exceeds the configured request input-byte budget.', setup_reason='local-input') instructions += '\n\n' + raw.decode('utf-8') instructions += '\nYour working directory is: ' + cwd except (OSError, ValueError): - return fail('SetupError', 'Cannot read the working directory, instructions, or credential file. Check the supplied paths and permissions.') + return fail('SetupError', 'Cannot read the working directory, instructions, or credential file. Check the supplied paths and permissions.', setup_reason='local-input') deadline = start + args.timeout children = 0 child_usage = [] @@ -380,9 +384,6 @@ async def delegate(task: str) -> str: preserve_raw_usage=True, parallel_tool_calls=False, retry=ModelRetrySettings(max_retries=0), extra_args={'max_tool_calls': 1, 'service_tier': 'default'})) - emit('start', model=args.model, cwd=cwd, limits=limits, - permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, - 'networkSandbox': False, 'freshChildConversation': True}) try: result = await asyncio.wait_for(Runner.run(make_agent(0), args.prompt, max_turns=args.max_turns, run_config=config, hooks=observation), remaining()) @@ -398,9 +399,9 @@ async def delegate(task: str) -> str: except Exception as error: name = type(error).__name__ if name in ('AuthenticationError', 'PermissionDeniedError'): - return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.') + return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.', setup_reason='credential-or-permission') if name == 'NotFoundError': - return fail('SetupError', 'The configured OpenAI model is unavailable to this account. Check the model selection and account access; no fallback was selected.') + return fail('SetupError', 'The configured OpenAI model is unavailable to this account. Check the model selection and account access; no fallback was selected.', setup_reason='model-unavailable') safe = name if name in ('MaxTurnsExceeded', 'TimeoutError', 'BudgetExceeded') else 'ExecutionError' return fail(safe, 'Execution stopped. Check configured limits, tool availability and OpenAI account access.') finally: diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index 06f9e51c..f6afb979 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -143,7 +143,28 @@ async def test_missing_key_fails_before_runner(self): self.assertEqual(await harness.run(self.args(directory)), 1) runner.assert_not_called() self.assertIn('OPENAI_API_KEY', output.getvalue()) - self.assertEqual(json.loads(output.getvalue())['error_type'], 'SetupError') + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'SetupError') + + async def test_local_setup_errors_start_before_failure_without_runner_or_client(self): + with tempfile.TemporaryDirectory() as directory: + oversized = Path(directory) / 'oversized-instructions' + oversized.write_bytes(b'x' * 257) + for instruction in (str(oversized), str(Path(directory) / 'absent-instructions')): + args = self.args(directory) + args.instructions = instruction + args.max_input_bytes = 256 + output = io.StringIO() + with patch.object(harness.Runner, 'run', AsyncMock()) as runner, patch.object(harness, 'AsyncOpenAI') as client, contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(args), 1) + runner.assert_not_called() + client.assert_not_called() + records = [json.loads(line) for line in output.getvalue().splitlines()] + self.assertEqual([record['type'] for record in records], ['start', 'error']) + self.assertEqual(records[0]['cwd'], directory) + self.assertEqual(records[0]['limits'], records[1]['limits']) + self.assertEqual(records[1]['error_type'], 'SetupError') + self.assertEqual(records[1]['setup_reason'], 'local-input') + self.assertEqual(records[1]['usageObservation']['startedCallCount'], 0) async def test_children_are_fresh_and_aggregate_usage(self): calls = [] @@ -390,6 +411,7 @@ async def test_auth_and_model_failures_do_not_leak_bodies(self): self.assertEqual(await harness.run(self.args(directory)), 1) self.assertNotIn('credential-value', output.getvalue()) self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'SetupError') + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['setup_reason'], 'model-unavailable' if name == 'NotFoundError' else 'credential-or-permission') class RetrievalTest(unittest.IsolatedAsyncioTestCase): From 7bedbc6d25dcc67970877602ffc40a7c7cfc24ff Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:30:27 -0400 Subject: [PATCH 08/64] Freeze release rehearsal at reviewed SDK corpus size --- cli/ci/rehearse_release.py | 2 +- cli/ci/test_rehearse_release.py | 8 ++++---- cli/protocol/OWNERSHIP.md | 2 ++ 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/cli/ci/rehearse_release.py b/cli/ci/rehearse_release.py index 983a8161..48afdb61 100644 --- a/cli/ci/rehearse_release.py +++ b/cli/ci/rehearse_release.py @@ -44,7 +44,7 @@ SAFE_BASENAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$") MAX_EVIDENCE_BYTES = 16 * 1024 * 1024 CONFORMANCE_PHASE = 7 -CONFORMANCE_CASES = 76 +CONFORMANCE_CASES = 100 CONFORMANCE_CANDIDATE_VALIDATIONS = CONFORMANCE_CASES * 3 CONFORMANCE_DIFFERENTIAL_VALIDATIONS = CONFORMANCE_CASES * 2 CONFORMANCE_TOTAL_VALIDATIONS = ( diff --git a/cli/ci/test_rehearse_release.py b/cli/ci/test_rehearse_release.py index 4e7929b0..cb6cffd7 100644 --- a/cli/ci/test_rehearse_release.py +++ b/cli/ci/test_rehearse_release.py @@ -592,10 +592,10 @@ def builder(**kwargs): self.assertIsInstance(benchmark.calls[0][4], float) mechanical = summary["bindings"]["mechanicalConformance"] self.assertEqual(7, mechanical["phase"]) - self.assertEqual(76, len(mechanical["caseIds"])) - self.assertEqual(228, mechanical["candidateCaseValidations"]) - self.assertEqual(152, mechanical["differentialValidations"]) - self.assertEqual(380, mechanical["totalValidations"]) + self.assertEqual(100, len(mechanical["caseIds"])) + self.assertEqual(300, mechanical["candidateCaseValidations"]) + self.assertEqual(200, mechanical["differentialValidations"]) + self.assertEqual(500, mechanical["totalValidations"]) self.assertEqual( rehearse_release.CONFORMANCE_CASES, len(mechanical["caseIds"]) ) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index d1ea47c0..4aa657c1 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1030,3 +1030,5 @@ IMP-098 correction oracle registration: root owns `cli/ci/check_architecture.py` IMP-097 measured adapter adversary admission: `config_assessment` owns `cli/conformance/adversarial/adapter-products/test_adapter_products.py` to diagnose the 16 configuration-before-readiness failures in full admission 13. Preserve credential canaries, strict schema/exit/readiness checks and no-spawn assertions. Correct valid fixture inputs only after measuring actual stdout. No product changes, Git or heavy builds. IMP-097 measured release rehearsal admission: `conformance_plumbing` owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` narrowly to update the frozen current-corpus count from 64 to the reviewed 76 configuration cases and exact 3N/2N/5N validation totals. Preserve historical snapshot replay, mutations, secretion/redaction, custody and wrong-count rejection. Existing full14 passed packaging and failed only this stale gate. No dynamic relaxation, Git or heavy real rehearsal builds. Root integrates SDK branch-specific 100-case count after merge. + +IMP-098 current rehearsal corpus freeze: root owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` for exact reviewed SDK 100-case count and 300/200/500 validation totals after real merge of the 097 76-case guard. Historical replay, wrong-count/mutation/redaction/custody checks remain. From 00fb4a713d3b3e8b251a4f08258f66fd41df3678 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:38:51 -0400 Subject: [PATCH 09/64] Preserve SDK recovery through CLI output and preflight credentials --- cli/bun/src/adapters/environment.ts | 1 + cli/bun/src/cli.ts | 22 +++-- cli/bun/src/core/output.ts | 4 +- cli/bun/test/sdk-production.test.ts | 21 +++- .../core/openprose-hosted-unavailable.json | 27 ++++- .../core/openprose-invalid-transport-run.json | 33 ++++++- .../cases/dx/dry-run-default-hosted.json | 29 +++++- .../cases/dx/jsonl-hosted-failure.json | 26 ++++- .../operations/config-explain-default.json | 88 ++++++++++++++--- .../doctor-default-unavailable.json | 99 ++++++++++++++++--- .../operations/harness-list-default.json | 26 ++++- .../openprose-invalid-transport-doctor.json | 33 ++++++- cli/conformance/runner/run.py | 36 ++++++- cli/conformance/runner/test_runner.py | 17 ++++ cli/protocol/OWNERSHIP.md | 8 ++ cli/rust/crates/prose-cli/src/main.rs | 9 ++ .../crates/prose-runner-core/src/error.rs | 55 +++++++++++ .../crates/prose-runner-core/src/runner.rs | 51 ++++++++++ .../fixtures/dx/dry-run-default-hosted.json | 4 +- 19 files changed, 519 insertions(+), 70 deletions(-) diff --git a/cli/bun/src/adapters/environment.ts b/cli/bun/src/adapters/environment.ts index 416b93ec..85e57160 100644 --- a/cli/bun/src/adapters/environment.ts +++ b/cli/bun/src/adapters/environment.ts @@ -33,6 +33,7 @@ export function buildInstalledAdapterEnvironment(input: AdapterEnvironmentInput) const error=failure("HARNESS_NEEDS_AUTH", { adapterId: input.definition.id, authProfile: input.credentialGroup, + ...(input.definition.id==="agents-sdk/jsonl"?{fallbackAttempted:false}:{}), }); if(input.definition.id==="agents-sdk/jsonl") { const {schema:_,...shape}=error.toJSON(); diff --git a/cli/bun/src/cli.ts b/cli/bun/src/cli.ts index 9993d97a..2b92c9ba 100644 --- a/cli/bun/src/cli.ts +++ b/cli/bun/src/cli.ts @@ -4,7 +4,7 @@ import { runWeaveHost, writeHostBytes, stopHostOutput } from "./core/weave-host" import { PUBLISHED_KERNEL_STARTUP } from "./core/build"; import { publishedKernel } from "./core/kernel-startup"; import {nativeOutputLimits} from "./adapters/output-budget"; -import {nativeLimits} from "./adapters/sdk-limits"; +import {nativeLimits,sdkSetupFailureAction} from "./adapters/sdk-limits"; import { nativeConfiguration } from "./adapters/native-profile"; import { parseEntrypoint, inferOutputMode, isValueOption } from "./core/args"; import { embeddedRuntimeImage } from "./assets/sentinel"; @@ -955,10 +955,7 @@ async function runInstalledProbeInvocation( transportTerminalObserved: true, fallbackAttempted: false, }) - : failure( - outcome.process.error.code, - installedProcessFailureDetails(outcome.process, adapterId), - ); + : installedProcessFailure(outcome.process,adapterId); if(outcome.sdkObservations!==undefined) { const {schema:_,...shape}=attemptedError.toJSON(); attemptedError=new RunnerFailure({...shape,details:{...attemptedError.details,...outcome.sdkObservations}}); @@ -1197,10 +1194,7 @@ async function runInstalledInvocation( } let attemptedError = outcome.process.error === null ? null - : failure( - outcome.process.error.code, - installedProcessFailureDetails(outcome.process, readiness.adapterId), - ); + : installedProcessFailure(outcome.process,readiness.adapterId); let terminalEnvelope: Record | null = null; const nativeOutput = config.values.outputContract === "native"; if (attemptedError === null && !nativeOutput) { @@ -1337,6 +1331,16 @@ function processEvents( return records; } +function installedProcessFailure(process:ProcessSupervisionResult,adapterId:InstalledAdapterId):RunnerFailure { + const problem=failure(process.error!.code,installedProcessFailureDetails(process,adapterId)); + if(problem.code!=="HARNESS_FAILED"||adapterId!=="agents-sdk/jsonl")return problem; + const nativeFailure=problem.details?.nativeFailure; + const action=nativeFailure!==null&&typeof nativeFailure==="object"&&!Array.isArray(nativeFailure)?sdkSetupFailureAction(nativeFailure as Record):undefined; + if(action===undefined)return problem; + const {schema:_schema,...shape}=problem.toJSON(); + return new RunnerFailure({...shape,action}); +} + export function installedProcessFailureDetails( process: ProcessSupervisionResult, adapterId: InstalledAdapterId, diff --git a/cli/bun/src/core/output.ts b/cli/bun/src/core/output.ts index bd0d29c8..00bafa0f 100644 --- a/cli/bun/src/core/output.ts +++ b/cli/bun/src/core/output.ts @@ -343,7 +343,9 @@ export function formatHumanError(error: RunnerErrorShape): string { && validRecoveryHandle(cleanupArgv[3]) ? `\nRecovery: preserve the original temporary-root environment, then run: ${humanRunnerCommand(`cli cleanup prime ${cleanupArgv[3]}`)}` : ""; - return `${humanSafeScalar(error.code)} at ${humanSafeScalar(error.boundary)}: ${humanSafeScalar(error.message)}${source}${reason}${version}\nAction: ${humanAction(error)}${recovery}\n`; + const nativeFailure=error.details?.nativeFailure as {kind?:unknown;setupReason?:unknown}|undefined; + const sdkSetup=error.code==="HARNESS_FAILED"&&error.details?.adapterId==="agents-sdk/jsonl"&&nativeFailure?.kind==="setup"&&typeof nativeFailure.setupReason==="string"&&["credential-or-permission","model-unavailable","local-input"].includes(nativeFailure.setupReason); + return `${humanSafeScalar(error.code)} at ${humanSafeScalar(error.boundary)}: ${humanSafeScalar(error.message)}${source}${reason}${version}\nAction: ${sdkSetup?humanSafeScalar(error.action):humanAction(error)}${recovery}\n`; } export function humanAction(error: Pick): string { diff --git a/cli/bun/test/sdk-production.test.ts b/cli/bun/test/sdk-production.test.ts index d1b75e5d..d9a3f148 100644 --- a/cli/bun/test/sdk-production.test.ts +++ b/cli/bun/test/sdk-production.test.ts @@ -5,6 +5,7 @@ import {mkdtemp,mkdir,writeFile,readFile,realpath,rm,symlink} from "node:fs/prom import {join} from "node:path"; import {tmpdir} from "node:os"; import oracle from "../../shared/fixtures/adapters/sdk-production.json"; +import taxonomy from "../../shared/errors/taxonomy.v1.json"; import {resolveInstalledExecutable} from "../src/adapters/executable"; import {sdkObservations} from "../src/adapters/sdk-observation"; import {installedProtocol} from "../src/adapters/protocols"; @@ -39,7 +40,14 @@ test("packaged SDK discovery follows the CLI symlink, relocates together and ref for(const key of [undefined,""," "])test(`SDK missing/blank key ${JSON.stringify(key)} fails before helper resolution`,async()=>{ const root=await workspace();let stdout="",stderr=""; const exit=await runCli(["--output","json","run","input.prose.md"],{processCwd:root,env:{HOME:join(root,"home"),...(key===undefined?{}:{OPENAI_API_KEY:key})},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:text=>{stderr+=text;}}); - expect(exit).toBe(10);expect(stderr).toBe("");expect(JSON.parse(stdout)).toMatchObject({code:"HARNESS_NEEDS_AUTH",details:{adapterId:"agents-sdk/jsonl",authProfile:"openai-api-key"}});expect(JSON.parse(stdout).action).toContain("OPENAI_API_KEY"); + expect(exit).toBe(10);expect(stderr).toBe("");expect(JSON.parse(stdout)).toMatchObject({code:"HARNESS_NEEDS_AUTH",details:{adapterId:"agents-sdk/jsonl",authProfile:"openai-api-key",fallbackAttempted:false}});expect(JSON.parse(stdout).action).toContain("OPENAI_API_KEY"); +}); +test("default SDK doctor exposes the complete missing-key error with no fallback",async()=>{ + const root=await workspace();let stdout="",stderr=""; + const exit=await runCli(["--output","json","cli","doctor"],{platform:"darwin",arch:"arm64",processCwd:root,userConfigPath:join(root,"absent.toml"),env:{},imageBundle:sentinelFixtureImage,clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:text=>{stderr+=text;}}); + const definition=taxonomy.errors.find(item=>item.code==="HARNESS_NEEDS_AUTH")!; + expect(exit).toBe(10);expect(stderr).toBe(""); + expect(JSON.parse(stdout).problems).toEqual([{schema:"openprose.runner-error/1",...definition,action:"Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.",details:{adapterId:"agents-sdk/jsonl",authProfile:"openai-api-key",fallbackAttempted:false}}]); }); test("pure exact SDK defaults resolve without credentials or executable discovery",async()=>{ const root=await workspace();let stdout=""; @@ -80,6 +88,17 @@ test("SDK setup recovery admits only recognized native reasons and retains compl expect(JSON.stringify(caught.toJSON())).not.toContain("sdk-provider-body-secret-sentinel"); } }); +for(const [setupReason,action] of Object.entries(oracle.setupFailure.reasons))for(const output of ["json","human"] as const)test(`actual CLI SDK ${setupReason} recovery survives ${output} rendering`,async()=>{ + const root=await workspace(),helper=join(root,"prose-agents-sdk"); + const terminal={type:"error",error_type:"SetupError",setup_reason:setupReason,message:"sdk-provider-body-secret-sentinel",limits:oracle.nativeLimits,usageObservation:oracle.observation.failureUsage,modelIdentity:oracle.observation.modelIdentity}; + await writeFile(helper,`#!/Users/mm/openprose/.scratch/imp-086-tools/bun-darwin-aarch64/bun\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));console.log(JSON.stringify(${JSON.stringify(terminal)}));process.exit(1);`,{mode:0o755}); + let stdout="",stderr=""; + const code=await runCli(["--output",output,"run","input.prose.md"],{platform:"darwin",arch:"arm64",processCwd:root,userConfigPath:join(root,"absent.toml"),env:{OPENAI_API_KEY:"fixture-key"},imageBundle:sentinelFixtureImage,installedAdapterProbe:{executable:helper,observationPath:join(root,"observation.json"),credentialGroup:"openai-api-key"},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:text=>{stderr+=text;}}); + expect(code).toBe(22);expect(stdout+stderr).not.toContain("sdk-provider-body-secret-sentinel"); + if(output==="json"){ + const result=JSON.parse(stdout);expect(result.error).toMatchObject({code:"HARNESS_FAILED",action,details:{nativeFailure:{kind:"setup",setupReason},usageObservation:oracle.observation.failureUsage,modelIdentity:oracle.observation.modelIdentity}});expect(stderr).toBe(""); + }else{expect(stdout).toBe("");expect(stderr).toContain(`Action: ${action}`);} +}); for(const scenario of ["success","failure","cancel"])test(`actual supervised SDK ${scenario} preserves completed observations`,async()=>{ const root=await workspace(),helper=join(root,"prose-agents-sdk"),image=await verifyRuntimeImage(sentinelFixtureImage); const observation=scenario==="success"?oracle.observation.completedUsage:oracle.observation.failureUsage; diff --git a/cli/conformance/cases/core/openprose-hosted-unavailable.json b/cli/conformance/cases/core/openprose-hosted-unavailable.json index 1485ecb2..4763cd19 100644 --- a/cli/conformance/cases/core/openprose-hosted-unavailable.json +++ b/cli/conformance/cases/core/openprose-hosted-unavailable.json @@ -1,9 +1,14 @@ { "schema": "openprose.runner-case/1", "id": "core.openprose-hosted-unavailable", - "summary": "The OpenProse-billed default fails actionably and never falls back.", + "summary": "The explicitly selected OpenProse-billed fails actionably and never falls back.", "phase": 1, - "tags": ["default", "billing", "no-fallback", "error"], + "tags": [ + "default", + "billing", + "no-fallback", + "error" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,14 +16,26 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "json", "run", "fixture.prose.md"], + "argv": [ + "--harness", + "openprose", + "--output", + "json", + "run", + "fixture.prose.md" + ], "cwd": "{{WORKSPACE}}", "environment": {} }, "expected": { "exitCode": 10, - "stdout": { "kind": "json", "schema": "openprose.runner-result/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.runner-result/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "errorCode": "HOSTED_UNAVAILABLE", "errorAction": "To use the hosted service, run `cli run submit FILE --preview`; running programs on this machine needs a local harness (`cli harness list`)." diff --git a/cli/conformance/cases/core/openprose-invalid-transport-run.json b/cli/conformance/cases/core/openprose-invalid-transport-run.json index 8e14ba64..c8e596d6 100644 --- a/cli/conformance/cases/core/openprose-invalid-transport-run.json +++ b/cli/conformance/cases/core/openprose-invalid-transport-run.json @@ -3,7 +3,14 @@ "id": "core.openprose-invalid-transport-run", "summary": "A language run rejects an explicit unsupported hosted transport before reporting hosted-service availability.", "phase": 6, - "tags": ["default", "agent-dx", "differential", "fail-closed", "machine-output", "transport"], + "tags": [ + "default", + "agent-dx", + "differential", + "fail-closed", + "machine-output", + "transport" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,14 +18,28 @@ "network": "denied" }, "invocation": { - "argv": ["--transport", "unsupported", "--output", "json", "run", "fixture.prose.md"], + "argv": [ + "--harness", + "openprose", + "--transport", + "unsupported", + "--output", + "json", + "run", + "fixture.prose.md" + ], "cwd": "{{WORKSPACE}}", "environment": {} }, "expected": { "exitCode": 20, - "stdout": { "kind": "json", "schema": "openprose.runner-error/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.runner-error/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultMatches": { "code": "TRANSPORT_UNSUPPORTED", @@ -30,7 +51,9 @@ "details": { "harness": "openprose", "requested": "unsupported", - "supported": ["hosted"] + "supported": [ + "hosted" + ] } } } diff --git a/cli/conformance/cases/dx/dry-run-default-hosted.json b/cli/conformance/cases/dx/dry-run-default-hosted.json index 5076f708..450b8a60 100644 --- a/cli/conformance/cases/dx/dry-run-default-hosted.json +++ b/cli/conformance/cases/dx/dry-run-default-hosted.json @@ -1,9 +1,15 @@ { "schema": "openprose.runner-case/1", "id": "dx.dry-run-default-hosted", - "summary": "The default dry run reports the hosted OpenProse identity and exact configuration provenance without starting a model.", + "summary": "The explicitly selected dry run reports the hosted OpenProse identity and exact configuration provenance without starting a model.", "phase": 7, - "tags": ["dry-run", "hosted", "configuration", "agent-dx", "differential"], + "tags": [ + "dry-run", + "hosted", + "configuration", + "agent-dx", + "differential" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,14 +17,27 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "json", "--dry-run", "write", "fixture request"], + "argv": [ + "--harness", + "openprose", + "--output", + "json", + "--dry-run", + "write", + "fixture request" + ], "cwd": "{{WORKSPACE}}", "environment": {} }, "expected": { "exitCode": 10, - "stdout": { "kind": "json", "schema": "openprose.runner-dry-run-report/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.runner-dry-run-report/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultFixture": "cli/shared/fixtures/dx/dry-run-default-hosted.json", "errorCode": "HOSTED_UNAVAILABLE", diff --git a/cli/conformance/cases/dx/jsonl-hosted-failure.json b/cli/conformance/cases/dx/jsonl-hosted-failure.json index 99f62041..0b311f6d 100644 --- a/cli/conformance/cases/dx/jsonl-hosted-failure.json +++ b/cli/conformance/cases/dx/jsonl-hosted-failure.json @@ -3,7 +3,14 @@ "id": "dx.jsonl-hosted-failure", "summary": "JSONL hosted refusal emits exactly one final failed terminal with the canonical actionable error.", "phase": 7, - "tags": ["jsonl", "hosted", "terminal", "error", "agent-dx", "differential"], + "tags": [ + "jsonl", + "hosted", + "terminal", + "error", + "agent-dx", + "differential" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,7 +18,14 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "jsonl", "run", "fixture.prose.md"], + "argv": [ + "--harness", + "openprose", + "--output", + "jsonl", + "run", + "fixture.prose.md" + ], "cwd": "{{WORKSPACE}}", "environment": {} }, @@ -19,10 +33,14 @@ "exitCode": 10, "stdout": { "kind": "jsonl", - "eventTypes": ["runner.failed"], + "eventTypes": [ + "runner.failed" + ], "terminalType": "runner.failed" }, - "stderr": { "kind": "empty" }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "errorCode": "HOSTED_UNAVAILABLE", "errorAction": "To use the hosted service, run `cli run submit FILE --preview`; running programs on this machine needs a local harness (`cli harness list`)." diff --git a/cli/conformance/cases/operations/config-explain-default.json b/cli/conformance/cases/operations/config-explain-default.json index 4d24690d..2be32656 100644 --- a/cli/conformance/cases/operations/config-explain-default.json +++ b/cli/conformance/cases/operations/config-explain-default.json @@ -3,7 +3,12 @@ "id": "operations.config-explain-default", "summary": "Both products expose the same source-attributed non-TTY configuration.", "phase": 6, - "tags": ["cli", "configuration", "agent-dx", "differential"], + "tags": [ + "cli", + "configuration", + "agent-dx", + "differential" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,7 +16,13 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "json", "cli", "config", "explain"], + "argv": [ + "--output", + "json", + "cli", + "config", + "explain" + ], "cwd": "{{WORKSPACE}}", "environment": { "HOME": "{{WORKSPACE}}/home", @@ -20,21 +31,74 @@ }, "expected": { "exitCode": 0, - "stdout": { "kind": "json", "schema": "openprose.configuration-explanation/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.configuration-explanation/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultMatches": { "schema": "openprose.configuration-explanation/1", "projectConfigPath": null, "values": { - "harness": { "value": "openprose" }, - "transport": { "value": "auto" }, - "model": { "value": null }, - "timeout": { "value": "10m" }, - "output": { "value": "json" }, - "color": { "value": false }, - "verbose": { "value": false }, - "authProfile": { "value": null } + "harness": { + "value": "agents-sdk", + "source": { + "kind": "default", + "location": "built-in" + } + }, + "transport": { + "value": "auto" + }, + "model": { + "value": "gpt-6.1-sol", + "source": { + "kind": "default", + "location": "built-in:agents-sdk" + } + }, + "timeout": { + "value": "10m" + }, + "output": { + "value": "json" + }, + "color": { + "value": false + }, + "verbose": { + "value": false + }, + "authProfile": { + "value": "openai-api-key", + "source": { + "kind": "default", + "location": "built-in:agents-sdk" + } + } + }, + "runtime": { + "transport": "jsonl", + "permissionMode": null, + "authProfile": "openai-api-key", + "billingOwner": "user-provider", + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + }, + "nativeOutputLimits": null } } } diff --git a/cli/conformance/cases/operations/doctor-default-unavailable.json b/cli/conformance/cases/operations/doctor-default-unavailable.json index 3fbf9994..2af62a58 100644 --- a/cli/conformance/cases/operations/doctor-default-unavailable.json +++ b/cli/conformance/cases/operations/doctor-default-unavailable.json @@ -1,9 +1,15 @@ { "schema": "openprose.runner-case/1", "id": "operations.doctor-default-unavailable", - "summary": "Doctor reports the unavailable OpenProse default identically and exits with its blocking status.", + "summary": "Doctor reports the SDK built-in and missing credentials without fallback or starting a helper.", "phase": 6, - "tags": ["cli", "doctor", "agent-dx", "differential", "no-fallback"], + "tags": [ + "cli", + "doctor", + "agent-dx", + "differential", + "no-fallback" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,7 +17,12 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "json", "cli", "doctor"], + "argv": [ + "--output", + "json", + "cli", + "doctor" + ], "cwd": "{{WORKSPACE}}", "environment": { "HOME": "{{WORKSPACE}}/home", @@ -20,23 +31,81 @@ }, "expected": { "exitCode": 10, - "stdout": { "kind": "json", "schema": "openprose.doctor-report/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.doctor-report/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultMatches": { "schema": "openprose.doctor-report/1", - "runner": { "version": "0.1.0", "commit": "development" }, - "build": { "profile": "development", "testSeamsEnabled": true }, + "runner": { + "version": "0.1.0", + "commit": "development" + }, + "build": { + "profile": "development", + "testSeamsEnabled": true + }, "ready": false, - "selectedHarness": "openprose", + "selectedHarness": "agents-sdk", "selectedHarnessVersion": null, - "selectedTransport": "hosted", - "selectedAdapterId": "openprose/hosted", - "promptPlacement": null, - "isolation": "unsupported", - "authCategory": "openprose-account", - "billingOwner": "openprose", - "image": { "releaseEligible": false } + "selectedTransport": "jsonl", + "selectedAdapterId": "agents-sdk/jsonl", + "promptPlacement": "system-append", + "isolation": "advisory", + "authCategory": "harness-managed", + "billingOwner": "user-provider", + "image": { + "releaseEligible": false + }, + "selectedAuthReadiness": "required", + "configuration": { + "values": { + "harness": { + "value": "agents-sdk" + }, + "model": { + "value": "gpt-6.1-sol" + }, + "authProfile": { + "value": "openai-api-key" + } + }, + "runtime": { + "nativeLimits": { + "maxTurns": 20, + "timeoutSeconds": 180, + "toolTimeoutSeconds": 30, + "maxOutputTokens": 12000, + "maxAggregateRequests": 20, + "maxAggregateHostedWebCalls": 20, + "maxAggregateFunctionTools": 80, + "maxObservedTotalTokens": 500000, + "maxRequestInputBytes": 256000, + "maxChildren": 8, + "maxChildDepth": 1 + } + } + }, + "problems": [ + { + "schema": "openprose.runner-error/1", + "code": "HARNESS_NEEDS_AUTH", + "boundary": "authentication", + "exitCode": 10, + "retryable": false, + "message": "The selected harness is not authenticated.", + "action": "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.", + "details": { + "adapterId": "agents-sdk/jsonl", + "authProfile": "openai-api-key", + "fallbackAttempted": false + } + } + ] } } } diff --git a/cli/conformance/cases/operations/harness-list-default.json b/cli/conformance/cases/operations/harness-list-default.json index 51fe9634..5b1cb52a 100644 --- a/cli/conformance/cases/operations/harness-list-default.json +++ b/cli/conformance/cases/operations/harness-list-default.json @@ -3,7 +3,12 @@ "id": "operations.harness-list-default", "summary": "Both products expose the same selected harness and ordered stable inventory.", "phase": 6, - "tags": ["cli", "harness", "agent-dx", "differential"], + "tags": [ + "cli", + "harness", + "agent-dx", + "differential" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,7 +16,13 @@ "network": "denied" }, "invocation": { - "argv": ["--output", "json", "cli", "harness", "list"], + "argv": [ + "--output", + "json", + "cli", + "harness", + "list" + ], "cwd": "{{WORKSPACE}}", "environment": { "HOME": "{{WORKSPACE}}/home", @@ -20,12 +31,17 @@ }, "expected": { "exitCode": 0, - "stdout": { "kind": "json", "schema": "openprose.harness-list/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.harness-list/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultMatches": { "schema": "openprose.harness-list/1", - "selected": "openprose" + "selected": "agents-sdk" } } } diff --git a/cli/conformance/cases/operations/openprose-invalid-transport-doctor.json b/cli/conformance/cases/operations/openprose-invalid-transport-doctor.json index 244d2cdd..94f16602 100644 --- a/cli/conformance/cases/operations/openprose-invalid-transport-doctor.json +++ b/cli/conformance/cases/operations/openprose-invalid-transport-doctor.json @@ -3,7 +3,14 @@ "id": "operations.openprose-invalid-transport-doctor", "summary": "Doctor rejects an explicit unsupported hosted transport before reporting hosted-service availability.", "phase": 6, - "tags": ["cli", "agent-dx", "differential", "fail-closed", "machine-output", "transport"], + "tags": [ + "cli", + "agent-dx", + "differential", + "fail-closed", + "machine-output", + "transport" + ], "controls": { "fixedInvocationId": "fixture-invocation-0001", "fixedTimestamp": "2025-01-01T00:00:00Z", @@ -11,14 +18,28 @@ "network": "denied" }, "invocation": { - "argv": ["--transport", "unsupported", "--output", "json", "cli", "doctor"], + "argv": [ + "--harness", + "openprose", + "--transport", + "unsupported", + "--output", + "json", + "cli", + "doctor" + ], "cwd": "{{WORKSPACE}}", "environment": {} }, "expected": { "exitCode": 20, - "stdout": { "kind": "json", "schema": "openprose.runner-error/1" }, - "stderr": { "kind": "empty" }, + "stdout": { + "kind": "json", + "schema": "openprose.runner-error/1" + }, + "stderr": { + "kind": "empty" + }, "startedHarness": false, "resultMatches": { "code": "TRANSPORT_UNSUPPORTED", @@ -30,7 +51,9 @@ "details": { "harness": "openprose", "requested": "unsupported", - "supported": ["hosted"] + "supported": [ + "hosted" + ] } } } diff --git a/cli/conformance/runner/run.py b/cli/conformance/runner/run.py index 55acc6ea..3d4ca87f 100755 --- a/cli/conformance/runner/run.py +++ b/cli/conformance/runner/run.py @@ -1724,6 +1724,40 @@ def validate_sdk_effects(observation: Observation) -> list[str]: return failures +def expand_fixture_global_cwd(arguments: list[str], workspace: Path) -> list[str]: + # The shared configuration manifest owns option names. Only a runner cwd + # macro before the language freeze point is expanded; task bytes stay opaque. + schema=json.loads((CLI/'shared/schemas/configuration-explanation.schema.json').read_text('utf-8')) + keys=schema['properties']['values']['properties'] + exceptions={'nativeAddDirs':'--native-add-dir','nativeAllowTools':'--native-allow-tool'} + values={'--cwd'} + for key in keys: + if key not in ('color','verbose'): + values.add(exceptions.get(key,'--'+re.sub(r'([A-Z])',lambda match:'-'+match.group(1).lower(),key))) + flags={'--no-color','--verbose','--dry-run'} + expanded=list(arguments) + index=0 + while index=len(expanded): + break + if name=='--cwd': + expanded[index+1]=expanded[index+1].replace('{{WORKSPACE}}',str(workspace.resolve())) + index+=2 + elif token in flags: + index+=1 + else: + break + return expanded + + def execute( product: Product, case: dict[str, Any], @@ -1805,7 +1839,7 @@ def execute( for name in case["controls"].get("omitEnvironment", []): environment.pop(name, None) completed = run_owned_process( - execution_product.execution_argv(case["invocation"]["argv"]), + execution_product.execution_argv(expand_fixture_global_cwd(case["invocation"]["argv"], canonical_workspace)), cwd=cwd, environment=environment, timeout_seconds=15, diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index 0dd80bf5..08c47890 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -111,6 +111,23 @@ def test_host_oracle_rejects_unsupported_host_without_skipping_case(self): self.assertTrue(runner.deep_subset({"terminal": {"classification": "success"}}, wanted["resultMatches"])) self.assertEqual(100, len(list(runner.case_paths(7, set())))) + def test_global_cwd_macro_expansion_respects_arity_and_language_freeze(self): + workspace=Path('/fixture workspace') + token='{{WORKSPACE}}/sub' + for original,wanted in [ + (['--output','json','--cwd',token,'run','--cwd',token],['--output','json','--cwd','/fixture workspace/sub','run','--cwd',token]), + (['--no-color','--cwd='+token,'--','--cwd',token],['--no-color','--cwd=/fixture workspace/sub','--','--cwd',token]), + (['--model','--cwd','run',token],['--model','--cwd','run',token]), + (['--unknown','--cwd',token],['--unknown','--cwd',token]), + (['--native-add-dir',token,'--verbose','--cwd',token],['--native-add-dir',token,'--verbose','--cwd','/fixture workspace/sub']), + (['--cwd'],['--cwd']), + (['--help','--cwd',token],['--help','--cwd',token]), + ]: + with self.subTest(original=original): + snapshot=list(original) + self.assertEqual(wanted,runner.expand_fixture_global_cwd(original,workspace)) + self.assertEqual(snapshot,original) + def test_sdk_installation_fixture_executes_canonical_clone_and_keeps_helper_off_path(self): for number in range(1,18): case = json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 4aa657c1..dd062713 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1032,3 +1032,11 @@ IMP-097 measured adapter adversary admission: `config_assessment` owns `cli/conf IMP-097 measured release rehearsal admission: `conformance_plumbing` owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` narrowly to update the frozen current-corpus count from 64 to the reviewed 76 configuration cases and exact 3N/2N/5N validation totals. Preserve historical snapshot replay, mutations, secretion/redaction, custody and wrong-count rejection. Existing full14 passed packaging and failed only this stale gate. No dynamic relaxation, Git or heavy real rehearsal builds. Root integrates SDK branch-specific 100-case count after merge. IMP-098 current rehearsal corpus freeze: root owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` for exact reviewed SDK 100-case count and 300/200/500 validation totals after real merge of the 097 76-case guard. Historical replay, wrong-count/mutation/redaction/custody checks remain. + +IMP-098 measured actual-100 candidate reconciliation: `conformance_plumbing` owns exact existing cases `cli/conformance/cases/core/openprose-hosted-unavailable.json`, `openprose-invalid-transport-run.json`, `service-word-correction-04.json`, `cli/conformance/cases/dx/dry-run-default-hosted.json`, `jsonl-hosted-failure.json`, `cli/conformance/cases/operations/config-explain-default.json`, `doctor-default-unavailable.json`, `harness-list-default.json`, `openprose-invalid-transport-doctor.json` and exact referenced shared fixture subobjects needed by those cases. Actual paired report sdk-paired-candidates-100-1.json establishes stale default expectations and workspace placeholder mismatch. Preserve explicit-hosted effects by selecting hosted, and true-default contract by frozen SDK expectation; never copy received reports or relax matching. Root owns product setup-action and Rust missing-key envelope repairs. No products/Git/heavy build/provider; use existing actual candidates for focused validation. + +IMP-098 Bun actual setup action follow-through: `configuration_contract` owns `cli/bun/src/adapters/runner.ts`, `protocols.ts`, `types.ts`, `cli/bun/src/cli.ts` plus existing SDK production tests to diagnose/repair fixed action propagation through final installed result and human render for measured SDK11–16 actual candidate failures. Root owns Rust `runner.rs`, `prose-cli/src/main.rs` and CLI tests for SDK missing-key bare error parity and human action projection. Preserve primary protocol/cancellation/cleanup errors, raw-provider exclusion and honest observations. No shared expectations changed for SDK02/03/11–16. + +IMP-098 measured follow-through extensions: `configuration_contract` additionally owns `cli/bun/src/core/output.ts` narrowly to show only recognized SDK HARNESS_FAILED setup fixed actions directly in human output, with scalar escaping and generic runner guidance retained for other errors. Root owns corresponding Rust `cli/rust/crates/prose-runner-core/src/error.rs`. `conformance_plumbing` additionally owns `cli/conformance/runner/run.py` and `test_runner.py` only to expand closed {{WORKSPACE}} macro in runner-owned --cwd global prefix values (separate or inline) for mechanical fixtures. Opaque language arguments, tails after -- and other values stay byte-identical. Test rejected/opaque/arity controls; preserve absolute-cwd case04 rather than narrowing coverage. + +IMP-098 SDK missing-key parity extension: `configuration_contract` owns `cli/bun/src/adapters/environment.ts` solely to add truthful fallbackAttempted:false to SDK missing-auth diagnostics shared by doctor and run. Other adapters remain unchanged. Preserve blank-key handling and prelaunchfailure; exact tests required. diff --git a/cli/rust/crates/prose-cli/src/main.rs b/cli/rust/crates/prose-cli/src/main.rs index 405780e3..d021d8a5 100644 --- a/cli/rust/crates/prose-cli/src/main.rs +++ b/cli/rust/crates/prose-cli/src/main.rs @@ -644,6 +644,15 @@ fn prepare( ) { return prose_runner_core::runner::configuration_outcome(&config, mode); } + if config.harness.value == "agents-sdk" + && matches!(parsed.action, Action::Forward { .. }) + && !parsed.globals.dry_run + { + let ambient = std::env::vars_os().collect::>(); + if let Err(error) = prose_runner_core::runner::sdk_credential_preflight(&config, &ambient) { + return error_outcome(error, mode, &clock, &ids); + } + } let image = match execution_image(&parsed, &config, cancellation) { Ok(image) => image, Err(error) => return error_outcome(error, mode, &clock, &ids), diff --git a/cli/rust/crates/prose-runner-core/src/error.rs b/cli/rust/crates/prose-runner-core/src/error.rs index a52b05b7..0a3d4a94 100644 --- a/cli/rust/crates/prose-runner-core/src/error.rs +++ b/cli/rust/crates/prose-runner-core/src/error.rs @@ -790,6 +790,21 @@ impl RunnerError { #[must_use] pub fn human_action(&self) -> String { + if self.code == ErrorCode::HarnessFailed { + if let Some(action) = self + .details + .as_ref() + .filter(|details| { + details.get("adapterId").and_then(Value::as_str) == Some("agents-sdk/jsonl") + }) + .and_then(|details| details.get("nativeFailure")) + .filter(|native| native.get("kind").and_then(Value::as_str) == Some("setup")) + .and_then(|native| native.get("setupReason").and_then(Value::as_str)) + .and_then(crate::installed_adapters::sdk_setup_action) + { + return human_safe_scalar(action); + } + } let executable = human_runner_executable(); let action = human_safe_scalar(&self.action); if executable == NON_COPYABLE_RUNNER_GUIDANCE { @@ -893,6 +908,46 @@ mod tests { use super::*; use serde_json::json; + #[test] + fn sdk_setup_human_actions_are_fixed_and_keep_primary_error_guards() { + for reason in [ + "credential-or-permission", + "model-unavailable", + "local-input", + ] { + let error = RunnerError::catalog(ErrorCode::HarnessFailed) + .with_detail("adapterId", "agents-sdk/jsonl") + .with_detail( + "nativeFailure", + serde_json::json!({"kind":"setup", "setupReason":reason}), + ); + let action = crate::installed_adapters::sdk_setup_action(reason).unwrap(); + assert_eq!(error.human_action(), action); + assert!(error.to_string().contains(&format!("\nAction: {action}"))); + let mut primary = error.clone(); + primary.code = ErrorCode::Cancelled; + assert_ne!(primary.human_action(), action); + let other = RunnerError::catalog(ErrorCode::HarnessFailed) + .with_detail("adapterId", "claude/print-stream-json") + .with_detail( + "nativeFailure", + serde_json::json!({"kind":"setup", "setupReason":reason}), + ); + assert_ne!(other.human_action(), action); + } + let unknown = RunnerError::catalog(ErrorCode::HarnessFailed) + .with_detail("adapterId", "agents-sdk/jsonl") + .with_detail( + "nativeFailure", + serde_json::json!({"kind":"setup", "setupReason":"arbitrary-provider-message"}), + ); + assert!( + !unknown + .human_action() + .contains("arbitrary-provider-message") + ); + } + #[test] fn quote_and_detail_match_the_shared_fixture() { let fixture: Value = serde_json::from_str(include_str!( diff --git a/cli/rust/crates/prose-runner-core/src/runner.rs b/cli/rust/crates/prose-runner-core/src/runner.rs index 412eb7b4..0c05a289 100644 --- a/cli/rust/crates/prose-runner-core/src/runner.rs +++ b/cli/rust/crates/prose-runner-core/src/runner.rs @@ -773,6 +773,27 @@ fn supported_transports(harness: &str) -> Option<&'static [&'static str]> { } } +/// Checks SDK credentials before image acquisition or native helper discovery. +/// +/// # Errors +/// +/// Returns the existing transport/configuration error for an invalid SDK route, +/// or `HARNESS_NEEDS_AUTH` for a missing or blank API key. No process is started. +pub fn sdk_credential_preflight( + config: &EffectiveConfig, + ambient: &[(std::ffi::OsString, std::ffi::OsString)], +) -> Result<(), RunnerError> { + if config.harness.value != "agents-sdk" { + return Ok(()); + } + validate_selected_transport(config)?; + let adapter = installed_adapters::InstalledAdapter::AgentsSdkJsonl; + let group = selected_auth_group(adapter, config)?; + installed_adapters::auth_readiness(adapter, &group, ambient) + .map(|_| ()) + .map_err(|error| error.with_detail("fallbackAttempted", false)) +} + fn validate_selected_transport(config: &EffectiveConfig) -> Result<(), RunnerError> { let Some(supported) = supported_transports(&config.harness.value) else { return Ok(()); @@ -4936,6 +4957,36 @@ mod tests { } } + #[test] + fn sdk_preflight_requires_nonblank_key_and_preserves_other_routes() { + let temp = TempDir::new().unwrap(); + let mut config = installed_config( + temp.path(), + "agents-sdk", + "jsonl", + Some("gpt-6.1-sol"), + "openai-api-key", + ); + for ambient in [Vec::new(), vec![("OPENAI_API_KEY".into(), " ".into())]] { + let error = sdk_credential_preflight(&config, &ambient).unwrap_err(); + assert_eq!(error.code, ErrorCode::HarnessNeedsAuth); + assert_eq!(error.details.as_ref().unwrap()["fallbackAttempted"], false); + assert_eq!( + error.action, + "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent." + ); + } + assert!( + sdk_credential_preflight( + &config, + &[("OPENAI_API_KEY".into(), "fixture-no-provider-key".into())] + ) + .is_ok() + ); + config.harness.value = "claude".into(); + assert!(sdk_credential_preflight(&config, &[]).is_ok()); + } + #[test] fn sdk_tool_timeout_arguments_preserve_parent_defaults() { let temp = TempDir::new().unwrap(); diff --git a/cli/shared/fixtures/dx/dry-run-default-hosted.json b/cli/shared/fixtures/dx/dry-run-default-hosted.json index 598c074a..fc9552a2 100644 --- a/cli/shared/fixtures/dx/dry-run-default-hosted.json +++ b/cli/shared/fixtures/dx/dry-run-default-hosted.json @@ -34,8 +34,8 @@ }, { "key": "harness", - "source": "default", - "location": "built-in", + "source": "flag", + "location": "--harness", "redacted": false }, { From 807a2bfc5974ccf2c90cbd94710ae776ab503a25 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:39:41 -0400 Subject: [PATCH 10/64] Separate SDK and Codex diagnostic qualification tests --- cli/protocol/OWNERSHIP.md | 2 ++ cli/rust/crates/prose-cli/tests/cli.rs | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index dd062713..9c86b6ff 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1040,3 +1040,5 @@ IMP-098 Bun actual setup action follow-through: `configuration_contract` owns `c IMP-098 measured follow-through extensions: `configuration_contract` additionally owns `cli/bun/src/core/output.ts` narrowly to show only recognized SDK HARNESS_FAILED setup fixed actions directly in human output, with scalar escaping and generic runner guidance retained for other errors. Root owns corresponding Rust `cli/rust/crates/prose-runner-core/src/error.rs`. `conformance_plumbing` additionally owns `cli/conformance/runner/run.py` and `test_runner.py` only to expand closed {{WORKSPACE}} macro in runner-owned --cwd global prefix values (separate or inline) for mechanical fixtures. Opaque language arguments, tails after -- and other values stay byte-identical. Test rejected/opaque/arity controls; preserve absolute-cwd case04 rather than narrowing coverage. IMP-098 SDK missing-key parity extension: `configuration_contract` owns `cli/bun/src/adapters/environment.ts` solely to add truthful fallbackAttempted:false to SDK missing-auth diagnostics shared by doctor and run. Other adapters remain unchanged. Preserve blank-key handling and prelaunchfailure; exact tests required. + +IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli/tests/cli.rs` only to split explicit Codex doctor assertions from `runner_diagnostics_and_identity_stay_local` (107/100 lines) into an independent test with its own isolated TempDir. Preserve every assertion and semantic qualification; no lint suppression. diff --git a/cli/rust/crates/prose-cli/tests/cli.rs b/cli/rust/crates/prose-cli/tests/cli.rs index 6e51884d..a632db0c 100644 --- a/cli/rust/crates/prose-cli/tests/cli.rs +++ b/cli/rust/crates/prose-cli/tests/cli.rs @@ -4866,7 +4866,11 @@ fn runner_diagnostics_and_identity_stay_local() { } ); assert_eq!(harnesses[6]["testOnly"], true); +} +#[test] +fn explicit_codex_doctor_stays_local_and_reports_missing_harness() { + let temp = TempDir::new().unwrap(); let installed_doctor = prose( temp.path(), &[ From 9959a780b9df903e9587decf8e46444ed6e2f961 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:48:10 -0400 Subject: [PATCH 11/64] IMP-098: assign genuine installed upgrade qualification paths --- cli/protocol/OWNERSHIP.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 9c86b6ff..bc865723 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1042,3 +1042,9 @@ IMP-098 measured follow-through extensions: `configuration_contract` additionall IMP-098 SDK missing-key parity extension: `configuration_contract` owns `cli/bun/src/adapters/environment.ts` solely to add truthful fallbackAttempted:false to SDK missing-auth diagnostics shared by doctor and run. Other adapters remain unchanged. Preserve blank-key handling and prelaunchfailure; exact tests required. IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli/tests/cli.rs` only to split explicit Codex doctor assertions from `runner_diagnostics_and_identity_stay_local` (107/100 lines) into an independent test with its own isolated TempDir. Preserve every assertion and semantic qualification; no lint suppression. + +## IMP-098 genuine installed upgrade qualification + +`/root/config_assessment` owns exact paths `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py` for genuine pinned published 0.15.0-rc.3 to verified local newer candidate upgrades, both selected implementations, installed SDK default/helper discovery, and isolated configuration preservation through upgrade/uninstall. Reuse established tap upgrade sequencing; retain no-replacement safety and strict candidate/prior custody. No Homebrew mutation, heavy builds, Git, credentials or provider calls by the agent. + +`/root/configuration_contract` owns new exact paths `cli/ci/qualify_installed_sdk.py` and `cli/ci/test_qualify_installed_sdk.py` for actual isolated fresh npm/npx/standalone execution, genuine prior-version upgrades, relocation, exact configuration preservation, built-in SDK diagnostics and saved alternatives. Prior release manifests/artifacts are independently pinned; receipts bind exact candidate bytes. No global installations, Git, heavy builds, credentials or provider calls by the agent. Root retains `cli/ci/build_kernel_rc.py`, `.github/workflows/cli-kernel-rc.yml`, `.github/workflows/cli-distribution-check.yml` for later integration after independent review. Consumer subprocesses must work without a checkout, consumer Python provisioning or harness PATH repair. Candidate local unsigned version is 0.15.0-rc.4, later than published rc.3; this does not authorize publication. From b6991c2299b78f3c8053c5d82883022f0f0bb37b Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:50:56 -0400 Subject: [PATCH 12/64] IMP-098: assign fixed prior release input qualification --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index bc865723..6db5bbbf 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1045,6 +1045,8 @@ IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli ## IMP-098 genuine installed upgrade qualification +`/root/conformance_plumbing` owns new exact paths `cli/ci/fetch_previous_release.py` and `cli/ci/test_fetch_previous_release.py` for fresh bounded downloads of the fixed pinned published RC3 manifest and exactly four native upgrade inputs. Preserve untouched bytes, reject unsafe paths/symlinks/tampering, use fixed HTTPS publication origin and checksum-bound receipts. Focused fake-download tests only; no real downloads, Git, heavy builds, credentials or paid calls by the agent. Root additionally owns `cli/ci/test_build_kernel_rc.py`, `cli/ci/check_workflows.py`, and `cli/ci/test_check_workflows.py` narrowly for integrating mandatory installed qualification and retaining native evidence. + `/root/config_assessment` owns exact paths `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py` for genuine pinned published 0.15.0-rc.3 to verified local newer candidate upgrades, both selected implementations, installed SDK default/helper discovery, and isolated configuration preservation through upgrade/uninstall. Reuse established tap upgrade sequencing; retain no-replacement safety and strict candidate/prior custody. No Homebrew mutation, heavy builds, Git, credentials or provider calls by the agent. `/root/configuration_contract` owns new exact paths `cli/ci/qualify_installed_sdk.py` and `cli/ci/test_qualify_installed_sdk.py` for actual isolated fresh npm/npx/standalone execution, genuine prior-version upgrades, relocation, exact configuration preservation, built-in SDK diagnostics and saved alternatives. Prior release manifests/artifacts are independently pinned; receipts bind exact candidate bytes. No global installations, Git, heavy builds, credentials or provider calls by the agent. Root retains `cli/ci/build_kernel_rc.py`, `.github/workflows/cli-kernel-rc.yml`, `.github/workflows/cli-distribution-check.yml` for later integration after independent review. Consumer subprocesses must work without a checkout, consumer Python provisioning or harness PATH repair. Candidate local unsigned version is 0.15.0-rc.4, later than published rc.3; this does not authorize publication. From a8828b017df4cdb3affbf972c4d35fd858e65b8e Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:58:38 -0400 Subject: [PATCH 13/64] IMP-098: qualify genuine installed upgrades and SDK discovery --- .github/workflows/cli-distribution-check.yml | 23 +- .github/workflows/cli-kernel-rc.yml | 23 +- cli/ci/check_workflows.py | 38 ++- cli/ci/fetch_previous_release.py | 192 +++++++++++ cli/ci/homebrew_rehearsal.py | 202 ++++++++++-- cli/ci/npm_alias_install.py | 2 +- cli/ci/qualify_installed_sdk.py | 324 +++++++++++++++++++ cli/ci/test_check_workflows.py | 28 ++ cli/ci/test_fetch_previous_release.py | 154 +++++++++ cli/ci/test_homebrew_rehearsal.py | 204 ++++++++++++ cli/ci/test_qualify_installed_sdk.py | 226 +++++++++++++ cli/protocol/OWNERSHIP.md | 2 + 12 files changed, 1390 insertions(+), 28 deletions(-) create mode 100644 cli/ci/fetch_previous_release.py create mode 100644 cli/ci/qualify_installed_sdk.py create mode 100644 cli/ci/test_fetch_previous_release.py create mode 100644 cli/ci/test_qualify_installed_sdk.py diff --git a/.github/workflows/cli-distribution-check.yml b/.github/workflows/cli-distribution-check.yml index 01f4007a..8067664a 100644 --- a/.github/workflows/cli-distribution-check.yml +++ b/.github/workflows/cli-distribution-check.yml @@ -109,6 +109,8 @@ jobs: python3 -m unittest discover -s cli/ci -p test_sign_macos.py python3 -m unittest discover -s cli/ci -p test_build_agents_sdk.py python3 -m unittest discover -s cli/ci -p test_build_kernel_rc.py + python3 -m unittest discover -s cli/ci -p test_fetch_previous_release.py + python3 -m unittest discover -s cli/ci -p test_qualify_installed_sdk.py python3 -m unittest cli.ci.test_package_local.PackagedSdkArtifactTests python3 -m unittest discover -s cli/ci -p test_dependency_evidence.py python3 -m unittest discover -s cli/ci -p test_check_workflows.py @@ -119,8 +121,22 @@ jobs: # This RC version is an unpublished construction fixture. Live model # qualification and reviewed publication remain separate gates. env: - RC_VERSION: '0.15.0-rc.1' + RC_VERSION: '0.15.0-rc.4' run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" + - name: Fetch pinned genuine previous release inputs + env: + UPGRADE_PLATFORM: ${{ matrix.os == 'ubuntu-22.04' && 'linux-x64-gnu' || matrix.os == 'ubuntu-22.04-arm' && 'linux-arm64-gnu' || matrix.os == 'macos-15-intel' && 'darwin-x64' || 'darwin-arm64' }} + run: python3 cli/ci/fetch_previous_release.py --platform "$UPGRADE_PLATFORM" --out "$RUNNER_TEMP/previous-rc3" + - name: Qualify actual SDK installations and genuine upgrades + env: + RC_VERSION: '0.15.0-rc.4' + EXPECTED_SOURCE: ${{ github.sha }} + run: | + python3 cli/ci/qualify_installed_sdk.py \ + --candidate-root "$RUNNER_TEMP/kernel-rc" --source "$EXPECTED_SOURCE" --version "$RC_VERSION" \ + --previous-release "$RUNNER_TEMP/previous-rc3" \ + --node "$(command -v node)" --npm "$(command -v npm)" \ + --out "$RUNNER_TEMP/kernel-rc/installed-qualification" - name: Test Homebrew candidate archive admission run: python3 -m unittest discover -s cli/ci -p test_homebrew_rehearsal.py - uses: Homebrew/actions/setup-homebrew@dc7099b3e807f1e2ecc61f3ecabc840eedd5586a @@ -131,12 +147,13 @@ jobs: --output "$RUNNER_TEMP/cli-homebrew-rehearsal" - name: Rehearse Homebrew against the production SDK archives env: - RC_VERSION: '0.15.0-rc.1' + RC_VERSION: '0.15.0-rc.4' EXPECTED_SOURCE: ${{ github.sha }} run: | "$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py \ --kernel-rc "$RUNNER_TEMP/kernel-rc" \ --expected-source "$EXPECTED_SOURCE" --expected-version "$RC_VERSION" \ + --previous-release "$RUNNER_TEMP/previous-rc3" \ --output "$RUNNER_TEMP/kernel-rc/homebrew" - name: Retain rehearsal evidence if: always() @@ -151,5 +168,7 @@ jobs: ${{ runner.temp }}/kernel-rc/logs ${{ runner.temp }}/kernel-rc/build-report.json ${{ runner.temp }}/kernel-rc/homebrew + ${{ runner.temp }}/kernel-rc/installed-qualification + ${{ runner.temp }}/previous-rc3 retention-days: 14 if-no-files-found: warn diff --git a/.github/workflows/cli-kernel-rc.yml b/.github/workflows/cli-kernel-rc.yml index b2915a37..a19bb040 100644 --- a/.github/workflows/cli-kernel-rc.yml +++ b/.github/workflows/cli-kernel-rc.yml @@ -77,19 +77,36 @@ jobs: run: | python3 -m unittest discover -s cli/ci -p test_build_kernel_rc.py python3 -m unittest discover -s cli/ci -p test_check_published_release.py + python3 -m unittest discover -s cli/ci -p test_fetch_previous_release.py + python3 -m unittest discover -s cli/ci -p test_qualify_installed_sdk.py + - name: Fetch pinned genuine previous release inputs + env: + UPGRADE_PLATFORM: ${{ matrix.platform }}${{ runner.os == 'Linux' && '-gnu' || '' }} + run: python3 cli/ci/fetch_previous_release.py --platform "$UPGRADE_PLATFORM" --out "$RUNNER_TEMP/previous-rc3" - name: Build and verify fresh standalone and npm installations env: - RC_VERSION: ${{ inputs.version || '0.15.0-rc.1' }} + RC_VERSION: ${{ inputs.version || '0.15.0-rc.4' }} run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" + - name: Qualify actual SDK installations and genuine upgrades + env: + RC_VERSION: ${{ inputs.version || '0.15.0-rc.4' }} + EXPECTED_SOURCE: ${{ github.sha }} + run: | + python3 cli/ci/qualify_installed_sdk.py \ + --candidate-root "$RUNNER_TEMP/kernel-rc" --source "$EXPECTED_SOURCE" --version "$RC_VERSION" \ + --previous-release "$RUNNER_TEMP/previous-rc3" \ + --node "$(command -v node)" --npm "$(command -v npm)" \ + --out "$RUNNER_TEMP/kernel-rc/installed-qualification" - uses: Homebrew/actions/setup-homebrew@dc7099b3e807f1e2ecc61f3ecabc840eedd5586a - name: Rehearse Homebrew against these exact release archives env: - RC_VERSION: ${{ inputs.version || '0.15.0-rc.1' }} + RC_VERSION: ${{ inputs.version || '0.15.0-rc.4' }} EXPECTED_SOURCE: ${{ github.sha }} run: | "$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py \ --kernel-rc "$RUNNER_TEMP/kernel-rc" \ --expected-source "$EXPECTED_SOURCE" --expected-version "$RC_VERSION" \ + --previous-release "$RUNNER_TEMP/previous-rc3" \ --output "$RUNNER_TEMP/kernel-rc/homebrew" - name: Retain packages and offline evidence if: always() @@ -101,5 +118,7 @@ jobs: ${{ runner.temp }}/kernel-rc/logs ${{ runner.temp }}/kernel-rc/build-report.json ${{ runner.temp }}/kernel-rc/homebrew + ${{ runner.temp }}/kernel-rc/installed-qualification + ${{ runner.temp }}/previous-rc3 retention-days: 14 if-no-files-found: warn diff --git a/cli/ci/check_workflows.py b/cli/ci/check_workflows.py index 9e4bd0df..5ec7208d 100644 --- a/cli/ci/check_workflows.py +++ b/cli/ci/check_workflows.py @@ -333,15 +333,16 @@ def require(condition: bool, message: str) -> None: sdk_homebrew = by_name["Rehearse Homebrew against the production SDK archives"] require(sdk_build.get("if") is None and sdk_homebrew.get("if") is None, "production SDK installations must be qualified unconditionally") - require(sdk_build.get("env") == {"RC_VERSION": "0.15.0-rc.1"} + require(sdk_build.get("env") == {"RC_VERSION": "0.15.0-rc.4"} and sdk_build.get("run") == 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3"', "production SDK must use the native source-bound RC builder") - require(sdk_homebrew.get("env") == {"RC_VERSION": "0.15.0-rc.1", "EXPECTED_SOURCE": "${{ github.sha }}"} + require(sdk_homebrew.get("env") == {"RC_VERSION": "0.15.0-rc.4", "EXPECTED_SOURCE": "${{ github.sha }}"} and sdk_homebrew.get("run") == '"$RUNNER_TEMP/distribution-python/bin/python3" cli/ci/homebrew_rehearsal.py \\\n' ' --kernel-rc "$RUNNER_TEMP/kernel-rc" \\\n' ' --expected-source "$EXPECTED_SOURCE" --expected-version "$RC_VERSION" \\\n' + ' --previous-release "$RUNNER_TEMP/previous-rc3" \\\n' ' --output "$RUNNER_TEMP/kernel-rc/homebrew"\n', "production Homebrew must bind current source and exact SDK archives") retention = next(step for step in steps if step.get("uses", "").startswith("actions/upload-artifact@")) @@ -355,6 +356,34 @@ def require(condition: bool, message: str) -> None: "freeze/install SDK before exact archive Homebrew checks and retention") if name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): kernel_rc = name == "cli-kernel-rc.yml" + previous = by_name["Fetch pinned genuine previous release inputs"] + installed = by_name["Qualify actual SDK installations and genuine upgrades"] + production_build = by_name["Build and verify fresh standalone and npm installations" if kernel_rc + else "Build and verify production SDK standalone and npm installations"] + production_homebrew = by_name["Rehearse Homebrew against these exact release archives" if kernel_rc + else "Rehearse Homebrew against the production SDK archives"] + require(previous.get("if") is None and installed.get("if") is None, + "genuine installed upgrade qualification must be unconditional") + platform = ("${{ matrix.platform }}${{ runner.os == 'Linux' && '-gnu' || '' }}" if kernel_rc + else "${{ matrix.os == 'ubuntu-22.04' && 'linux-x64-gnu' || matrix.os == 'ubuntu-22.04-arm' && 'linux-arm64-gnu' || matrix.os == 'macos-15-intel' && 'darwin-x64' || 'darwin-arm64' }}") + require(previous.get("env") == {"UPGRADE_PLATFORM": platform} + and previous.get("run") == + 'python3 cli/ci/fetch_previous_release.py --platform "$UPGRADE_PLATFORM" --out "$RUNNER_TEMP/previous-rc3"', + "prior upgrade inputs must use the fixed native published release") + version = "${{ inputs.version || '0.15.0-rc.4' }}" if kernel_rc else "0.15.0-rc.4" + require(installed.get("env") == {"RC_VERSION": version, "EXPECTED_SOURCE": "${{ github.sha }}"} + and installed.get("run") == + 'python3 cli/ci/qualify_installed_sdk.py \\\n' + ' --candidate-root "$RUNNER_TEMP/kernel-rc" --source "$EXPECTED_SOURCE" --version "$RC_VERSION" \\\n' + ' --previous-release "$RUNNER_TEMP/previous-rc3" \\\n' + ' --node "$(command -v node)" --npm "$(command -v npm)" \\\n' + ' --out "$RUNNER_TEMP/kernel-rc/installed-qualification"\n', + "installed qualification must bind exact source, version and genuine prior bytes") + require(steps.index(previous) < steps.index(installed) + and steps.index(production_build) < steps.index(installed) < steps.index(production_homebrew), + "qualify current native installations before genuine Homebrew upgrades") + require('--previous-release "$RUNNER_TEMP/previous-rc3"' in production_homebrew.get("run", ""), + "production Homebrew must exercise a genuine prior-version upgrade") label = ("Rehearse Homebrew against these exact release archives" if kernel_rc else "Rehearse Homebrew against this build's verified native archives") homebrew = by_name[label] @@ -368,7 +397,7 @@ def require(condition: bool, message: str) -> None: if kernel_rc: fragments += ['--kernel-rc "$RUNNER_TEMP/kernel-rc"', '--expected-source "$EXPECTED_SOURCE"', '--expected-version "$RC_VERSION"', '--output "$RUNNER_TEMP/kernel-rc/homebrew"'] - require(homebrew.get("env") == {"RC_VERSION": "${{ inputs.version || '0.15.0-rc.1' }}", + require(homebrew.get("env") == {"RC_VERSION": "${{ inputs.version || '0.15.0-rc.4' }}", "EXPECTED_SOURCE": "${{ github.sha }}"}, "exact release Homebrew admission must bind workflow source/version") evidence_path = "${{ runner.temp }}/kernel-rc/homebrew" @@ -379,6 +408,9 @@ def require(condition: bool, message: str) -> None: "Homebrew admission must use pinned Python and verified archive custody") require(evidence_path in retention.get("with", {}).get("path", "").splitlines(), "retain Homebrew custody and cleanup results") + require(all(path in retention.get("with", {}).get("path", "").splitlines() for path in + ("${{ runner.temp }}/kernel-rc/installed-qualification", "${{ runner.temp }}/previous-rc3")), + "retain exact prior upgrade inputs and installed qualification evidence") require( any( step.get("uses", "").startswith("actions/upload-artifact@") diff --git a/cli/ci/fetch_previous_release.py b/cli/ci/fetch_previous_release.py new file mode 100644 index 00000000..b656c7b2 --- /dev/null +++ b/cli/ci/fetch_previous_release.py @@ -0,0 +1,192 @@ +#!/usr/bin/env python3 +"""Fetch fixed, checksum-pinned RC3 inputs for native upgrade qualification.""" +from __future__ import annotations +import argparse +import hashlib +import json +import os +from pathlib import Path +import re +import subprocess +import stat +import sys +from typing import BinaryIO, Callable + +VERSION = '0.15.0-rc.3' +SOURCE = '1941a34c3503f9a1417aebea1bc19ecada91e58c' +MANIFEST_SHA256 = '571eb285f964ea980d4e3aec6aa574f782b9d51f2a6934d219bc2ae7d8755e4f' +BASE_URL = 'https://pkg.prose.md/cli/releases/' + VERSION + '/' +PLATFORMS = ('darwin-arm64', 'darwin-x64', 'linux-arm64-gnu', 'linux-x64-gnu') +MAX_MANIFEST = 1024 * 1024 +MAX_ARTIFACT = 256 * 1024 * 1024 +SAFE_NAME = re.compile(r'[A-Za-z0-9][A-Za-z0-9._-]{0,180}') +SHA256 = re.compile(r'[0-9a-f]{64}') + +class BoundedWriter: + def __init__(self, stream: BinaryIO, maximum: int): + self.stream, self.maximum, self.count = stream, maximum, 0 + def write(self, data: bytes) -> int: + if self.count + len(data) > self.maximum: + raise ValueError('Previous release download exceeded its byte bound') + count = self.stream.write(data) + if count != len(data): + raise ValueError('Previous release download write failed') + self.count += count + return count + +def curl_download(url: str, sink: BoundedWriter, maximum: int) -> None: + # No curlrc, redirects, ambient credentials/proxies, shell, or diagnostic bodies. + process = subprocess.Popen([ + '/usr/bin/curl', '--disable', '--fail', '--silent', '--proto', '=https', + '--tlsv1.2', '--connect-timeout', '20', '--max-time', '120', + '--max-filesize', str(maximum), url, + ], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, + env={'PATH': '/usr/bin:/bin', 'LC_ALL': 'C'}, start_new_session=True) + try: + assert process.stdout is not None + while True: + block = process.stdout.read(65536) + if not block: + break + sink.write(block) + if process.wait(timeout=5) != 0: + raise ValueError('Previous release download failed') + finally: + if process.poll() is None: + process.kill() + process.wait(timeout=5) + if process.stdout is not None: + process.stdout.close() + +def decode_manifest(data: bytes) -> dict: + if not data or len(data) > MAX_MANIFEST or hashlib.sha256(data).hexdigest() != MANIFEST_SHA256: + raise ValueError('Previous published manifest identity mismatch') + try: + manifest = json.loads(data) + except (ValueError, UnicodeDecodeError): + raise ValueError('Previous published manifest is invalid') from None + if not isinstance(manifest, dict) or manifest.get('schema') != 'openprose.cli-distribution/1' or manifest.get('version') != VERSION or manifest.get('source') != SOURCE: + raise ValueError('Expected the fixed published RC3 manifest') + return manifest + +def select_artifacts(manifest: dict, platform: str) -> list[dict]: + if platform not in PLATFORMS: + raise ValueError('Unsupported previous native platform') + artifacts = manifest.get('artifacts') + if not isinstance(artifacts, list): + raise ValueError('Previous manifest artifact inventory is invalid') + wanted = {('standalone','rust',platform), ('standalone','bun',platform), + ('npm','bun',platform), ('npm','bun','all')} + selected: dict[tuple, dict] = {} + names: set[str] = set() + for item in artifacts: + if not isinstance(item, dict): + raise ValueError('Previous manifest artifact record is invalid') + identity = (item.get('kind'),item.get('implementation'),item.get('platform')) + if not all(isinstance(value,str) for value in identity): + raise ValueError('Previous manifest artifact classification is invalid') + if identity not in wanted: + continue + name, digest, size = item.get('name'),item.get('sha256'),item.get('size') + if identity in selected or not isinstance(name,str) or not SAFE_NAME.fullmatch(name) or '..' in name or name in {'manifest.json','download-receipt.json'} or name in names: + raise ValueError('Previous selected artifact name or selection is invalid') + if not isinstance(digest,str) or not SHA256.fullmatch(digest) or type(size) is not int or not 0 < size <= MAX_ARTIFACT: + raise ValueError('Previous selected artifact identity is invalid') + selected[identity] = item + names.add(name) + if set(selected) != wanted: + raise ValueError('Exactly four previous platform and meta artifacts are required') + return [selected[key] for key in [('standalone','rust',platform),('standalone','bun',platform),('npm','bun','all'),('npm','bun',platform)]] + +def fetch(platform: str, out: Path, downloader: Callable = curl_download) -> dict: + if platform not in PLATFORMS: + raise ValueError('Unsupported previous native platform') + out = out.absolute() + for parent in (out.parent,*out.parent.parents): + if parent.is_symlink(): + raise ValueError('Previous release destination parents must not be symlinks') + if not out.parent.is_dir(): + raise ValueError('Previous release destination parent must exist') + try: + out.mkdir(mode=0o700) + except FileExistsError: + raise ValueError('Previous release destination must be fresh') from None + directory = os.open(out,os.O_RDONLY|os.O_DIRECTORY|os.O_NOFOLLOW) + identity = os.fstat(directory) + def custody() -> None: + current = out.lstat() + if out.is_symlink() or (current.st_dev,current.st_ino)!=(identity.st_dev,identity.st_ino): + raise ValueError('Previous release destination custody changed') + def write(name: str, maximum: int, url: str | None = None, content: bytes | None = None) -> bytes: + custody() + fd = os.open(name,os.O_RDWR|os.O_CREAT|os.O_EXCL|os.O_NOFOLLOW,0o600,dir_fd=directory) + with os.fdopen(fd,'w+b') as stream: + sink=BoundedWriter(stream,maximum) + if content is not None: + sink.write(content) + else: + try: + downloader(url,sink,maximum) + except Exception: + raise ValueError('Previous release download failed or exceeded its byte bound') from None + stream.flush() + os.fsync(stream.fileno()) + stat=os.fstat(stream.fileno()) + named=os.stat(name,dir_fd=directory,follow_symlinks=False) + if (stat.st_dev,stat.st_ino,stat.st_size)!=(named.st_dev,named.st_ino,named.st_size): + raise ValueError('Previous release file custody changed') + stream.seek(0) + data=stream.read(maximum+1) + if len(data)!=sink.count or len(data)>maximum: + raise ValueError('Previous release file size changed') + custody() + return data + try: + data=write('manifest.json',MAX_MANIFEST,BASE_URL+'manifest.json') + write_manifest_bytes=data + manifest=decode_manifest(data) + rows=[] + for item in select_artifacts(manifest,platform): + data=write(item['name'],item['size'],BASE_URL+item['name']) + digest=hashlib.sha256(data).hexdigest() + if len(data)!=item['size'] or digest!=item['sha256']: + raise ValueError('Previous artifact byte identity mismatch') + rows.append({'name':item['name'],'sha256':digest,'byteLength':len(data)}) + # Recheck the whole retained inventory after the last download. A later + # download must not silently replace an earlier verified input. + for row in [{'name':'manifest.json','sha256':MANIFEST_SHA256,'byteLength':len(write_manifest_bytes)},*rows]: + fd=os.open(row['name'],os.O_RDONLY|os.O_NOFOLLOW,dir_fd=directory) + with os.fdopen(fd,'rb') as stream: + before=os.fstat(stream.fileno()) + if not stat.S_ISREG(before.st_mode) or before.st_nlink!=1 or before.st_size!=row['byteLength']: + raise ValueError('Previous retained input custody mismatch') + digest=hashlib.sha256() + for block in iter(lambda:stream.read(65536),b''): + digest.update(block) + after=os.fstat(stream.fileno()) + named=os.stat(row['name'],dir_fd=directory,follow_symlinks=False) + if (before.st_dev,before.st_ino,before.st_size,before.st_mtime_ns,before.st_ctime_ns)!=(after.st_dev,after.st_ino,after.st_size,after.st_mtime_ns,after.st_ctime_ns) or (named.st_dev,named.st_ino)!=(after.st_dev,after.st_ino) or digest.hexdigest()!=row['sha256']: + raise ValueError('Previous retained input identity mismatch') + custody() + receipt={'schema':'openprose.prior-release-qualification-inputs/1','version':VERSION, + 'source':SOURCE,'platform':platform,'manifestSha256':MANIFEST_SHA256,'artifacts':rows} + write('download-receipt.json',MAX_MANIFEST,content=(json.dumps(receipt,sort_keys=True,indent=2)+'\n').encode()) + return receipt + finally: + os.close(directory) + +def main(argv: list[str] | None = None) -> int: + parser=argparse.ArgumentParser(description=__doc__) + parser.add_argument('--platform',choices=PLATFORMS,required=True) + parser.add_argument('--out',type=Path,required=True) + args=parser.parse_args(argv) + try: + fetch(args.platform,args.out) + except (ValueError,OSError,subprocess.SubprocessError): + print('Previous RC3 input fetch failed; use a fresh output directory and verify fixed release availability.',file=sys.stderr) + return 1 + print('Pinned previous RC3 platform inputs verified.') + return 0 + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 48c61577..80af326d 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -19,6 +19,57 @@ SAFE_VERSION = re.compile(r'[0-9]+\.[0-9]+\.[0-9]+(?:-[A-Za-z0-9.-]+)?') PLATFORMS = {'darwin-arm64', 'darwin-x64', 'linux-arm64-gnu', 'linux-x64-gnu'} TAP = 'openprose/cli-rehearsal' +PREVIOUS_VERSION = '0.15.0-rc.3' +PREVIOUS_MANIFEST_SHA256 = '571eb285f964ea980d4e3aec6aa574f782b9d51f2a6934d219bc2ae7d8755e4f' + + +def read_previous_manifest(root: Path) -> dict[str, Any]: + """Read the unchanged, independently pinned published RC3 manifest.""" + if root.is_symlink() or not root.is_dir(): + raise ValueError('A regular previous release directory is required') + path = root / 'manifest.json' + if path.is_symlink() or not path.is_file() or path.stat().st_size > 1024 * 1024: + raise ValueError('A regular bounded previous manifest is required') + if pub.digest(path) != PREVIOUS_MANIFEST_SHA256: + raise ValueError('Previous published manifest identity mismatch') + manifest = pub.read_json(path) + if manifest.get('schema') != 'openprose.cli-distribution/1' or manifest.get('version') != PREVIOUS_VERSION: + raise ValueError('Expected the pinned published RC3 manifest') + return manifest + + +def verify_previous_release(root: Path, platform: str, candidate_version: str) -> dict[str, Any]: + previous = read_previous_manifest(root) + match = re.fullmatch(r'0\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)-rc\.(0|[1-9][0-9]*)', candidate_version) + if match is None or tuple(map(int, match.groups())) <= (15, 0, 3): + raise ValueError('The candidate must be a newer RC than the genuine previous release') + if platform not in PLATFORMS: + raise ValueError('Unsupported previous native platform') + archives = {} + binary_hashes = {} + for item in previous.get('artifacts', []): + if item.get('kind') != 'standalone' or item.get('platform') != platform: + continue + implementation, name = item.get('implementation'), item.get('name') + if implementation not in {'bun', 'rust'} or implementation in archives: + raise ValueError('Exactly one previous archive per implementation is required') + if not isinstance(name, str) or not SAFE_NAME.fullmatch(name) or '..' in name: + raise ValueError('Unsafe previous archive name') + archive = root / name + if archive.is_symlink() or not archive.is_file(): + raise ValueError('A regular previous archive is required') + if type(item.get('size')) is not int or not 0 < item['size'] <= pub.MAX_BYTES or archive.stat().st_size != item['size'] or pub.digest(archive) != item.get('sha256'): + raise ValueError('Previous archive identity mismatch') + members = pub.archive_members(archive) + binaries = [data for path, data in members.items() if Path(path).name == 'prose' and len(Path(path).parts) == 2] + if len(binaries) != 1: + raise ValueError('Previous archive requires one prose executable') + archives[implementation] = dict(item, path=name, byteLength=item['size']) + binary_hashes[implementation] = hashlib.sha256(binaries[0]).hexdigest() + if set(archives) != {'bun', 'rust'}: + raise ValueError('Both previous native implementations are required') + return {'version': PREVIOUS_VERSION, 'manifestSha256': PREVIOUS_MANIFEST_SHA256, + 'root': root.resolve(), 'archives': archives, 'binaryHashes': binary_hashes} def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, Any]]]: @@ -179,28 +230,73 @@ def run(rehearsal: Path, output: Path, brew: str) -> dict[str, Any]: return exercise(rehearsal.resolve() / 'package', manifest, archives, verified, output, brew) -def run_kernel_rc(root: Path, output: Path, brew: str, expected_source: str, expected_version: str) -> dict[str, Any]: +def run_kernel_rc(root: Path, output: Path, brew: str, expected_source: str, expected_version: str, + previous_release: Path | None = None) -> dict[str, Any]: verified, manifest, archives = verify_kernel_rc(root, expected_source, expected_version) - return exercise(root.resolve() / 'package', manifest, archives, verified, output, brew) + previous = verify_previous_release(previous_release, manifest['platform'], manifest['version']) if previous_release else None + if previous and not isinstance(manifest.get('agentsSdk'), dict): + raise ValueError('Genuine SDK upgrade qualification requires a packaged SDK candidate') + return exercise(root.resolve() / 'package', manifest, archives, verified, output, brew, previous=previous) + + +def qualify_installed_sdk(active: Path, manifest: dict[str, Any], verified: dict[str, Any], + command: Any, label: str, *, expected_timeout: str = '10m') -> None: + """Probe the real installed default; synthetic-key dry runs never infer.""" + helper = active.resolve(strict=True).parent / 'prose-agents-sdk' + sdk = manifest.get('agentsSdk') + if not isinstance(sdk, dict) or not helper.is_file() or helper.is_symlink() or pub.digest(helper) != sdk.get('sha256'): + raise ValueError('Homebrew SDK helper differs from the verified package') + command(label + '-sdk-imports', [str(helper), '--packaged-self-test']) + command(label + '-sdk-tools', [str(helper), '--packaged-tool-self-test']) + def report(suffix: str, argv: list[str], code: int = 0, additions: dict[str, str] | None = None) -> dict[str, Any]: + result = command(label + suffix, [str(active), *argv], expected_code=code, additions=additions) + if result.stderr: + raise ValueError('Installed SDK probe emitted unexpected diagnostics') + return json.loads(result.stdout) + explanation = report('-default-explain', ['cli', 'config', 'explain', '--json', '--', 'run', 'input.prose.md']) + if explanation.get('schema') != 'openprose.configuration-explanation/1' or explanation.get('diagnostics'): + raise ValueError('Installed default explanation failed') + for key, value in (('harness', 'agents-sdk'), ('model', 'gpt-6.1-sol'), ('authProfile', 'openai-api-key')): + if explanation.get('values', {}).get(key, {}).get('value') != value: + raise ValueError('Installed SDK default selection differs') + if explanation.get('values', {}).get('timeout', {}).get('value') != expected_timeout: + raise ValueError('Installed SDK explanation did not inherit the expected user settings') + missing = report('-missing-key', ['--output', 'json', 'run', 'input.prose.md'], 10) + if missing.get('schema') != 'openprose.runner-error/1' or missing.get('code') != 'HARNESS_NEEDS_AUTH' or missing.get('details', {}).get('fallbackAttempted') is not False: + raise ValueError('Installed missing-key preflight did not fail safely') + # This is an inert canary, never an account credential. Only --dry-run gets it. + dry = report('-default-dry-run', ['--output', 'json', '--dry-run', 'run', 'input.prose.md'], + additions={'OPENAI_API_KEY': 'provider-free-installation-canary'}) + if dry.get('schema') != 'openprose.runner-dry-run-report/1' or dry.get('wouldStartModel') is not False or dry.get('readiness') != 'ready': + raise ValueError('Installed SDK dry run did not establish provider-free readiness') + selection = dry.get('selection', {}) + if any(selection.get(key) != value for key, value in (('harness', 'agents-sdk'), ('adapterId', 'agents-sdk/jsonl'), + ('transport', 'jsonl'), ('runtimeVersion', '0.1.0'), ('model', 'gpt-6.1-sol'))): + raise ValueError('Installed command did not discover the packaged SDK default') + if dry.get('billingOwner') != 'user-provider' or dry.get('blockingError') is not None: + raise ValueError('Installed SDK dry run changed billing or reported a blocker') def exercise(package: Path, manifest: dict[str, Any], archives: dict[str, Any], verified: dict[str, Any], - output: Path, brew: str) -> dict[str, Any]: + output: Path, brew: str, *, previous: dict[str, Any] | None = None) -> dict[str, Any]: output.mkdir(exist_ok=False) output = output.resolve() env = {key: os.environ[key] for key in ('PATH', 'HOME', 'USER', 'LOGNAME', 'TMPDIR', 'DEVELOPER_DIR', 'SDKROOT') if key in os.environ} env.update(HOMEBREW_NO_AUTO_UPDATE='1', HOMEBREW_NO_ANALYTICS='1', HOMEBREW_NO_INSTALL_CLEANUP='1', HOMEBREW_CACHE=str(output / 'cache'), HOMEBREW_LOGS=str(output / 'logs'), - HOMEBREW_TEMP=str(output / 'tmp'), XDG_CONFIG_HOME=str(output / 'trust')) - for name in ('cache', 'logs', 'tmp', 'trust'): + HOMEBREW_TEMP=str(output / 'tmp'), XDG_CONFIG_HOME=str(output / 'trust'), + PROSE_CONFIG_DIR=str(output / 'user-settings')) + for name in ('cache', 'logs', 'tmp', 'trust', 'user-settings'): (output / name).mkdir() checks = [] - def command(label: str, argv: list[str], *, expected_failure: bool = False) -> subprocess.CompletedProcess[str]: - result = subprocess.run(argv, env=env, text=True, capture_output=True, timeout=180) + def command(label: str, argv: list[str], *, expected_failure: bool = False, + expected_code: int = 0, additions: dict[str, str] | None = None) -> subprocess.CompletedProcess[str]: + result = subprocess.run(argv, env=dict(env, **(additions or {})), cwd=output, + text=True, capture_output=True, timeout=180) (output / f'{label}.log').write_text(result.stdout + result.stderr) checks.append({'name': label, 'exitCode': result.returncode, 'expectedFailure': expected_failure}) - if (result.returncode != 0) != expected_failure: + if (expected_failure and result.returncode == 0) or (not expected_failure and result.returncode != expected_code): raise ValueError(f'Homebrew check failed: {label}; inspect its retained log') return result @@ -208,12 +304,15 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s if TAP in command('existing-taps', [brew, 'tap']).stdout.splitlines(): raise ValueError('Rehearsal refuses to replace an existing tap') installed = command('existing-installations', [brew, 'list', '--formula', '--versions']).stdout - if any(line.split()[0] in {'prose-bun', 'prose-rust'} for line in installed.splitlines() if line.split()): + if any(line.split()[0].split('/')[-1] in {'prose-bun', 'prose-rust'} for line in installed.splitlines() if line.split()): raise ValueError('Rehearsal requires a fresh Homebrew installation without Prose packages') # Keep an existing command unchanged; CI runners should have no prefix/bin/prose. active = prefix / 'bin/prose' if active.exists() or active.is_symlink(): raise ValueError('Rehearsal refuses to overwrite an existing prose command') + active_helper = prefix / 'bin/prose-agents-sdk' + if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): + raise ValueError('Rehearsal refuses to overwrite an existing SDK helper command') tap = output / 'tap' (tap / 'Formula').mkdir(parents=True) for implementation, item in archives.items(): @@ -225,6 +324,15 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s command('tap', [brew, 'tap', TAP, str(tap)]) completed = False try: + # A local tap is cloned by Homebrew. Upgrade the owned installed checkout, + # not the separate source repository used when initially creating it. + installed_tap = Path(command('installed-tap-repository', [brew, '--repo', TAP]).stdout.strip()) + if installed_tap.is_symlink() or not installed_tap.is_dir(): + raise ValueError('Homebrew did not create a regular owned tap checkout') + for implementation in ('bun', 'rust'): + installed_formula = installed_tap / 'Formula' / f'prose-{implementation}.rb' + if installed_formula.is_symlink() or not installed_formula.is_file() or installed_formula.read_bytes() != (tap / 'Formula' / installed_formula.name).read_bytes(): + raise ValueError('Homebrew installed tap differs from the newly owned formulas') for implementation in ('bun', 'rust'): name = f'{TAP}/prose-{implementation}' command(f'install-{implementation}', [brew, 'install', '--build-from-source', name]) @@ -235,13 +343,8 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s digest = hashlib.sha256(active.read_bytes()).hexdigest() if digest != verified['packageIdentity'][f'{implementation}BinarySha256']: raise ValueError('Homebrew executable differs from the verified candidate') - sdk_record = manifest.get('agentsSdk') - if isinstance(sdk_record, dict): - helper = active.resolve().parent / 'prose-agents-sdk' - if not helper.is_file() or hashlib.sha256(helper.read_bytes()).hexdigest() != sdk_record.get('sha256'): - raise ValueError('Homebrew SDK helper differs from the verified package') - command(f'sdk-self-test-{implementation}', [str(helper), '--packaged-self-test']) - command(f'sdk-tools-self-test-{implementation}', [str(helper), '--packaged-tool-self-test']) + if isinstance(manifest.get('agentsSdk'), dict): + qualify_installed_sdk(active, manifest, verified, command, 'fresh-' + implementation) command(f'unlink-{implementation}', [brew, 'unlink', name]) command('link-bun', [brew, 'link', f'{TAP}/prose-bun']) original = hashlib.sha256(active.read_bytes()).hexdigest() @@ -254,6 +357,55 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s command('switch-link-rust', [brew, 'link', f'{TAP}/prose-rust']) if hashlib.sha256(active.read_bytes()).hexdigest() != verified['packageIdentity']['rustBinarySha256']: raise ValueError('Rust link switch did not select the verified executable') + if previous is not None: + command('uninstall-fresh-before-upgrade', [brew, 'uninstall', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + settings = output / 'user-settings/cli.toml' + settings_bytes = b'# explicit upgrade preservation\ntimeout = "9m"\n' + settings.write_bytes(settings_bytes) + def preserve(label: str) -> None: + if settings.is_symlink() or not settings.is_file() or settings.read_bytes() != settings_bytes: + raise ValueError('Homebrew changed explicit user settings: ' + label) + checks.append({'name': label, 'exitCode': 0, 'sha256': hashlib.sha256(settings_bytes).hexdigest()}) + for selected in ('bun', 'rust'): + for implementation, item in previous['archives'].items(): + (installed_tap / 'Formula' / f'prose-{implementation}.rb').write_text( + formula(previous['version'], implementation, previous['root'] / item['path'], item['sha256'])) + command(selected + '-install-base-bun', [brew, 'install', '--build-from-source', f'{TAP}/prose-bun']) + command(selected + '-unlink-base-bun', [brew, 'unlink', f'{TAP}/prose-bun']) + command(selected + '-install-base-rust', [brew, 'install', '--build-from-source', f'{TAP}/prose-rust']) + if selected == 'bun': + command(selected + '-unlink-base-rust', [brew, 'unlink', f'{TAP}/prose-rust']) + command(selected + '-link-base-bun', [brew, 'link', f'{TAP}/prose-bun']) + if pub.digest(active) != previous['binaryHashes'][selected]: + raise ValueError('Selected prior executable differs from genuine published bytes') + base_banner = command(selected + '-base-version', [str(active), '--version']).stdout.strip() + if base_banner != f'prose {previous["version"]} ({selected})': + raise ValueError('The upgrade did not start from the genuine previous version') + preserve(selected + '-settings-before-upgrade') + for implementation, item in archives.items(): + (installed_tap / 'Formula' / f'prose-{implementation}.rb').write_text( + formula(manifest['version'], implementation, package / item['path'], item['sha256'])) + other = 'rust' if selected == 'bun' else 'bun' + command(selected + '-unlink-selected-before-upgrade', [brew, 'unlink', f'{TAP}/prose-{selected}']) + command(selected + '-upgrade-inactive', [brew, 'upgrade', f'{TAP}/prose-{other}']) + command(selected + '-test-upgraded-inactive', [brew, 'test', f'{TAP}/prose-{other}']) + if pub.digest(active) != verified['packageIdentity'][other + 'BinarySha256']: + raise ValueError('Inactive implementation did not upgrade to exact candidate bytes') + qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-inactive', expected_timeout='9m') + preserve(selected + '-settings-after-inactive-upgrade') + command(selected + '-unlink-upgraded-inactive', [brew, 'unlink', f'{TAP}/prose-{other}']) + command(selected + '-upgrade-selected', [brew, 'upgrade', f'{TAP}/prose-{selected}']) + command(selected + '-restore-selected-link', [brew, 'link', f'{TAP}/prose-{selected}']) + command(selected + '-test-upgraded-selected', [brew, 'test', f'{TAP}/prose-{selected}']) + selected_prefix = Path(command(selected + '-selected-prefix', [brew, '--prefix', f'{TAP}/prose-{selected}']).stdout.strip()) + if active.resolve() != (selected_prefix / 'bin/prose').resolve() or pub.digest(active) != verified['packageIdentity'][selected + 'BinarySha256']: + raise ValueError('Upgrade changed the selected implementation or candidate identity') + qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-selected', expected_timeout='9m') + preserve(selected + '-settings-after-selected-upgrade') + command(selected + '-uninstall-upgraded', [brew, 'uninstall', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + preserve(selected + '-settings-after-uninstall') + if active.exists() or active.is_symlink(): + raise ValueError('Upgraded command survived uninstall') completed = True finally: # Remove only packages under the newly owned rehearsal tap. @@ -265,15 +417,24 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s if TAP in command('remaining-taps', [brew, 'tap']).stdout.splitlines(): raise ValueError('The rehearsal tap did not uninstall cleanly') remaining = command('remaining-installations', [brew, 'list', '--formula', '--versions']).stdout - if any(line.split()[0] in {'prose-bun', 'prose-rust'} for line in remaining.splitlines() if line.split()): + if any(line.split()[0].split('/')[-1] in {'prose-bun', 'prose-rust'} for line in remaining.splitlines() if line.split()): raise ValueError('The rehearsal packages did not uninstall cleanly') if active.exists() or active.is_symlink(): raise ValueError('The rehearsal command did not uninstall cleanly') + if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): + raise ValueError('The rehearsal SDK helper command did not uninstall cleanly') + if previous: + preserve('settings-after-final-cleanup') receipt = {'schema': 'openprose.homebrew-rehearsal/1', 'status': ('passed-native-rc-packaging-check' if verified['custodyKind'] == 'kernel-rc-native' else 'passed-development-packaging-check') if completed else 'failed', 'version': manifest['version'], 'platform': manifest['platform'], 'packageIdentity': verified['packageIdentity'], 'source': verified.get('source', verified['packageIdentity'].get('sourceRevision', manifest.get('source', {}).get('revision'))), 'custodyKind': verified['custodyKind'], 'rehearsalManifestSha256': verified['manifestSha256'], 'checks': checks, 'modelCalls': 0, 'publicationAuthorized': False, 'releaseQualification': False, 'uninstallPassed': True} + receipt['upgradeQualification'] = 'passed-genuine-upgrade-both-selections' if previous else 'not-requested-fresh-install-only' + receipt['kernelRetrieval'] = 'published-kernel-may-be-acquired-by-provider-free-dry-run' if manifest.get('agentsSdk') else 'not-measured' + if previous: + receipt['previousRelease'] = {'version': previous['version'], 'manifestSha256': previous['manifestSha256'], + 'archives': list(previous['archives'].values())} if 'nativeReportSha256' in verified: receipt['nativeReportSha256'] = verified['nativeReportSha256'] receipt['archiveIdentities'] = verified['archiveIdentities'] @@ -288,15 +449,16 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False) -> s inputs.add_argument('--kernel-rc', type=Path) parser.add_argument('--expected-source') parser.add_argument('--expected-version') + parser.add_argument('--previous-release', type=Path, help='Untouched pinned RC3 manifest.json and current-platform archives; enables genuine upgrades') parser.add_argument('--output', type=Path, required=True) parser.add_argument('--brew', default='brew') args = parser.parse_args() if args.kernel_rc: if not args.expected_source or not args.expected_version: parser.error('--kernel-rc requires --expected-source and --expected-version') - result = run_kernel_rc(args.kernel_rc, args.output, args.brew, args.expected_source, args.expected_version) + result = run_kernel_rc(args.kernel_rc, args.output, args.brew, args.expected_source, args.expected_version, args.previous_release) else: - if args.expected_source or args.expected_version: - parser.error('Expected source/version are only supported with --kernel-rc') + if args.expected_source or args.expected_version or args.previous_release: + parser.error('Expected source/version and previous release are only supported with --kernel-rc') result = run(args.rehearsal, args.output, args.brew) print(json.dumps(result, indent=2)) diff --git a/cli/ci/npm_alias_install.py b/cli/ci/npm_alias_install.py index dbc8b53b..377a830c 100644 --- a/cli/ci/npm_alias_install.py +++ b/cli/ci/npm_alias_install.py @@ -50,6 +50,6 @@ def log_message(self, *args): offline_env = dict(cache_env, npm_config_offline='true') command(['npm', 'install', '--global', '--prefix', prefix, '--offline', '--ignore-scripts', '--no-audit', '--no-fund', meta], env=offline_env, cwd=cwd, log=log, timeout=120) - return {'registryScope': 'loopback-only', 'cacheRequests': requests, + return {'registryScope': 'loopback-only', 'registryUrl': registry, 'cacheRequests': requests, 'installation': 'offline-original-root-tarball', 'platformVersion': version, 'platformSha256': hashlib.sha256(payload).hexdigest()} diff --git a/cli/ci/qualify_installed_sdk.py b/cli/ci/qualify_installed_sdk.py new file mode 100644 index 00000000..89407906 --- /dev/null +++ b/cli/ci/qualify_installed_sdk.py @@ -0,0 +1,324 @@ +#!/usr/bin/env python3 +"""Qualify original installed RC bytes in isolated consumer roots; never publish.""" +from __future__ import annotations +import argparse +import hashlib +import json +import os +from pathlib import Path +import re +import shutil +import sys + +import homebrew_rehearsal as custody +import kernel_rc_evidence +import npm_alias_install +import package_local +import publication as pub + +PREVIOUS_MANIFEST_SHA256 = '571eb285f964ea980d4e3aec6aa574f782b9d51f2a6934d219bc2ae7d8755e4f' +SETUP_ACTION = 'Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.' +SDK_NAMES = ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt') +AUTH_CANARY = 'provider-free-installation-canary' + + +def offline_registry_environment(env: dict[str, str], installation: dict) -> dict[str, str]: + """npm keys cached metadata by registry URL, including its loopback port.""" + registry = installation.get('registryUrl') + pub.require(isinstance(registry, str) and re.fullmatch(r'http://127\.0\.0\.1:[1-9][0-9]{0,4}', registry) and int(registry.rsplit(':', 1)[1]) <= 65535, 'Exact loopback cache registry URL is required') + return dict(env, npm_config_registry=registry, npm_config_offline='true') + + +def regular(path: Path) -> None: + pub.require(path.is_file() and not path.is_symlink(), 'Expected regular file: ' + str(path)) + + +def rc_version(version: str) -> tuple[int, ...]: + pub.require(isinstance(version, str) and re.fullmatch(r'(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)-rc\.(0|[1-9][0-9]*)', version), 'Exact RC version required') + return tuple(int(x) for x in re.split(r'\.|-rc\.', version)) + + +def artifact(root: Path, record: dict, *, previous=False) -> Path: + name = record['name' if previous else 'path'] + pub.require(pub.safe_name(name), 'Unsafe artifact name') + path = root / name + regular(path) + length = record['size' if previous else 'byteLength'] + pub.require(type(length) is int and 0 < length <= pub.MAX_BYTES and path.stat().st_size == length and pub.digest(path) == record['sha256'], 'Artifact bytes differ: ' + name) + return path + + +def npm_inputs(root: Path, manifest: dict, platform: str, *, previous=False, expected_binary=None) -> tuple[Path, Path]: + kinds = ('npm', 'npm') if previous else ('npm-meta', 'npm-platform') + platforms = ('all', platform) if previous else (None, platform) + selected = [] + for kind, target in zip(kinds, platforms): + records = [a for a in manifest['artifacts'] if a['kind'] == kind and a.get('platform') == target] + pub.require(len(records) == 1, 'Exactly one npm artifact per role is required') + selected.append(artifact(root, records[0], previous=previous)) + meta, native = [pub.archive_members(path) for path in selected] + for members in (meta, native): + pub.require('package/package.json' in members, 'Missing npm package metadata') + root_package, platform_package = [json.loads(m['package/package.json'], object_pairs_hook=pub.object_pairs) for m in (meta, native)] + version = manifest['version'] + source = manifest['source'] if previous else manifest['source']['revision'] + pub.require(root_package.get('name') == platform_package.get('name') == '@openprose/prose-cli' and root_package.get('version') == version and platform_package.get('version') == package_local.npm_payload_version(version, platform), 'Original same-name npm identities differ') + for record in (root_package, platform_package): + pub.require(not record.get('scripts') and record.get('openproseCohort', {}).get('version') == version and record['openproseCohort'].get('sourceRevision') == source, 'npm source/cohort or lifecycle differs') + alias = '@openprose/prose-cli-' + platform + pub.require(root_package.get('optionalDependencies', {}).get(alias) == 'npm:@openprose/prose-cli@' + platform_package['version'], 'npm platform alias differs') + pub.require('package/bin/prose.js' in meta and 'package/bin/prose' in native, 'Missing npm launcher/native bytes') + if expected_binary is not None: + pub.require(hashlib.sha256(native['package/bin/prose']).hexdigest() == expected_binary, 'npm native payload differs from verified standalone') + return tuple(selected) + + +def extract_standalone(archive: Path, directory: Path, *, sdk: bool) -> dict: + """Extract only verified regular payloads, never archive metadata or links.""" + pub.require(not directory.exists() and not directory.is_symlink(), 'Installation must be fresh') + members = pub.archive_members(archive) + executables = [name for name in members if Path(name).name == 'prose'] + pub.require(len(executables) == 1 and len(Path(executables[0]).parts) == 2, 'Unique package-root CLI required') + prefix = executables[0].rsplit('/', 1)[0] + '/' + names = ('prose', *SDK_NAMES) if sdk else ('prose',) + pub.require(all(prefix + name in members for name in names), 'Required executable siblings missing') + directory.mkdir(parents=True) + receipt = {} + for name in names: + data = members[prefix + name] + path = directory / name + with path.open('xb') as stream: + stream.write(data) + path.chmod(0o755 if name in ('prose', 'prose-agents-sdk') else 0o644) + receipt[name] = {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + return receipt + + +class Qualification: + def __init__(self, output: Path, node: Path, npm: Path): + pub.require(not output.exists() and not output.is_symlink(), 'Output must be fresh') + regular(node); regular(npm) + pub.require(os.access(node, os.X_OK), 'Node must be executable') + self.output, self.node, self.npm = output.absolute(), node, npm + output.mkdir(parents=True) + self.logs = output / 'logs'; self.logs.mkdir() + self.tools = output / 'node-only'; self.tools.mkdir() + (self.tools / 'node').symlink_to(node) + self.commands = []; self.checks = []; self.payloads = [] + + def environment(self, root: Path, *, npm=False, explicit_config=False) -> dict[str, str]: + # A boundary marker stops project configuration discovery at this + # isolated consumer root even when evidence lives beneath a checkout. + # It contains no Git history or source files and never invokes Git. + boundary = root / '.git'; boundary.mkdir(exist_ok=True) + home = root / 'home'; home.mkdir(parents=True, exist_ok=True) + temporary = root / 'tmp'; temporary.mkdir(exist_ok=True) + env = {'HOME': str(home), 'TMPDIR': str(temporary), 'LANG': 'C', 'LC_ALL': 'C', 'PATH': str(self.tools) if npm else ''} + if explicit_config: + config = root / 'preferences'; config.mkdir(exist_ok=True) + env['PROSE_CONFIG_DIR'] = str(config) + if npm: + userconfig = root / 'user.npmrc'; userconfig.write_text('') + globalconfig = root / 'global.npmrc'; globalconfig.write_text('') + env.update(npm_config_userconfig=str(userconfig), npm_config_globalconfig=str(globalconfig), npm_config_cache=str(root / 'npm-cache'), npm_config_registry='http://127.0.0.1:9', npm_config_offline='true', npm_config_update_notifier='false', npm_config_script_shell='/bin/sh') + return env + + def execute(self, argv, *, env, cwd, label, expected=0, timeout=120): + args = [str(x) for x in argv] + if args[0] == 'npm': + args = [str(self.node), str(self.npm), *args[1:]] + result = package_local.run_bounded(args, cwd=cwd, environment=env, timeout_seconds=timeout, label=label) + record = {'argv': args, 'exitCode': result.returncode, 'stdout': result.stdout.decode('utf-8', errors='strict'), 'stderr': result.stderr.decode('utf-8', errors='strict')} + path = self.logs / (str(len(self.commands)).zfill(3) + '-' + label + '.json') + path.write_text(json.dumps(record, sort_keys=True) + '\n') + self.commands.append({'label': label, 'path': str(path.relative_to(self.output)), 'sha256': pub.digest(path), 'byteLength': path.stat().st_size}) + pub.require(result.returncode == expected, 'Unexpected exit for ' + label + '; inspect ' + str(path)) + return record + + def npm_command(self, args, *, env, cwd, log, timeout=120): + return self.execute(args, env=env, cwd=cwd, label=log.stem, timeout=timeout) + + def json_command(self, command, args, root, env, label, expected=0): + result = self.execute([*command, *args], cwd=root, env=env, label=label, expected=expected) + pub.require(result['stderr'] == '', 'Unexpected diagnostic stderr for ' + label) + return json.loads(result['stdout'], object_pairs_hook=pub.object_pairs) + + def identity(self, command, runner, version, source, root, env, label): + banner = self.execute([*command, '--version'], cwd=root, env=env, label=label + '-version') + pub.require(banner['stdout'].strip() == f'prose {version} ({runner})' and not banner['stderr'], 'Installed version/runner mismatch') + report = self.json_command(command, ['--output=json', 'cli', 'doctor'], root, env, label + '-doctor', 10) + pub.require(report.get('runner') == {'name': runner, 'version': version, 'commit': source}, 'Installed source identity mismatch') + pub.require(report.get('build', {}).get('profile') == 'release' and report['build'].get('testSeamsEnabled') is False, 'Production release without seams required') + return report + + def defaults(self, command, runner, version, source, root, env, label): + doctor = self.identity(command, runner, version, source, root, env, label) + pub.require(doctor.get('selectedHarness') == 'agents-sdk' and doctor.get('selectedTransport') == 'jsonl', 'SDK must be the installed default') + auth = doctor.get('problems', []) + pub.require(len(auth) == 1 and auth[0].get('code') == 'HARNESS_NEEDS_AUTH' and auth[0].get('action') == SETUP_ACTION and auth[0].get('details') == {'adapterId': 'agents-sdk/jsonl', 'authProfile': 'openai-api-key', 'fallbackAttempted': False}, 'Missing-key doctor differs') + explain = self.json_command(command, ['cli', 'config', 'explain', '--json', '--', 'run', 'missing.prose.md'], root, env, label + '-explain') + for key, value in {'harness': 'agents-sdk', 'model': 'gpt-6.1-sol', 'authProfile': 'openai-api-key'}.items(): + pub.require(explain['values'][key]['value'] == value and explain['values'][key]['source']['kind'] == 'default', 'Defaults must inherit without writing overrides') + path = Path(env.get('PROSE_CONFIG_DIR', str(Path(env['HOME']) / '.prose'))) / 'cli.toml' + pub.require(not path.exists() and not path.is_symlink(), 'Read-only diagnostics persisted settings') + missing = self.json_command(command, ['--output=json', 'run', 'missing.prose.md'], root, env, label + '-missing-key', 10) + pub.require(missing.get('schema') == 'openprose.runner-error/1' and missing.get('code') == 'HARNESS_NEEDS_AUTH' and missing.get('action') == SETUP_ACTION, 'Missing-key run must fail before inference/image acquisition') + dry = self.json_command(command, ['--output=json', '--dry-run', 'run', 'missing.prose.md'], root, env, label + '-dry-run', 10) + pub.require(dry.get('schema') == 'openprose.runner-dry-run-report/1' and dry.get('blockingError', {}).get('code') == 'HARNESS_NEEDS_AUTH', 'Installed default dry-run must diagnose absent key') + ready = self.json_command(command, ['--output=json', '--dry-run', 'run', 'missing.prose.md'], root, dict(env, OPENAI_API_KEY=AUTH_CANARY), label + '-helper-discovery') + selection = ready.get('selection', {}) + pub.require(ready.get('schema') == 'openprose.runner-dry-run-report/1' and ready.get('wouldStartModel') is False and ready.get('readiness') == 'ready' and ready.get('blockingError') is None and ready.get('billingOwner') == 'user-provider', 'Synthetic-key dry-run must be ready without inference') + pub.require(all(selection.get(key) == value for key, value in {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}.items()), 'Installed CLI did not discover its packaged SDK sibling') + pub.require(not path.exists(), 'Dry-run persisted settings') + self.checks.append({'name': label, 'status': 'pass', 'modelCalls': 0, 'settingsPath': str(path)}) + + def save_preferences(self, command, root, env, label): + path = Path(env.get('PROSE_CONFIG_DIR', str(Path(env['HOME']) / '.prose'))) / 'cli.toml' + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b'# retained preference\ntimeout = "1m"\n') + self.json_command(command, ['--output=json', 'cli', 'harness', 'use', 'claude', '--model', 'claude-sonnet-4-6', '--auth-profile', 'anthropic-api-key'], root, env, label + '-save') + data = path.read_bytes() + pub.require(b'# retained preference' in data and b'timeout = "1m"' in data, 'Preference writer lost unrelated settings') + return path, data + + def preferences(self, command, root, env, label, saved): + path, data = saved + pub.require(path.read_bytes() == data, 'Upgrade changed explicit preferences') + report = self.json_command(command, ['--output=json', 'cli', 'config', 'explain'], root, env, label + '-saved-explain') + for key, value in {'harness': 'claude', 'model': 'claude-sonnet-4-6', 'authProfile': 'anthropic-api-key', 'timeout': '1m'}.items(): + pub.require(report['values'][key]['value'] == value and report['values'][key]['source']['kind'] == 'user-config', 'Saved alternative or override lost') + override = self.json_command(command, ['cli', 'config', 'explain', '--json', '--', '--harness', 'agents-sdk', '--model', 'gpt-6.1-sol', '--auth-profile', 'openai-api-key', 'run', 'missing.prose.md'], root, env, label + '-explicit-override') + pub.require(override['values']['harness']['value'] == 'agents-sdk' and override['values']['harness']['source']['kind'] == 'flag' and path.read_bytes() == data, 'Explicit override persisted or failed') + + def payload(self, directory, expected_binary, sdk_record): + regular(directory / 'prose') + pub.require(pub.digest(directory / 'prose') == expected_binary, 'Installed CLI bytes differ') + for name, key in zip(SDK_NAMES, ('sha256', 'receiptSha256', 'noticesSha256')): + regular(directory / name) + pub.require(pub.digest(directory / name) == sdk_record[key], 'Installed SDK sibling bytes differ') + for argument, expected in (('--packaged-self-test', kernel_rc_evidence.SDK_IMPORT_TEST), ('--packaged-tool-self-test', kernel_rc_evidence.SDK_TOOL_TEST)): + env = {'PATH': '', 'HOME': str(self.output), 'TMPDIR': str(self.output), 'LANG': 'C', 'LC_ALL': 'C'} + record = self.execute([directory / 'prose-agents-sdk', argument], env=env, cwd=self.output, label=directory.parent.name + '-' + argument.lstrip('-'), timeout=45) + pub.require(not record['stderr'] and json.loads(record['stdout']) == expected, 'Installed SDK self-test differs') + identities = {name: {'sha256': pub.digest(directory / name), 'byteLength': (directory / name).stat().st_size} for name in ('prose', *SDK_NAMES)} + pub.require(identities['prose']['sha256'] == expected_binary and all(identities[name]['sha256'] == sdk_record[key] for name, key in zip(SDK_NAMES, ('sha256', 'receiptSha256', 'noticesSha256'))), 'Installed bytes changed during probes') + self.payloads.append({'directory': str(directory.relative_to(self.output)), 'members': identities}) + + +def qualify(candidate_root, previous_release, output, source, version, node, npm): + """All writes and installations are beneath fresh output; prior bytes stay original.""" + verified, manifest, archives = custody.verify_kernel_rc(candidate_root, source, version) + kernel_rc_evidence.validate_sdk_archives(manifest, lambda name: pub.archive_members(candidate_root / 'package' / name)) + previous = custody.verify_previous_release(previous_release, manifest['platform'], version) + previous_plan = custody.read_previous_manifest(previous_release) + pub.require(pub.digest(previous_release / 'manifest.json') == PREVIOUS_MANIFEST_SHA256, 'Previous manifest is not the pinned published release') + pub.require(rc_version(version) > rc_version(previous_plan['version']), 'A genuine version upgrade is required') + pub.require(manifest['platform'] == package_local.current_platform_id(), 'Candidate must match the native host') + candidate_npm = npm_inputs(candidate_root / 'package', manifest, manifest['platform'], expected_binary=verified['packageIdentity']['bunBinarySha256']) + previous_npm = npm_inputs(previous_release, previous_plan, manifest['platform'], previous=True, expected_binary=previous['binaryHashes']['bun']) + q = Qualification(output, node, npm) + inputs = [candidate_root / 'package' / a['path'] for a in manifest['artifacts']] + [candidate_root / 'package/release-manifest.json', candidate_root / 'build-report.json', previous_release / 'manifest.json', *previous_npm, node, npm] + inputs += [previous_release / a['name'] for a in previous['archives'].values()] + input_hashes = {str(path): pub.digest(path) for path in inputs} + for runner in ('bun', 'rust'): + root = output / ('standalone-' + runner); root.mkdir() + env = q.environment(root, explicit_config=runner == 'rust') + fresh = root / 'fresh' + extract_standalone(candidate_root / 'package' / archives[runner]['path'], fresh, sdk=True) + q.payload(fresh, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) + q.defaults([fresh / 'prose'], runner, version, source, root, env, runner + '-fresh') + relocated = root / 'relocated'; fresh.rename(relocated) + q.payload(relocated, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) + q.defaults([relocated / 'prose'], runner, version, source, root, env, runner + '-relocated') + saved = q.save_preferences([relocated / 'prose'], root, env, runner) + upgrade = root / 'upgrade' + extract_standalone(previous_release / previous['archives'][runner]['name'], upgrade, sdk=False) + pub.require(pub.digest(upgrade / 'prose') == previous['binaryHashes'][runner], 'Installed previous bytes differ') + q.identity([upgrade / 'prose'], runner, previous_plan['version'], previous_plan['source'], root, env, runner + '-previous') + shutil.rmtree(upgrade) + extract_standalone(candidate_root / 'package' / archives[runner]['path'], upgrade, sdk=True) + q.payload(upgrade, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) + q.identity([upgrade / 'prose'], runner, version, source, root, env, runner + '-upgraded') + q.preferences([upgrade / 'prose'], root, env, runner + '-upgraded', saved) + q.payload(upgrade, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) + shutil.rmtree(upgrade); shutil.rmtree(relocated) + pub.require(saved[0].read_bytes() == saved[1], 'Standalone uninstall removed user settings') + q.checks.append({'name': runner + '-upgrade-uninstall', 'status': 'pass', 'settingsSha256': hashlib.sha256(saved[1]).hexdigest()}) + root = output / 'npm'; root.mkdir() + env = q.environment(root, npm=True, explicit_config=True) + prefix = root / 'prefix' + def install(packages, label): + return npm_alias_install.install(*packages, prefix, env=env, cwd=root, command=q.npm_command, log=q.logs / (label + '.log')) + installs = [install(candidate_npm, 'npm-fresh')] + launcher = prefix / 'bin/prose' + def npm_payload(): + native = list((prefix / 'lib/node_modules').rglob('prose-agents-sdk')) + pub.require(len(native) == 1, 'Exactly one installed npm SDK helper is required') + q.payload(native[0].parent, verified['packageIdentity']['bunBinarySha256'], manifest['agentsSdk']) + meta = pub.archive_members(candidate_npm[0]) + pub.require(pub.digest(launcher.resolve(strict=True)) == hashlib.sha256(meta['package/bin/prose.js']).hexdigest(), 'Installed npm launcher differs') + npm_payload() + q.defaults([node, launcher], 'bun', version, source, root, env, 'npm-fresh') + saved = q.save_preferences([node, launcher], root, env, 'npm') + q.execute(['npm', 'uninstall', '--global', '--prefix', prefix, '--offline', '--ignore-scripts', '--no-audit', '--no-fund', '@openprose/prose-cli'], env=env, cwd=root, label='npm-remove-fresh') + installs.append(install(previous_npm, 'npm-previous')) + q.identity([node, launcher], 'bun', previous_plan['version'], previous_plan['source'], root, env, 'npm-previous') + installs.append(install(candidate_npm, 'npm-upgrade')) + npm_payload() + q.identity([node, launcher], 'bun', version, source, root, env, 'npm-upgraded') + q.preferences([node, launcher], root, env, 'npm-upgraded', saved) + npm_payload() + q.execute(['npm', 'uninstall', '--global', '--prefix', prefix, '--offline', '--ignore-scripts', '--no-audit', '--no-fund', '@openprose/prose-cli'], env=env, cwd=root, label='npm-uninstall') + pub.require(saved[0].read_bytes() == saved[1] and not launcher.exists(), 'npm uninstall changed settings or left launcher') + q.checks.append({'name': 'npm-genuine-upgrade-uninstall', 'status': 'pass', 'settingsSha256': hashlib.sha256(saved[1]).hexdigest()}) + root = output / 'npx'; root.mkdir() + env = q.environment(root, npm=True) + # Populate only the exact platform package using the established offline + # installer; npm exec itself installs the original root tarball in _npx. + install_record = npm_alias_install.install(*candidate_npm, root / 'cache-primer-prefix', env=env, cwd=root, command=q.npm_command, log=q.logs / 'npx-prime.log') + env = offline_registry_environment(env, install_record) + shutil.rmtree(root / 'cache-primer-prefix') + command = [node, npm, 'exec', '--offline', '--yes', '--ignore-scripts', '--no-audit', '--no-fund', '--package', candidate_npm[0], '--', 'prose'] + q.defaults(command, 'bun', version, source, root, env, 'npx-cache') + cache_launchers = list((root / 'npm-cache' / '_npx').rglob('@openprose/prose-cli/bin/prose.js')) + pub.require(len(cache_launchers) == 1, 'Actual npx cache launcher is required') + cache_launcher = cache_launchers[0] + meta = pub.archive_members(candidate_npm[0]) + pub.require(pub.digest(cache_launcher) == hashlib.sha256(meta['package/bin/prose.js']).hexdigest(), 'npx launcher bytes differ') + helpers = list((root / 'npm-cache' / '_npx').rglob('prose-agents-sdk')) + pub.require(len(helpers) == 1, 'Actual npx cache SDK helper required') + q.payload(helpers[0].parent, verified['packageIdentity']['bunBinarySha256'], manifest['agentsSdk']) + saved = q.save_preferences(command, root, env, 'npx') + q.preferences(command, root, env, 'npx-saved', saved) + q.payload(helpers[0].parent, verified['packageIdentity']['bunBinarySha256'], manifest['agentsSdk']) + shutil.rmtree(root / 'npm-cache' / '_npx') + pub.require(saved[0].read_bytes() == saved[1], 'npx cache removal changed settings') + q.checks.append({'name': 'npx-cache-removal-settings', 'status': 'pass', 'settingsSha256': hashlib.sha256(saved[1]).hexdigest()}) + for path, digest in input_hashes.items(): + pub.require(pub.digest(Path(path)) == digest, 'Original qualification input changed') + report = {'schema': 'openprose.installed-sdk-qualification/1', 'status': 'pass', 'sourceRevision': source, 'version': version, 'platform': manifest['platform'], 'candidate': verified, 'previous': {'version': previous_plan['version'], 'source': previous_plan['source'], 'manifestSha256': PREVIOUS_MANIFEST_SHA256}, 'inputSha256': input_hashes, 'node': {'path': str(node), 'sha256': pub.digest(node)}, 'npm': {'path': str(npm), 'sha256': pub.digest(npm)}, 'checks': q.checks, 'commands': q.commands, 'installedPayloads': q.payloads, 'npmInstallations': [*installs, install_record], 'modelCalls': 0, 'networkScope': 'loopback npm cache priming and public published-kernel retrieval only; no provider credentials', 'networkIsolation': 'not-enforced', 'publicationAuthorized': False, 'globalUserStateModified': False} + (output / 'qualification.json').write_text(json.dumps(report, indent=2, sort_keys=True) + '\n') + return report + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--candidate-root', type=Path, required=True) + parser.add_argument('--source', required=True) + parser.add_argument('--version', required=True) + parser.add_argument('--previous-release', type=Path, required=True) + parser.add_argument('--previous-manifest-sha256', choices=[PREVIOUS_MANIFEST_SHA256], default=PREVIOUS_MANIFEST_SHA256) + parser.add_argument('--node', type=Path, required=True) + parser.add_argument('--npm', type=Path, required=True, help='Exact npm-cli.js file, invoked by explicit Node') + parser.add_argument('--out', type=Path, required=True) + args = parser.parse_args(argv) + try: + report = qualify(args.candidate_root.absolute(), args.previous_release.absolute(), args.out.absolute(), args.source, args.version, args.node.resolve(strict=True), args.npm.resolve(strict=True)) + print(json.dumps({'status': report['status'], 'report': str(args.out.absolute() / 'qualification.json')}, sort_keys=True)) + except (ValueError, OSError, KeyError, package_local.PackageError) as error: + parser.exit(2, str(error) + '\n') + return 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/cli/ci/test_check_workflows.py b/cli/ci/test_check_workflows.py index e099cbce..10752d33 100644 --- a/cli/ci/test_check_workflows.py +++ b/cli/ci/test_check_workflows.py @@ -204,6 +204,34 @@ def test_release_homebrew_custody_cannot_select_another_source_or_version(self): self.changed(name, lambda w, j, i=index, a=argument: j["steps"][i].update( run=j["steps"][i]["run"].replace(a, "--wrong-argument"))) + def test_genuine_installed_upgrade_gates_cannot_be_bypassed(self): + for name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): + steps = self.workflows[name]["jobs"][JOBS[name]]["steps"] + for label in ("Fetch pinned genuine previous release inputs", + "Qualify actual SDK installations and genuine upgrades"): + index = next(i for i, step in enumerate(steps) if step.get("name") == label) + self.changed(name, lambda w, j, i=index: j["steps"].pop(i)) + self.changed(name, lambda w, j, i=index: j["steps"][i].update({"if": "false"})) + self.changed(name, lambda w, j, i=index: j["steps"][i].update(run="echo skipped")) + installed = next(i for i, step in enumerate(steps) + if step.get("name") == "Qualify actual SDK installations and genuine upgrades") + for key in ("EXPECTED_SOURCE", "RC_VERSION"): + self.changed(name, lambda w, j, i=installed, k=key: + j["steps"][i]["env"].update({k: "unreviewed"})) + for argument in ("--candidate-root", "--source", "--version", "--previous-release", "--node", "--npm"): + self.changed(name, lambda w, j, i=installed, a=argument: + j["steps"][i].update(run=j["steps"][i]["run"].replace(a, "--wrong-argument"))) + label = ("Rehearse Homebrew against these exact release archives" if name == "cli-kernel-rc.yml" + else "Rehearse Homebrew against the production SDK archives") + homebrew = next(i for i, step in enumerate(steps) if step.get("name") == label) + self.changed(name, lambda w, j, i=homebrew: + j["steps"][i].update(run=j["steps"][i]["run"].replace("--previous-release", "--wrong-argument"))) + retention = next(i for i, step in enumerate(steps) + if step.get("uses", "").startswith("actions/upload-artifact@")) + for path in ("${{ runner.temp }}/kernel-rc/installed-qualification", "${{ runner.temp }}/previous-rc3"): + self.changed(name, lambda w, j, i=retention, p=path: + j["steps"][i]["with"].update(path=j["steps"][i]["with"]["path"].replace(p, "omitted"))) + def test_locked_dependency_guards_and_required_commands(self): for name in JOBS: if name == "cli-publish.yml": diff --git a/cli/ci/test_fetch_previous_release.py b/cli/ci/test_fetch_previous_release.py new file mode 100644 index 00000000..ce6cf97f --- /dev/null +++ b/cli/ci/test_fetch_previous_release.py @@ -0,0 +1,154 @@ +from __future__ import annotations +import copy +import hashlib +import io +import json +from pathlib import Path +import tempfile +import unittest +from unittest.mock import patch +import fetch_previous_release as fetcher + + +def digest(value): return hashlib.sha256(value).hexdigest() + +def fixture(): + objects={} + artifacts=[] + for platform in fetcher.PLATFORMS: + for kind,implementation in [('standalone','rust'),('standalone','bun'),('npm','bun')]: + name=f'{kind}-{implementation}-{platform}.tgz' + body=name.encode() + objects[fetcher.BASE_URL+name]=body + artifacts.append(dict(kind=kind,implementation=implementation,platform=platform,name=name,size=len(body),sha256=digest(body))) + name='npm-bun-all.tgz'; body=b'meta';objects[fetcher.BASE_URL+name]=body + artifacts.append(dict(kind='npm',implementation='bun',platform='all',name=name,size=len(body),sha256=digest(body))) + manifest=dict(schema='openprose.cli-distribution/1',version=fetcher.VERSION,source=fetcher.SOURCE,artifacts=artifacts) + return manifest,objects + +class FetchPreviousReleaseTest(unittest.TestCase): + def run_fetch(self, root, manifest=None, mutate=None, platform='darwin-arm64'): + default,objects=fixture();manifest=manifest or default + encoded=(json.dumps(manifest,sort_keys=True)+'\n').encode() + objects[fetcher.BASE_URL+'manifest.json']=encoded + calls=[] + def download(url,sink,maximum): + calls.append(url) + if mutate: mutate(url,sink,objects) + sink.write(objects[url]) + with patch.object(fetcher,'MANIFEST_SHA256',digest(encoded)): + receipt=fetcher.fetch(platform,root,download) + return receipt,calls,encoded + + def test_all_four_platforms_select_exact_bytes_and_fixed_same_origin(self): + for platform in fetcher.PLATFORMS: + with self.subTest(platform=platform),tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'fresh';receipt,calls,manifest=self.run_fetch(root,platform=platform) + self.assertEqual(5,len(calls));self.assertEqual(4,len(receipt['artifacts'])) + self.assertEqual(manifest,(root/'manifest.json').read_bytes()) + self.assertEqual(platform,receipt['platform']) + self.assertEqual(digest(manifest),receipt['manifestSha256']) + self.assertTrue(all(url.startswith(fetcher.BASE_URL) for url in calls)) + self.assertTrue(all(platform in row['name'] or row['name']=='npm-bun-all.tgz' for row in receipt['artifacts'])) + self.assertEqual(receipt,json.loads((root/'download-receipt.json').read_text())) + for row in receipt['artifacts']: + data=(root/row['name']).read_bytes() + self.assertEqual(row['sha256'],digest(data));self.assertEqual(row['byteLength'],len(data)) + + def test_fixed_manifest_hash_and_metadata_required(self): + manifest,_=fixture();encoded=json.dumps(manifest).encode() + with self.assertRaisesRegex(ValueError,'identity'):fetcher.decode_manifest(encoded) + for key,value in [('schema','wrong'),('version','0.15.0-rc.4'),('source','0'*40)]: + changed=copy.deepcopy(manifest);changed[key]=value;data=json.dumps(changed).encode() + with patch.object(fetcher,'MANIFEST_SHA256',digest(data)),self.assertRaisesRegex(ValueError,'fixed published'):fetcher.decode_manifest(data) + for body in (b'not JSON',b'[]'): + with patch.object(fetcher,'MANIFEST_SHA256',digest(body)),self.assertRaises(ValueError):fetcher.decode_manifest(body) + + def test_missing_duplicate_and_cross_platform_selection_rejected(self): + manifest,_=fixture() + for change in ('missing','duplicate','wrong-platform'): + altered=copy.deepcopy(manifest) + if change=='missing':altered['artifacts'].pop(0) + elif change=='duplicate':altered['artifacts'].append(altered['artifacts'][0]) + else:altered['artifacts'][0]['platform']='linux-x64-musl' + with self.subTest(change=change),self.assertRaises(ValueError):fetcher.select_artifacts(altered,'darwin-arm64') + with self.assertRaises(ValueError):fetcher.select_artifacts(manifest,'linux-x64-musl') + + def test_unsafe_names_sizes_digests_and_classifications_rejected(self): + original,_=fixture() + changes=[('name','../escape'),('name','https://evil.test/a'),('name','manifest.json'),('name','download-receipt.json'),('name','a?query'),('size',True),('size',0),('size',fetcher.MAX_ARTIFACT+1),('sha256','bad'),('platform',[])] + for key,value in changes: + changed=copy.deepcopy(original);changed['artifacts'][0][key]=value + with self.subTest(key=key,value=value),self.assertRaises(ValueError):fetcher.select_artifacts(changed,'darwin-arm64') + + def test_output_must_be_fresh_and_never_overwrites(self): + with tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'existing';root.mkdir();(root/'keep').write_bytes(b'keep') + with self.assertRaisesRegex(ValueError,'fresh'):self.run_fetch(root) + self.assertEqual(b'keep',(root/'keep').read_bytes()) + link=Path(tmp).resolve()/'link';link.symlink_to(root,target_is_directory=True) + with self.assertRaises(ValueError):self.run_fetch(link) + with self.assertRaisesRegex(ValueError,'symlinks'):self.run_fetch(link/'child') + self.assertFalse((root/'child').exists()) + + def test_wrong_artifact_bytes_and_short_download_fail_without_receipt(self): + for body in (b'wrong',b''): + with self.subTest(body=body),tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'fresh' + def corrupt(url,sink,objects): + if url.endswith('standalone-rust-darwin-arm64.tgz'):objects[url]=body + with self.assertRaises(ValueError):self.run_fetch(root,mutate=corrupt) + self.assertFalse((root/'download-receipt.json').exists()) + + def test_bounded_download_enforces_length_even_without_content_length(self): + stream=io.BytesIO();writer=fetcher.BoundedWriter(stream,4);writer.write(b'ab') + with self.assertRaises(ValueError):writer.write(b'cde') + self.assertEqual(b'ab',stream.getvalue()) + with tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'fresh' + def excessive(url,sink,objects): + if url.endswith('.tgz'):sink.write(b'x'*1024) + with self.assertRaises(ValueError):self.run_fetch(root,mutate=excessive) + self.assertFalse((root/'download-receipt.json').exists()) + + def test_later_download_cannot_tamper_with_prior_verified_bytes(self): + with tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'fresh' + def tamper(url,sink,objects): + if url.endswith('npm-bun-darwin-arm64.tgz'):(root/'standalone-rust-darwin-arm64.tgz').write_bytes(b'tampered') + with self.assertRaisesRegex(ValueError,'retained'):self.run_fetch(root,mutate=tamper) + self.assertFalse((root/'download-receipt.json').exists()) + + def test_precreated_symlink_cannot_redirect_artifact_write(self): + with tempfile.TemporaryDirectory() as tmp: + root=Path(tmp).resolve()/'fresh';victim=Path(tmp).resolve()/'victim';victim.write_bytes(b'keep') + def inject(url,sink,objects): + if url.endswith('manifest.json'):(root/'standalone-rust-darwin-arm64.tgz').symlink_to(victim) + with self.assertRaises(FileExistsError):self.run_fetch(root,mutate=inject) + self.assertEqual(b'keep',victim.read_bytes()) + + def test_curl_is_fixed_https_without_redirects_or_ambient_credentials(self): + class Process: + def __init__(self): + self.stdout=io.BytesIO(b'bytes');self.returncode=0 + def wait(self,timeout):return self.returncode + def poll(self):return self.returncode + stream=io.BytesIO() + with patch.object(fetcher.subprocess,'Popen',return_value=Process()) as popen: + fetcher.curl_download(fetcher.BASE_URL+'manifest.json',fetcher.BoundedWriter(stream,5),5) + args=popen.call_args.args[0];options=popen.call_args.kwargs + self.assertEqual('/usr/bin/curl',args[0]);self.assertEqual('--disable',args[1]) + self.assertIn('--max-time',args);self.assertIn('--max-filesize',args) + self.assertEqual('=https',args[args.index('--proto')+1]) + self.assertNotIn('--location',args);self.assertNotIn('-L',args) + self.assertEqual({'PATH':'/usr/bin:/bin','LC_ALL':'C'},options['env']) + self.assertEqual(fetcher.subprocess.DEVNULL,options['stderr']) + self.assertTrue(options['start_new_session']) + self.assertEqual(b'bytes',stream.getvalue()) + + def test_cli_download_errors_do_not_leak_provider_or_local_diagnostics(self): + with patch.object(fetcher,'fetch',side_effect=ValueError('secret-body /private/path')),patch('sys.stderr',new_callable=io.StringIO) as err: + self.assertEqual(1,fetcher.main(['--platform','darwin-arm64','--out','fresh'])) + self.assertNotIn('secret-body',err.getvalue());self.assertNotIn('/private/path',err.getvalue()) + +if __name__=='__main__':unittest.main() diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index c44c588e..a13fa53e 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -5,9 +5,11 @@ import json from pathlib import Path import sys +import subprocess import tarfile import tempfile import unittest +from unittest.mock import patch sys.path.insert(0, str(Path(__file__).resolve().parent)) import homebrew_rehearsal as rehearsal @@ -268,3 +270,205 @@ def test_unsafe_report_path_rejected_before_read(self): self.report['evidence']['../outside'] = {'sha256': '0' * 64, 'byteLength': 1} (self.root / 'build-report.json').write_text(json.dumps(self.report)) with self.assertRaisesRegex(ValueError, 'Unsafe evidence path'): self.verify() + + +class PreviousReleaseTests(unittest.TestCase): + def setUp(self): + temp = tempfile.TemporaryDirectory() + self.addCleanup(temp.cleanup) + self.root = Path(temp.name) + self.manifest = {'schema': 'openprose.cli-distribution/1', 'version': rehearsal.PREVIOUS_VERSION, 'artifacts': []} + for implementation in ('bun', 'rust'): + path = self.root / (implementation + '.tar.gz') + with tarfile.open(path, 'w:gz') as archive: + member = tarfile.TarInfo('package/prose'); member.size = len(implementation) + archive.addfile(member, io.BytesIO(implementation.encode())) + self.manifest['artifacts'].append({'kind': 'standalone', 'implementation': implementation, + 'platform': 'darwin-arm64', 'name': path.name, 'size': path.stat().st_size, + 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()}) + (self.root / 'manifest.json').write_text(json.dumps(self.manifest)) + self.pin = hashlib.sha256((self.root / 'manifest.json').read_bytes()).hexdigest() + + def verify(self, version='0.15.0-rc.4'): + with patch.object(rehearsal, 'PREVIOUS_MANIFEST_SHA256', self.pin): + return rehearsal.verify_previous_release(self.root, 'darwin-arm64', version) + + def test_actual_prior_bytes_have_no_sdk_requirement(self): + previous = self.verify() + self.assertEqual(previous['version'], '0.15.0-rc.3') + self.assertEqual(set(previous['archives']), {'bun', 'rust'}) + self.assertEqual(previous['binaryHashes']['bun'], hashlib.sha256(b'bun').hexdigest()) + + def test_mutated_manifest_or_archive_is_refused(self): + for name in ('manifest.json', 'bun.tar.gz'): + path = self.root / name; original = path.read_bytes() + path.write_bytes(original + b' ') + with self.subTest(name=name), self.assertRaisesRegex(ValueError, 'identity mismatch'): + self.verify() + path.write_bytes(original) + + def test_same_version_downgrade_and_non_rc_candidate_are_refused(self): + for version in ('0.15.0-rc.3', '0.15.0-rc.2', '0.14.0-rc.9', '0.15.0'): + with self.subTest(version=version), self.assertRaisesRegex(ValueError, 'newer RC'): + self.verify(version) + + def test_symlinked_prior_archive_refused_before_mutation(self): + path = self.root / 'bun.tar.gz'; path.rename(self.root / 'real.tar.gz'); path.symlink_to('real.tar.gz') + with self.assertRaisesRegex(ValueError, 'regular previous archive'): + self.verify() + + +class NativeExerciseTests(unittest.TestCase): + """Exercise genuine install/upgrade transitions with an inert fake brew.""" + def setUp(self): + temp = tempfile.TemporaryDirectory(); self.addCleanup(temp.cleanup) + self.root = Path(temp.name); self.prefix = self.root / 'brew' + (self.prefix / 'bin').mkdir(parents=True) + self.package = self.root / 'candidate'; self.package.mkdir() + self.output = self.root / 'receipt' + self.installed_tap = self.prefix / 'Homebrew/Library/Taps/openprose/homebrew-cli-rehearsal' + self.active = self.prefix / 'bin/prose' + self.manifest = {'version': '0.15.0-rc.4', 'platform': 'darwin-arm64', + 'agentsSdk': {'sha256': hashlib.sha256(b'sdk').hexdigest()}} + self.archives = {i: {'path': i + '.tar.gz', 'sha256': 'a' * 64} for i in ('bun', 'rust')} + self.verified = {'custodyKind': 'kernel-rc-native', 'manifestSha256': 'b' * 64, + 'source': 'c' * 40, 'packageIdentity': { + i + 'BinarySha256': hashlib.sha256(('new-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} + self.previous = {'version': '0.15.0-rc.3', 'root': self.root, 'manifestSha256': 'd' * 64, + 'archives': self.archives, + 'binaryHashes': {i: hashlib.sha256(('old-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} + self.kegs = {}; self.tapped = False; self.calls = []; self.corrupt_settings = False + self.bad_dry_run = False + + def execute(self, argv, **kwargs): + self.calls.append((list(argv), dict(kwargs['env']))) + out = ''; code = 0 + if argv[0] == 'brew': + verb = argv[1] + if verb == '--prefix': + out = str(self.prefix if len(argv) == 2 else self.kegs[argv[2].split('-')[-1]][0]) + '\n' + elif verb == '--repo': out = str(self.installed_tap) + '\n' + elif verb == 'tap': + if len(argv) == 2: out = rehearsal.TAP + '\n' if self.tapped else '' + else: + self.tapped = True + (self.installed_tap / 'Formula').mkdir(parents=True) + for implementation in ('bun', 'rust'): + filename = 'prose-' + implementation + '.rb' + (self.installed_tap / 'Formula' / filename).write_bytes((self.output / 'tap/Formula' / filename).read_bytes()) + elif verb == 'untap': self.tapped = False + elif verb == 'list': out = ''.join('prose-' + i + ' ' + state[1] + '\n' for i, state in self.kegs.items()) + elif verb in {'install', 'upgrade'}: + implementation = argv[-1].split('-')[-1] + formula = (self.installed_tap / 'Formula' / ('prose-' + implementation + '.rb')).read_text() + version = rehearsal.re.search(r' version "([^"]+)"', formula).group(1) + if verb == 'upgrade': + self.assertIn(implementation, self.kegs, 'Upgrade requires an installed previous keg') + self.assertEqual(self.kegs[implementation][1], '0.15.0-rc.3', 'Upgrade must replace an actual previous version') + keg = self.prefix / 'Cellar' / implementation / version / 'bin'; keg.mkdir(parents=True, exist_ok=True) + binary = keg / 'prose' + binary.write_bytes((('new-' if version.endswith('.4') else 'old-') + implementation).encode()) + if version.endswith('.4'): (keg / 'prose-agents-sdk').write_bytes(b'sdk') + self.kegs[implementation] = (keg.parent, version) + self.assertFalse(self.active.exists() or self.active.is_symlink(), 'Install/upgrade must vacate shared command') + self.active.symlink_to(binary) + if self.corrupt_settings and verb == 'upgrade': + (Path(kwargs['env']['PROSE_CONFIG_DIR']) / 'cli.toml').write_text('timeout = "1m"\n') + elif verb == 'unlink': + if self.active.is_symlink(): self.active.unlink() + elif verb == 'link': + binary = self.kegs[argv[-1].split('-')[-1]][0] / 'bin/prose' + if self.active.is_symlink(): + if self.active.resolve() != binary.resolve(): out = 'Could not symlink'; code = 1 + else: self.active.symlink_to(binary) + elif verb == 'uninstall': + for name in argv[2:]: + implementation = name.split('-')[-1] + if implementation in self.kegs: + if self.active.is_symlink() and self.active.resolve().parent.parent == self.kegs[implementation][0].resolve(): self.active.unlink() + del self.kegs[implementation] + elif verb != 'test': self.fail('Unexpected brew command: ' + str(argv)) + elif argv[0] == 'git': pass # Inert fake: no Git operation occurs in this test. + elif Path(argv[0]).name == 'prose-agents-sdk': out = '{}\n' + elif Path(argv[0]).name == 'prose': + resolved = Path(argv[0]).resolve(); version = resolved.parent.parent.name; implementation = resolved.parent.parent.parent.name + if argv[1:] == ['--version']: out = f'prose {version} ({implementation})\n' + elif 'explain' in argv: + out = json.dumps({'schema': 'openprose.configuration-explanation/1', 'diagnostics': [], 'values': { + k: {'value': v} for k, v in [('harness', 'agents-sdk'), ('model', 'gpt-6.1-sol'), ('authProfile', 'openai-api-key'), + ('timeout', '9m' if (Path(kwargs['env']['PROSE_CONFIG_DIR']) / 'cli.toml').exists() else '10m')]}}) + elif '--dry-run' in argv: + self.assertEqual(kwargs['env'].get('OPENAI_API_KEY'), 'provider-free-installation-canary') + out = json.dumps({'schema': 'openprose.runner-dry-run-report/1', 'wouldStartModel': self.bad_dry_run, + 'readiness': 'ready', 'billingOwner': 'user-provider', 'blockingError': None, + 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}}) + else: + self.assertNotIn('OPENAI_API_KEY', kwargs['env']) + out = json.dumps({'schema': 'openprose.runner-error/1', 'code': 'HARNESS_NEEDS_AUTH', 'details': {'fallbackAttempted': False}}); code = 10 + else: self.fail('Unexpected process: ' + str(argv)) + return subprocess.CompletedProcess(argv, code, out, '') + + def exercise(self, previous=True): + with patch.object(rehearsal.subprocess, 'run', side_effect=self.execute): + return rehearsal.exercise(self.package, self.manifest, self.archives, self.verified, + self.output, 'brew', previous=self.previous if previous else None) + + def test_both_genuine_upgrades_preserve_selection_settings_and_scrub_credentials(self): + with patch.dict(rehearsal.os.environ, {'OPENAI_API_KEY': 'ambient-secret', 'HTTPS_PROXY': 'ambient-proxy'}): + receipt = self.exercise() + self.assertEqual(receipt['upgradeQualification'], 'passed-genuine-upgrade-both-selections') + self.assertEqual(receipt['modelCalls'], 0) + self.assertTrue(receipt['uninstallPassed']) + self.assertEqual((self.output / 'user-settings/cli.toml').read_bytes(), b'# explicit upgrade preservation\ntimeout = "9m"\n') + names = [c['name'] for c in receipt['checks']] + for selected in ('bun', 'rust'): + positions = [names.index(selected + suffix) for suffix in ('-unlink-selected-before-upgrade', '-upgrade-inactive', + '-unlink-upgraded-inactive', '-upgrade-selected', '-restore-selected-link', '-settings-after-uninstall')] + self.assertEqual(positions, sorted(positions)) + for argv, env in self.calls: + self.assertNotIn('HTTPS_PROXY', env) + if '--dry-run' not in argv: self.assertNotIn('OPENAI_API_KEY', env) + self.assertFalse(self.active.exists() or self.active.is_symlink()) + self.assertNotEqual(self.installed_tap, self.output / 'tap') + + def test_changed_settings_fail_instead_of_claiming_upgrade_success(self): + self.corrupt_settings = True + with self.assertRaisesRegex(ValueError, 'changed explicit user settings'): + self.exercise() + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_model_starting_dry_run_is_refused(self): + self.bad_dry_run = True + with self.assertRaisesRegex(ValueError, 'provider-free readiness'): + self.exercise() + + def test_existing_command_is_not_replaced(self): + self.active.write_bytes(b'existing') + with self.assertRaisesRegex(ValueError, 'overwrite an existing prose command'): + self.exercise() + self.assertEqual(self.active.read_bytes(), b'existing') + self.assertFalse(any(argv[:2] == ['brew', 'install'] for argv, _ in self.calls)) + + def test_existing_tap_is_not_replaced(self): + self.tapped = True + with self.assertRaisesRegex(ValueError, 'replace an existing tap'): + self.exercise() + self.assertFalse(any(argv[:2] == ['brew', 'install'] for argv, _ in self.calls)) + + def test_existing_helper_command_is_not_replaced(self): + helper = self.prefix / 'bin/prose-agents-sdk'; helper.write_bytes(b'existing-helper') + with self.assertRaisesRegex(ValueError, 'overwrite an existing SDK helper'): + self.exercise() + self.assertEqual(helper.read_bytes(), b'existing-helper') + self.assertFalse(any(argv[:2] == ['brew', 'install'] for argv, _ in self.calls)) + + def test_existing_fully_qualified_prose_keg_is_not_replaced(self): + self.kegs['bun'] = (self.root, '0.15.0-rc.3') + with self.assertRaisesRegex(ValueError, 'fresh Homebrew installation'): + self.exercise() + self.assertFalse(any(argv[:2] == ['brew', 'install'] for argv, _ in self.calls)) + + def test_fresh_only_receipt_does_not_claim_genuine_upgrade(self): + receipt = self.exercise(previous=False) + self.assertEqual(receipt['upgradeQualification'], 'not-requested-fresh-install-only') + self.assertFalse(any(argv[:2] == ['brew', 'upgrade'] for argv, _ in self.calls)) diff --git a/cli/ci/test_qualify_installed_sdk.py b/cli/ci/test_qualify_installed_sdk.py new file mode 100644 index 00000000..85486361 --- /dev/null +++ b/cli/ci/test_qualify_installed_sdk.py @@ -0,0 +1,226 @@ +"""Hermetic custody and consumer-environment checks for installation qualification.""" +import hashlib +import io +import json +import os +from pathlib import Path +import subprocess +import sys +import tarfile +import tempfile +import unittest +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import qualify_installed_sdk as installed + + +def archive(path, members): + with tarfile.open(path, 'w:gz') as target: + for name, data in members.items(): + member = tarfile.TarInfo(name); member.size = len(data); member.mode = 0o755 + target.addfile(member, io.BytesIO(data)) + + +class InstallationQualificationTests(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory(); self.addCleanup(self.temporary.cleanup) + self.root = Path(self.temporary.name) + self.node = self.root / 'node'; self.node.write_bytes(b'node-fixture'); self.node.chmod(0o755) + self.npm = self.root / 'npm-cli.js'; self.npm.write_bytes(b'npm-fixture') + + def qualification(self): + return installed.Qualification(self.root / 'output', self.node, self.npm) + + def test_nonfresh_output_and_symlink_inputs_rejected(self): + output = self.root / 'output'; output.mkdir() + with self.assertRaisesRegex(ValueError, 'fresh'): + installed.Qualification(output, self.node, self.npm) + linked = self.root / 'linked'; linked.symlink_to(self.node) + with self.assertRaisesRegex(ValueError, 'regular'): + installed.Qualification(self.root / 'new', linked, self.npm) + + def test_consumer_environment_has_no_provider_python_checkout_or_ambient_settings(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir() + with patch.dict(os.environ, {'OPENAI_API_KEY': 'secret', 'PROSE_HARNESS': 'codex', 'PROSE_CONFIG_DIR': '/user-settings', 'PYTHONPATH': '/checkout', 'npm_config_userconfig': '/user-npmrc'}, clear=False): + native = q.environment(root) + npm = q.environment(root, npm=True, explicit_config=True) + self.assertEqual(native['PATH'], '') + self.assertTrue((root / '.git').is_dir()) + self.assertEqual(npm['PATH'], str(q.tools)) + for env in (native, npm): + self.assertNotIn('OPENAI_API_KEY', env); self.assertNotIn('PROSE_HARNESS', env); self.assertNotIn('PYTHONPATH', env) + self.assertNotIn('HTTPS_PROXY', env); self.assertTrue(Path(env['HOME']).is_relative_to(q.output)) + self.assertEqual(npm['PROSE_CONFIG_DIR'], str(root / 'preferences')) + self.assertEqual(sorted(p.name for p in q.tools.iterdir()), ['node']) + self.assertEqual(npm['npm_config_offline'], 'true') + self.assertEqual(npm['npm_config_script_shell'], '/bin/sh') + self.assertEqual(Path(npm['npm_config_userconfig']).read_bytes(), b'') + + def test_exact_rc_order_and_invalid_versions(self): + self.assertGreater(installed.rc_version('0.15.0-rc.4'), installed.rc_version('0.15.0-rc.3')) + self.assertLess(installed.rc_version('0.15.0-rc.2'), installed.rc_version('0.15.0-rc.3')) + for bad in ('0.15.0', '0.15.0-rc.04', 'latest', '0.15.0-rc.3-dev'): + with self.assertRaises(ValueError): installed.rc_version(bad) + + def test_npx_preserves_exact_priming_registry_cache_key_and_remains_offline(self): + env = {'npm_config_registry': 'http://127.0.0.1:9', 'npm_config_offline': 'false', 'HOME': '/isolated'} + actual = installed.offline_registry_environment(env, {'registryUrl': 'http://127.0.0.1:45321'}) + self.assertEqual(actual, {'npm_config_registry': 'http://127.0.0.1:45321', 'npm_config_offline': 'true', 'HOME': '/isolated'}) + self.assertEqual(env['npm_config_registry'], 'http://127.0.0.1:9') + for registry in (None, 'https://registry.npmjs.org', 'http://localhost:1234', 'http://127.0.0.1:0', 'http://127.0.0.1:65536'): + with self.assertRaisesRegex(ValueError, 'loopback'): + installed.offline_registry_environment(env, {'registryUrl': registry}) + + def test_standalone_extracts_bound_siblings_and_relocation_preserves_bytes(self): + path = self.root / 'package.tgz' + data = {'payload/prose': b'cli', **{'payload/' + n: n.encode() for n in installed.SDK_NAMES}} + archive(path, data) + receipt = installed.extract_standalone(path, self.root / 'install', sdk=True) + (self.root / 'install').rename(self.root / 'relocated') + self.assertEqual(set(receipt), {'prose', *installed.SDK_NAMES}) + for name, record in receipt.items(): + self.assertEqual(installed.pub.digest(self.root / 'relocated' / name), record['sha256']) + with self.assertRaisesRegex(ValueError, 'fresh'): + installed.extract_standalone(path, self.root / 'relocated', sdk=True) + + def test_archive_link_and_missing_helper_rejected_before_execution(self): + path = self.root / 'bad.tgz' + with tarfile.open(path, 'w:gz') as target: + member = tarfile.TarInfo('payload/prose'); member.type = tarfile.SYMTYPE; member.linkname = '/user/bin/prose'; target.addfile(member) + with self.assertRaisesRegex(ValueError, 'Links'): + installed.extract_standalone(path, self.root / 'install', sdk=True) + archive(path, {'payload/prose': b'cli'}) + with self.assertRaisesRegex(ValueError, 'siblings'): + installed.extract_standalone(path, self.root / 'install', sdk=True) + self.assertFalse((self.root / 'install').exists()) + + def test_artifact_digest_and_regular_file_binding(self): + path = self.root / 'native.tgz'; path.write_bytes(b'original') + record = {'path': path.name, 'byteLength': 8, 'sha256': installed.pub.digest(path)} + self.assertEqual(installed.artifact(self.root, record), path) + path.write_bytes(b'modified') + with self.assertRaisesRegex(ValueError, 'bytes differ'): + installed.artifact(self.root, record) + + def test_npm_original_alias_cohort_and_native_binding(self): + source = '1' * 40; version = '0.15.0-rc.4'; platform = 'darwin-arm64'; binary = b'compiled-bun' + payload_version = installed.package_local.npm_payload_version(version, platform) + cohort = {'sourceRevision': source, 'version': version} + meta = {'name': '@openprose/prose-cli', 'version': version, 'openproseCohort': cohort, 'optionalDependencies': {'@openprose/prose-cli-' + platform: 'npm:@openprose/prose-cli@' + payload_version}} + native = {'name': '@openprose/prose-cli', 'version': payload_version, 'openproseCohort': cohort} + archive(self.root / 'meta.tgz', {'package/package.json': json.dumps(meta).encode(), 'package/bin/prose.js': b'launcher'}) + archive(self.root / 'native.tgz', {'package/package.json': json.dumps(native).encode(), 'package/bin/prose': binary}) + manifest = {'source': {'revision': source}, 'version': version, 'artifacts': [{'kind': kind, 'platform': target, 'path': path.name, 'byteLength': path.stat().st_size, 'sha256': installed.pub.digest(path)} for kind, target, path in [('npm-meta', None, self.root / 'meta.tgz'), ('npm-platform', platform, self.root / 'native.tgz')]]} + self.assertEqual(installed.npm_inputs(self.root, manifest, platform, expected_binary=hashlib.sha256(binary).hexdigest()), (self.root / 'meta.tgz', self.root / 'native.tgz')) + with self.assertRaisesRegex(ValueError, 'differs from verified'): + installed.npm_inputs(self.root, manifest, platform, expected_binary='0' * 64) + native['scripts'] = {'postinstall': 'touch bad'} + archive(self.root / 'native.tgz', {'package/package.json': json.dumps(native).encode(), 'package/bin/prose': binary}) + manifest['artifacts'][1].update(byteLength=(self.root / 'native.tgz').stat().st_size, sha256=installed.pub.digest(self.root / 'native.tgz')) + with self.assertRaisesRegex(ValueError, 'lifecycle'): + installed.npm_inputs(self.root, manifest, platform) + + def test_execute_uses_explicit_node_and_checksum_bound_receipt(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root, npm=True) + with patch.object(installed.package_local, 'run_bounded', return_value=subprocess.CompletedProcess([], 0, b'ok\n', b'')) as run: + q.execute(['npm', '--version'], cwd=root, env=env, label='npm-version') + self.assertEqual(run.call_args.args[0][:2], [str(self.node), str(self.npm)]) + self.assertEqual(run.call_args.kwargs['cwd'], root) + evidence = q.output / q.commands[0]['path'] + self.assertEqual(installed.pub.digest(evidence), q.commands[0]['sha256']) + self.assertEqual(json.loads(evidence.read_text())['stdout'], 'ok\n') + + def test_failed_command_keeps_receipt_and_does_not_claim_pass(self): + q = self.qualification(); env = {'PATH': ''} + with patch.object(installed.package_local, 'run_bounded', return_value=subprocess.CompletedProcess([], 7, b'', b'failure')): + with self.assertRaisesRegex(ValueError, 'Unexpected exit'): + q.execute(['/fixture/prose'], cwd=q.output, env=env, label='failed') + self.assertEqual(q.checks, []); self.assertEqual(len(q.commands), 1) + + def test_defaults_reject_stale_identity_and_persisted_builtin_settings(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + with patch.object(q, 'identity', return_value={'selectedHarness': 'openprose', 'selectedTransport': 'hosted'}): + with self.assertRaisesRegex(ValueError, 'installed default'): + q.defaults(['/fixture'], 'bun', '0.15.0-rc.4', '1' * 40, root, env, 'stale') + self.assertEqual(q.checks, []) + + def consumer_responses(self, q, root, env, *, persist=False, source='1' * 40): + doctor = {'runner': {'name': 'bun', 'version': '0.15.0-rc.4', 'commit': source}, 'build': {'profile': 'release', 'testSeamsEnabled': False}, 'selectedHarness': 'agents-sdk', 'selectedTransport': 'jsonl', 'problems': [{'code': 'HARNESS_NEEDS_AUTH', 'action': installed.SETUP_ACTION, 'details': {'adapterId': 'agents-sdk/jsonl', 'authProfile': 'openai-api-key', 'fallbackAttempted': False}}]} + def run(argv, **kwargs): + self.assertEqual(kwargs['cwd'], root) + if kwargs['environment'].get('OPENAI_API_KEY') == installed.AUTH_CANARY: + self.assertIn('--dry-run', argv) + self.assertEqual(kwargs['environment'], dict(env, OPENAI_API_KEY=installed.AUTH_CANARY)) + else: self.assertEqual(kwargs['environment'], env) + self.assertEqual(argv[0], '/installed/prose') + if '--version' in argv: return subprocess.CompletedProcess(argv, 0, b'prose 0.15.0-rc.4 (bun)\n', b'') + if 'doctor' in argv: value, code = doctor, 10 + elif 'explain' in argv: + value, code = {'values': {k: {'value': v, 'source': {'kind': 'default'}} for k, v in {'harness': 'agents-sdk', 'model': 'gpt-6.1-sol', 'authProfile': 'openai-api-key'}.items()}}, 0 + if persist: + path = Path(env['HOME']) / '.prose/cli.toml'; path.parent.mkdir(); path.write_text('harness="agents-sdk"\n') + elif '--dry-run' in argv: + if kwargs['environment'].get('OPENAI_API_KEY') == installed.AUTH_CANARY: + value, code = {'schema': 'openprose.runner-dry-run-report/1', 'wouldStartModel': False, 'readiness': 'ready', 'blockingError': None, 'billingOwner': 'user-provider', 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}}, 0 + else: value, code = {'schema': 'openprose.runner-dry-run-report/1', 'blockingError': {'code': 'HARNESS_NEEDS_AUTH'}}, 10 + else: value, code = {'schema': 'openprose.runner-error/1', 'code': 'HARNESS_NEEDS_AUTH', 'action': installed.SETUP_ACTION}, 10 + return subprocess.CompletedProcess(argv, code, json.dumps(value).encode() + b'\n', b'') + return run + + def test_actual_consumer_probe_matrix_checks_identity_defaults_and_zero_persistence(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + with patch.object(installed.package_local, 'run_bounded', side_effect=self.consumer_responses(q, root, env)) as run: + q.defaults(['/installed/prose'], 'bun', '0.15.0-rc.4', '1' * 40, root, env, 'fresh') + self.assertEqual(run.call_count, 6) + self.assertEqual([x['label'] for x in q.commands], ['fresh-version', 'fresh-doctor', 'fresh-explain', 'fresh-missing-key', 'fresh-dry-run', 'fresh-helper-discovery']) + self.assertEqual(len(q.checks), 1) + self.assertFalse((Path(env['HOME']) / '.prose/cli.toml').exists()) + + def test_readonly_explain_cannot_silently_persist_builtin_preferences(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + with patch.object(installed.package_local, 'run_bounded', side_effect=self.consumer_responses(q, root, env, persist=True)): + with self.assertRaisesRegex(ValueError, 'persisted settings'): + q.defaults(['/installed/prose'], 'bun', '0.15.0-rc.4', '1' * 40, root, env, 'fresh') + self.assertEqual(q.checks, []) + + def test_actual_consumer_probe_rejects_wrong_embedded_source(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + with patch.object(installed.package_local, 'run_bounded', side_effect=self.consumer_responses(q, root, env, source='2' * 40)): + with self.assertRaisesRegex(ValueError, 'source identity'): + q.defaults(['/installed/prose'], 'bun', '0.15.0-rc.4', '1' * 40, root, env, 'fresh') + self.assertEqual(q.checks, []) + + def test_outer_helper_discovery_requires_no_inference_and_exact_packaged_runtime(self): + for index, mutation in enumerate(({'wouldStartModel': True}, {'readiness': 'blocked'}, {'selection': {'harness': 'agents-sdk', 'transport': 'jsonl', 'runtimeVersion': '9.9.9'}}, {'blockingError': {'code': 'HARNESS_UNAVAILABLE'}})): + with self.subTest(mutation=mutation): + q = installed.Qualification(self.root / ('output-' + str(index)), self.node, self.npm) + root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + normal = self.consumer_responses(q, root, env) + def run(argv, **kwargs): + result = normal(argv, **kwargs) + if kwargs['environment'].get('OPENAI_API_KEY') == installed.AUTH_CANARY: + value = json.loads(result.stdout); value.update(mutation) + result = subprocess.CompletedProcess(argv, result.returncode, json.dumps(value).encode(), b'') + return result + with patch.object(installed.package_local, 'run_bounded', side_effect=run): + with self.assertRaisesRegex(ValueError, 'Synthetic-key|packaged SDK sibling'): + q.defaults(['/installed/prose'], 'bun', '0.15.0-rc.4', '1' * 40, root, env, 'fresh') + self.assertEqual(q.checks, []) + + def test_saved_preferences_must_survive_byte_exactly(self): + q = self.qualification(); root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) + path = root / 'preferences.toml'; path.write_bytes(b'# original\ntimeout="1m"\n') + saved = (path, path.read_bytes()); path.write_bytes(b'changed') + with self.assertRaisesRegex(ValueError, 'Upgrade changed'): + q.preferences(['/fixture'], root, env, 'upgrade', saved) + + def test_candidate_custody_failure_prevents_output_and_execution(self): + with patch.object(installed.custody, 'verify_kernel_rc', side_effect=ValueError('custody rejected')) as verifier: + with self.assertRaisesRegex(ValueError, 'custody rejected'): + installed.qualify(self.root / 'candidate', self.root / 'previous', self.root / 'output', '1' * 40, '0.15.0-rc.4', self.node, self.npm) + verifier.assert_called_once(); self.assertFalse((self.root / 'output').exists()) + + +if __name__ == '__main__': + unittest.main() diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 6db5bbbf..416e08d1 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1045,6 +1045,8 @@ IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli ## IMP-098 genuine installed upgrade qualification +Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exact loopback registry URL already used for checksum-bound cache priming. The installed qualification runner must reuse that offline cache identity for actual npx execution; no publication, credential, network or package-selection behavior changes are authorized by this extension. + `/root/conformance_plumbing` owns new exact paths `cli/ci/fetch_previous_release.py` and `cli/ci/test_fetch_previous_release.py` for fresh bounded downloads of the fixed pinned published RC3 manifest and exactly four native upgrade inputs. Preserve untouched bytes, reject unsafe paths/symlinks/tampering, use fixed HTTPS publication origin and checksum-bound receipts. Focused fake-download tests only; no real downloads, Git, heavy builds, credentials or paid calls by the agent. Root additionally owns `cli/ci/test_build_kernel_rc.py`, `cli/ci/check_workflows.py`, and `cli/ci/test_check_workflows.py` narrowly for integrating mandatory installed qualification and retaining native evidence. `/root/config_assessment` owns exact paths `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py` for genuine pinned published 0.15.0-rc.3 to verified local newer candidate upgrades, both selected implementations, installed SDK default/helper discovery, and isolated configuration preservation through upgrade/uninstall. Reuse established tap upgrade sequencing; retain no-replacement safety and strict candidate/prior custody. No Homebrew mutation, heavy builds, Git, credentials or provider calls by the agent. From c3ff0bebab99eac26b486c2f995dccf16b7884a0 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 20:59:29 -0400 Subject: [PATCH 14/64] IMP-098: assign measured generated SDK oracle reconciliation --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 416e08d1..c997ac77 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1045,6 +1045,8 @@ IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli ## IMP-098 genuine installed upgrade qualification +IMP-098 measured full-admission oracle reconciliation: `/root/conformance_plumbing` owns `cli/shared/fixtures/adapters/generate.py` and `cli/shared/fixtures/adapters/bin/adapter_probe.py` narrowly to make generated SDK scenario bytes and executable provider-free probe agree with the already reviewed SDK observation records. Full admission 1 failed only the generated-fixture drift and exact SDK probe comparisons at this gate. Preserve the existing scenario observations, exact oracle tests, raw usage/model identity controls, other adapter output and launch/canary/privacy checks. No product changes, Git, heavy builds or providers; root retains scenario expectations and the oracle test. + Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exact loopback registry URL already used for checksum-bound cache priming. The installed qualification runner must reuse that offline cache identity for actual npx execution; no publication, credential, network or package-selection behavior changes are authorized by this extension. `/root/conformance_plumbing` owns new exact paths `cli/ci/fetch_previous_release.py` and `cli/ci/test_fetch_previous_release.py` for fresh bounded downloads of the fixed pinned published RC3 manifest and exactly four native upgrade inputs. Preserve untouched bytes, reject unsafe paths/symlinks/tampering, use fixed HTTPS publication origin and checksum-bound receipts. Focused fake-download tests only; no real downloads, Git, heavy builds, credentials or paid calls by the agent. Root additionally owns `cli/ci/test_build_kernel_rc.py`, `cli/ci/check_workflows.py`, and `cli/ci/test_check_workflows.py` narrowly for integrating mandatory installed qualification and retaining native evidence. From 65f09be5d2753a899879972c7ce7d45e97e38b4f Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:01:58 -0400 Subject: [PATCH 15/64] IMP-098: keep SDK generated oracle and probe observations coherent --- cli/shared/fixtures/adapters/bin/adapter_probe.py | 9 ++++++++- cli/shared/fixtures/adapters/generate.py | 9 +++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/cli/shared/fixtures/adapters/bin/adapter_probe.py b/cli/shared/fixtures/adapters/bin/adapter_probe.py index d12ab9c1..e735f6e6 100755 --- a/cli/shared/fixtures/adapters/bin/adapter_probe.py +++ b/cli/shared/fixtures/adapters/bin/adapter_probe.py @@ -260,9 +260,16 @@ def main() -> int: ] ) elif adapter_id == "agents-sdk/jsonl": + # Fixture observations come from the reviewed SDK authority, independently + # of the scenario bytes whose exact launch/output oracle checks this probe. + sdk_observation = json.loads( + (Path(__file__).resolve().parents[1] / "sdk-production.json").read_text("utf-8") + )["observation"] emit([ {"type": "start", "model": "fixture", "cwd": "fixture"}, - {"type": "final", "output": assistant_text}, + {"type": "final", "output": assistant_text, + "usageObservation": sdk_observation["completedUsage"], + "modelIdentity": sdk_observation["modelIdentity"]}, ]) elif adapter_id == "claude/print-stream-json": emit( diff --git a/cli/shared/fixtures/adapters/generate.py b/cli/shared/fixtures/adapters/generate.py index fdb136b8..bdbea834 100644 --- a/cli/shared/fixtures/adapters/generate.py +++ b/cli/shared/fixtures/adapters/generate.py @@ -176,9 +176,12 @@ def expected_outputs() -> dict[Path, bytes]: "source": "cli/shared/image/echo-v0/manifest.json#modelVisibleBytes", "mode": "0600", "byteLength": len(image), "sha256": sha256(image), }] + sdk_observation = load_json(ADAPTERS / "sdk-production.json")["observation"] scenario["fakeStdout"] = [ {"type": "start", "model": "fixture", "cwd": "fixture"}, - {"type": "final", "output": assistant_text}, + {"type": "final", "output": assistant_text, + "usageObservation": sdk_observation["completedUsage"], + "modelIdentity": sdk_observation["modelIdentity"]}, ] elif adapter_id in {"claude/print-stream-json", "omp/rpc"}: scenario["files"] = [ @@ -250,8 +253,10 @@ def expected_outputs() -> dict[Path, bytes]: }, ] scenario["expectedLanguageTerminal"] = terminal + # Preserve the reviewed SDK scenario's ASCII-escaped byte serialization. + # Other adapters retain their existing UTF-8 serialization. outputs[path] = ( - json.dumps(scenario, ensure_ascii=False, indent=2).encode("utf-8") + b"\n" + json.dumps(scenario, ensure_ascii=(adapter_id == "agents-sdk/jsonl"), indent=2).encode("utf-8") + b"\n" ) return outputs From 0ce12ca3d4658d05a442500118ac147d6be9291b Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:05:48 -0400 Subject: [PATCH 16/64] IMP-098: make hosted build identity failure route explicit --- cli/protocol/OWNERSHIP.md | 2 ++ cli/rust/crates/prose-cli/tests/cli.rs | 9 ++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index c997ac77..909075ac 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1045,6 +1045,8 @@ IMP-098 measured full-target Clippy repair: root owns `cli/rust/crates/prose-cli ## IMP-098 genuine installed upgrade qualification +IMP-098 measured full-admission Rust identity reconciliation: root owns only `compiled_commit_identity_is_exact_in_success_failure_and_stream_results` in `cli/rust/crates/prose-cli/tests/cli.rs` to select the existing hosted failure explicitly. Full admission 2 passed 82 CLI tests and failed this runner-result commit assertion because the new SDK missing-key preflight returns its reviewed bare error. Preserve every commit/build/stream assertion; shared SDK credential cases remain exact. No product behavior or schema change. + IMP-098 measured full-admission oracle reconciliation: `/root/conformance_plumbing` owns `cli/shared/fixtures/adapters/generate.py` and `cli/shared/fixtures/adapters/bin/adapter_probe.py` narrowly to make generated SDK scenario bytes and executable provider-free probe agree with the already reviewed SDK observation records. Full admission 1 failed only the generated-fixture drift and exact SDK probe comparisons at this gate. Preserve the existing scenario observations, exact oracle tests, raw usage/model identity controls, other adapter output and launch/canary/privacy checks. No product changes, Git, heavy builds or providers; root retains scenario expectations and the oracle test. Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exact loopback registry URL already used for checksum-bound cache priming. The installed qualification runner must reuse that offline cache identity for actual npx execution; no publication, credential, network or package-selection behavior changes are authorized by this extension. diff --git a/cli/rust/crates/prose-cli/tests/cli.rs b/cli/rust/crates/prose-cli/tests/cli.rs index a632db0c..6ea780c0 100644 --- a/cli/rust/crates/prose-cli/tests/cli.rs +++ b/cli/rust/crates/prose-cli/tests/cli.rs @@ -4525,7 +4525,14 @@ fn compiled_commit_identity_is_exact_in_success_failure_and_stream_results() { let failure = prose( temp.path(), - &["--output", "json", "run", "fixture.prose.md"], + &[ + "--harness", + "openprose", + "--output", + "json", + "run", + "fixture.prose.md", + ], ); assert_eq!(failure.status.code(), Some(10)); assert_eq!(json_stdout(&failure)["runner"]["commit"], expected); From 851ebab18b3b07f50a1a098ec398673444a71eda Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:16:41 -0400 Subject: [PATCH 17/64] Lease measured SDK dependency inventory admission repairs --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 909075ac..bb5df1f1 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1056,3 +1056,5 @@ Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exac `/root/config_assessment` owns exact paths `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py` for genuine pinned published 0.15.0-rc.3 to verified local newer candidate upgrades, both selected implementations, installed SDK default/helper discovery, and isolated configuration preservation through upgrade/uninstall. Reuse established tap upgrade sequencing; retain no-replacement safety and strict candidate/prior custody. No Homebrew mutation, heavy builds, Git, credentials or provider calls by the agent. `/root/configuration_contract` owns new exact paths `cli/ci/qualify_installed_sdk.py` and `cli/ci/test_qualify_installed_sdk.py` for actual isolated fresh npm/npx/standalone execution, genuine prior-version upgrades, relocation, exact configuration preservation, built-in SDK diagnostics and saved alternatives. Prior release manifests/artifacts are independently pinned; receipts bind exact candidate bytes. No global installations, Git, heavy builds, credentials or provider calls by the agent. Root retains `cli/ci/build_kernel_rc.py`, `.github/workflows/cli-kernel-rc.yml`, `.github/workflows/cli-distribution-check.yml` for later integration after independent review. Consumer subprocesses must work without a checkout, consumer Python provisioning or harness PATH repair. Candidate local unsigned version is 0.15.0-rc.4, later than published rc.3; this does not authorize publication. + +IMP-098 measured package-lifecycle admission repair: `/root/configuration_contract` owns exact paths `cli/ci/dependency_evidence.py`, `cli/ci/test_dependency_evidence.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py` narrowly for truthful closed SDK Python dependency inventory, original pinned Python version grammar, normalized deterministic ordering, and direct invocation of SDK tests. Full admission 3 exposed unsupported Python inventory shape; independent diagnosis also found legitimate two-component pinned versions and normalization order rejected by Cargo/Bun assumptions. Preserve strict non-Python contracts, exact original package versions/hashes/source/scopes, and malformed input rejection. No fabricated lockfileVersion or weakened generic validator. No Git/heavy builds/provider calls; root owns lifecycle rerun and integration. From 35638ae61903230f027853b2b069744077874d49 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:18:02 -0400 Subject: [PATCH 18/64] Lease direct SDK test discovery corrections --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index bb5df1f1..6ea8f2d2 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1058,3 +1058,5 @@ Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exac `/root/configuration_contract` owns new exact paths `cli/ci/qualify_installed_sdk.py` and `cli/ci/test_qualify_installed_sdk.py` for actual isolated fresh npm/npx/standalone execution, genuine prior-version upgrades, relocation, exact configuration preservation, built-in SDK diagnostics and saved alternatives. Prior release manifests/artifacts are independently pinned; receipts bind exact candidate bytes. No global installations, Git, heavy builds, credentials or provider calls by the agent. Root retains `cli/ci/build_kernel_rc.py`, `.github/workflows/cli-kernel-rc.yml`, `.github/workflows/cli-distribution-check.yml` for later integration after independent review. Consumer subprocesses must work without a checkout, consumer Python provisioning or harness PATH repair. Candidate local unsigned version is 0.15.0-rc.4, later than published rc.3; this does not authorize publication. IMP-098 measured package-lifecycle admission repair: `/root/configuration_contract` owns exact paths `cli/ci/dependency_evidence.py`, `cli/ci/test_dependency_evidence.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py` narrowly for truthful closed SDK Python dependency inventory, original pinned Python version grammar, normalized deterministic ordering, and direct invocation of SDK tests. Full admission 3 exposed unsupported Python inventory shape; independent diagnosis also found legitimate two-component pinned versions and normalization order rejected by Cargo/Bun assumptions. Preserve strict non-Python contracts, exact original package versions/hashes/source/scopes, and malformed input rejection. No fabricated lockfileVersion or weakened generic validator. No Git/heavy builds/provider calls; root owns lifecycle rerun and integration. + +IMP-098 direct test invocation coverage: `/root/conformance_plumbing` owns only relocation of the existing `if __name__ == '__main__': unittest.main()` block to physical EOF in `cli/ci/test_build_kernel_rc.py`, `cli/ci/test_check_workflows.py`, and `cli/ci/test_package_local.py`. Read-only AST audit found SDK extraction, workflow admission and artifact tests declared after these blocks; module/discovery admission already includes them, but direct execution skips six methods. Preserve every test body and module/discovery behavior. Dependency test placement is separately owned by configuration_contract. No product/Git/heavy builds/provider calls. From 7e916a2b50ad59b29ee99a027f767ce41eda209b Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:18:52 -0400 Subject: [PATCH 19/64] Include SDK artifact tests in direct Python suite execution --- cli/ci/test_build_kernel_rc.py | 6 ++++-- cli/ci/test_check_workflows.py | 6 ++++-- cli/ci/test_package_local.py | 6 ++++-- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index 0807d63e..6647e932 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -110,8 +110,6 @@ def test_tampered_artifact_is_rejected(self): rc.verified_artifacts(root) -if __name__ == '__main__': - unittest.main() class KernelRCSdkExtractionTests(unittest.TestCase): archive = KernelRCBuildTests.archive @@ -131,3 +129,7 @@ def test_production_extracts_closed_sdk_payload(self): executable = rc.extract_binary(archive, root / 'install/prose', require_sdk=True) self.assertEqual({p.name for p in executable.parent.iterdir()}, set(names)) self.assertTrue(os.access(executable.parent / 'prose-agents-sdk', os.X_OK)) + + +if __name__ == '__main__': + unittest.main() diff --git a/cli/ci/test_check_workflows.py b/cli/ci/test_check_workflows.py index 10752d33..f10fd568 100644 --- a/cli/ci/test_check_workflows.py +++ b/cli/ci/test_check_workflows.py @@ -393,8 +393,6 @@ def test_publication_verification_and_bootstrap_are_mandatory(self): self.changed(name, lambda w, j: j["steps"].reverse()) -if __name__ == "__main__": - unittest.main() class HomebrewActionAdmissionTest(unittest.TestCase): @@ -405,3 +403,7 @@ def test_only_the_reviewed_homebrew_setup_pin_is_admitted(self): self.assertEqual([], audit_workflow("cli-distribution-check.yml", text)) for replacement in ("Homebrew/actions/setup-homebrew@main", "Homebrew/actions/setup-homebrew@" + "0" * 40): self.assertTrue(audit_workflow("cli-distribution-check.yml", text.replace(reviewed, replacement))) + + +if __name__ == "__main__": + unittest.main() diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index 149f54b4..4b0b254f 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -4824,8 +4824,6 @@ def test_arbitrary_gate_files_only_create_an_unpublishable_release_candidate( self.assertFalse(manifest["externalGates"]["authorityValidatedByPackager"]) -if __name__ == "__main__": - unittest.main() class PackagedSdkArtifactTests(unittest.TestCase): def test_release_requires_packaged_helper_before_other_work(self): @@ -4866,3 +4864,7 @@ def test_sdk_payload_is_sibling_in_standalone_and_npm(self): self.assertTrue(member.isfile()) self.assertEqual(member.mode, mode) self.assertEqual(source.extractfile(member).read(), data) + + +if __name__ == "__main__": + unittest.main() From 8a61ea404e361e7b17d8b6435194776c8a6617ee Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:20:35 -0400 Subject: [PATCH 20/64] Admit truthful hash-locked SDK Python dependency evidence --- cli/benchmarks/installed/benchmark.py | 35 +++++++++-- cli/benchmarks/installed/test_benchmark.py | 67 +++++++++++++++++++++- cli/ci/dependency_evidence.py | 13 +++++ cli/ci/test_dependency_evidence.py | 33 ++++++++++- 4 files changed, 140 insertions(+), 8 deletions(-) diff --git a/cli/benchmarks/installed/benchmark.py b/cli/benchmarks/installed/benchmark.py index fe7a9409..c94983ae 100644 --- a/cli/benchmarks/installed/benchmark.py +++ b/cli/benchmarks/installed/benchmark.py @@ -48,6 +48,16 @@ MAX_INSTALLED_TREE_BYTES = 768 * 1024 * 1024 MAX_INSTALLED_TREE_PATH_BYTES = 4_096 SHA256_RE = re.compile(r"^[0-9a-f]{64}$") +# Exact pinned Python releases use PEP 440 release segments, not npm/Cargo SemVer. +# Epochs are excluded by the hash-locked requirements token grammar. +PYTHON_VERSION_RE = re.compile( + r"v?[0-9]+(?:\.[0-9]+)*" + r"(?:[-_.]?(?:a|b|c|rc|alpha|beta|pre|preview)[-_.]?[0-9]*)?" + r"(?:(?:-[0-9]+)|(?:[-_.]?(?:post|rev|r)[-_.]?[0-9]*))?" + r"(?:[-_.]?dev[-_.]?[0-9]*)?" + r"(?:\+[a-z0-9]+(?:[-_.][a-z0-9]+)*)?", + re.IGNORECASE | re.ASCII, +) SEMVER_RE = re.compile( r"^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)" r"(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?" @@ -776,7 +786,7 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: ), } if "agentsSdkPython" in inventories: - inventory_shapes["agentsSdkPython"] = ({"lockfileVersion", "scopeBasis", "packages"}, None, None, 1, + inventory_shapes["agentsSdkPython"] = ({"scopeBasis", "packages"}, None, None, None, "hash-locked-four-platform-wheel-candidates") allowed_scopes = {"runtime", "development", "build", "workspace", "component", "frozen-sdk-build"} for inventory_name, ( @@ -789,8 +799,13 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: inventory = require_object(inventories.get(inventory_name), inventory_name) require_exact_keys(inventory, keys, f"dependency inventory {inventory_name}") if ( - inventory.get("lockfileVersion") != expected_lock_version - or isinstance(inventory.get("lockfileVersion"), bool) + ( + expected_lock_version is not None + and ( + inventory.get("lockfileVersion") != expected_lock_version + or isinstance(inventory.get("lockfileVersion"), bool) + ) + ) or inventory.get("scopeBasis") != expected_scope_basis ): fail( @@ -823,10 +838,15 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: package.get("version"), "dependency package version" ) source = require_string(package.get("source"), "dependency package source") + python_package = inventory_name == "agentsSdkPython" + valid_version = ( + len(version) <= 128 and PYTHON_VERSION_RE.fullmatch(version) is not None + if python_package else is_exact_semver(version) + ) if ( DEPENDENCY_NAME_RE.fullmatch(name) is None or ".." in name - or not is_exact_semver(version) + or not valid_version or len(source) > 1024 or any(ord(character) < 0x20 for character in source) ): @@ -873,6 +893,13 @@ def validate_dependency_evidence(value: Any) -> dict[str, Any]: fail( "EVIDENCE_MALFORMED", f"unsupported integrity status for {identity}" ) + if python_package and ( + scopes != ["frozen-sdk-build"] + or integrity.get("status") != "declared" + or integrity.get("algorithm") != "sha256" + or source != "pypi:wheel-sha256:" + integrity.get("digest", "") + ): + fail("EVIDENCE_MALFORMED", "SDK Python wheel provenance differs") if identities != sorted(identities): fail("EVIDENCE_MALFORMED", f"packages must be sorted for {inventory_name}") diff --git a/cli/benchmarks/installed/test_benchmark.py b/cli/benchmarks/installed/test_benchmark.py index 5d2fb4cf..9edb5080 100644 --- a/cli/benchmarks/installed/test_benchmark.py +++ b/cli/benchmarks/installed/test_benchmark.py @@ -720,7 +720,7 @@ def append_sdk(members): dependency_path = output / 'dependency-evidence.json'; dependency = json.loads(dependency_path.read_text()) dependency['sources'].append({'path': 'harnesses/agents-sdk/requirements-build.txt', 'byteLength': 1, 'sha256': sdk['dependencyLockSha256']}) dependency['inventories']['agentsSdkPython'] = { - 'lockfileVersion': 1, 'scopeBasis': 'hash-locked-four-platform-wheel-candidates', + 'scopeBasis': 'hash-locked-four-platform-wheel-candidates', 'packages': [{'name': 'fixture', 'version': '1.0.0', 'source': 'pypi:wheel-sha256:' + 'e'*64, 'scopes': ['frozen-sdk-build'], 'integrity': {'status': 'declared', 'algorithm': 'sha256', 'digest': 'e'*64}}]} dependency_path.write_text(json.dumps(dependency)) @@ -793,6 +793,71 @@ def rewrite_tar(path: Path, mutate) -> None: write_tar(path, members) +class PythonSdkDependencyValidationTests(unittest.TestCase): + @staticmethod + def generated_report(): + script = ROOT / "cli/ci/dependency_evidence.py" + spec = importlib.util.spec_from_file_location("sdk_generated_dependency_evidence", script) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module.build_report(ROOT) + + def test_original_generated_sdk_inventory_is_admitted_without_rewriting(self): + report = self.generated_report() + before = copy.deepcopy(report) + BENCHMARK.validate_dependency_evidence(report) + self.assertEqual(report, before) + # Exercise the lifecycle's separate source/hash/SBOM verifier, without a build. + script = ROOT / "cli/ci/package_local.py" + spec = importlib.util.spec_from_file_location("sdk_dependency_package_local", script) + packager = importlib.util.module_from_spec(spec) + spec.loader.exec_module(packager) + encoded, independently_verified = packager.dependency_evidence() + self.assertEqual(json.loads(encoded), report) + self.assertEqual(independently_verified, report) + BENCHMARK.validate_dependency_evidence(independently_verified) + inventory = report["inventories"]["agentsSdkPython"] + self.assertEqual(set(inventory), {"scopeBasis", "packages"}) + versions = {package["version"] for package in inventory["packages"]} + self.assertTrue({"3.20", "26.3", "3.0", "2026.8"}.issubset(versions)) + + def test_python_evidence_remains_closed_and_hash_bound(self): + report = self.generated_report() + for poison in ("lock-version", "scope-basis", "version", "source", "digest", "scopes", "integrity", "order", "duplicate"): + with self.subTest(poison=poison): + value = copy.deepcopy(report) + inventory = value["inventories"]["agentsSdkPython"] + package = inventory["packages"][0] + if poison == "lock-version": + inventory["lockfileVersion"] = 1 + elif poison == "scope-basis": + inventory["scopeBasis"] = "unverified" + elif poison == "version": + package["version"] = "3..20" + elif poison == "source": + package["source"] = "pypi:wheel-sha256:" + "0" * 64 + elif poison == "digest": + package["integrity"]["digest"] = "0" * 63 + elif poison == "scopes": + package["scopes"] = ["runtime"] + elif poison == "integrity": + package["integrity"] = {"status": "not-applicable", "reason": "missing"} + elif poison == "order": + inventory["packages"].reverse() + else: + inventory["packages"].append(copy.deepcopy(package)) + with self.assertRaises(BENCHMARK.BenchmarkError): + BENCHMARK.validate_dependency_evidence(value) + + def test_python_versions_do_not_weaken_cargo_or_bun_semver(self): + for inventory_name in ("bun", "cargo", "windowsProcessHostCargo"): + with self.subTest(inventory=inventory_name): + value = dependency_fixture() + value["inventories"][inventory_name]["packages"][0]["version"] = "3.20" + with self.assertRaises(BENCHMARK.BenchmarkError): + BENCHMARK.validate_dependency_evidence(value) + + class InstalledPackageBenchmarkTests(unittest.TestCase): def test_release_cannot_omit_sdk_identity_after_rehashing(self): with tempfile.TemporaryDirectory() as temporary: diff --git a/cli/ci/dependency_evidence.py b/cli/ci/dependency_evidence.py index 6129b7e7..367c77e0 100644 --- a/cli/ci/dependency_evidence.py +++ b/cli/ci/dependency_evidence.py @@ -32,6 +32,16 @@ CRATE_NAME_RE = re.compile(r"^[A-Za-z0-9_-]+$") VERSION_RE = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+(?:[-+][0-9A-Za-z.-]+)?$") SHA256_RE = re.compile(r"^[0-9a-f]{64}$") +# Exact pinned Python releases use PEP 440 release segments, not npm/Cargo SemVer. +# Epochs are excluded by the hash-locked requirements token grammar. +PYTHON_VERSION_RE = re.compile( + r"v?[0-9]+(?:\.[0-9]+)*" + r"(?:[-_.]?(?:a|b|c|rc|alpha|beta|pre|preview)[-_.]?[0-9]*)?" + r"(?:(?:-[0-9]+)|(?:[-_.]?(?:post|rev|r)[-_.]?[0-9]*))?" + r"(?:[-_.]?dev[-_.]?[0-9]*)?" + r"(?:\+[a-z0-9]+(?:[-_.][a-z0-9]+)*)?", + re.IGNORECASE | re.ASCII, +) class EvidenceError(Exception): @@ -873,6 +883,8 @@ def build_report(root: Path) -> dict[str, Any]: if match is None: fail("SOURCE_MALFORMED", "SDK Python requirements must be exact and hash locked") name, version, hashes = match.groups() + if len(version) > 128 or PYTHON_VERSION_RE.fullmatch(version) is None: + fail("SOURCE_MALFORMED", "SDK Python requirement version must be an exact pinned Python release") name = name.lower().replace("_", "-") if name in names: fail("SOURCE_DUPLICATE", "duplicate SDK Python requirement") @@ -884,6 +896,7 @@ def build_report(root: Path) -> dict[str, Any]: "integrity": {"status": "declared", "algorithm": "sha256", "digest": digest}}) if not packages: fail("SOURCE_MALFORMED", "SDK Python requirements are empty") + packages.sort(key=lambda package: (package["name"], package["version"], package["source"])) python_inventory = {"scopeBasis": "hash-locked-four-platform-wheel-candidates", "packages": packages} python_sources = [(python_lock, lock_bytes)] source_values = [ diff --git a/cli/ci/test_dependency_evidence.py b/cli/ci/test_dependency_evidence.py index e7425a77..eb233d45 100644 --- a/cli/ci/test_dependency_evidence.py +++ b/cli/ci/test_dependency_evidence.py @@ -486,9 +486,6 @@ def test_malformed_cli_is_machine_readable_and_has_no_override_switch(self) -> N self.assertNotIn("license", result.stderr.decode().lower()) -if __name__ == "__main__": - unittest.main() - class PythonSdkDependencyEvidenceTests(unittest.TestCase): def test_python_lock_candidates_bind_hashes_and_reject_unpinned_input(self): with tempfile.TemporaryDirectory() as temporary: @@ -503,3 +500,33 @@ def test_python_lock_candidates_bind_hashes_and_reject_unpinned_input(self): lock.write_text(text) with self.assertRaises(EVIDENCE.EvidenceError): EVIDENCE.build_report(root) + + + def test_python_versions_and_normalized_sort_preserve_original_lock_identity(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + make_fixture(root) + lock = root / "harnesses/agents-sdk/requirements-build.txt" + lock.parent.mkdir(parents=True) + versions = ("3.20", "26.3", "3.0", "2026.8", "1.0rc1", "1.0.post2", "1.0.dev3+linux.1") + # Deliberately unsorted; underscore normalization changes identity ordering. + lock.write_text("typing_extensions==3.20 --hash=sha256:" + SHA256_A + "\n" + + "typing-z==26.3 --hash=sha256:" + SHA256_B + "\n" + + "".join(f"fixture-{index}=={version} --hash=sha256:{SHA256_C}\n" + for index, version in enumerate(versions[2:]))) + report = EVIDENCE.build_report(root) + inventory = report["inventories"]["agentsSdkPython"] + self.assertEqual(set(inventory), {"scopeBasis", "packages"}) + packages = inventory["packages"] + identities = [(p["name"], p["version"], p["source"]) for p in packages] + self.assertEqual(identities, sorted(identities)) + self.assertEqual({p["version"] for p in packages}, set(versions)) + for version in ("nonsense", "1..2", "1.*", "1.0+", "1.0rcx"): + with self.subTest(version=version): + lock.write_text(f"fixture=={version} --hash=sha256:{SHA256_A}\n") + with self.assertRaises(EVIDENCE.EvidenceError): + EVIDENCE.build_report(root) + + +if __name__ == "__main__": + unittest.main() From 9a884490929a2671274b2cd794d7cb5527120a9d Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:29:17 -0400 Subject: [PATCH 21/64] Lease measured Bun default fixture reconciliation --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 6ea8f2d2..d88d93f0 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1060,3 +1060,5 @@ Root additionally owns `cli/ci/npm_alias_install.py` narrowly to return the exac IMP-098 measured package-lifecycle admission repair: `/root/configuration_contract` owns exact paths `cli/ci/dependency_evidence.py`, `cli/ci/test_dependency_evidence.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py` narrowly for truthful closed SDK Python dependency inventory, original pinned Python version grammar, normalized deterministic ordering, and direct invocation of SDK tests. Full admission 3 exposed unsupported Python inventory shape; independent diagnosis also found legitimate two-component pinned versions and normalization order rejected by Cargo/Bun assumptions. Preserve strict non-Python contracts, exact original package versions/hashes/source/scopes, and malformed input rejection. No fabricated lockfileVersion or weakened generic validator. No Git/heavy builds/provider calls; root owns lifecycle rerun and integration. IMP-098 direct test invocation coverage: `/root/conformance_plumbing` owns only relocation of the existing `if __name__ == '__main__': unittest.main()` block to physical EOF in `cli/ci/test_build_kernel_rc.py`, `cli/ci/test_check_workflows.py`, and `cli/ci/test_package_local.py`. Read-only AST audit found SDK extraction, workflow admission and artifact tests declared after these blocks; module/discovery admission already includes them, but direct execution skips six methods. Preserve every test body and module/discovery behavior. Dependency test placement is separately owned by configuration_contract. No product/Git/heavy builds/provider calls. + +IMP-098 measured full-admission Bun default reconciliation: `/root/configuration_contract` owns narrowly the four failing assertions/call sites in `cli/bun/test/build-identity.test.ts`, `cli/bun/test/standalone.test.ts`, and `cli/bun/test/shared-contract.test.ts`. Full admission 4 passed both Rust variants and actual package lifecycle, then Bun reported 796 passed/4 failed: old hosted default inventory, intended hosted identity failure implicitly using new SDK default, hostile-cwd no-auto-load test implicitly using old hosted default, and hosted service suggested argv retaining now correctly omitted local --harness. Keep actual default inventory asserting SDK; select hosted explicitly only for tests whose purpose is hosted failure/identity or no-auto-load; match the existing closed shared suggested-command contract. Preserve artifact reproducibility, exact version/commit, hostile environment and no-invocation assertions. No product behavior or shared oracle edits, Git, heavy builds or provider calls by agent; root runs actual three-suite replay. From 4bf95046cbcad6af79e517bd90e51345fcc651e2 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:30:49 -0400 Subject: [PATCH 22/64] Preserve default-path hostile environment canary coverage --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index d88d93f0..2a90709d 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1062,3 +1062,5 @@ IMP-098 measured package-lifecycle admission repair: `/root/configuration_contra IMP-098 direct test invocation coverage: `/root/conformance_plumbing` owns only relocation of the existing `if __name__ == '__main__': unittest.main()` block to physical EOF in `cli/ci/test_build_kernel_rc.py`, `cli/ci/test_check_workflows.py`, and `cli/ci/test_package_local.py`. Read-only AST audit found SDK extraction, workflow admission and artifact tests declared after these blocks; module/discovery admission already includes them, but direct execution skips six methods. Preserve every test body and module/discovery behavior. Dependency test placement is separately owned by configuration_contract. No product/Git/heavy builds/provider calls. IMP-098 measured full-admission Bun default reconciliation: `/root/configuration_contract` owns narrowly the four failing assertions/call sites in `cli/bun/test/build-identity.test.ts`, `cli/bun/test/standalone.test.ts`, and `cli/bun/test/shared-contract.test.ts`. Full admission 4 passed both Rust variants and actual package lifecycle, then Bun reported 796 passed/4 failed: old hosted default inventory, intended hosted identity failure implicitly using new SDK default, hostile-cwd no-auto-load test implicitly using old hosted default, and hosted service suggested argv retaining now correctly omitted local --harness. Keep actual default inventory asserting SDK; select hosted explicitly only for tests whose purpose is hosted failure/identity or no-auto-load; match the existing closed shared suggested-command contract. Preserve artifact reproducibility, exact version/commit, hostile environment and no-invocation assertions. No product behavior or shared oracle edits, Git, heavy builds or provider calls by agent; root runs actual three-suite replay. + +The preceding Bun lease's hostile-cwd instruction is superseded by independent review: this test must retain no harness override, because both `.env` and preload canaries set `PROSE_HARNESS=mock` and an explicit flag would mask auto-loading. `configuration_contract` keeps the default path and asserts the closed SDK bare missing-auth error, fixed action, exact adapter/profile, no fallback and empty stderr. Hosted identity tests remain explicitly hosted; production behavior and shared oracle stay unchanged. From 2dcc191ebd3866b24aebd4d1d092adf6c6f1f07f Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:31:32 -0400 Subject: [PATCH 23/64] Reconcile Bun SDK defaults while preserving hostile environment coverage --- cli/bun/test/build-identity.test.ts | 6 +++--- cli/bun/test/shared-contract.test.ts | 2 +- cli/bun/test/standalone.test.ts | 13 ++++++++++--- 3 files changed, 14 insertions(+), 7 deletions(-) diff --git a/cli/bun/test/build-identity.test.ts b/cli/bun/test/build-identity.test.ts index 703f43e7..bcb30fb8 100644 --- a/cli/bun/test/build-identity.test.ts +++ b/cli/bun/test/build-identity.test.ts @@ -127,7 +127,7 @@ describe("standalone build identity and seam exclusion", () => { const humanInventory = await run([first, "cli", "harness", "list"], root); expect(humanInventory).toMatchObject({ exitCode: 0, stderr: "" }); expect(humanInventory.stdout).toContain( - "* openprose availability=not-implemented transport=hosted (selected)\n", + "* agents-sdk availability=missing transport=jsonl (selected)\n", ); const exactFirst = await realpath(first); expect(humanInventory.stdout).toContain(`Choose Codex: '${exactFirst}' cli harness use codex\n`); @@ -244,7 +244,7 @@ describe("standalone build identity and seam exclusion", () => { runner: { name: "bun", version: "0.1.0", commit }, build: { profile: "release", testSeamsEnabled: false }, }); - const failure = await run([first, "--output=json", "run", "example.prose.md"], root); + const failure = await run([first, "--harness=openprose", "--output=json", "run", "example.prose.md"], root); expect(failure.exitCode).toBe(10); expect(JSON.parse(failure.stdout)).toMatchObject({ schema: "openprose.runner-result/1", @@ -320,7 +320,7 @@ describe("standalone build identity and seam exclusion", () => { runner: { name: "bun", version, commit: "version-test-commit" }, }); - const failure = await run([executable, "--output=json", "run", "example.prose.md"], root); + const failure = await run([executable, "--harness=openprose", "--output=json", "run", "example.prose.md"], root); expect(failure.exitCode).toBe(10); expect(JSON.parse(failure.stdout)).toMatchObject({ runner: { name: "bun", version, commit: "version-test-commit" }, diff --git a/cli/bun/test/shared-contract.test.ts b/cli/bun/test/shared-contract.test.ts index 0cecc0df..4a0250da 100644 --- a/cli/bun/test/shared-contract.test.ts +++ b/cli/bun/test/shared-contract.test.ts @@ -160,7 +160,7 @@ describe("shared runner contracts", () => { error: { code: "HOSTED_UNAVAILABLE", action: "To use the hosted service, run `cli run submit FILE --preview`; running programs on this machine needs a local harness (`cli harness list`).", - details: { suggestedArgv: ["--harness", "openprose", "--output", "json", "cli", "run", "submit", "fixture.prose.md", "--preview"] }, + details: { suggestedArgv: ["--output", "json", "cli", "run", "submit", "fixture.prose.md", "--preview"] }, }, }); expect(io.invocations).toHaveLength(0); diff --git a/cli/bun/test/standalone.test.ts b/cli/bun/test/standalone.test.ts index c24ddd52..862f396a 100644 --- a/cli/bun/test/standalone.test.ts +++ b/cli/bun/test/standalone.test.ts @@ -156,9 +156,16 @@ describe("standalone executable", () => { ]); expect(exit).toBe(10); expect(JSON.parse(stdout)).toMatchObject({ - schema: "openprose.runner-result/1", - error: { code: "HOSTED_UNAVAILABLE" }, - runnerExitCode: 10, + schema: "openprose.runner-error/1", + code: "HARNESS_NEEDS_AUTH", + boundary: "authentication", + exitCode: 10, + action: "Set OPENAI_API_KEY to an OpenAI API key in the process environment, then retry. No model request was sent.", + details: { + adapterId: "agents-sdk/jsonl", + authProfile: "openai-api-key", + fallbackAttempted: false, + }, }); expect(stderr).toBe(""); }); From cd0e143fe58b25491ced694444d58ace2dfe8332 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:41:08 -0400 Subject: [PATCH 24/64] Lease strict installed helper banner assertion repair --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 2a90709d..f4d9c859 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1064,3 +1064,5 @@ IMP-098 direct test invocation coverage: `/root/conformance_plumbing` owns only IMP-098 measured full-admission Bun default reconciliation: `/root/configuration_contract` owns narrowly the four failing assertions/call sites in `cli/bun/test/build-identity.test.ts`, `cli/bun/test/standalone.test.ts`, and `cli/bun/test/shared-contract.test.ts`. Full admission 4 passed both Rust variants and actual package lifecycle, then Bun reported 796 passed/4 failed: old hosted default inventory, intended hosted identity failure implicitly using new SDK default, hostile-cwd no-auto-load test implicitly using old hosted default, and hosted service suggested argv retaining now correctly omitted local --harness. Keep actual default inventory asserting SDK; select hosted explicitly only for tests whose purpose is hosted failure/identity or no-auto-load; match the existing closed shared suggested-command contract. Preserve artifact reproducibility, exact version/commit, hostile environment and no-invocation assertions. No product behavior or shared oracle edits, Git, heavy builds or provider calls by agent; root runs actual three-suite replay. The preceding Bun lease's hostile-cwd instruction is superseded by independent review: this test must retain no harness override, because both `.env` and preload canaries set `PROSE_HARNESS=mock` and an explicit flag would mask auto-loading. `configuration_contract` keeps the default path and asserts the closed SDK bare missing-auth error, fixed action, exact adapter/profile, no fallback and empty stderr. Hosted identity tests remain explicitly hosted; production behavior and shared oracle stay unchanged. + +IMP-098 actual installed helper version assertion: `/root/configuration_contract` owns `cli/ci/qualify_installed_sdk.py`, `cli/ci/test_qualify_installed_sdk.py`, `cli/ci/homebrew_rehearsal.py`, and `cli/ci/test_homebrew_rehearsal.py` narrowly to require the frozen raw helper banner `prose-agents-sdk 0.1.0` in installed dry-run selection. Actual RC4 source `2dcc191` built successfully and the installed ready/no-model dry-run discovered its sibling but rejected the gate's mistaken bare `0.1.0` expectation. Shared recipe and existing Prime reporting establish raw validated banner semantics; product/schema/oracle remain unchanged. Add complete valid-selection poison controls rejecting bare/wrong version/wrong helper identity. This supersedes completed Homebrew source ownership only for this assertion/tests; no global installs, Git, heavy builds or provider calls by agent. Full admission 5 has ended before source changes. From 6c47e861b1e5256c680028cafbcdf2ed25abf0d8 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:42:08 -0400 Subject: [PATCH 25/64] Lease explicit hosted service framework fixtures --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index f4d9c859..b1cf2652 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1066,3 +1066,5 @@ IMP-098 measured full-admission Bun default reconciliation: `/root/configuration The preceding Bun lease's hostile-cwd instruction is superseded by independent review: this test must retain no harness override, because both `.env` and preload canaries set `PROSE_HARNESS=mock` and an explicit flag would mask auto-loading. `configuration_contract` keeps the default path and asserts the closed SDK bare missing-auth error, fixed action, exact adapter/profile, no fallback and empty stderr. Hosted identity tests remain explicitly hosted; production behavior and shared oracle stay unchanged. IMP-098 actual installed helper version assertion: `/root/configuration_contract` owns `cli/ci/qualify_installed_sdk.py`, `cli/ci/test_qualify_installed_sdk.py`, `cli/ci/homebrew_rehearsal.py`, and `cli/ci/test_homebrew_rehearsal.py` narrowly to require the frozen raw helper banner `prose-agents-sdk 0.1.0` in installed dry-run selection. Actual RC4 source `2dcc191` built successfully and the installed ready/no-model dry-run discovered its sibling but rejected the gate's mistaken bare `0.1.0` expectation. Shared recipe and existing Prime reporting establish raw validated banner semantics; product/schema/oracle remain unchanged. Add complete valid-selection poison controls rejecting bare/wrong version/wrong helper identity. This supersedes completed Homebrew source ownership only for this assertion/tests; no global installs, Git, heavy builds or provider calls by agent. Full admission 5 has ended before source changes. + +IMP-098 measured service framework corpus reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json`, `no-cli-examples-list-human.json`, `no-cli-examples-list-json.json`, `no-cli-file-forwards-with-hint.json`, `no-cli-help-command-json.json`, `no-cli-help-human.json`, `no-cli-help-json.json`, `no-cli-language-command-hint.json`, `no-cli-run-existing-file-forwards.json`, `no-cli-status-file-forwards.json`, `no-cli-status-json.json`, and `no-cli-status-jsonl.json`. Full admission 5 passed all 100 differential cases then found 12 old-default hosted-branch expectations among 1,089 service cases. Independent diagnosis confirms SPEC language routing and both products already condition these exact grammar/hint/refusal branches on explicitly selected openprose. Add only `environment.PROSE_HARNESS=openprose` and truthful descriptions; preserve every argv/output/hint/languageForwarded/no-request expectation. No product/oracle taxonomy changes, Git, builds or providers. Root owns the corresponding SPEC 7.1/missing-cli prose replacing historical hosted-default claims with explicit selection, and root runs both real service corpus suites. From 29f41d4086762e8d3346681ce184bd9b2ed26e98 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:43:25 -0400 Subject: [PATCH 26/64] Lease closed hosted selector in service fixture schema --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index b1cf2652..a8bd98a1 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1068,3 +1068,5 @@ The preceding Bun lease's hostile-cwd instruction is superseded by independent r IMP-098 actual installed helper version assertion: `/root/configuration_contract` owns `cli/ci/qualify_installed_sdk.py`, `cli/ci/test_qualify_installed_sdk.py`, `cli/ci/homebrew_rehearsal.py`, and `cli/ci/test_homebrew_rehearsal.py` narrowly to require the frozen raw helper banner `prose-agents-sdk 0.1.0` in installed dry-run selection. Actual RC4 source `2dcc191` built successfully and the installed ready/no-model dry-run discovered its sibling but rejected the gate's mistaken bare `0.1.0` expectation. Shared recipe and existing Prime reporting establish raw validated banner semantics; product/schema/oracle remain unchanged. Add complete valid-selection poison controls rejecting bare/wrong version/wrong helper identity. This supersedes completed Homebrew source ownership only for this assertion/tests; no global installs, Git, heavy builds or provider calls by agent. Full admission 5 has ended before source changes. IMP-098 measured service framework corpus reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json`, `no-cli-examples-list-human.json`, `no-cli-examples-list-json.json`, `no-cli-file-forwards-with-hint.json`, `no-cli-help-command-json.json`, `no-cli-help-human.json`, `no-cli-help-json.json`, `no-cli-language-command-hint.json`, `no-cli-run-existing-file-forwards.json`, `no-cli-status-file-forwards.json`, `no-cli-status-json.json`, and `no-cli-status-jsonl.json`. Full admission 5 passed all 100 differential cases then found 12 old-default hosted-branch expectations among 1,089 service cases. Independent diagnosis confirms SPEC language routing and both products already condition these exact grammar/hint/refusal branches on explicitly selected openprose. Add only `environment.PROSE_HARNESS=openprose` and truthful descriptions; preserve every argv/output/hint/languageForwarded/no-request expectation. No product/oracle taxonomy changes, Git, builds or providers. Root owns the corresponding SPEC 7.1/missing-cli prose replacing historical hosted-default claims with explicit selection, and root runs both real service corpus suites. + +The twelve-fixture reconciliation additionally leases `/root/conformance_plumbing` `cli/conformance/runner/service-fixture.schema.json` only to allow the case environment key `PROSE_HARNESS` with the closed constant value `openprose`, and `cli/shared/tests/test_service_contract.py` for schema regression controls accepting that one selector and rejecting arbitrary harnesses/unknown environment keys. Corpus validation proved the existing environment allowlist rejects the explicit selector. Keep all other environment and corpus contracts unchanged; no general provider/harness admission or production change. Root retains Git, actual corpus replay and integration. From 3eeab982271234a5630b23e624e02f823a7d2bcd Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:46:34 -0400 Subject: [PATCH 27/64] Qualify exact SDK banner and explicit hosted service framework routes --- cli/SPEC.md | 17 +++++++++++------ cli/ci/homebrew_rehearsal.py | 2 +- cli/ci/qualify_installed_sdk.py | 2 +- cli/ci/test_homebrew_rehearsal.py | 18 +++++++++++++++++- cli/ci/test_qualify_installed_sdk.py | 10 ++++++++-- .../no-cli-double-dash-forwards.json | 7 +++++-- .../framework/no-cli-examples-list-human.json | 5 ++++- .../framework/no-cli-examples-list-json.json | 5 ++++- .../no-cli-file-forwards-with-hint.json | 7 +++++-- .../framework/no-cli-help-command-json.json | 5 ++++- .../service/framework/no-cli-help-human.json | 5 ++++- .../service/framework/no-cli-help-json.json | 5 ++++- .../no-cli-language-command-hint.json | 5 ++++- .../no-cli-run-existing-file-forwards.json | 5 ++++- .../no-cli-status-file-forwards.json | 7 +++++-- .../service/framework/no-cli-status-json.json | 5 ++++- .../framework/no-cli-status-jsonl.json | 5 ++++- .../runner/service-fixture.schema.json | 8 +++++++- cli/shared/tests/test_service_contract.py | 19 +++++++++++++++++++ 19 files changed, 115 insertions(+), 27 deletions(-) diff --git a/cli/SPEC.md b/cli/SPEC.md index 9c63356d..009643a8 100644 --- a/cli/SPEC.md +++ b/cli/SPEC.md @@ -381,7 +381,7 @@ of forwarding. The same applies to a lone word that a `nounSynonyms` entry maps to exactly one command (`login`, `whoami`, `models`), to a verb before its group (`list jobs`) and to a `commandRewrites` word (`stop`, `delete`, `share`, `cron`). A language command that also names a service command -(`status`, `help`, `examples`) is forwarded unless the default hosted harness, +(`status`, `help`, `examples`) is forwarded unless the explicitly selected hosted harness, which runs no language command, would refuse it; `run FILE` is always forwarded, and `prose -- ` always forwards. @@ -2166,21 +2166,26 @@ unconsumed tokens: to the OpenProse language instead, put `--` before them`` (``is not a command`` for a synonym). The Action reads ``Insert cli before the service command: `prose ... cli ...`.`` (``Use `cli `: ...`` for a - synonym). `details.suggestedArgv` is the original argv with `cli` and the - command words in place of the typed words, with every global kept in place. + synonym). `details.suggestedArgv` inserts `cli` and the corrected command + words. Executable service corrections retain display options, apply `--cwd` + to the file operand, retain `--model` only when supported, and map `--dry-run` + to supported `--preview` without duplication. Local harness, transport, + authentication and timeout globals are omitted. Alias-only corrections + retain the original opaque arguments and native globals. - **Language commands.** `grammar.intentInference.languageCommands` is the SPEC 7.1 list (a contract test compares them). A language command that also names a service command (`status` → `service triage`, `help [COMMAND]` → `--help`, `examples` → `example list`) is forwarded when a local harness is - selected. Under the default `openprose` harness, which runs no language + selected. Under an explicitly selected `openprose` harness, which runs no language command, it is the `INVOCATION_INVALID` rejection instead (reason `` is a language command, which runs only with a local harness``). - `run FILE` is always forwarded: its `HOSTED_UNAVAILABLE` refusal keeps the + `run FILE` is always forwarded: the explicitly selected hosted harness's + `HOSTED_UNAVAILABLE` refusal keeps the frozen Action, and `details.suggestedArgv` is `prose ... cli run submit FILE --preview`. - **A file of that name.** When one of the words names an existing file or directory, the argv is a language command and is forwarded unchanged. If - the default `openprose` harness then refuses with `HOSTED_UNAVAILABLE`, the + an explicitly selected `openprose` harness then refuses with `HOSTED_UNAVAILABLE`, the Action gains ``If you meant the hosted service command, use `prose cli ...`.`` and `details.suggestedArgv` carries that argv. - **Global aliases before `cli`.** An `optionAliases` entry whose target is a diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 80af326d..86b13522 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -271,7 +271,7 @@ def report(suffix: str, argv: list[str], code: int = 0, additions: dict[str, str raise ValueError('Installed SDK dry run did not establish provider-free readiness') selection = dry.get('selection', {}) if any(selection.get(key) != value for key, value in (('harness', 'agents-sdk'), ('adapterId', 'agents-sdk/jsonl'), - ('transport', 'jsonl'), ('runtimeVersion', '0.1.0'), ('model', 'gpt-6.1-sol'))): + ('transport', 'jsonl'), ('runtimeVersion', 'prose-agents-sdk 0.1.0'), ('model', 'gpt-6.1-sol'))): raise ValueError('Installed command did not discover the packaged SDK default') if dry.get('billingOwner') != 'user-provider' or dry.get('blockingError') is not None: raise ValueError('Installed SDK dry run changed billing or reported a blocker') diff --git a/cli/ci/qualify_installed_sdk.py b/cli/ci/qualify_installed_sdk.py index 89407906..3b6ba595 100644 --- a/cli/ci/qualify_installed_sdk.py +++ b/cli/ci/qualify_installed_sdk.py @@ -168,7 +168,7 @@ def defaults(self, command, runner, version, source, root, env, label): ready = self.json_command(command, ['--output=json', '--dry-run', 'run', 'missing.prose.md'], root, dict(env, OPENAI_API_KEY=AUTH_CANARY), label + '-helper-discovery') selection = ready.get('selection', {}) pub.require(ready.get('schema') == 'openprose.runner-dry-run-report/1' and ready.get('wouldStartModel') is False and ready.get('readiness') == 'ready' and ready.get('blockingError') is None and ready.get('billingOwner') == 'user-provider', 'Synthetic-key dry-run must be ready without inference') - pub.require(all(selection.get(key) == value for key, value in {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}.items()), 'Installed CLI did not discover its packaged SDK sibling') + pub.require(all(selection.get(key) == value for key, value in {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': 'prose-agents-sdk 0.1.0', 'model': 'gpt-6.1-sol'}.items()), 'Installed CLI did not discover its packaged SDK sibling') pub.require(not path.exists(), 'Dry-run persisted settings') self.checks.append({'name': label, 'status': 'pass', 'modelCalls': 0, 'settingsPath': str(path)}) diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index a13fa53e..ca270d6b 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -339,6 +339,7 @@ def setUp(self): 'binaryHashes': {i: hashlib.sha256(('old-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} self.kegs = {}; self.tapped = False; self.calls = []; self.corrupt_settings = False self.bad_dry_run = False + self.runtime_banner = 'prose-agents-sdk 0.1.0' def execute(self, argv, **kwargs): self.calls.append((list(argv), dict(kwargs['env']))) @@ -401,7 +402,7 @@ def execute(self, argv, **kwargs): self.assertEqual(kwargs['env'].get('OPENAI_API_KEY'), 'provider-free-installation-canary') out = json.dumps({'schema': 'openprose.runner-dry-run-report/1', 'wouldStartModel': self.bad_dry_run, 'readiness': 'ready', 'billingOwner': 'user-provider', 'blockingError': None, - 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}}) + 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': self.runtime_banner, 'model': 'gpt-6.1-sol'}}) else: self.assertNotIn('OPENAI_API_KEY', kwargs['env']) out = json.dumps({'schema': 'openprose.runner-error/1', 'code': 'HARNESS_NEEDS_AUTH', 'details': {'fallbackAttempted': False}}); code = 10 @@ -442,6 +443,21 @@ def test_model_starting_dry_run_is_refused(self): with self.assertRaisesRegex(ValueError, 'provider-free readiness'): self.exercise() + def assert_runtime_banner_refused(self, banner): + self.runtime_banner = banner + with self.assertRaisesRegex(ValueError, 'discover the packaged SDK default'): + self.exercise() + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_complete_ready_selection_rejects_bare_runtime_version(self): + self.assert_runtime_banner_refused('0.1.0') + + def test_complete_ready_selection_rejects_wrong_runtime_version(self): + self.assert_runtime_banner_refused('prose-agents-sdk 9.9.9') + + def test_complete_ready_selection_rejects_wrong_helper_identity(self): + self.assert_runtime_banner_refused('other-helper 0.1.0') + def test_existing_command_is_not_replaced(self): self.active.write_bytes(b'existing') with self.assertRaisesRegex(ValueError, 'overwrite an existing prose command'): diff --git a/cli/ci/test_qualify_installed_sdk.py b/cli/ci/test_qualify_installed_sdk.py index 85486361..ce344e55 100644 --- a/cli/ci/test_qualify_installed_sdk.py +++ b/cli/ci/test_qualify_installed_sdk.py @@ -162,7 +162,7 @@ def run(argv, **kwargs): path = Path(env['HOME']) / '.prose/cli.toml'; path.parent.mkdir(); path.write_text('harness="agents-sdk"\n') elif '--dry-run' in argv: if kwargs['environment'].get('OPENAI_API_KEY') == installed.AUTH_CANARY: - value, code = {'schema': 'openprose.runner-dry-run-report/1', 'wouldStartModel': False, 'readiness': 'ready', 'blockingError': None, 'billingOwner': 'user-provider', 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': '0.1.0', 'model': 'gpt-6.1-sol'}}, 0 + value, code = {'schema': 'openprose.runner-dry-run-report/1', 'wouldStartModel': False, 'readiness': 'ready', 'blockingError': None, 'billingOwner': 'user-provider', 'selection': {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', 'runtimeVersion': 'prose-agents-sdk 0.1.0', 'model': 'gpt-6.1-sol'}}, 0 else: value, code = {'schema': 'openprose.runner-dry-run-report/1', 'blockingError': {'code': 'HARNESS_NEEDS_AUTH'}}, 10 else: value, code = {'schema': 'openprose.runner-error/1', 'code': 'HARNESS_NEEDS_AUTH', 'action': installed.SETUP_ACTION}, 10 return subprocess.CompletedProcess(argv, code, json.dumps(value).encode() + b'\n', b'') @@ -192,7 +192,13 @@ def test_actual_consumer_probe_rejects_wrong_embedded_source(self): self.assertEqual(q.checks, []) def test_outer_helper_discovery_requires_no_inference_and_exact_packaged_runtime(self): - for index, mutation in enumerate(({'wouldStartModel': True}, {'readiness': 'blocked'}, {'selection': {'harness': 'agents-sdk', 'transport': 'jsonl', 'runtimeVersion': '9.9.9'}}, {'blockingError': {'code': 'HARNESS_UNAVAILABLE'}})): + valid_selection = {'harness': 'agents-sdk', 'adapterId': 'agents-sdk/jsonl', 'transport': 'jsonl', + 'runtimeVersion': 'prose-agents-sdk 0.1.0', 'model': 'gpt-6.1-sol'} + mutations = [{'wouldStartModel': True}, {'readiness': 'blocked'}, + {'blockingError': {'code': 'HARNESS_UNAVAILABLE'}}] + mutations.extend({'selection': dict(valid_selection, runtimeVersion=banner)} for banner in + ('0.1.0', 'prose-agents-sdk 9.9.9', 'other-helper 0.1.0')) + for index, mutation in enumerate(mutations): with self.subTest(mutation=mutation): q = installed.Qualification(self.root / ('output-' + str(index)), self.node, self.npm) root = q.output / 'consumer'; root.mkdir(); env = q.environment(root) diff --git a/cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json b/cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json index f8bcdc14..a62b98af 100644 --- a/cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json +++ b/cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json @@ -2,7 +2,7 @@ "id": "no-cli-double-dash-forwards", "feature": "framework", "operation": "job.list", - "description": "`prose -- job list` always forwards to the language, with no service hint.", + "description": "`prose -- job list` always forwards to the language, with no service hint. This fixture explicitly selects the openprose hosted harness through PROSE_HARNESS.", "argv": [ "--", "job", @@ -26,5 +26,8 @@ "forbid": [ "`prose cli job list`" ], - "languageForwarded": true + "languageForwarded": true, + "environment": { + "PROSE_HARNESS": "openprose" + } } diff --git a/cli/conformance/cases/service/framework/no-cli-examples-list-human.json b/cli/conformance/cases/service/framework/no-cli-examples-list-human.json index e48f452b..565da655 100644 --- a/cli/conformance/cases/service/framework/no-cli-examples-list-human.json +++ b/cli/conformance/cases/service/framework/no-cli-examples-list-human.json @@ -2,7 +2,7 @@ "id": "no-cli-examples-list-human", "feature": "framework", "operation": "example.list", - "description": "`prose examples list` is a language command, but the default hosted harness runs none, so it exits 2 with `prose cli example list`; nothing is forwarded.", + "description": "`prose examples list` is a language command, but the explicitly selected hosted harness runs none, so it exits 2 with `prose cli example list`; nothing is forwarded.", "argv": [ "examples", "list" @@ -21,5 +21,8 @@ }, "stderr": { "text": "OpenProse: INVOCATION_INVALID: That command isn't quite right.\nDetail: `examples list` is a language command, which runs only with a local harness; did you mean `prose cli example list`? Nothing was forwarded or sent. To run the language command, select a local harness with `prose cli harness use `\nAction: Use `cli example list`: `prose cli example list`.\n" + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-examples-list-json.json b/cli/conformance/cases/service/framework/no-cli-examples-list-json.json index a338b763..aefc04b5 100644 --- a/cli/conformance/cases/service/framework/no-cli-examples-list-json.json +++ b/cli/conformance/cases/service/framework/no-cli-examples-list-json.json @@ -2,7 +2,7 @@ "id": "no-cli-examples-list-json", "feature": "framework", "operation": "example.list", - "description": "`prose --output json examples list` under the default hosted harness prints the INVOCATION_INVALID envelope with `prose cli example list`.", + "description": "`prose --output json examples list` under the explicitly selected hosted harness prints the INVOCATION_INVALID envelope with `prose cli example list`.", "argv": [ "--output", "json", @@ -44,5 +44,8 @@ "result": null, "schema": "openprose.service-operation/1" } + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-file-forwards-with-hint.json b/cli/conformance/cases/service/framework/no-cli-file-forwards-with-hint.json index c242ab02..c16437ec 100644 --- a/cli/conformance/cases/service/framework/no-cli-file-forwards-with-hint.json +++ b/cli/conformance/cases/service/framework/no-cli-file-forwards-with-hint.json @@ -2,7 +2,7 @@ "id": "no-cli-file-forwards-with-hint", "feature": "framework", "operation": "run.submit", - "description": "A file named like the command word makes the words a language command: they are forwarded unchanged, and the HOSTED_UNAVAILABLE Action also names the exact `prose cli ...` command.", + "description": "A file named like the command word makes the words a language command: they are forwarded unchanged, and the HOSTED_UNAVAILABLE Action also names the exact `prose cli ...` command. This fixture explicitly selects the openprose hosted harness through PROSE_HARNESS.", "files": [ { "path": "submit", @@ -32,5 +32,8 @@ "To use the hosted service, run `prose cli run submit hello.prose`; running programs on this machine needs a local harness (`prose cli harness list`)." ] }, - "languageForwarded": true + "languageForwarded": true, + "environment": { + "PROSE_HARNESS": "openprose" + } } diff --git a/cli/conformance/cases/service/framework/no-cli-help-command-json.json b/cli/conformance/cases/service/framework/no-cli-help-command-json.json index 70c399ef..1fd9d1de 100644 --- a/cli/conformance/cases/service/framework/no-cli-help-command-json.json +++ b/cli/conformance/cases/service/framework/no-cli-help-command-json.json @@ -2,7 +2,7 @@ "id": "no-cli-help-command-json", "feature": "framework", "operation": "run.submit", - "description": "`prose help run submit` under the default hosted harness suggests that command's help, `prose cli run submit --help`.", + "description": "`prose help run submit` under the explicitly selected hosted harness suggests that command's help, `prose cli run submit --help`.", "argv": [ "--output", "json", @@ -46,5 +46,8 @@ "result": null, "schema": "openprose.service-operation/1" } + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-help-human.json b/cli/conformance/cases/service/framework/no-cli-help-human.json index 1ec3c208..1efb12a4 100644 --- a/cli/conformance/cases/service/framework/no-cli-help-human.json +++ b/cli/conformance/cases/service/framework/no-cli-help-human.json @@ -2,7 +2,7 @@ "id": "no-cli-help-human", "feature": "framework", "operation": "service.operations", - "description": "`prose help` is a language command, but the default hosted harness runs none, so it exits 2 with `prose --help`; nothing is forwarded.", + "description": "`prose help` is a language command, but the explicitly selected hosted harness runs none, so it exits 2 with `prose --help`; nothing is forwarded.", "argv": [ "help" ], @@ -20,5 +20,8 @@ }, "stderr": { "text": "OpenProse: INVOCATION_INVALID: That command isn't quite right.\nDetail: `help` is a language command, which runs only with a local harness; did you mean `prose --help`? Nothing was forwarded or sent. To run the language command, select a local harness with `prose cli harness use `\nAction: Show the runner help: `prose --help`.\n" + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-help-json.json b/cli/conformance/cases/service/framework/no-cli-help-json.json index d8c8cde2..87c90887 100644 --- a/cli/conformance/cases/service/framework/no-cli-help-json.json +++ b/cli/conformance/cases/service/framework/no-cli-help-json.json @@ -2,7 +2,7 @@ "id": "no-cli-help-json", "feature": "framework", "operation": "service.operations", - "description": "`prose --output json help` under the default hosted harness prints the INVOCATION_INVALID envelope with `prose --help`.", + "description": "`prose --output json help` under the explicitly selected hosted harness prints the INVOCATION_INVALID envelope with `prose --help`.", "argv": [ "--output", "json", @@ -41,5 +41,8 @@ "result": null, "schema": "openprose.service-operation/1" } + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-language-command-hint.json b/cli/conformance/cases/service/framework/no-cli-language-command-hint.json index 9564b80d..1196255b 100644 --- a/cli/conformance/cases/service/framework/no-cli-language-command-hint.json +++ b/cli/conformance/cases/service/framework/no-cli-language-command-hint.json @@ -2,7 +2,7 @@ "id": "no-cli-language-command-hint", "feature": "framework", "operation": "service.status", - "description": "`status` is a language command (SPEC 7.1), but the default hosted harness runs no language command, so `prose status` exits 2 with `prose cli service triage`; nothing is forwarded.", + "description": "`status` is a language command (SPEC 7.1), but the explicitly selected hosted harness runs no language command, so `prose status` exits 2 with `prose cli service triage`; nothing is forwarded.", "argv": [ "status" ], @@ -20,5 +20,8 @@ }, "stderr": { "text": "OpenProse: INVOCATION_INVALID: That command isn't quite right.\nDetail: `status` is a language command, which runs only with a local harness; did you mean `prose cli service triage`? Nothing was forwarded or sent. To run the language command, select a local harness with `prose cli harness use `\nAction: Use `cli service triage`: `prose cli service triage`.\n" + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-run-existing-file-forwards.json b/cli/conformance/cases/service/framework/no-cli-run-existing-file-forwards.json index 961302cd..d208fe32 100644 --- a/cli/conformance/cases/service/framework/no-cli-run-existing-file-forwards.json +++ b/cli/conformance/cases/service/framework/no-cli-run-existing-file-forwards.json @@ -2,7 +2,7 @@ "id": "no-cli-run-existing-file-forwards", "feature": "framework", "operation": "run.submit", - "description": "`prose run FILE` stays the language command: the default hosted harness refuses it with HOSTED_UNAVAILABLE (exit 10, never a fallback) and its frozen Action; in JSON its details.suggestedArgv names `cli run submit FILE --preview`.", + "description": "`prose run FILE` stays the language command: the explicitly selected hosted harness refuses it with HOSTED_UNAVAILABLE (exit 10, never a fallback) and its frozen Action; in JSON its details.suggestedArgv names `cli run submit FILE --preview`.", "files": [ { "path": "hello.prose.md", @@ -31,5 +31,8 @@ "HOSTED_UNAVAILABLE", "running programs on this machine needs a local harness (`cli harness list`).\n" ] + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-status-file-forwards.json b/cli/conformance/cases/service/framework/no-cli-status-file-forwards.json index f912167f..22f7593f 100644 --- a/cli/conformance/cases/service/framework/no-cli-status-file-forwards.json +++ b/cli/conformance/cases/service/framework/no-cli-status-file-forwards.json @@ -2,7 +2,7 @@ "id": "no-cli-status-file-forwards", "feature": "framework", "operation": "service.status", - "description": "A file named `status` makes `prose status` the language command: it is forwarded, and the HOSTED_UNAVAILABLE Action also names `prose cli service triage`.", + "description": "A file named `status` makes `prose status` the language command: it is forwarded, and the HOSTED_UNAVAILABLE Action also names `prose cli service triage`. This fixture explicitly selects the openprose hosted harness through PROSE_HARNESS.", "files": [ { "path": "status", @@ -30,5 +30,8 @@ "To use the hosted service, run `prose cli service triage`; running programs on this machine needs a local harness" ] }, - "languageForwarded": true + "languageForwarded": true, + "environment": { + "PROSE_HARNESS": "openprose" + } } diff --git a/cli/conformance/cases/service/framework/no-cli-status-json.json b/cli/conformance/cases/service/framework/no-cli-status-json.json index 53dfeaa8..5f6b68a2 100644 --- a/cli/conformance/cases/service/framework/no-cli-status-json.json +++ b/cli/conformance/cases/service/framework/no-cli-status-json.json @@ -2,7 +2,7 @@ "id": "no-cli-status-json", "feature": "framework", "operation": "service.triage", - "description": "`prose --output json status` under the default hosted harness prints the INVOCATION_INVALID envelope with `prose cli service triage`.", + "description": "`prose --output json status` under the explicitly selected hosted harness prints the INVOCATION_INVALID envelope with `prose cli service triage`.", "argv": [ "--output", "json", @@ -43,5 +43,8 @@ "result": null, "schema": "openprose.service-operation/1" } + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/cases/service/framework/no-cli-status-jsonl.json b/cli/conformance/cases/service/framework/no-cli-status-jsonl.json index 43bfb1b1..90a387b1 100644 --- a/cli/conformance/cases/service/framework/no-cli-status-jsonl.json +++ b/cli/conformance/cases/service/framework/no-cli-status-jsonl.json @@ -2,7 +2,7 @@ "id": "no-cli-status-jsonl", "feature": "framework", "operation": "service.triage", - "description": "`prose --output jsonl status` under the default hosted harness prints the INVOCATION_INVALID envelope with `prose --output jsonl cli service triage`.", + "description": "`prose --output jsonl status` under the explicitly selected hosted harness prints the INVOCATION_INVALID envelope with `prose --output jsonl cli service triage`.", "argv": [ "--output", "jsonl", @@ -43,5 +43,8 @@ "result": null, "schema": "openprose.service-operation/1" } + }, + "environment": { + "PROSE_HARNESS": "openprose" } } diff --git a/cli/conformance/runner/service-fixture.schema.json b/cli/conformance/runner/service-fixture.schema.json index ced86a42..497dfbd9 100644 --- a/cli/conformance/runner/service-fixture.schema.json +++ b/cli/conformance/runner/service-fixture.schema.json @@ -395,11 +395,17 @@ "OPENPROSE_API_KEY", "OPENPROSE_API_URL", "PROSE_OUTPUT", - "PROSE_DEBUG" + "PROSE_DEBUG", + "PROSE_HARNESS" ] }, "additionalProperties": { "type": "string" + }, + "properties": { + "PROSE_HARNESS": { + "const": "openprose" + } } }, "files": { diff --git a/cli/shared/tests/test_service_contract.py b/cli/shared/tests/test_service_contract.py index 10577e6e..b07dd62b 100644 --- a/cli/shared/tests/test_service_contract.py +++ b/cli/shared/tests/test_service_contract.py @@ -235,6 +235,25 @@ def test_runner_error_details_carry_service_fields(self): self.assert_invalid("runner-error.schema.json", self.failure("SERVICE_REQUEST_REJECTED", **bad)) # ------------------------------------------------------------- environments + def test_fixture_environment_admits_only_the_explicit_hosted_selector(self): + schema=deepcopy(self.documents[RUNNER/'service-fixture.schema.json']) + schema['$ref']='#/$defs/case' + case=load_json(CLI/'conformance/cases/service/framework/no-cli-help-json.json') + self.assertEqual('openprose',case['environment']['PROSE_HARNESS']) + self.assertEqual([],self.errors(schema,case)) + for value in ('agents-sdk','mock','claude','',None,'openprose '): + invalid=deepcopy(case);invalid['environment']['PROSE_HARNESS']=value + with self.subTest(selector=value): + self.assertTrue(self.errors(schema,invalid)) + invalid=deepcopy(case);invalid['environment']['UNKNOWN_CASE_ENV']='value' + self.assertTrue(self.errors(schema,invalid)) + for key in ('OPENPROSE_API_KEY','OPENPROSE_API_URL','PROSE_OUTPUT','PROSE_DEBUG'): + allowed=deepcopy(case);allowed['environment'][key]='fixture-string' + with self.subTest(existing_key=key): + self.assertEqual([],self.errors(schema,allowed)) + allowed['environment'][key]=1 + self.assertTrue(self.errors(schema,allowed)) + def test_account_results_are_envelope_results(self): """`cli auth ...` and `cli org list` print the service-operation/1 envelope; their results are closed and name no service environment.""" From a645341106a45b9a1a404a41c60a9382f8b24cbd Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 21:59:15 -0400 Subject: [PATCH 28/64] Lease measured Homebrew and portable SDK fixture fixes --- cli/protocol/OWNERSHIP.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index a8bd98a1..d4e00bdc 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1070,3 +1070,7 @@ IMP-098 actual installed helper version assertion: `/root/configuration_contract IMP-098 measured service framework corpus reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/cases/service/framework/no-cli-double-dash-forwards.json`, `no-cli-examples-list-human.json`, `no-cli-examples-list-json.json`, `no-cli-file-forwards-with-hint.json`, `no-cli-help-command-json.json`, `no-cli-help-human.json`, `no-cli-help-json.json`, `no-cli-language-command-hint.json`, `no-cli-run-existing-file-forwards.json`, `no-cli-status-file-forwards.json`, `no-cli-status-json.json`, and `no-cli-status-jsonl.json`. Full admission 5 passed all 100 differential cases then found 12 old-default hosted-branch expectations among 1,089 service cases. Independent diagnosis confirms SPEC language routing and both products already condition these exact grammar/hint/refusal branches on explicitly selected openprose. Add only `environment.PROSE_HARNESS=openprose` and truthful descriptions; preserve every argv/output/hint/languageForwarded/no-request expectation. No product/oracle taxonomy changes, Git, builds or providers. Root owns the corresponding SPEC 7.1/missing-cli prose replacing historical hosted-default claims with explicit selection, and root runs both real service corpus suites. The twelve-fixture reconciliation additionally leases `/root/conformance_plumbing` `cli/conformance/runner/service-fixture.schema.json` only to allow the case environment key `PROSE_HARNESS` with the closed constant value `openprose`, and `cli/shared/tests/test_service_contract.py` for schema regression controls accepting that one selector and rejecting arbitrary harnesses/unknown environment keys. Corpus validation proved the existing environment allowlist rejects the explicit selector. Keep all other environment and corpus contracts unchanged; no general provider/harness admission or production change. Root retains Git, actual corpus replay and integration. + +IMP-098 actual retained-keg Homebrew repair: root owns applying the independently reviewed scratch copies to `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py`. Actual first rehearsal retained RC3 after uninstalling RC4; the reviewed repair confines all-version uninstall to the two newly owned fully qualified formulas, checks absent kegs/CLI/helper between rounds and deterministically links the selected genuine baseline. Forty unit controls and actual repaired scratch-driver replay pass all 94 checks at artifact source `3eeab98`, including both genuine upgrades/settings/cleanup. Source SHA256 `089b0cb7853fa2df4e4471769544fe2d8cc4662319763ce8ccbc1d749c87c0d1`, tests `1653e9d5d191080b3974e4767e54f1d793a3b742270bea3cdf6407bc48674fbd`; original hashes and patch retained under scratch. Full admission 6 has ended before application. + +IMP-098 measured public-surface repair: root owns only the two fixture helper shebang strings in `cli/bun/test/sdk-production.test.ts`, replacing committed machine-specific Bun paths with the actual test process executable interpolation. Full admission 6 passed both service and registry suites, then public-surface-files rejected developer home and ticket paths in these test literals. Preserve every setup/usage/cancellation/action assertion and executable test behavior; no allowlist, product or shared oracle changes. Root runs exact SDK suite and strict public-source scan before independent review. From 23b21673e01a13c9a1e8afbd800b87b443a4cda0 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:00:55 -0400 Subject: [PATCH 29/64] Remove owned Homebrew versions completely and keep SDK fixtures portable --- cli/bun/test/sdk-production.test.ts | 4 +- cli/ci/homebrew_rehearsal.py | 18 +++++-- cli/ci/test_homebrew_rehearsal.py | 76 +++++++++++++++++++++++++++-- 3 files changed, 89 insertions(+), 9 deletions(-) diff --git a/cli/bun/test/sdk-production.test.ts b/cli/bun/test/sdk-production.test.ts index d9a3f148..de471357 100644 --- a/cli/bun/test/sdk-production.test.ts +++ b/cli/bun/test/sdk-production.test.ts @@ -91,7 +91,7 @@ test("SDK setup recovery admits only recognized native reasons and retains compl for(const [setupReason,action] of Object.entries(oracle.setupFailure.reasons))for(const output of ["json","human"] as const)test(`actual CLI SDK ${setupReason} recovery survives ${output} rendering`,async()=>{ const root=await workspace(),helper=join(root,"prose-agents-sdk"); const terminal={type:"error",error_type:"SetupError",setup_reason:setupReason,message:"sdk-provider-body-secret-sentinel",limits:oracle.nativeLimits,usageObservation:oracle.observation.failureUsage,modelIdentity:oracle.observation.modelIdentity}; - await writeFile(helper,`#!/Users/mm/openprose/.scratch/imp-086-tools/bun-darwin-aarch64/bun\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));console.log(JSON.stringify(${JSON.stringify(terminal)}));process.exit(1);`,{mode:0o755}); + await writeFile(helper,`#!${process.execPath}\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));console.log(JSON.stringify(${JSON.stringify(terminal)}));process.exit(1);`,{mode:0o755}); let stdout="",stderr=""; const code=await runCli(["--output",output,"run","input.prose.md"],{platform:"darwin",arch:"arm64",processCwd:root,userConfigPath:join(root,"absent.toml"),env:{OPENAI_API_KEY:"fixture-key"},imageBundle:sentinelFixtureImage,installedAdapterProbe:{executable:helper,observationPath:join(root,"observation.json"),credentialGroup:"openai-api-key"},clock:{now:()=>"2025-01-01T00:00:00Z",monotonicMs:()=>0},ids:{invocationId:()=>"fixture-invocation-0001"},writeStdout:text=>{stdout+=text;},writeStderr:text=>{stderr+=text;}}); expect(code).toBe(22);expect(stdout+stderr).not.toContain("sdk-provider-body-secret-sentinel"); @@ -103,7 +103,7 @@ for(const scenario of ["success","failure","cancel"])test(`actual supervised SDK const root=await workspace(),helper=join(root,"prose-agents-sdk"),image=await verifyRuntimeImage(sentinelFixtureImage); const observation=scenario==="success"?oracle.observation.completedUsage:oracle.observation.failureUsage; const terminal={type:scenario==="success"?"final":"error",output:"native result",error_type:scenario==="cancel"?"CancelledError":"ExecutionError",limits:oracle.nativeLimits,usageObservation:observation,modelIdentity:oracle.observation.modelIdentity}; - const source=`#!/Users/mm/openprose/.scratch/imp-086-tools/bun-darwin-aarch64/bun\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));\nconst terminal=${JSON.stringify(terminal)};\n${scenario==="cancel"?"console.log(JSON.stringify({...terminal,type:'tool_result',name:'execute_shell'}));process.on('SIGTERM',()=>{console.log(JSON.stringify(terminal));process.exit(1)});setInterval(()=>{},1000);":"console.log(JSON.stringify(terminal));process.exit("+(scenario==="success"?0:1)+");"}`; + const source=`#!${process.execPath}\nconsole.log(JSON.stringify({type:'start',model:'gpt-6.1-sol',cwd:${JSON.stringify(root)}}));\nconst terminal=${JSON.stringify(terminal)};\n${scenario==="cancel"?"console.log(JSON.stringify({...terminal,type:'tool_result',name:'execute_shell'}));process.on('SIGTERM',()=>{console.log(JSON.stringify(terminal));process.exit(1)});setInterval(()=>{},1000);":"console.log(JSON.stringify(terminal));process.exit("+(scenario==="success"?0:1)+");"}`; await writeFile(helper,source,{mode:0o755});const task={schema:image.manifest.taskEnvelope.schemaId,argv:["prose","run","input.prose.md"],interactionMode:"non-interactive" as const}; const invocation:RunnerInvocation={schema:"openprose.runner-invocation/1",invocationId:"fixture-invocation-0001",cwd:root,languageImage:{formatVersion:image.manifest.imageFormatVersion,version:image.manifest.imageVersion,sha256:image.aggregateSha256},runner:{name:"bun",version:"test",commit:"test"},harness:"agents-sdk",transport:"jsonl",recursionToken:"fixture-recursion",task,taskDigestSha256:await sha256(canonicalJson(task))}; const controller=new AbortController(),capture=join(root,"native.jsonl"); diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 86b13522..0b6d4fca 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -358,7 +358,16 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False, if hashlib.sha256(active.read_bytes()).hexdigest() != verified['packageIdentity']['rustBinarySha256']: raise ValueError('Rust link switch did not select the verified executable') if previous is not None: - command('uninstall-fresh-before-upgrade', [brew, 'uninstall', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + # Initial admission established exclusive ownership of both formulas. + # Upgrade retains old kegs; ordinary uninstall removes only one version. + def require_empty_owned_installation(label: str) -> None: + installed = command(label, [brew, 'list', '--formula', '--versions']).stdout + if any(line.split()[0].split('/')[-1] in {'prose-bun', 'prose-rust'} for line in installed.splitlines() if line.split()): + raise ValueError('Owned rehearsal kegs survived all-version uninstall') + if active.exists() or active.is_symlink() or active_helper.exists() or active_helper.is_symlink(): + raise ValueError('Owned rehearsal commands survived all-version uninstall') + command('uninstall-fresh-before-upgrade', [brew, 'uninstall', '--force', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + require_empty_owned_installation('remaining-kegs-before-upgrades') settings = output / 'user-settings/cli.toml' settings_bytes = b'# explicit upgrade preservation\ntimeout = "9m"\n' settings.write_bytes(settings_bytes) @@ -375,7 +384,7 @@ def preserve(label: str) -> None: command(selected + '-install-base-rust', [brew, 'install', '--build-from-source', f'{TAP}/prose-rust']) if selected == 'bun': command(selected + '-unlink-base-rust', [brew, 'unlink', f'{TAP}/prose-rust']) - command(selected + '-link-base-bun', [brew, 'link', f'{TAP}/prose-bun']) + command(selected + '-link-base-selected', [brew, 'link', f'{TAP}/prose-{selected}']) if pub.digest(active) != previous['binaryHashes'][selected]: raise ValueError('Selected prior executable differs from genuine published bytes') base_banner = command(selected + '-base-version', [str(active), '--version']).stdout.strip() @@ -402,7 +411,8 @@ def preserve(label: str) -> None: raise ValueError('Upgrade changed the selected implementation or candidate identity') qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-selected', expected_timeout='9m') preserve(selected + '-settings-after-selected-upgrade') - command(selected + '-uninstall-upgraded', [brew, 'uninstall', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + command(selected + '-uninstall-upgraded', [brew, 'uninstall', '--force', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) + require_empty_owned_installation(selected + '-remaining-kegs-after-upgrade') preserve(selected + '-settings-after-uninstall') if active.exists() or active.is_symlink(): raise ValueError('Upgraded command survived uninstall') @@ -410,7 +420,7 @@ def preserve(label: str) -> None: finally: # Remove only packages under the newly owned rehearsal tap. for implementation in ('bun', 'rust'): - cleaned = subprocess.run([brew, 'uninstall', f'{TAP}/prose-{implementation}'], env=env, text=True, capture_output=True, timeout=180) + cleaned = subprocess.run([brew, 'uninstall', '--force', f'{TAP}/prose-{implementation}'], env=env, text=True, capture_output=True, timeout=180) (output / f'cleanup-{implementation}.log').write_text(cleaned.stdout + cleaned.stderr) untapped = subprocess.run([brew, 'untap', TAP], env=env, text=True, capture_output=True, timeout=180) (output / 'cleanup-tap.log').write_text(untapped.stdout + untapped.stderr) diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index ca270d6b..4d9467b3 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -337,6 +337,7 @@ def setUp(self): self.previous = {'version': '0.15.0-rc.3', 'root': self.root, 'manifestSha256': 'd' * 64, 'archives': self.archives, 'binaryHashes': {i: hashlib.sha256(('old-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} + self.retained_versions = {}; self.retained_before_force = []; self.leave_retained_keg = False self.kegs = {}; self.tapped = False; self.calls = []; self.corrupt_settings = False self.bad_dry_run = False self.runtime_banner = 'prose-agents-sdk 0.1.0' @@ -358,11 +359,14 @@ def execute(self, argv, **kwargs): filename = 'prose-' + implementation + '.rb' (self.installed_tap / 'Formula' / filename).write_bytes((self.output / 'tap/Formula' / filename).read_bytes()) elif verb == 'untap': self.tapped = False - elif verb == 'list': out = ''.join('prose-' + i + ' ' + state[1] + '\n' for i, state in self.kegs.items()) + elif verb == 'list': + out = ''.join('prose-' + i + ' ' + ' '.join(sorted(self.retained_versions.get(i, {state[1]}))) + '\n' for i, state in self.kegs.items()) elif verb in {'install', 'upgrade'}: implementation = argv[-1].split('-')[-1] formula = (self.installed_tap / 'Formula' / ('prose-' + implementation + '.rb')).read_text() version = rehearsal.re.search(r' version "([^"]+)"', formula).group(1) + if verb == 'install' and version in self.retained_versions.get(implementation, set()): + return subprocess.CompletedProcess(argv, 0, 'Already installed, not linked\n', '') if verb == 'upgrade': self.assertIn(implementation, self.kegs, 'Upgrade requires an installed previous keg') self.assertEqual(self.kegs[implementation][1], '0.15.0-rc.3', 'Upgrade must replace an actual previous version') @@ -371,6 +375,7 @@ def execute(self, argv, **kwargs): binary.write_bytes((('new-' if version.endswith('.4') else 'old-') + implementation).encode()) if version.endswith('.4'): (keg / 'prose-agents-sdk').write_bytes(b'sdk') self.kegs[implementation] = (keg.parent, version) + self.retained_versions.setdefault(implementation, set()).add(version) self.assertFalse(self.active.exists() or self.active.is_symlink(), 'Install/upgrade must vacate shared command') self.active.symlink_to(binary) if self.corrupt_settings and verb == 'upgrade': @@ -383,11 +388,24 @@ def execute(self, argv, **kwargs): if self.active.resolve() != binary.resolve(): out = 'Could not symlink'; code = 1 else: self.active.symlink_to(binary) elif verb == 'uninstall': - for name in argv[2:]: + force = '--force' in argv + for name in [arg for arg in argv[2:] if arg != '--force']: + self.assertTrue(name.startswith(rehearsal.TAP + '/prose-'), 'All-version cleanup must stay in the owned tap') implementation = name.split('-')[-1] if implementation in self.kegs: + versions = self.retained_versions.get(implementation, {self.kegs[implementation][1]}) + if force: self.retained_before_force.append((implementation, set(versions))) if self.active.is_symlink() and self.active.resolve().parent.parent == self.kegs[implementation][0].resolve(): self.active.unlink() - del self.kegs[implementation] + remaining = set() if force else versions - {self.kegs[implementation][1]} + if self.leave_retained_keg and force and len(versions) > 1: + remaining = {'0.15.0-rc.3'} + if remaining: + self.retained_versions[implementation] = remaining + version = sorted(remaining)[-1] + self.kegs[implementation] = (self.prefix / 'Cellar' / implementation / version, version) + else: + self.retained_versions.pop(implementation, None) + del self.kegs[implementation] elif verb != 'test': self.fail('Unexpected brew command: ' + str(argv)) elif argv[0] == 'git': pass # Inert fake: no Git operation occurs in this test. elif Path(argv[0]).name == 'prose-agents-sdk': out = '{}\n' @@ -432,6 +450,58 @@ def test_both_genuine_upgrades_preserve_selection_settings_and_scrub_credentials self.assertFalse(self.active.exists() or self.active.is_symlink()) self.assertNotEqual(self.installed_tap, self.output / 'tap') + def test_genuine_upgrade_retains_old_kegs_until_guarded_all_version_uninstall(self): + receipt = self.exercise() + retained = [(implementation, versions) for implementation, versions in self.retained_before_force + if versions == {'0.15.0-rc.3', '0.15.0-rc.4'}] + self.assertEqual([implementation for implementation, _ in retained], ['bun', 'rust', 'bun', 'rust']) + self.assertEqual(self.retained_versions, {}) + names = [check['name'] for check in receipt['checks']] + self.assertIn('bun-remaining-kegs-after-upgrade', names) + self.assertIn('rust-remaining-kegs-after-upgrade', names) + for argv, _ in self.calls: + if argv[:2] == ['brew', 'uninstall']: + self.assertIn('--force', argv) + self.assertTrue(all(arg.startswith(rehearsal.TAP + '/prose-') for arg in argv[2:] if arg != '--force')) + + def test_retained_old_keg_after_force_fails_before_second_round(self): + self.leave_retained_keg = True + with self.assertRaisesRegex(ValueError, 'kegs survived all-version uninstall'): + self.exercise() + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + # The second round must not disguise retained RC3 as a fresh installation. + labels = [p.name for p in self.output.glob('rust-install-base-*.log')] + self.assertEqual(labels, []) + + def test_lingering_helper_after_all_version_uninstall_is_refused(self): + original_execute = self.execute + def leave_helper(argv, **kwargs): + had_old_and_new = any(len(versions) > 1 for versions in self.retained_versions.values()) + result = original_execute(argv, **kwargs) + if argv[:2] == ['brew', 'uninstall'] and '--force' in argv and had_old_and_new: + (self.prefix / 'bin/prose-agents-sdk').write_bytes(b'lingering-helper') + return result + with patch.object(self, 'execute', side_effect=leave_helper): + with self.assertRaisesRegex(ValueError, 'commands survived all-version uninstall'): + self.exercise() + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_ordinary_uninstall_retains_old_unlinked_keg_and_reinstall_does_not_link(self): + self.exercise() + for version in ('0.15.0-rc.3', '0.15.0-rc.4'): + (self.prefix / 'Cellar/bun' / version / 'bin').mkdir(parents=True, exist_ok=True) + self.retained_versions['bun'] = {'0.15.0-rc.3', '0.15.0-rc.4'} + self.kegs['bun'] = (self.prefix / 'Cellar/bun/0.15.0-rc.4', '0.15.0-rc.4') + env = {'PROSE_CONFIG_DIR': str(self.output / 'user-settings')} + self.execute(['brew', 'uninstall', rehearsal.TAP + '/prose-bun'], env=env) + self.assertEqual(self.retained_versions['bun'], {'0.15.0-rc.3'}) + formula = self.installed_tap / 'Formula/prose-bun.rb' + formula.write_text(formula.read_text().replace('0.15.0-rc.4', '0.15.0-rc.3')) + result = self.execute(['brew', 'install', '--build-from-source', rehearsal.TAP + '/prose-bun'], env=env) + self.assertEqual(result.returncode, 0) + self.assertIn('Already installed, not linked', result.stdout) + self.assertFalse(self.active.exists() or self.active.is_symlink()) + def test_changed_settings_fail_instead_of_claiming_upgrade_success(self): self.corrupt_settings = True with self.assertRaisesRegex(ValueError, 'changed explicit user settings'): From 96faa954c4ae13de8a2dd4466c381570c6333d6f Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:14:49 -0400 Subject: [PATCH 30/64] Lease measured SDK packaging admission repairs --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index d4e00bdc..ab3b50dc 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1074,3 +1074,5 @@ The twelve-fixture reconciliation additionally leases `/root/conformance_plumbin IMP-098 actual retained-keg Homebrew repair: root owns applying the independently reviewed scratch copies to `cli/ci/homebrew_rehearsal.py` and `cli/ci/test_homebrew_rehearsal.py`. Actual first rehearsal retained RC3 after uninstalling RC4; the reviewed repair confines all-version uninstall to the two newly owned fully qualified formulas, checks absent kegs/CLI/helper between rounds and deterministically links the selected genuine baseline. Forty unit controls and actual repaired scratch-driver replay pass all 94 checks at artifact source `3eeab98`, including both genuine upgrades/settings/cleanup. Source SHA256 `089b0cb7853fa2df4e4471769544fe2d8cc4662319763ce8ccbc1d749c87c0d1`, tests `1653e9d5d191080b3974e4767e54f1d793a3b742270bea3cdf6407bc48674fbd`; original hashes and patch retained under scratch. Full admission 6 has ended before application. IMP-098 measured public-surface repair: root owns only the two fixture helper shebang strings in `cli/bun/test/sdk-production.test.ts`, replacing committed machine-specific Bun paths with the actual test process executable interpolation. Full admission 6 passed both service and registry suites, then public-surface-files rejected developer home and ticket paths in these test literals. Preserve every setup/usage/cancellation/action assertion and executable test behavior; no allowlist, product or shared oracle changes. Root runs exact SDK suite and strict public-source scan before independent review. + +IMP-098 measured package-local admission reconciliation: `/root/configuration_contract` owns only `cli/ci/test_package_local.py` for the four failures in full admission7. Keep bare fresh/hostile-cwd execution and SDK default assertions, all package bytes/identity/redaction checks, and exact dependency component coverage including the existing 67 Python candidates. Supply the existing valid fake SDK build to sentinel/missing-external-gate subprocesses so they reach their original intended strict rejection; do not accept the earlier missing-helper error or weaken production prerequisites. Preserve separate explicit missing-SDK rejection. No production/oracle edits, Git, heavy builds or provider/key access by agent. Root runs focused and full admission and owns integration; full admission7 has ended before this lease. From 6b4507de991ef059fc1ff6ea84d7e1a0e43358cf Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:15:50 -0400 Subject: [PATCH 31/64] Clarify cross-platform early packaging rejection coverage --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index ab3b50dc..fd1adf3d 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1076,3 +1076,5 @@ IMP-098 actual retained-keg Homebrew repair: root owns applying the independentl IMP-098 measured public-surface repair: root owns only the two fixture helper shebang strings in `cli/bun/test/sdk-production.test.ts`, replacing committed machine-specific Bun paths with the actual test process executable interpolation. Full admission 6 passed both service and registry suites, then public-surface-files rejected developer home and ticket paths in these test literals. Preserve every setup/usage/cancellation/action assertion and executable test behavior; no allowlist, product or shared oracle changes. Root runs exact SDK suite and strict public-source scan before independent review. IMP-098 measured package-local admission reconciliation: `/root/configuration_contract` owns only `cli/ci/test_package_local.py` for the four failures in full admission7. Keep bare fresh/hostile-cwd execution and SDK default assertions, all package bytes/identity/redaction checks, and exact dependency component coverage including the existing 67 Python candidates. Supply the existing valid fake SDK build to sentinel/missing-external-gate subprocesses so they reach their original intended strict rejection; do not accept the earlier missing-helper error or weaken production prerequisites. Preserve separate explicit missing-SDK rejection. No production/oracle edits, Git, heavy builds or provider/key access by agent. Root runs focused and full admission and owns integration; full admission7 has ended before this lease. + +The preceding packaging test lease is clarified for the cross-platform early-refusal tests: pass an explicitly uninspected absent SDK path and assert it remains absent, preserving original image/external-gate errors and absent output. These branches intentionally reject before inspecting the native helper prerequisite. This supersedes the valid-native-fixture suggestion, avoids POSIX/compiler or alphabetical test-order coupling, and does not claim SDK validation. Dedicated missing-SDK and verified-helper controls remain unchanged; no test-helper refactor is leased. From 1e74dccd330c5309956391eb9529d22ea71c840f Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:18:06 -0400 Subject: [PATCH 32/64] Reconcile packaging acceptance with the packaged SDK default --- cli/ci/test_package_local.py | 51 +++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 7 deletions(-) diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index 4b0b254f..7098ac11 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -2844,10 +2844,12 @@ def test_unpacked_rust_and_bun_archives_run_the_public_machine_surface( report = json.loads(result.stdout) self.assertEqual(report["schema"], schema) if schema == "openprose.configuration-explanation/1": - self.assertEqual(report["values"]["harness"]["value"], "openprose") + self.assertEqual(report["values"]["harness"]["value"], "agents-sdk") + self.assertEqual(report["values"]["model"]["value"], "gpt-6.1-sol") + self.assertEqual(report["values"]["authProfile"]["value"], "openai-api-key") self.assertFalse(report["values"]["color"]["value"]) if schema == "openprose.doctor-report/1": - self.assertEqual(report["selectedHarness"], "openprose") + self.assertEqual(report["selectedHarness"], "agents-sdk") self.assertEqual(report["image"]["version"], image["imageVersion"]) self.assertEqual( report["image"]["sha256"], image["aggregateSha256"]["sha256"] @@ -2881,9 +2883,10 @@ def test_npm_meta_and_platform_packages_install_with_scripts_disabled(self) -> N self.root / "npm-config-env", ) self.assertEqual(explained.returncode, 0, explained.stderr) - self.assertEqual( - json.loads(explained.stdout)["values"]["harness"]["value"], "openprose" - ) + values = json.loads(explained.stdout)["values"] + self.assertEqual(values["harness"]["value"], "agents-sdk") + self.assertEqual(values["model"]["value"], "gpt-6.1-sol") + self.assertEqual(values["authProfile"]["value"], "openai-api-key") self.assertNotIn("must-not-load", explained.stdout.decode("utf-8")) account = run_artifact( executable, @@ -4272,8 +4275,36 @@ def test_evidence_scaffolding_is_bound_and_explicitly_non_authoritative( for item in component["properties"] ) ] - # The Rust kernel HTTPS loader adds 51 locked dependencies. - self.assertEqual(len(dependency_components), 127 + 14 + 13) + # Include every hash-locked platform wheel candidate for the packaged SDK. + sdk_packages = dependency["inventories"]["agentsSdkPython"]["packages"] + self.assertEqual(len(sdk_packages), 67) + sdk_components = [component for component in dependency_components + if component["group"] == "agents-sdk-python"] + self.assertEqual(len(sdk_components), 67) + for package in sdk_packages: + self.assertEqual(package["scopes"], ["frozen-sdk-build"]) + self.assertEqual(package["integrity"]["status"], "declared") + self.assertEqual(package["integrity"]["algorithm"], "sha256") + digest = package["integrity"]["digest"] + self.assertRegex(digest, r"^[0-9a-f]{64}$") + self.assertEqual(package["source"], "pypi:wheel-sha256:" + digest) + actual_sdk_identities = set() + for component in sdk_components: + properties = {item["name"]: item["value"] for item in component["properties"]} + self.assertEqual(component["type"], "library") + self.assertEqual(properties["openprose:component"], "agents-sdk-python") + self.assertEqual(properties["openprose:scopes"], "frozen-sdk-build") + self.assertEqual(properties["openprose:integrity-status"], "declared") + self.assertEqual(len(component["hashes"]), 1) + self.assertEqual(component["hashes"][0]["alg"], "SHA-256") + actual_sdk_identities.add((component["name"], component["version"], + properties["openprose:source"], component["hashes"][0]["content"])) + self.assertEqual( + actual_sdk_identities, + {(package["name"], package["version"], package["source"], package["integrity"]["digest"]) + for package in sdk_packages}, + ) + self.assertEqual(len(dependency_components), 127 + 14 + 13 + 67) self.assertEqual( len({component["bom-ref"] for component in dependency_components}), len(dependency_components), @@ -4420,6 +4451,8 @@ def test_release_mode_rejects_sentinel_and_missing_external_gates(self) -> None: str(self.rust_binary), "--bun-binary", str(self.bun_binary), + "--agents-sdk-build", + str(self.root / "uninspected-sdk"), "--image-manifest", str(IMAGE_MANIFEST), "--out", @@ -4454,6 +4487,8 @@ def test_release_mode_rejects_sentinel_and_missing_external_gates(self) -> None: str(self.rust_binary), "--bun-binary", str(self.bun_binary), + "--agents-sdk-build", + str(self.root / "uninspected-sdk"), "--image-manifest", str(eligible), *readelf_args(), @@ -4470,6 +4505,8 @@ def test_release_mode_rejects_sentinel_and_missing_external_gates(self) -> None: self.assertIn("--canonical-profile", missing.stderr) self.assertIn("--release-evidence", missing.stderr) self.assertFalse((self.root / "release-missing-gates").exists()) + # These intended early refusals must precede inspection of SDK input bytes. + self.assertFalse((self.root / "uninspected-sdk").exists()) @unittest.skipIf( platform.system() == "Windows", "executable fixture uses a POSIX shebang" From 6d9838ca3f374ba66ddaa0d4168ffed4f8c8967d Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:34:25 -0400 Subject: [PATCH 33/64] Lease authenticated npm SDK rehearsal coverage --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index fd1adf3d..0e3bd44a 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1078,3 +1078,5 @@ IMP-098 measured public-surface repair: root owns only the two fixture helper sh IMP-098 measured package-local admission reconciliation: `/root/configuration_contract` owns only `cli/ci/test_package_local.py` for the four failures in full admission7. Keep bare fresh/hostile-cwd execution and SDK default assertions, all package bytes/identity/redaction checks, and exact dependency component coverage including the existing 67 Python candidates. Supply the existing valid fake SDK build to sentinel/missing-external-gate subprocesses so they reach their original intended strict rejection; do not accept the earlier missing-helper error or weaken production prerequisites. Preserve separate explicit missing-SDK rejection. No production/oracle edits, Git, heavy builds or provider/key access by agent. Root runs focused and full admission and owns integration; full admission7 has ended before this lease. The preceding packaging test lease is clarified for the cross-platform early-refusal tests: pass an explicitly uninspected absent SDK path and assert it remains absent, preserving original image/external-gate errors and absent output. These branches intentionally reject before inspecting the native helper prerequisite. This supersedes the valid-native-fixture suggestion, avoids POSIX/compiler or alphabetical test-order coupling, and does not claim SDK validation. Dedicated missing-SDK and verified-helper controls remain unchanged; no test-helper refactor is leased. + +IMP-098 measured installed npm mechanical rehearsal repair: /root/conformance_plumbing owns cli/conformance/runner/run.py, cli/conformance/runner/test_runner.py, cli/ci/rehearse_release.py, and cli/ci/test_rehearse_release.py only for the actual final full-admission8 failure. The real installed npm launcher uses authenticated Node; SDK fixture relocation currently blanket-rejects interpreters. Freeze focused tests before plumbing an explicit benchmark-bound npm platform context into SDK fixture preparation. Verify original launcher/Node and native platform binary/manifest byte identities; clone the minimal actual npm package layout, execute the unchanged launcher snapshot with relocated module context, place the provider-free fixture helper beside its actual native child, and check closure before/after execution. Native-only direct product admission remains strict; no arbitrary interpreted exemption, launcher bypass or skipped SDK cases. Retain all 100 cases across three installed candidates and exact 300/200/500 totals, original non-SDK execution routes, safe paths, source/custody/mutation controls, and separate semantic/release nonclaims. Internal fixture custody may be retained without changing the public report shape; stop if any schema or cross-lease change is necessary. No production CLI/runtime edits, Git, builds or provider/key access. Root owns actual focused/rehearsal/full replay and integration; full admission8 has ended. From 18b9bfb22786dc777213d0e3ebaf5d79b89f6894 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 22:44:23 -0400 Subject: [PATCH 34/64] Qualify SDK relocation through the authenticated installed npm launcher --- cli/ci/rehearse_release.py | 41 +++++++++ cli/ci/test_rehearse_release.py | 40 ++++++++ cli/conformance/runner/run.py | 128 ++++++++++++++++++++++++-- cli/conformance/runner/test_runner.py | 91 ++++++++++++++++++ 4 files changed, 291 insertions(+), 9 deletions(-) diff --git a/cli/ci/rehearse_release.py b/cli/ci/rehearse_release.py index 48afdb61..1c2a3b2e 100644 --- a/cli/ci/rehearse_release.py +++ b/cli/ci/rehearse_release.py @@ -766,6 +766,42 @@ def _conformance_inputs( return candidates, node_path, node_digest +def _npm_conformance_context(install: Path, raw_report: Mapping[str, Any]) -> dict[str, Any]: + platform=raw_report.get('platform') + if not isinstance(platform,str) or not re.fullmatch('[a-z0-9-]+',platform): + raise RehearsalError('npm conformance platform is invalid') + records=raw_report.get('installations') + npm=[row for row in records or [] if row.get('surface')=='npm-launcher'] + if len(npm)!=1: + raise RehearsalError('npm conformance requires the benchmark-bound installed tree') + entries=npm[0].get('treeIdentity',{}).get('entries',[]) + prefix=install/'npm-prefix' + meta='lib/node_modules/@openprose/prose-cli' + child='lib/node_modules/@openprose/prose-cli-'+platform + relative={'metaManifest':meta+'/package.json','launcher':meta+'/bin/prose.js', + 'platformManifest':child+'/package.json','native':child+'/bin/prose'} + files={} + for key,name in relative.items(): + matching=[row for row in entries if row.get('path')==name] + if len(matching)!=1 or matching[0].get('type')!='regular': + raise RehearsalError('npm conformance closure lacks one benchmark-bound regular file') + row=matching[0] + digest=require_sha(row.get('sha256'),'npm '+key) + length=row.get('byteLength') + if type(length) is not int or not 0 None: _build, raw, analysis = reports() benchmark = FakeBenchmark(raw, analysis) diff --git a/cli/conformance/runner/run.py b/cli/conformance/runner/run.py index 3d4ca87f..a11ab1ac 100755 --- a/cli/conformance/runner/run.py +++ b/cli/conformance/runner/run.py @@ -5,7 +5,7 @@ import argparse from copy import deepcopy -from dataclasses import dataclass +from dataclasses import dataclass, replace import hashlib import json import os @@ -84,9 +84,9 @@ const fs = require("node:fs"); const path = require("node:path"); const Module = require("node:module"); -const [snapshot, sourceFilename, ...opaqueArgv] = process.argv.slice(1); +const [snapshot, sourceFilename, launchFilename, ...opaqueArgv] = process.argv.slice(1); if (!snapshot || !sourceFilename) throw new Error("missing conformance snapshot context"); -process.argv = [process.execPath, sourceFilename, ...opaqueArgv]; +process.argv = [process.execPath, launchFilename, ...opaqueArgv]; const candidate = new Module(sourceFilename, module); candidate.filename = sourceFilename; candidate.paths = Module._nodeModulePaths(path.dirname(sourceFilename)); @@ -103,6 +103,8 @@ class Product: execution_executable: Path | None = None execution_interpreter: Path | None = None execution_source_context: Path | None = None + npm_context: dict[str, Any] | None = None + execution_launch_path: Path | None = None @property def expected_runner_name(self) -> str: @@ -119,6 +121,7 @@ def execution_argv(self, arguments: list[str]) -> list[str]: "--", str(executable), str(self.execution_source_context), + str(self.execution_launch_path or self.execution_source_context), *arguments, ] return [ @@ -408,6 +411,7 @@ def snapshot_product( executable, interpreter, source_context, + product.npm_context, ) @@ -1623,12 +1627,81 @@ def isolate_workspace(workspace: Path) -> None: raise ValueError("mechanical workspace Git boundary must be a directory or file") +def npm_context_paths(context: dict[str, Any]) -> dict[str, Path]: + if not isinstance(context,dict) or set(context) != {'prefix','platform','files'} or not isinstance(context['platform'],str) or context['platform'] not in {'darwin-arm64','darwin-x64','linux-arm64-gnu','linux-x64-gnu'}: + raise ValueError('SDK npm context must be an explicit closed native platform') + if not isinstance(context['prefix'],str): + raise ValueError('SDK npm context prefix must be a string') + prefix=Path(context['prefix']) + if not prefix.is_absolute() or prefix.resolve()!=prefix or prefix.is_symlink() or not prefix.is_dir(): + raise ValueError('SDK npm context prefix must be a regular absolute directory') + meta='lib/node_modules/@openprose/prose-cli' + platform='lib/node_modules/@openprose/prose-cli-'+context['platform'] + required={'metaManifest':meta+'/package.json','launcher':meta+'/bin/prose.js', + 'platformManifest':platform+'/package.json','native':platform+'/bin/prose'} + files=context['files'] + if not isinstance(files,dict) or set(files)!=set(required): + raise ValueError('SDK npm context closure is incomplete') + paths={} + for key,relative in required.items(): + row=files[key] + if not isinstance(row,dict) or set(row)!={'path','byteLength','sha256'} or row['path']!=relative or type(row['byteLength']) is not int or not 0 list[Product]: + contexts={} + for label,encoded in specifications: + if label in contexts or label not in {product.name for product in products}: + raise ValueError('SDK npm context label is duplicate or unknown') + contexts[label]=json.loads(encoded) + for product in products: + if product.name in contexts: + if product.interpreter is None or product.expected_runner_name!='bun': + raise ValueError('SDK npm context requires an explicit Bun launcher interpreter') + npm_context_paths(contexts[product.name]) + return [replace(product,npm_context=contexts.get(product.name,product.npm_context)) for product in products] + + def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: Path, environment_root: Path) -> tuple[Product, dict[str, Any], Path]: """Exercise real sibling discovery using an exact native candidate byte copy.""" workspace = workspace.resolve() environment_root = environment_root.resolve() - if product.interpreter is not None or product.execution_interpreter is not None: + interpreted=product.interpreter is not None or product.execution_interpreter is not None + if interpreted and product.npm_context is None: raise ValueError("SDK installation cases require a native compiled candidate") oracle = json.loads((CLI / 'shared/fixtures/adapters/sdk-production.json').read_text('utf-8')) entries = [entry for entry in oracle['cases'] if entry['id'] == case['id']] @@ -1641,10 +1714,26 @@ def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: install = environment_root / 'relocated installation' / 'real bin' install.mkdir(parents=True, exist_ok=False) source = product.execution_executable or product.executable - native = install / 'prose' + npm_paths=None + if interpreted: + npm_paths=npm_context_paths(product.npm_context) + if product.execution_source_context != npm_paths['launcher'].resolve() or not _uses_node_commonjs_source_context(npm_paths['launcher'],product.execution_interpreter or product.interpreter): + raise ValueError('SDK npm context requires the exact authenticated CommonJS launcher') + if source.read_bytes()!=npm_paths['launcher'].read_bytes(): + raise ValueError('SDK npm launcher snapshot differs from its authenticated source') + prefix=install/'npm-prefix' + for key in ('metaManifest','launcher','platformManifest'): + destination=prefix/product.npm_context['files'][key]['path'] + destination.parent.mkdir(parents=True,exist_ok=True) + shutil.copyfile((product.execution_executable or product.executable) if key=='launcher' else npm_paths[key],destination) + native=prefix/product.npm_context['files']['native']['path'] + source=npm_paths['native'] + else: + native = install / 'prose' + native.parent.mkdir(parents=True,exist_ok=True) shutil.copyfile(source, native) native.chmod(0o700) - helper = install / 'prose-agents-sdk' + helper = native.parent / 'prose-agents-sdk' shutil.copyfile(INSTALLED_ADAPTER_HARNESS, helper) helper.chmod(0o700) poison = environment_root / 'foreign-path' @@ -1652,13 +1741,15 @@ def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: shutil.copyfile(INSTALLED_ADAPTER_HARNESS, poison / 'prose-agents-sdk') (poison / 'prose-agents-sdk').chmod(0o700) (poison / 'python3').symlink_to(Path(sys.executable).resolve()) + launcher_context=(prefix/product.npm_context['files']['launcher']['path']) if interpreted else None + launch_target=launcher_context if interpreted else native if fixture['installation'] == 'symlink': shim = environment_root / 'user bin' shim.mkdir() target = shim / 'prose' - target.symlink_to(native) + target.symlink_to(launch_target) else: - target = native + target = launch_target if fixture['userConfig'] is not None: user = workspace / 'home/.prose/cli.toml' user.parent.mkdir(parents=True) @@ -1676,8 +1767,15 @@ def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: setup = dict(scenario=fixture['scenario'],expectedHelper=str(helper.resolve()),model=oracle['defaults']['model'],limits=limits,usageObservation=usage,modelIdentity=identity,wireNumberLexemes=fixture.get('wireNumberLexemes', {}),errorType=fixture.get('rawErrorType','ExecutionError'),setupReason=fixture.get('rawSetupReason'),errorMessage=fixture.get('rawMessage','safe fixture failure')) (workspace / '.sdk-compatibility-fixture.json').write_text(json.dumps(setup), encoding='utf-8') execution = Product(product.name, product.executable, product.runner_name, - execution_executable=target) + product.interpreter, (product.execution_executable or product.executable) if interpreted else target, product.execution_interpreter, + launcher_context, product.npm_context, target if interpreted else None) fixture = {**fixture, 'nativePath': str(native), 'nativeSha256': sha256(source.read_bytes())} + if interpreted: + fixture['npmContext']=product.npm_context + fixture['npmCloneFiles']=[{'path':str(prefix/row['path']),'sha256':row['sha256'],'byteLength':row['byteLength']} for row in product.npm_context['files'].values()] + preflight=validate_sdk_effects(Observation(product,case,0,b'',b'',workspace=workspace,sdk_fixture=fixture)) + if preflight: + raise ValueError('; '.join(preflight)) return execution, fixture, poison @@ -1702,6 +1800,15 @@ def validate_sdk_effects(observation: Observation) -> list[str]: failures.append('SDK default upgrade changed an explicit alternative harness preference') except OSError: failures.append('SDK fixture effect bytes unavailable') + if 'npmContext' in fixture: + try: + npm_context_paths(fixture['npmContext']) + for row in fixture['npmCloneFiles']: + path=Path(row['path']) + if any(member.is_symlink() for member in (path,*path.parents)) or not path.is_file() or path.stat().st_size!=row['byteLength'] or sha256(path.read_bytes())!=row['sha256']: + failures.append('SDK npm relocated closure bytes changed') + except (OSError,ValueError,TypeError,KeyError): + failures.append('SDK npm original closure custody changed') for group in fixture.get('absentObservationGroups', []): parsed = observation.parsed error = parsed.get('error') if isinstance(parsed, dict) else None @@ -2358,6 +2465,7 @@ def main(argv: list[str] | None = None) -> int: "the interpreter bytes are bound into the report" ), ) + parser.add_argument('--candidate-npm-context',action='append',nargs=2,default=[],metavar=('LABEL','JSON'),help='Explicit benchmark-bound npm native child closure for SDK installation fixtures') args = parser.parse_args(argv) if args.candidate and (args.rust is not None or args.bun is not None): parser.error("--candidate cannot be combined with --rust or --bun") @@ -2377,6 +2485,7 @@ def main(argv: list[str] | None = None) -> int: ] ) products = attach_candidate_interpreters(products, args.candidate_interpreter) + products = attach_npm_contexts(products,args.candidate_npm_context) except ValueError as error: parser.error(str(error)) verified_products: list[Product] = [] @@ -2392,6 +2501,7 @@ def main(argv: list[str] | None = None) -> int: if product.interpreter is not None else None ), + npm_context=product.npm_context, ) try: identity = capture_candidate_identity(verified) diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index 08c47890..e4e6ac46 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -63,6 +63,97 @@ def test_discovers_new_contracts_and_fails_closed_on_unknown_or_duplicate(self): class RunnerUnitTest(unittest.TestCase): + def npm_fixture(self, root): + prefix=root.resolve()/'installed' + base='lib/node_modules/@openprose/prose-cli' + child=base+'-darwin-arm64' + rows={'launcher':base+'/bin/prose.js','metaManifest':base+'/package.json','platformManifest':child+'/package.json','native':child+'/bin/prose'} + launcher=b"const fs=require('node:fs'); const path=require('node:path'); process.stdout.write(JSON.stringify({argv:process.argv.slice(1),child:fs.existsSync(path.resolve(__dirname,'../../prose-cli-darwin-arm64/bin/prose'))}));" + launcher+=b" if(process.env.SDK_FIXTURE_EXECUTE){const cp=require('node:child_process'); const child=path.resolve(__dirname,'../../prose-cli-darwin-arm64/bin/prose'); const native=cp.spawnSync(child,[],{encoding:'utf8'}); if(native.status!==0)process.exit(91); const helper=cp.spawnSync(process.env.SDK_FIXTURE_PYTHON,[path.join(path.dirname(child),'prose-agents-sdk'),'--cwd',process.cwd(),'--instructions',path.join(process.cwd(),'instructions'),'--model','gpt-6.1-sol','--prompt',JSON.stringify({argv:['prose','run','input.prose.md']})],{encoding:'utf8'}); if(helper.status!==0){process.stderr.write(helper.stderr);process.exit(92);} process.stderr.write(helper.stdout); }" + native=Path('/usr/bin/true').read_bytes() + cohort={'schema':'openprose.npm-cohort/3','version':'0.16.0-rc.1','sourceRevision':'a'*40,'admittedPlatforms':['darwin-arm64']} + meta={'name':'@openprose/prose-cli','version':cohort['version'],'type':'commonjs','bin':{'prose':'bin/prose.js'},'openproseCohort':cohort,'optionalDependencies':{'@openprose/prose-cli-darwin-arm64':'npm:@openprose/prose-cli@0.16.0-0.rc.1-darwin-arm64'},'openproseLauncher':{'path':'bin/prose.js','sha256':runner.sha256(launcher),'byteLength':len(launcher)}} + platform={'name':'@openprose/prose-cli','version':'0.16.0-0.rc.1-darwin-arm64','openproseCohort':cohort,'openprosePlatform':'darwin-arm64','openproseSourceRevision':'a'*40,'openproseBinary':'bin/prose','openproseBinarySha256':runner.sha256(native),'openproseBinaryByteLength':len(native)} + contents={'launcher':launcher,'native':native,'metaManifest':json.dumps(meta).encode(),'platformManifest':json.dumps(platform).encode()} + files={} + for key,relative in rows.items(): + path=prefix/relative; path.parent.mkdir(parents=True,exist_ok=True); path.write_bytes(contents[key]); path.chmod(0o700) + files[key]={'path':relative,'sha256':runner.sha256(contents[key]),'byteLength':len(contents[key])} + return {'prefix':str(prefix),'platform':'darwin-arm64','files':files} + + def test_npm_context_rejects_malformed_escape_symlink_and_custody_changes(self): + with tempfile.TemporaryDirectory() as temporary: + context=self.npm_fixture(Path(temporary)) + self.assertEqual(4,len(runner.npm_context_paths(context))) + for mutation in ('null','missing','escape','length','unknown','platform','files-list','row-list','prefix-type'): + changed=json.loads(json.dumps(context)) + if mutation=='null': changed=None + elif mutation=='missing': del changed['files']['native'] + elif mutation=='escape': changed['files']['native']['path']='../native' + elif mutation=='length': changed['files']['native']['byteLength']=True + elif mutation=='unknown': changed['secret']='unaccepted' + elif mutation=='files-list': changed['files']=[] + elif mutation=='row-list': changed['files']['native']=[] + elif mutation=='prefix-type': changed['prefix']=None + else: changed['platform']='linux-x64-musl' + with self.subTest(mutation=mutation), self.assertRaises(ValueError): runner.npm_context_paths(changed) + path=Path(context['prefix'])/context['files']['native']['path'] + data=path.read_bytes(); path.write_bytes(data+b'changed') + with self.assertRaisesRegex(ValueError,'bytes changed'): runner.npm_context_paths(context) + path.write_bytes(data); foreign=path.with_name('foreign'); path.rename(foreign); path.symlink_to(foreign) + with self.assertRaisesRegex(ValueError,'symlink'): runner.npm_context_paths(context) + + def test_npm_context_rejects_metadata_rebound_to_foreign_cohort(self): + for field in ('name','version','openprosePlatform','openproseSourceRevision','openproseCohort','openproseBinarySha256'): + with self.subTest(field=field), tempfile.TemporaryDirectory() as temporary: + context=self.npm_fixture(Path(temporary)); row=context['files']['platformManifest']; path=Path(context['prefix'])/row['path'] + payload=json.loads(path.read_text()); payload[field]={'schema':'foreign'} if field=='openproseCohort' else 'foreign' + path.write_text(json.dumps(payload)); row.update(sha256=runner.sha256(path.read_bytes()),byteLength=path.stat().st_size) + with self.assertRaises(ValueError): runner.npm_context_paths(context) + + def test_npm_context_rejects_unbound_interpreter_labels_and_metadata_shapes(self): + with tempfile.TemporaryDirectory() as temporary: + context=self.npm_fixture(Path(temporary)); launcher=Path(context['prefix'])/context['files']['launcher']['path'] + product=runner.Product('fixture',launcher,'bun',Path(sys.executable)) + for specifications in ([['unknown',json.dumps(context)]],[['fixture',json.dumps(context)],['fixture',json.dumps(context)]],[['fixture','[]']]): + with self.assertRaises(ValueError): runner.attach_npm_contexts([product],specifications) + with self.assertRaisesRegex(ValueError,'explicit Bun launcher'): + runner.attach_npm_contexts([runner.Product('fixture',launcher,'bun')],[['fixture',json.dumps(context)]]) + for key in ('metaManifest','platformManifest'): + with self.subTest(key=key), tempfile.TemporaryDirectory() as temporary: + context=self.npm_fixture(Path(temporary)); row=context['files'][key]; path=Path(context['prefix'])/row['path']; path.write_bytes(b'[]') + row.update(sha256=runner.sha256(b'[]'),byteLength=2) + with self.assertRaises(ValueError): runner.npm_context_paths(context) + + @unittest.skipUnless(shutil.which('node'),'Node is required for npm closure fixture') + def test_sdk_npm_executes_immutable_snapshot_with_minimal_relocated_closure(self): + for number in (4,8): + with self.subTest(case=number), tempfile.TemporaryDirectory() as temporary: + root=Path(temporary); context=self.npm_fixture(root) + launcher=Path(context['prefix'])/context['files']['launcher']['path'] + product=runner.Product('npm-launcher',launcher,'bun',Path(shutil.which('node'))) + product=runner.attach_npm_contexts([product],[['npm-launcher',json.dumps(context)]])[0] + snapshot=runner.snapshot_product(product,runner.capture_candidate_identity(product),root/'snapshot') + environment,workspace=runner.product_roots(root/'case','npm-launcher'); runner.isolate_workspace(workspace) + case=json.loads((runner.CASES/'adapters'/f'sdk-production-{number:02}.json').read_text()) + execution,fixture,poison=runner.prepare_sdk_installation(snapshot,case,workspace,environment) + (workspace/'instructions').write_bytes(b'OPENPROSE_SENTINEL_IMAGE_V1') + process_environment={'PATH':str(poison),'SDK_FIXTURE_EXECUTE':'1','SDK_FIXTURE_PYTHON':str(Path(sys.executable).resolve())} + self.assertEqual(snapshot.execution_executable,execution.execution_executable) + self.assertEqual(number==8,execution.execution_launch_path.is_symlink()) + native=Path(fixture['nativePath']); self.assertTrue((native.parent/'prose-agents-sdk').is_file()) + self.assertEqual(4,len(fixture['npmCloneFiles'])) + result=runner.run_owned_process(execution.execution_argv(['opaque']),cwd=workspace,environment=process_environment,timeout_seconds=3) + self.assertEqual(0,result.exit_code,result.stderr) + records=[json.loads(line) for line in result.stderr.splitlines()]; self.assertEqual('final',records[-1]['type']); self.assertTrue((workspace/'.sdk-harness-started').is_file()) + payload=json.loads(result.stdout); self.assertTrue(payload['child']); self.assertEqual([str(execution.execution_launch_path),'opaque'],payload['argv']) + observation=runner.Observation(product,case,0,b'',b'',workspace=workspace,sdk_fixture=fixture) + self.assertEqual([],runner.validate_sdk_effects(observation)) + execution.execution_source_context.write_text("throw Error('mutable clone executed')") + result=runner.run_owned_process(execution.execution_argv(['opaque']),cwd=workspace,environment=process_environment,timeout_seconds=3) + self.assertEqual(0,result.exit_code,result.stderr) + self.assertIn('SDK npm relocated closure bytes changed',runner.validate_sdk_effects(observation)) + def test_host_oracle_keeps_admitted_expectations_and_requires_rejection(self): for path in runner.case_paths(7, set()): case = json.loads(path.read_text("utf-8")) From 46a6a348276f8b8a89ad1c7e226db6aa8ebd05e4 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:07:17 -0400 Subject: [PATCH 35/64] Lease measured Linux SDK cache construction repair --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 0e3bd44a..f71281a7 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1080,3 +1080,5 @@ IMP-098 measured package-local admission reconciliation: `/root/configuration_co The preceding packaging test lease is clarified for the cross-platform early-refusal tests: pass an explicitly uninspected absent SDK path and assert it remains absent, preserving original image/external-gate errors and absent output. These branches intentionally reject before inspecting the native helper prerequisite. This supersedes the valid-native-fixture suggestion, avoids POSIX/compiler or alphabetical test-order coupling, and does not claim SDK validation. Dedicated missing-SDK and verified-helper controls remain unchanged; no test-helper refactor is leased. IMP-098 measured installed npm mechanical rehearsal repair: /root/conformance_plumbing owns cli/conformance/runner/run.py, cli/conformance/runner/test_runner.py, cli/ci/rehearse_release.py, and cli/ci/test_rehearse_release.py only for the actual final full-admission8 failure. The real installed npm launcher uses authenticated Node; SDK fixture relocation currently blanket-rejects interpreters. Freeze focused tests before plumbing an explicit benchmark-bound npm platform context into SDK fixture preparation. Verify original launcher/Node and native platform binary/manifest byte identities; clone the minimal actual npm package layout, execute the unchanged launcher snapshot with relocated module context, place the provider-free fixture helper beside its actual native child, and check closure before/after execution. Native-only direct product admission remains strict; no arbitrary interpreted exemption, launcher bypass or skipped SDK cases. Retain all 100 cases across three installed candidates and exact 300/200/500 totals, original non-SDK execution routes, safe paths, source/custody/mutation controls, and separate semantic/release nonclaims. Internal fixture custody may be retained without changing the public report shape; stop if any schema or cross-lease change is necessary. No production CLI/runtime edits, Git, builds or provider/key access. Root owns actual focused/rehearsal/full replay and integration; full admission8 has ended. + +IMP-098 measured Linux SDK construction repair: `/root/conformance_plumbing` owns only `cli/ci/build_agents_sdk.py` and `cli/ci/test_build_agents_sdk.py` for the first PR54 Linux build failure. Retained build-sdk.log reports absent owned pyinstaller-cache during disposable cache cleanup after freezing. PyInstaller need not populate that directory on Linux. Establish the isolated owned cache directory explicitly during environment preparation and add a narrow platform-independent control verifying fresh cache ownership and credential-free environment; preserve strict fresh-output, cleanup, runtime/library tests and artifact custody. Diagnose retained logs before edits; do not suppress arbitrary cleanup failures or weaken Linux glibc admission. No Git, heavy builds, providers or key access. Root owns focused/native/full admission and integration; full admission9 completed before this lease. From 6c54009337a7f236c7721e7bcae40eacf171d6cd Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:08:38 -0400 Subject: [PATCH 36/64] Create the owned SDK freeze cache before strict cleanup --- cli/ci/build_agents_sdk.py | 4 +++- cli/ci/test_build_agents_sdk.py | 36 +++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/cli/ci/build_agents_sdk.py b/cli/ci/build_agents_sdk.py index b93b6081..8729be2e 100644 --- a/cli/ci/build_agents_sdk.py +++ b/cli/ci/build_agents_sdk.py @@ -111,9 +111,11 @@ def environment(output, ambient): env = {k: ambient[k] for k in ('PATH', 'DEVELOPER_DIR', 'SDKROOT', 'SYSTEMROOT') if k in ambient} home = output / 'home'; home.mkdir() tmp = output / 'tmp'; tmp.mkdir() + # PyInstaller may leave this cache unused on Linux; cleanup still owns it. + cache = output / 'pyinstaller-cache'; cache.mkdir() env.update(HOME=str(home), TMPDIR=str(tmp), LANG='C.UTF-8', PYTHONHASHSEED='0', PYTHONNOUSERSITE='1', PYTHONDONTWRITEBYTECODE='1', SOURCE_DATE_EPOCH='0', - PYINSTALLER_CONFIG_DIR=str(output / 'pyinstaller-cache')) + PYINSTALLER_CONFIG_DIR=str(cache)) return env diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py index a71fccd1..080000b4 100644 --- a/cli/ci/test_build_agents_sdk.py +++ b/cli/ci/test_build_agents_sdk.py @@ -37,6 +37,42 @@ def test_build_environment_excludes_credentials_configuration_and_runtime_inject self.assertNotIn('LD_LIBRARY_PATH', env) self.assertEqual(env['HOME'], str(Path(temp) / 'home')) + def test_build_environment_owns_fresh_cache_even_when_freezer_never_uses_it(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp) + ambient_cache = root / 'ambient-cache'; ambient_cache.mkdir() + marker = ambient_cache / 'preserve'; marker.write_bytes(b'ambient') + for name in ('first', 'second'): + output = root / name; output.mkdir() + env = sdk.environment(output, {'PATH': '/usr/bin:/bin', + 'PYINSTALLER_CONFIG_DIR': str(ambient_cache), 'OPENAI_API_KEY': 'fixture-secret'}) + cache = output / 'pyinstaller-cache' + self.assertEqual(str(cache), env['PYINSTALLER_CONFIG_DIR']) + self.assertTrue(cache.is_dir()) + self.assertFalse(cache.is_symlink()) + self.assertEqual([], list(cache.iterdir())) + self.assertNotIn('OPENAI_API_KEY', env) + # The production cleanup remains strict even if PyInstaller wrote nothing. + sdk.shutil.rmtree(cache) + self.assertFalse(cache.exists()) + with self.assertRaises(FileNotFoundError): + sdk.shutil.rmtree(cache) + self.assertEqual(b'ambient', marker.read_bytes()) + + def test_build_environment_refuses_preexisting_cache_directory_or_symlink(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp) + foreign = root / 'foreign'; foreign.mkdir() + marker = foreign / 'preserve'; marker.write_bytes(b'foreign') + for kind in ('directory', 'symlink'): + output = root / kind; output.mkdir() + cache = output / 'pyinstaller-cache' + if kind == 'directory': cache.mkdir() + else: cache.symlink_to(foreign, target_is_directory=True) + with self.subTest(kind=kind), self.assertRaises(FileExistsError): + sdk.environment(output, {'PATH': '/usr/bin:/bin'}) + self.assertEqual(b'foreign', marker.read_bytes()) + def test_release_builder_rejects_wrong_python_before_writing(self): with tempfile.TemporaryDirectory() as temp: path = Path(temp) / 'out' From 3dba2ce2dd8f3adcc9ff32ba799e917e30bdab28 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:12:02 -0400 Subject: [PATCH 37/64] Lease native SDK inventory host oracle reconciliation --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index f71281a7..09a1c437 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1082,3 +1082,5 @@ The preceding packaging test lease is clarified for the cross-platform early-ref IMP-098 measured installed npm mechanical rehearsal repair: /root/conformance_plumbing owns cli/conformance/runner/run.py, cli/conformance/runner/test_runner.py, cli/ci/rehearse_release.py, and cli/ci/test_rehearse_release.py only for the actual final full-admission8 failure. The real installed npm launcher uses authenticated Node; SDK fixture relocation currently blanket-rejects interpreters. Freeze focused tests before plumbing an explicit benchmark-bound npm platform context into SDK fixture preparation. Verify original launcher/Node and native platform binary/manifest byte identities; clone the minimal actual npm package layout, execute the unchanged launcher snapshot with relocated module context, place the provider-free fixture helper beside its actual native child, and check closure before/after execution. Native-only direct product admission remains strict; no arbitrary interpreted exemption, launcher bypass or skipped SDK cases. Retain all 100 cases across three installed candidates and exact 300/200/500 totals, original non-SDK execution routes, safe paths, source/custody/mutation controls, and separate semantic/release nonclaims. Internal fixture custody may be retained without changing the public report shape; stop if any schema or cross-lease change is necessary. No production CLI/runtime edits, Git, builds or provider/key access. Root owns actual focused/rehearsal/full replay and integration; full admission8 has ended. IMP-098 measured Linux SDK construction repair: `/root/conformance_plumbing` owns only `cli/ci/build_agents_sdk.py` and `cli/ci/test_build_agents_sdk.py` for the first PR54 Linux build failure. Retained build-sdk.log reports absent owned pyinstaller-cache during disposable cache cleanup after freezing. PyInstaller need not populate that directory on Linux. Establish the isolated owned cache directory explicitly during environment preparation and add a narrow platform-independent control verifying fresh cache ownership and credential-free environment; preserve strict fresh-output, cleanup, runtime/library tests and artifact custody. Diagnose retained logs before edits; do not suppress arbitrary cleanup failures or weaken Linux glibc admission. No Git, heavy builds, providers or key access. Root owns focused/native/full admission and integration; full admission9 completed before this lease. + +IMP-098 measured native inventory oracle reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/fixtures/adapter-host-expectations.json` and `cli/conformance/runner/test_runner.py`. First Linux ARM distribution rehearsal failed the two Codex inventory cases across all three candidates. Source diagnosis proves the host fixture incorrectly rewrites the missing SDK helper to incompatible on newly supported hosts, contradicting the separately frozen four-platform GNU SDK recipe/production oracle. Update only SDK normalized admitted hosts and exact GNU supported platforms; reconcile exact complete inventory tests while preserving every other harness host, blocked Codex, all 100 cases and strict 300/200/500 totals. Add recipe/production-oracle agreement and unsupported-host rejection controls. Raw actual arrays were not retained, so successful real Linux/native rerun must confirm this diagnosis; do not assert an unobserved exact delta. No product edits, Git, heavy builds or providers/key access. Root owns independent review and full/native reruns; no new full admission is active. From d7fa4af8f0266dbecb83b282f4128716a06a63c3 Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:13:27 -0400 Subject: [PATCH 38/64] Align SDK inventory host oracle with frozen four-platform contract --- .../fixtures/adapter-host-expectations.json | 10 ++++- cli/conformance/runner/test_runner.py | 40 +++++++++++++++++-- 2 files changed, 45 insertions(+), 5 deletions(-) diff --git a/cli/conformance/fixtures/adapter-host-expectations.json b/cli/conformance/fixtures/adapter-host-expectations.json index 1ffaa5c9..0883713d 100644 --- a/cli/conformance/fixtures/adapter-host-expectations.json +++ b/cli/conformance/fixtures/adapter-host-expectations.json @@ -18,7 +18,10 @@ "linux-x64" ], "agents-sdk/jsonl": [ - "darwin-arm64" + "darwin-arm64", + "darwin-x64", + "linux-arm64", + "linux-x64" ] }, "cases": { @@ -53,7 +56,10 @@ "linux-x64" ], "agents-sdk/jsonl": [ - "darwin-arm64" + "darwin-arm64", + "darwin-x64", + "linux-arm64-gnu", + "linux-x64-gnu" ] }, "inventoryCases": [ diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index e4e6ac46..6ee0dad1 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -180,9 +180,9 @@ def test_inventory_oracle_freezes_all_harnesses_and_codex_blocked_on_each_host(s case = json.loads((runner.CASES / f"adapters/{name}.json").read_text()) for os_name, arch, incompatible in [ ("darwin", "arm64", set()), - ("darwin", "x86_64", {"prime", "omp", "claude", "agents-sdk"}), - ("linux", "aarch64", {"prime", "omp", "claude", "agents-sdk"}), - ("linux", "x86_64", {"prime", "claude", "agents-sdk"}), + ("darwin", "x86_64", {"prime", "omp", "claude"}), + ("linux", "aarch64", {"prime", "omp", "claude"}), + ("linux", "x86_64", {"prime", "claude"}), ]: wanted = json.loads(json.dumps(case["expected"])) for harness in wanted["resultMatches"]["harnesses"]: @@ -195,6 +195,40 @@ def test_inventory_oracle_freezes_all_harnesses_and_codex_blocked_on_each_host(s self.assertFalse(actual["startedHarness"]) + def test_sdk_inventory_host_oracle_matches_four_platform_frozen_contracts(self): + host=json.loads((runner.CLI/'conformance/fixtures/adapter-host-expectations.json').read_text()) + recipe=json.loads((runner.CLI/'shared/capabilities/adapters/recipes/agents-sdk-jsonl.v1.json').read_text()) + production=json.loads((runner.CLI/'shared/fixtures/adapters/sdk-production.json').read_text()) + supported=['darwin-arm64','darwin-x64','linux-arm64-gnu','linux-x64-gnu'] + normalized=['darwin-arm64','darwin-x64','linux-arm64','linux-x64'] + self.assertEqual(supported,recipe['support']['platforms']) + self.assertEqual(supported,production['discovery']['admissionPlatforms']) + self.assertEqual(supported,host['supportedPlatforms']['agents-sdk/jsonl']) + self.assertEqual(normalized,production['discovery']['platforms']) + self.assertEqual(normalized,host['admittedHosts']['agents-sdk/jsonl']) + for name in ('codex-doctor-probe-failed','codex-list-probe-failed'): + case=json.loads((runner.CASES/f'adapters/{name}.json').read_text()) + original=json.loads(json.dumps(case)) + for os_name,arch in [('darwin','arm64'),('darwin','x86_64'),('linux','aarch64'),('linux','x86_64')]: + with self.subTest(case=name,os=os_name,arch=arch): + expected=runner.expected_for_host(case,os_name,arch) + sdk=next(row for row in expected['resultMatches']['harnesses'] if row['id']=='agents-sdk') + self.assertEqual(next(row for row in case['expected']['resultMatches']['harnesses'] if row['id']=='agents-sdk'),sdk) + self.assertEqual('missing',sdk['availability']) + self.assertEqual(original,case) + self.assertEqual(100,len(list(runner.case_paths(7,set())))) + + def test_inventory_host_oracle_preserves_complete_arrays_on_unsupported_hosts(self): + for name in ('codex-doctor-probe-failed','codex-list-probe-failed'): + case=json.loads((runner.CASES/f'adapters/{name}.json').read_text()) + for os_name,arch in [('freebsd','x86_64'),('win32','AMD64'),('darwin','riscv64'),('linux','riscv64')]: + with self.subTest(case=name,os=os_name,arch=arch): + wanted=json.loads(json.dumps(case['expected'])) + for harness in wanted['resultMatches']['harnesses']: + if harness['runtime']=='installed-process': harness['availability']='incompatible' + self.assertEqual(wanted,runner.expected_for_host(case,os_name,arch)) + self.assertFalse(wanted['startedHarness']) + def test_host_oracle_rejects_unsupported_host_without_skipping_case(self): case = json.loads((runner.CASES / "adapters/claude-functional-alpha.json").read_text()) wanted = runner.expected_for_host(case, "linux", "aarch64") From ca16ffc2382c820767130066266e5a9b98da859b Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:29:32 -0400 Subject: [PATCH 39/64] Record reviewed cold SDK version-probe repair scope --- cli/protocol/OWNERSHIP.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 09a1c437..6047847a 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1084,3 +1084,5 @@ IMP-098 measured installed npm mechanical rehearsal repair: /root/conformance_pl IMP-098 measured Linux SDK construction repair: `/root/conformance_plumbing` owns only `cli/ci/build_agents_sdk.py` and `cli/ci/test_build_agents_sdk.py` for the first PR54 Linux build failure. Retained build-sdk.log reports absent owned pyinstaller-cache during disposable cache cleanup after freezing. PyInstaller need not populate that directory on Linux. Establish the isolated owned cache directory explicitly during environment preparation and add a narrow platform-independent control verifying fresh cache ownership and credential-free environment; preserve strict fresh-output, cleanup, runtime/library tests and artifact custody. Diagnose retained logs before edits; do not suppress arbitrary cleanup failures or weaken Linux glibc admission. No Git, heavy builds, providers or key access. Root owns focused/native/full admission and integration; full admission9 completed before this lease. IMP-098 measured native inventory oracle reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/fixtures/adapter-host-expectations.json` and `cli/conformance/runner/test_runner.py`. First Linux ARM distribution rehearsal failed the two Codex inventory cases across all three candidates. Source diagnosis proves the host fixture incorrectly rewrites the missing SDK helper to incompatible on newly supported hosts, contradicting the separately frozen four-platform GNU SDK recipe/production oracle. Update only SDK normalized admitted hosts and exact GNU supported platforms; reconcile exact complete inventory tests while preserving every other harness host, blocked Codex, all 100 cases and strict 300/200/500 totals. Add recipe/production-oracle agreement and unsupported-host rejection controls. Raw actual arrays were not retained, so successful real Linux/native rerun must confirm this diagnosis; do not assert an unobserved exact delta. No product edits, Git, heavy builds or providers/key access. Root owns independent review and full/native reruns; no new full admission is active. + +IMP-098 measured cold packaged helper version-probe repair: root owns applying the independently reviewed scratch copies to `cli/ci/build_agents_sdk.py` and `cli/ci/test_build_agents_sdk.py`. Native Mac x64 installed cold discovery failed with STARTUP_TIMEOUT during the unchanged five-second --version probe after construction checks passed. The generated entry unnecessarily imported the provider SDK before argparse handled version. Add an exact-argv sys-only banner path; preserve every original remaining self-test/library/runtime dispatch byte and the fixed probe deadline. Reviewed scratch builder SHA256 `ac37b50f67cf8f22bad8a8bf29aee5bd10b43a53dbcfbd5f985a813bf8aefad8`, tests `2a165d5a537d8852cfbccae94285905b68b8a9a075a9ef2cbda1548eb9a473a6`; eleven focused tests pass. Removal of unnecessary imports does not prove their measured contribution versus bootloader/OS loading; rebuilt actual native cold installed evidence must confirm the repair. Full admission10 completed all 53 gates before applying changes; root owns fresh source/native qualification. No timeout increase, retries, case skips, provider/key access or publication. From 578c4184b6f7a84b34be594b41108bac485208ec Mon Sep 17 00:00:00 2001 From: mm Date: Tue, 6 Oct 2026 23:29:41 -0400 Subject: [PATCH 40/64] Keep cold SDK identity probes free of provider imports --- cli/ci/build_agents_sdk.py | 48 ++++++++++++++++----------- cli/ci/test_build_agents_sdk.py | 57 +++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+), 19 deletions(-) diff --git a/cli/ci/build_agents_sdk.py b/cli/ci/build_agents_sdk.py index 8729be2e..116e7d87 100644 --- a/cli/ci/build_agents_sdk.py +++ b/cli/ci/build_agents_sdk.py @@ -21,6 +21,34 @@ RECEIPT = 'agents-sdk-build.json' NOTICES = 'AGENTS-SDK-NOTICES.txt' MAX_BYTES = 256 * 1024 * 1024 +# Identity probes must not initialize the provider SDK before the fixed probe deadline. +ENTRY_SOURCE = ( + 'import sys\n' + "if sys.argv[1:] == ['--version']:\n" + " print('prose-agents-sdk 0.1.0')\n" + "elif '--packaged-self-test' in sys.argv:\n" + ' import json, ssl, pathlib, certifi, agents, openai, pydantic, jiter\n' + ' from importlib.metadata import version\n' + ' ssl.create_default_context(cafile=certifi.where())\n' + " print(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':version('openai-agents'),'openai':version('openai'),'certificates':pathlib.Path(certifi.where()).is_file(),'modelCalls':0},sort_keys=True))\n" + "elif '--packaged-tool-self-test' in sys.argv:\n" + ' import sdk_tool_selftest\n' + ' sdk_tool_selftest.main()\n' + "elif '--packaged-library-test' in sys.argv:\n" + ' import json, pathlib, re\n' + ' versions=[]; count=0\n' + " for path in pathlib.Path(sys._MEIPASS).rglob('*'):\n" + ' if path.is_file() and not path.is_symlink() and path.stat().st_size<=268435456:\n' + " with path.open('rb') as stream:\n" + " if stream.read(4)!=b'\\x7fELF': continue\n" + ' data=stream.read()\n' + ' count+=1\n' + " versions.extend(tuple(int(x) for x in v.split(b'.')) for v in re.findall(rb'GLIBC_([0-9]+\\.[0-9]+(?:\\.[0-9]+)?)',data))\n" + " print(json.dumps({'schema':'openprose.sdk-packaged-libraries/1','elfCount':count,'requiredGlibcMaximum':'.'.join(str(x) for x in max(versions)) if versions else None,'modelCalls':0},sort_keys=True))\n" + 'else:\n' + ' import runpy\n' + " runpy.run_module('prose_sdk_runtime',run_name='__main__')\n" +) TOOL_SELF_TEST_SOURCE = r''' import asyncio import json @@ -165,25 +193,7 @@ def build(output, *, epoch=0, codesign_identity=None): # A frozen module cannot run python -c; this explicit self-test entrypoint is bundled # separately and exercises imported SDK modules/certificate resources without API calls. entry = output / 'sdk-entry.py' - entry.write_text("import sys\nif '--packaged-self-test' in sys.argv:\n" - " import json, ssl, pathlib, certifi, agents, openai, pydantic, jiter\n" - " from importlib.metadata import version\n" - " ssl.create_default_context(cafile=certifi.where())\n" - " print(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':version('openai-agents'),'openai':version('openai'),'certificates':pathlib.Path(certifi.where()).is_file(),'modelCalls':0},sort_keys=True))\n" - "elif '--packaged-tool-self-test' in sys.argv:\n" - " import sdk_tool_selftest\n sdk_tool_selftest.main()\n" - "elif '--packaged-library-test' in sys.argv:\n" - " import json, pathlib, re\n" - " versions=[]; count=0\n" - " for path in pathlib.Path(sys._MEIPASS).rglob('*'):\n" - " if path.is_file() and not path.is_symlink() and path.stat().st_size<=268435456:\n" - " with path.open('rb') as stream:\n" - " if stream.read(4)!=b'\\x7fELF': continue\n" - " data=stream.read()\n" - " count+=1\n" - " versions.extend(tuple(int(x) for x in v.split(b'.')) for v in re.findall(rb'GLIBC_([0-9]+\\.[0-9]+(?:\\.[0-9]+)?)',data))\n" - " print(json.dumps({'schema':'openprose.sdk-packaged-libraries/1','elfCount':count,'requiredGlibcMaximum':'.'.join(str(x) for x in max(versions)) if versions else None,'modelCalls':0},sort_keys=True))\n" - "else:\n import runpy\n runpy.run_module('prose_sdk_runtime',run_name='__main__')\n") + entry.write_text(ENTRY_SOURCE) # Snapshot the concurrently maintained runtime before freezing. Receipt binds exact bytes. source_copy = output / 'prose_sdk_runtime.py'; source_copy.write_bytes(SOURCE.read_bytes()) source_sha = digest(source_copy) diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py index 080000b4..d8a4ba0c 100644 --- a/cli/ci/test_build_agents_sdk.py +++ b/cli/ci/test_build_agents_sdk.py @@ -1,5 +1,8 @@ import importlib.util import os +import json +import subprocess +import sys from pathlib import Path import tempfile import unittest @@ -9,6 +12,60 @@ class BuildSdkTests(unittest.TestCase): + def test_generated_exact_version_probe_avoids_all_sdk_and_runtime_imports(self): + # Execute the bytes actually bundled by the builder, with heavy imports fatal. + wrapper = """import builtins, sys +original_import = builtins.__import__ +def lightweight_import(name, *args, **kwargs): + if name != 'sys': + raise RuntimeError('Unexpected version-probe import: ' + name) + return original_import(name, *args, **kwargs) +builtins.__import__ = lightweight_import +sys.argv = ['prose-agents-sdk', '--version'] +""" + "exec(compile(" + repr(sdk.ENTRY_SOURCE) + ", 'sdk-entry.py', 'exec'))\n" + result = subprocess.run([sys.executable, '-I', '-c', wrapper], + capture_output=True, timeout=5, + env={'PATH': '', 'PYTHONDONTWRITEBYTECODE': '1'}) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout, b'prose-agents-sdk 0.1.0\n') + self.assertEqual(result.stderr, b'') + + def test_generated_nonversion_and_mixed_argv_keep_runtime_dispatch(self): + for arguments in ([], ['--cwd', '/work', '--prompt', 'task'], + ['--version', '--model', 'chosen'], ['--version', '--version']): + with self.subTest(arguments=arguments): + wrapper = """import json, sys, types +runtime_dispatch = types.ModuleType('runpy') +def run_module(name, run_name): + print(json.dumps({'module': name, 'runName': run_name, 'argv': sys.argv[1:]})) +runtime_dispatch.run_module = run_module +sys.modules['runpy'] = runtime_dispatch +""" + 'sys.argv = ' + repr(['prose-agents-sdk', *arguments]) + '\n' + wrapper += "exec(compile(" + repr(sdk.ENTRY_SOURCE) + ", 'sdk-entry.py', 'exec'))\n" + result = subprocess.run([sys.executable, '-I', '-c', wrapper], + capture_output=True, timeout=5, + env={'PATH': '', 'PYTHONDONTWRITEBYTECODE': '1'}) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(json.loads(result.stdout), + {'module': 'prose_sdk_runtime', 'runName': '__main__', 'argv': arguments}) + self.assertEqual(result.stderr, b'') + + def test_generated_packaged_tool_selftest_keeps_precedence_for_mixed_version_argv(self): + for arguments in (['--packaged-tool-self-test'], ['--version', '--packaged-tool-self-test']): + with self.subTest(arguments=arguments): + wrapper = """import sys, types +selftest = types.ModuleType('sdk_tool_selftest') +selftest.main = lambda: print('tool-self-test-dispatched') +sys.modules['sdk_tool_selftest'] = selftest +""" + 'sys.argv = ' + repr(['prose-agents-sdk', *arguments]) + '\n' + wrapper += "exec(compile(" + repr(sdk.ENTRY_SOURCE) + ", 'sdk-entry.py', 'exec'))\n" + result = subprocess.run([sys.executable, '-I', '-c', wrapper], + capture_output=True, timeout=5, + env={'PATH': '', 'PYTHONDONTWRITEBYTECODE': '1'}) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(result.stdout, b'tool-self-test-dispatched\n') + self.assertEqual(result.stderr, b'') + def test_lock_is_closed_and_contains_all_runtime_and_build_requirements(self): packages = sdk.lock_packages() self.assertEqual(len(packages), 46) From acd1822c4a328103f3f1d6b83bb5bbbfd1aa2813 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:15:48 -0400 Subject: [PATCH 41/64] Lease safe native release probe diagnostics --- cli/protocol/OWNERSHIP.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 6047847a..1efbb807 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1086,3 +1086,6 @@ IMP-098 measured Linux SDK construction repair: `/root/conformance_plumbing` own IMP-098 measured native inventory oracle reconciliation: `/root/conformance_plumbing` owns only `cli/conformance/fixtures/adapter-host-expectations.json` and `cli/conformance/runner/test_runner.py`. First Linux ARM distribution rehearsal failed the two Codex inventory cases across all three candidates. Source diagnosis proves the host fixture incorrectly rewrites the missing SDK helper to incompatible on newly supported hosts, contradicting the separately frozen four-platform GNU SDK recipe/production oracle. Update only SDK normalized admitted hosts and exact GNU supported platforms; reconcile exact complete inventory tests while preserving every other harness host, blocked Codex, all 100 cases and strict 300/200/500 totals. Add recipe/production-oracle agreement and unsupported-host rejection controls. Raw actual arrays were not retained, so successful real Linux/native rerun must confirm this diagnosis; do not assert an unobserved exact delta. No product edits, Git, heavy builds or providers/key access. Root owns independent review and full/native reruns; no new full admission is active. IMP-098 measured cold packaged helper version-probe repair: root owns applying the independently reviewed scratch copies to `cli/ci/build_agents_sdk.py` and `cli/ci/test_build_agents_sdk.py`. Native Mac x64 installed cold discovery failed with STARTUP_TIMEOUT during the unchanged five-second --version probe after construction checks passed. The generated entry unnecessarily imported the provider SDK before argparse handled version. Add an exact-argv sys-only banner path; preserve every original remaining self-test/library/runtime dispatch byte and the fixed probe deadline. Reviewed scratch builder SHA256 `ac37b50f67cf8f22bad8a8bf29aee5bd10b43a53dbcfbd5f985a813bf8aefad8`, tests `2a165d5a537d8852cfbccae94285905b68b8a9a075a9ef2cbda1548eb9a473a6`; eleven focused tests pass. Removal of unnecessary imports does not prove their measured contribution versus bootloader/OS loading; rebuilt actual native cold installed evidence must confirm the repair. Full admission10 completed all 53 gates before applying changes; root owns fresh source/native qualification. No timeout increase, retries, case skips, provider/key access or publication. + + +IMP-098 second native Mac Intel probe diagnosis: root owns only `cli/ci/check_published_release.py` and `cli/ci/test_check_published_release.py` to name the closed offline probe and measured exit/stderr-presence when its existing strict contract fails. Source578 native installed npm construction failed with an anonymous exit/stderr mismatch; actual stdout/stderr were not retained, so no startup or provider cause is asserted. Preserve all four probes, exact expected codes, immutable bytes, no credentials, 15-second invocation bounds and five-second helper discovery deadline. Emit no raw stdout/stderr, paths or arbitrary argv. Add independent safe-diagnostic controls before native replay. No timeout increase, retries, gate relaxation or publication. Full admission12 completed all53 gates before this lease. From 4153930ceddeec3190f5bbe303f4dc2ab021eafd Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:17:01 -0400 Subject: [PATCH 42/64] Identify failed offline release probes without disclosing output --- cli/ci/check_published_release.py | 13 +++++++------ cli/ci/test_check_published_release.py | 26 +++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 7 deletions(-) diff --git a/cli/ci/check_published_release.py b/cli/ci/check_published_release.py index d524f646..e0fa7f3d 100644 --- a/cli/ci/check_published_release.py +++ b/cli/ci/check_published_release.py @@ -40,22 +40,23 @@ def check(binary: Path, runner: str, commit: str, version: str, execute=None, *, 'TMPDIR': directory, 'LANG': 'C', 'LC_ALL': 'C', 'HTTP_PROXY': 'http://127.0.0.1:9', 'HTTPS_PROXY': 'http://127.0.0.1:9', 'ALL_PROXY': 'http://127.0.0.1:9', 'NO_PROXY': ''} - def run(args, expected_code): + def run(probe, args, expected_code): call = execute or (lambda argv, cwd, environment: build_local.execute_bounded(argv, cwd, environment, timeout_seconds=15)) result = call(([str(node)] if node else []) + [str(binary), *args], root, env) if result.returncode != expected_code or result.stderr: - raise ValueError('offline release probe failed its exit/stderr contract') + raise ValueError(f'offline release probe {probe} failed its exit/stderr contract ' + f'(exit {result.returncode}; stderr present: {bool(result.stderr)})') return result.stdout.decode('utf-8') - if run(['--version'], 0).strip() != f'prose {version} ({runner})': + if run('version', ['--version'], 0).strip() != f'prose {version} ({runner})': raise ValueError('version banner does not match requested release') - doctor = json.loads(run(['--output=json', 'cli', 'doctor'], 10)) + doctor = json.loads(run('doctor', ['--output=json', 'cli', 'doctor'], 10)) if not subset(doctor, contract['doctor']) or not subset(doctor.get('runner'), {'name': runner, 'commit': commit, 'version': version}): raise ValueError('release doctor does not attest published startup without test seams') - inventory = json.loads(run(['--output=json', 'cli', 'harness', 'list'], 0)) + inventory = json.loads(run('harness-list', ['--output=json', 'cli', 'harness', 'list'], 0)) mocks = [item for item in inventory['harnesses'] if item['id'] == 'mock'] if len(mocks) != 1 or not subset(mocks[0], contract['mock']): raise ValueError('mock execution must be unavailable in release binaries') - result = json.loads(run(['--harness=mock', '--output=json', 'run', 'hello'], contract['mockRunExitCode'])) + result = json.loads(run('mock-run', ['--harness=mock', '--output=json', 'run', 'hello'], contract['mockRunExitCode'])) if result.get('error', result).get('code') != contract['mockRunErrorCode']: raise ValueError('mock request did not fail closed') if hashlib.sha256(binary.read_bytes()).hexdigest() != before: diff --git a/cli/ci/test_check_published_release.py b/cli/ci/test_check_published_release.py index a143f2b0..ad90682b 100644 --- a/cli/ci/test_check_published_release.py +++ b/cli/ci/test_check_published_release.py @@ -7,7 +7,7 @@ class PublishedReleaseTests(unittest.TestCase): - def check_fixture(self, mutation=None, use_node=False): + def check_fixture(self, mutation=None, use_node=False, failed_args=None, failed_code=21, failed_stderr=b''): with tempfile.TemporaryDirectory() as directory: binary = Path(directory) / 'prose' binary.write_bytes(b'fixture executable') @@ -25,6 +25,8 @@ def execute(argv, cwd, env): self.assertEqual(str(node.resolve()), argv[0]) args = argv[2:] if node else argv[1:] calls.append(args) + if args == failed_args: + return subprocess.CompletedProcess(argv, failed_code, b'private output body', failed_stderr) if args == ['--version']: return subprocess.CompletedProcess(argv, 0, b'prose 0.15.0-rc.1 (bun)\n', b'') if args[-1] == 'doctor': @@ -59,3 +61,25 @@ def test_development_or_test_seams_fail_closed(self): def test_source_mismatch_fails_closed(self): with self.assertRaises(ValueError): self.check_fixture(lambda value: value['runner'].update(commit='b' * 40)) + + def test_each_failed_offline_probe_has_safe_measured_identity(self): + probes = [ + ('version', ['--version']), + ('doctor', ['--output=json', 'cli', 'doctor']), + ('harness-list', ['--output=json', 'cli', 'harness', 'list']), + ('mock-run', ['--harness=mock', '--output=json', 'run', 'hello']), + ] + for name, args in probes: + for use_node in (False, True): + with self.subTest(probe=name, node=use_node): + with self.assertRaises(ValueError) as error: + self.check_fixture(use_node=use_node, failed_args=args) + self.assertEqual(f'offline release probe {name} failed its exit/stderr contract ' + '(exit 21; stderr present: False)', str(error.exception)) + + def test_expected_exit_with_stderr_still_fails_without_disclosing_bodies(self): + with self.assertRaises(ValueError) as error: + self.check_fixture(failed_args=['--output=json', 'cli', 'doctor'], failed_code=10, + failed_stderr=b'private stderr credential body') + self.assertEqual('offline release probe doctor failed its exit/stderr contract ' + '(exit 10; stderr present: True)', str(error.exception)) From 5139e2d1423232b65311176ae3f1b9aa887ca8fb Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:26:53 -0400 Subject: [PATCH 43/64] Lease native SDK provenance and compatible library integration --- cli/protocol/OWNERSHIP.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 1efbb807..6ef9156f 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1089,3 +1089,6 @@ IMP-098 measured cold packaged helper version-probe repair: root owns applying t IMP-098 second native Mac Intel probe diagnosis: root owns only `cli/ci/check_published_release.py` and `cli/ci/test_check_published_release.py` to name the closed offline probe and measured exit/stderr-presence when its existing strict contract fails. Source578 native installed npm construction failed with an anonymous exit/stderr mismatch; actual stdout/stderr were not retained, so no startup or provider cause is asserted. Preserve all four probes, exact expected codes, immutable bytes, no credentials, 15-second invocation bounds and five-second helper discovery deadline. Emit no raw stdout/stderr, paths or arbitrary argv. Add independent safe-diagnostic controls before native replay. No timeout increase, retries, gate relaxation or publication. Full admission12 completed all53 gates before this lease. + + +IMP-098 native GNU 2.34 builder wave: `/root/configuration_contract` owns `cli/ci/build_agents_sdk.py`, `cli/ci/test_build_agents_sdk.py`, new `cli/ci/sdk_native_inventory.py` and new `cli/ci/test_sdk_native_inventory.py`. Follow workspace plan `work/plans/IMP-098/linux-sdk-native-build.md`. Implement the explicit same-architecture, digest-bound libgcc supplier/native-input hooks, exact frozen membership/hash and required-symbol closure, ELF/machine/native origin evidence, matching supplier notices and existing receipt extension. Preserve the five-second version probe, conservative GNU 2.34 gate, all default/runtime/special argument dispatch, fresh owned output and ambient credential exclusion. Source metadata, wheel records and exact supplier package/source/license facts establish origins; unknown origins and mismatched bytes fail. Reuse the retained archive-only ELF analysis as reviewed evidence, not a copied oracle. Freeze a strict common native-input/receipt interface with root before dependent packaging work. Agents run lightweight controls only; no Git, Docker/images/heavy builds, providers or key access. Root owns architecture registration, integrated/native/full checks and commits. Full admission12 ended before this wave; prior receipts keep their original source scope. From 5a99d38502d5c30dacd4316eec39eaa24d32b2ec Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:37:36 -0400 Subject: [PATCH 44/64] Route native Linux SDK construction through pinned compatible suppliers --- cli/ci/agents-sdk-linux-build.lock.json | 48 ++++ cli/ci/build_agents_sdk_linux.py | 337 ++++++++++++++++++++++++ cli/ci/build_kernel_rc.py | 31 ++- cli/ci/run_local.py | 2 + cli/ci/test_build_agents_sdk_linux.py | 189 +++++++++++++ cli/ci/test_build_kernel_rc.py | 33 +++ cli/protocol/OWNERSHIP.md | 3 + 7 files changed, 638 insertions(+), 5 deletions(-) create mode 100644 cli/ci/agents-sdk-linux-build.lock.json create mode 100644 cli/ci/build_agents_sdk_linux.py create mode 100644 cli/ci/test_build_agents_sdk_linux.py diff --git a/cli/ci/agents-sdk-linux-build.lock.json b/cli/ci/agents-sdk-linux-build.lock.json new file mode 100644 index 00000000..2f2477cf --- /dev/null +++ b/cli/ci/agents-sdk-linux-build.lock.json @@ -0,0 +1,48 @@ +{ + "platforms": { + "linux-arm64-gnu": { + "dockerPlatform": "linux/arm64", + "freezer": { + "configSha256": "6953b8d045c926e0101ef1e0845a7b7b63c308f7335fab933502dd52d0e221c5", + "image": "quay.io/pypa/manylinux_2_34_aarch64@sha256:59827f7c1914f4c0ba114cfa2d0f83104a6b17a8eafb9c2abd45f929db090107", + "metadataUrl": "https://quay.io/api/v1/repository/pypa/manylinux_2_34_aarch64/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + }, + "machine": "aarch64", + "pythonArchive": { + "byteLength": 65985139, + "metadataUrl": "https://api.github.com/repos/astral-sh/python-build-standalone/releases/tags/20260807", + "sha256": "9201b2d72f8ea0250d594716cfefd8cd85f7b21a8741ba473fafcde58776c619", + "targetTriple": "aarch64-unknown-linux-gnu", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260807/cpython-3.10.20%2B20260807-aarch64-unknown-linux-gnu-pgo%2Blto-full.tar.zst" + }, + "supplier": { + "configSha256": "83284f80eba5a876e1441d216f0e1d0143659b05642a75b34cc213270d83c9b9", + "image": "quay.io/pypa/manylinux_2_28_aarch64@sha256:f5f03dedf61b47d69d0742d36047db78adeee3bdb7a9910ff52304750ecf2514", + "metadataUrl": "https://quay.io/api/v1/repository/pypa/manylinux_2_28_aarch64/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + } + }, + "linux-x64-gnu": { + "dockerPlatform": "linux/amd64", + "freezer": { + "configSha256": "4b9a4dadc04042a4a007afeb3b4eacea081f7fddffe79a2d7d461ce135a628b3", + "image": "quay.io/pypa/manylinux_2_34_x86_64@sha256:e4ca528d41ab66b3654110295d7a2c7bfa599ddbb6efb269ec21f7f73b88f1b6", + "metadataUrl": "https://quay.io/api/v1/repository/pypa/manylinux_2_34_x86_64/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + }, + "machine": "x86_64", + "pythonArchive": { + "byteLength": 66186516, + "metadataUrl": "https://api.github.com/repos/astral-sh/python-build-standalone/releases/tags/20260807", + "sha256": "9e57324fd5e25f485fa5c8c587a2fe02f34a869092abd463e8e816c3cb1d48f2", + "targetTriple": "x86_64-unknown-linux-gnu", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20260807/cpython-3.10.20%2B20260807-x86_64-unknown-linux-gnu-pgo%2Blto-full.tar.zst" + }, + "supplier": { + "configSha256": "a5950a162818fbcab20deb88978805e262ff2ba1442de746e982fc2877c6b382", + "image": "quay.io/pypa/manylinux_2_28_x86_64@sha256:39df0042d5cc900b085aa25a0659368b42a0006c54c474299b785b44c1b4ff82", + "metadataUrl": "https://quay.io/api/v1/repository/pypa/manylinux_2_28_x86_64/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + } + } + }, + "python": "3.10.20", + "schema": "openprose.sdk-linux-build-inputs/1" +} diff --git a/cli/ci/build_agents_sdk_linux.py b/cli/ci/build_agents_sdk_linux.py new file mode 100644 index 00000000..625e0558 --- /dev/null +++ b/cli/ci/build_agents_sdk_linux.py @@ -0,0 +1,337 @@ +"""Native, digest-bound Linux SDK construction driver. + +Uses the explicit supplier hooks and requires native source and symbol custody +in the frozen receipt before accepting construction. +""" +from __future__ import annotations +import argparse +import hashlib +import json +import os +from pathlib import Path +import platform +import re +import subprocess +import selectors +import signal +import time +import uuid +import shutil +import sys + +LOCK = Path(__file__).with_name('agents-sdk-linux-build.lock.json') +SOURCES = ('harnesses/agents-sdk/run.py', 'harnesses/agents-sdk/requirements-build.txt', + 'cli/ci/build_agents_sdk.py', 'harnesses/agents-sdk/test_run.py', 'cli/ci/sdk_native_inventory.py') +NAME = 'prose-agents-sdk' +RECEIPT = 'agents-sdk-build.json' +MIN_FREE_BYTES = 8 * 1024**3 +MAX_RETAINED_BYTES = 4 * 1024**3 +MAX_FILES = 100000 +MAX_LOG_BYTES = 32 * 1024**2 +OWNER_LABEL = 'org.openprose.sdk-builder.owner' + + +def require(ok, message): + if not ok: raise ValueError(message) + +def sha(path): + h=hashlib.sha256() + with Path(path).open('rb') as f: + for chunk in iter(lambda:f.read(1024*1024),b''): h.update(chunk) + return h.hexdigest() + +def load_lock(path=None): + path=LOCK if path is None else path + x=json.loads(path.read_text()) + require(set(x)=={'schema','python','platforms'} and x['schema']=='openprose.sdk-linux-build-inputs/1' + and x['python']=='3.10.20', 'Invalid Linux input lock') + require(set(x['platforms'])=={'linux-x64-gnu','linux-arm64-gnu'}, 'Closed native target set required') + for target, machine, oci in [('linux-x64-gnu','x86_64','amd64'),('linux-arm64-gnu','aarch64','arm64')]: + r=x['platforms'][target] + require(set(r)=={'machine','dockerPlatform','freezer','supplier','pythonArchive'} and + r['machine']==machine and r['dockerPlatform']=='linux/'+oci, 'Native target identity mismatch') + for key,floor in [('freezer','2_34'),('supplier','2_28')]: + v=r[key] + require(set(v)=={'image','configSha256','metadataUrl'} and re.fullmatch( + 'quay.io/pypa/manylinux_'+floor+'_'+machine+'@sha256:[0-9a-f]{64}',v['image']) + and re.fullmatch('[0-9a-f]{64}',v['configSha256']), 'Digest-bound image required') + p=r['pythonArchive'];triple=machine+'-unknown-linux-gnu' + require(set(p)=={'url','sha256','byteLength','targetTriple','metadataUrl'} and p['targetTriple']==triple + and re.fullmatch('https://github.com/astral-sh/python-build-standalone/releases/download/[0-9]{8}/cpython-3[.]10[.]20%2B[0-9]{8}-'+triple+'-pgo%2Blto-full[.]tar[.]zst',p['url']) + and re.fullmatch('[0-9a-f]{64}',p['sha256']) and type(p['byteLength']) is int + and 0=0 and type(epoch) is int,'Nonnegative epoch required') + x=load_lock();require(target in x['platforms'],'Unsupported native target');r=x['platforms'][target] + if native: require(sys.platform=='linux' and platform.machine()==r['machine'],'Native Linux architecture required') + verify_archive(archive,r) + require(shutil.disk_usage(output.parent).free>=MIN_FREE_BYTES,'Linux freeze requires at least8GiB free owned scratch') + snapshot=source_hashes(source) + owner=uuid.uuid4().hex + return r,snapshot,[container_command(r,'supplier',source,output,'supplier.sh',owner=owner), + container_command(r,'freezer',source,output,'prepare.sh',network='bridge',owner=owner), + container_command(r,'freezer',source,output,'freeze.sh',owner=owner)] + +def stop_process_tree(process): + try: os.killpg(process.pid,signal.SIGTERM) + except ProcessLookupError: pass + try: process.wait(timeout=2) + except subprocess.TimeoutExpired: + try: os.killpg(process.pid,signal.SIGKILL) + except ProcessLookupError: pass + process.wait(timeout=2) + # The direct child may exit while its descendants keep a pipe open. + try: os.killpg(process.pid,signal.SIGKILL) + except ProcessLookupError: pass + +def process(command, output, label, *, timeout=1200, max_bytes=MAX_LOG_BYTES): + env={'PATH':os.environ['PATH'],'HOME':str(output/'home'),'LANG':'C.UTF-8'} + p=subprocess.Popen(command,env=env,stdin=subprocess.DEVNULL,stdout=subprocess.PIPE, + stderr=subprocess.STDOUT,start_new_session=True) + deadline=time.monotonic()+timeout; total=0; next_audit=0 + try: + with (output/(label+'.log')).open('wb') as log, selectors.DefaultSelector() as events: + events.register(p.stdout,selectors.EVENT_READ) + while events.get_map(): + require(time.monotonic()=next_audit: + check_output(output) + require(shutil.disk_usage(output).free>=512*1024**2,'Owned scratch reserve exhausted') + next_audit=time.monotonic()+1 + for key,_ in events.select(min(.1,max(0,deadline-time.monotonic()))): + data=os.read(key.fileobj.fileno(),65536) + if not data: events.unregister(key.fileobj); continue + log.write(data[:max(0,max_bytes-total)]) + total+=len(data) + require(total<=max_bytes,'Process output limit exceeded: '+label) + p.wait(timeout=max(.01,deadline-time.monotonic())) + require(p.returncode==0,'Inspect retained '+label+'.log') + except BaseException: + stop_process_tree(p); raise + finally: + if p.stdout: p.stdout.close() + return (output/(label+'.log')).read_bytes() + +def cleanup_owned(command, output, label): + if '--name' not in command: return + name=command[command.index('--name')+1] + owner=command[command.index('--label')+1].split('=',1)[1] + require(re.fullmatch('prose-sdk-'+owner+'-[a-z]+',name), 'Unowned container cleanup forbidden') + prefix=['docker','--config',str(output/'docker-config')] + try: + actual=process(prefix+['container','inspect','--format','{{index .Config.Labels "'+OWNER_LABEL+'"}}',name], + output,label+'-cleanup-inspect',timeout=10,max_bytes=65536).decode().strip() + require(actual==owner,'Container ownership label differs; cleanup refused') + process(prefix+['container','rm','--force',name],output,label+'-cleanup-remove',timeout=10,max_bytes=65536) + except Exception as e: + (output/(label+'-cleanup-error.txt')).write_text(type(e).__name__+': '+str(e)+'\n') + # Preserve the original construction failure; never claim cleanup passed. + +def run(command, output, label, *, timeout=1200, max_bytes=MAX_LOG_BYTES): + try: return process(command,output,label,timeout=timeout,max_bytes=max_bytes) + except BaseException: + cleanup_owned(command,output,label); raise + +def build(source,output,target,archive,*,epoch=0): + source=checked_path(source);output=checked_path(output) + r,snapshot,commands=plan(source,output,target,archive,epoch=epoch) + output.mkdir();(output/'home').mkdir();(output/'docker-config').mkdir() + def guarded(command,label,**kwargs): + assert_stable(source,snapshot);check_output(output) + try: return run(command,output,label,**kwargs) + finally: + assert_stable(source,snapshot);check_output(output) + (output/'target.json').write_text(json.dumps(r));(output/'epoch.txt').write_text(str(epoch)) + # Caller downloads exact archive beforehand; never copy an unverified input. + assert_stable(source,snapshot) + shutil.copyfile(archive,output/'python-full.tar.zst') + verify_archive(output/'python-full.tar.zst',r) + assert_stable(source,snapshot) + for name,content in [('supplier.sh',SUPPLIER),('supplier.py',SUPPLIER_PY),('prepare.sh',PREPARE),('freeze.sh',FREEZE)]: + (output/name).write_text(content) + for key in ('supplier','freezer'): + # Pull by digest during preparation only; verify local config identity. + guarded(['docker','--config',str(output/'docker-config'),'pull','--platform',r['dockerPlatform'],r[key]['image']],'pull-'+key) + raw=guarded(['docker','--config',str(output/'docker-config'),'image','inspect','--format','{{json .}}',r[key]['image']], + 'inspect-'+key,timeout=15,max_bytes=65536) + validate_image(json.loads(raw),r,key) + guarded(commands[0],'supplier') + origin=json.loads((output/'supplier/origin.json').read_text()) + validate_origin(origin,output/'supplier',r) + inputs={'schema':'openprose.sdk-linux-native-input/1','target':target,'images':{k:r[k] for k in ('supplier','freezer')}, + 'pythonArchive':r['pythonArchive'],'sourceSnapshot':snapshot,'libgcc':origin, + 'driverSha256':snapshot['driverSha256'],'lockSha256':snapshot['lockSha256']} + (output/'native-input.json').write_text(json.dumps(inputs,indent=2,sort_keys=True)+'\n') + verify_archive(output/'python-full.tar.zst',r) + guarded(commands[1],'prepare') + metadata_path=output/'python-full/python/PYTHON.json' + require(metadata_path.is_file() and metadata_path.stat().st_size<=8*1024*1024,'Retained full Python metadata required') + provider=json.loads(metadata_path.read_text()) + require(provider['python_version']=='3.10.20' and provider['target_triple']==r['pythonArchive']['targetTriple'] + and provider['libpython_link_mode']=='shared','Retained Python metadata identity mismatch') + inputs['pythonDistribution']={'metadataPath':'/job/python-full/python/PYTHON.json', + 'root':'/job/python-full/python','metadataSha256':sha(metadata_path)} + (output/'native-input.json').write_text(json.dumps(inputs,indent=2,sort_keys=True)+'\n') + verify_archive(output/'python-full.tar.zst',r) + guarded(commands[2],'freeze') + receipt=json.loads((output/'frozen'/RECEIPT).read_text()) + native=receipt.get('nativeDependencies',{}) + require(native.get('symbolClosureVerified') is True,'Actual native symbol closure required') + members=native.get('libraries',[]) + require(sum(m.get('path')=='libgcc_s.so.1' and m.get('sha256')==origin['library']['sha256'] for m in members)==1, + 'Embedded libgcc must equal supplier; no host fallback') + require(receipt.get('linuxBuildInputSha256')==sha(output/'native-input.json'),'Frozen receipt must bind input custody') + require(receipt.get('linuxBuildSourceSnapshot')==snapshot,'Frozen receipt must bind exact source snapshot') + require(receipt['helper']['sha256']==sha(output/'frozen'/NAME),'Helper bytes changed') + assert_stable(source,snapshot);check_output(output) + return inputs + +if __name__=='__main__': + p=argparse.ArgumentParser(description=__doc__);p.add_argument('--source',type=Path,required=True) + p.add_argument('--out',type=Path,required=True);p.add_argument('--platform',required=True) + p.add_argument('--python-archive',type=Path,required=True);p.add_argument('--source-date-epoch',type=int,default=0) + a=p.parse_args() + print(json.dumps(build(a.source,a.out,a.platform,a.python_archive,epoch=a.source_date_epoch),sort_keys=True)) diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index 439283ba..04729a45 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -4,6 +4,7 @@ import hashlib import json import os +import platform from pathlib import Path, PurePosixPath import re import shutil @@ -127,7 +128,26 @@ def verified_artifacts(package): return manifest -def build(version, output, *, agents_sdk_python=None): +def build_sdk(output, *, epoch, env, logs, agents_sdk_python=None, linux_python_archive=None): + sdk_output = output / 'agents-sdk' + if sys.platform.startswith('linux'): + require(linux_python_archive is not None, 'Linux SDK construction requires the pinned full Python archive') + targets = {'x86_64': 'linux-x64-gnu', 'aarch64': 'linux-arm64-gnu'} + require(platform.machine() in targets, 'Unsupported native Linux SDK architecture') + from build_agents_sdk_linux import build as build_linux + stage = output / 'agents-sdk-native' + build_linux(ROOT, stage, targets[platform.machine()], linux_python_archive, epoch=int(epoch)) + sdk_output.mkdir() + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + shutil.copy2(stage / 'frozen' / name, sdk_output / name) + else: + require(sys.platform == 'darwin', 'SDK construction requires a supported native release host') + command([agents_sdk_python or sys.executable, ROOT / 'cli/ci/build_agents_sdk.py', '--out', sdk_output, + '--source-date-epoch', epoch], env=env, cwd=ROOT, log=logs / 'build-sdk.log') + return sdk_output + + +def build(version, output, *, agents_sdk_python=None, linux_python_archive=None): require(RC.fullmatch(version) is not None, 'Use an exact X.Y.Z-rc.N version') require(not output.exists() and not output.is_symlink(), 'Output must be fresh') revision = subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip() @@ -140,9 +160,8 @@ def build(version, output, *, agents_sdk_python=None): env.update(OPENPROSE_BUILD_COMMIT=revision, OPENPROSE_BUILD_VERSION=version, OPENPROSE_REQUIRE_RELEASE_IMAGE='1', CARGO_TARGET_DIR=str(output / 'cargo-target')) binaries = output / 'binaries'; binaries.mkdir() - sdk_output = output / 'agents-sdk' - command([agents_sdk_python or sys.executable, ROOT / 'cli/ci/build_agents_sdk.py', '--out', sdk_output, - '--source-date-epoch', epoch], env=env, cwd=ROOT, log=logs / 'build-sdk.log') + sdk_output = build_sdk(output, epoch=epoch, env=env, logs=logs, + agents_sdk_python=agents_sdk_python, linux_python_archive=linux_python_archive) command(['bun', '--no-env-file', 'scripts/image-bundle.ts', 'build', '--require-release-eligible', '--outfile', binaries / 'prose-bun'], env=env, cwd=ROOT / 'cli/bun', log=logs / 'build-bun.log') command(['cargo', 'build', '--manifest-path', ROOT / 'cli/rust/Cargo.toml', '--release', '--locked', @@ -209,8 +228,10 @@ def check(binary, runner, label, node=None): parser.add_argument('--version', required=True) parser.add_argument('--out', type=Path, required=True) parser.add_argument('--agents-sdk-python', type=Path, help='Separate hash-locked SDK build interpreter (defaults to current interpreter)') + parser.add_argument('--agents-sdk-linux-python-archive', type=Path, help='Exact pinned full shared Python archive for native Linux construction') args = parser.parse_args() try: - print(json.dumps(build(args.version, args.out.absolute(), agents_sdk_python=args.agents_sdk_python), sort_keys=True)) + print(json.dumps(build(args.version, args.out.absolute(), agents_sdk_python=args.agents_sdk_python, + linux_python_archive=args.agents_sdk_linux_python_archive), sort_keys=True)) except (ValueError, OSError, subprocess.SubprocessError) as error: parser.exit(2, str(error) + '\n') diff --git a/cli/ci/run_local.py b/cli/ci/run_local.py index 10c4c1c2..f019a379 100644 --- a/cli/ci/run_local.py +++ b/cli/ci/run_local.py @@ -136,6 +136,8 @@ def gates() -> tuple[Gate, ...]: "test_npm_identity", "test_npm_same_name", "test_build_kernel_rc", + "test_build_agents_sdk_linux", + "test_sdk_native_inventory", "test_assemble_kernel_rc", "test_kernel_rc_evidence", "test_check_published_release", diff --git a/cli/ci/test_build_agents_sdk_linux.py b/cli/ci/test_build_agents_sdk_linux.py new file mode 100644 index 00000000..e32cf895 --- /dev/null +++ b/cli/ci/test_build_agents_sdk_linux.py @@ -0,0 +1,189 @@ +import copy,json,tempfile,unittest +from pathlib import Path +from unittest.mock import patch +import build_agents_sdk_linux as d + +class DriverControls(unittest.TestCase): + def test_real_closed_pins(self): + x=d.load_lock();self.assertEqual(len(x['platforms']),2) + def test_floating_or_wrong_arch_image_rejected(self): + for key in ('freezer','supplier'): + for value in ('quay.io/pypa/manylinux_2_34_x86_64:latest','quay.io/pypa/manylinux_2_34_aarch64@sha256:'+'a'*64): + x=copy.deepcopy(d.load_lock());x['platforms']['linux-x64-gnu'][key]['image']=value + with tempfile.TemporaryDirectory() as t: + p=Path(t).resolve()/'lock';p.write_text(json.dumps(x)) + with self.assertRaises(ValueError):d.load_lock(p) + def test_variant_or_install_only_python_rejected(self): + for field,value in [('targetTriple','x86_64_v2-unknown-linux-gnu'),('url','https://example.com/untrusted.tar.zst'),('byteLength',True)]: + x=copy.deepcopy(d.load_lock());x['platforms']['linux-x64-gnu']['pythonArchive'][field]=value + with tempfile.TemporaryDirectory() as t: + p=Path(t).resolve()/'lock';p.write_text(json.dumps(x)) + with self.assertRaises(ValueError):d.load_lock(p) + def test_mounts_are_closed_and_offline_freeze(self): + r=d.load_lock()['platforms']['linux-x64-gnu'];c=d.container_command(r,'freezer',Path('/source-owned'),Path('/job-owned'),'freeze.sh') + self.assertIn('--network=none',c);self.assertIn('--pull=never',c);self.assertIn('--read-only',c) + self.assertIn('type=bind,src=/source-owned,dst=/source,readonly',c) + self.assertNotIn('--privileged',c);self.assertNotIn('--env-file',c) + self.assertEqual(c.count('--mount'),2);self.assertIn('--cap-drop=ALL',c) + def test_no_ambient_credentials_passed(self): + import sys + with tempfile.TemporaryDirectory() as t, patch.dict(d.os.environ,{'OPENAI_API_KEY':'sentinel','GH_TOKEN':'sentinel'}): + out=Path(t).resolve();(out/'home').mkdir() + result=d.process([sys.executable,'-c','import os,json;print(json.dumps(dict(os.environ)))'],out,'env') + env=json.loads(result);self.assertNotIn('OPENAI_API_KEY',env);self.assertNotIn('GH_TOKEN',env) + self.assertEqual(env['HOME'],str(out/'home')) + def test_host_or_emulated_target_cannot_be_native(self): + with tempfile.TemporaryDirectory() as t: + source=Path(t).resolve()/'source';source.mkdir() + with patch.object(d.sys,'platform','darwin'),self.assertRaisesRegex(ValueError,'Native Linux'): + d.plan(source,Path(t).resolve()/'out','linux-x64-gnu',Path(t).resolve()/'absent') + def test_unknown_target_and_unsafe_mount_rejected(self): + with tempfile.TemporaryDirectory() as t: + source=Path(t).resolve()/'source';source.mkdir() + with self.assertRaises(ValueError):d.plan(source,Path(t).resolve()/'out','windows-x64',Path(t).resolve()/'absent',native=False) + with self.assertRaises(ValueError):d.checked_path(Path(t).resolve()/'bad,mount') + (Path(t).resolve()/'link').symlink_to(source) + with self.assertRaises(ValueError):d.checked_path(Path(t).resolve()/'link') + def test_wrong_python_bytes_rejected_before_any_command(self): + with tempfile.TemporaryDirectory() as t,patch.object(d.subprocess,'run') as run: + source=Path(t).resolve()/'source';source.mkdir();archive=Path(t).resolve()/'python';archive.write_bytes(b'wrong') + with self.assertRaisesRegex(ValueError,'archive differs'):d.plan(source,Path(t).resolve()/'out','linux-x64-gnu',archive,native=False) + run.assert_not_called();self.assertFalse((Path(t).resolve()/'out').exists()) + def test_required_selection_and_custody_hooks(self): + self.assertIn('--linux-libgcc /job/supplier/libgcc_s.so.1',d.FREEZE) + self.assertIn('--linux-native-origin /job/native-input.json',d.FREEZE) + self.assertNotIn('exclude',d.FREEZE);self.assertNotIn('LD_LIBRARY_PATH',d.FREEZE) + self.assertIn("digests.get(str(p))==digest",d.SUPPLIER_PY) + self.assertIn('GCC RUNTIME LIBRARY EXCEPTION',d.SUPPLIER_PY) + self.assertIn("x['libpython_link_mode']=='shared'",d.PREPARE) + def test_preparation_network_is_explicit_and_freeze_is_not(self): + r=d.load_lock()['platforms']['linux-arm64-gnu'] + self.assertIn('--network=bridge',d.container_command(r,'freezer',Path('/source'),Path('/job'),'prepare.sh',network='bridge')) + with self.assertRaises(ValueError):d.container_command(r,'freezer',Path('/source'),Path('/job'),'freeze.sh',network='host') + + + +class RealOrchestratorControls(unittest.TestCase): + def setup_inputs(self,root): + source=root/'source';source.mkdir() + for name in d.SOURCES: + p=source/name;p.parent.mkdir(parents=True,exist_ok=True);p.write_text('fixture source') + driver=root/'driver.py';driver.write_text('fixture driver') + archive=root/'python';archive.write_bytes(b'not an executed archive') + lock=d.load_lock();r=lock['platforms']['linux-x64-gnu'];r['pythonArchive']['byteLength']=archive.stat().st_size;r['pythonArchive']['sha256']=d.sha(archive) + path=root/'lock.json';path.write_text(json.dumps(lock)) + return source,driver,archive,path,lock + def exercise(self,mutation): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();source,driver,archive,lockpath,lock=self.setup_inputs(root);output=root/'out';calls=[] + original=d.shutil.copyfile + def copy(src,dest): + original(src,dest) + if mutation=='copy':Path(dest).write_bytes(b'mutated after plan') + def fake(command,out,label,**kw): + calls.append(label) + if label=='pull-supplier': + if mutation=='source':(source/d.SOURCES[0]).write_text('changed source') + if mutation=='tests':(source/'harnesses/agents-sdk/test_run.py').write_text('changed tests') + if mutation=='native-module':(source/'cli/ci/sdk_native_inventory.py').write_text('changed native inventory module') + if mutation=='driver':driver.write_text('changed driver') + if mutation=='lock':lockpath.write_text(lockpath.read_text()+' ') + if mutation=='wrong-image':return b'' + if label.startswith('inspect-'): + key=label.removeprefix('inspect-');r=lock['platforms']['linux-x64-gnu'] + return json.dumps({'Id':'sha256:'+r[key]['configSha256'],'Os':'linux','Architecture':'arm64', + 'RepoDigests':[r[key]['image']]}).encode() + return b'' + with patch.object(d,'__file__',str(driver)),patch.object(d,'LOCK',lockpath),patch.object(d.sys,'platform','linux'),patch.object(d.platform,'machine',return_value='x86_64'),patch.object(d.shutil,'disk_usage') as disk,patch.object(d.shutil,'copyfile',side_effect=copy),patch.object(d,'run',side_effect=fake): + disk.return_value.free=16*1024**3 + with self.assertRaises(ValueError):d.build(source,output,'linux-x64-gnu',archive) + return calls + def test_copied_archive_mutation_refused_before_container(self):self.assertEqual(self.exercise('copy'),[]) + def test_sources_tests_driver_and_lock_are_rechecked(self): + for kind in ('source','tests','native-module','driver','lock'): + with self.subTest(kind=kind):self.assertEqual(self.exercise(kind),['pull-supplier']) + def test_native_inventory_module_mutation_refused_before_inspect(self):self.assertEqual(self.exercise('native-module'),['pull-supplier']) + def test_actual_inspect_architecture_is_checked(self):self.assertEqual(self.exercise('wrong-image'),['pull-supplier','inspect-supplier']) + def test_actual_process_timeout_kills_delayed_descendant_effect(self): + import sys,time + with tempfile.TemporaryDirectory() as t: + out=Path(t).resolve();(out/'home').mkdir();effect=out/'late-effect' + child='import time,pathlib;time.sleep(.6);pathlib.Path('+repr(str(effect))+').write_text("escaped")' + code='import subprocess,sys,time;subprocess.Popen([sys.executable,"-c",'+repr(child)+']);time.sleep(20)' + with self.assertRaisesRegex(ValueError,'deadline'):d.process([sys.executable,'-c',code],out,'timeout',timeout=.1) + time.sleep(.7);self.assertFalse(effect.exists()) + def test_timeout_only_removes_label_verified_owned_container(self): + with tempfile.TemporaryDirectory() as t: + out=Path(t).resolve();row=d.load_lock()['platforms']['linux-x64-gnu'];command=d.container_command(row,'freezer',Path('/source'),out,'freeze.sh',owner='a'*32);calls=[] + def fake(c,o,label,**kw): + calls.append((c,label,kw)) + if label=='freeze':raise subprocess.TimeoutExpired(c,.1) + if label.endswith('inspect'):return ('a'*32+'\n').encode() + return b'' + import subprocess + with patch.object(d,'process',side_effect=fake),self.assertRaises(subprocess.TimeoutExpired):d.run(command,out,'freeze',timeout=.1) + self.assertEqual([x[1] for x in calls],['freeze','freeze-cleanup-inspect','freeze-cleanup-remove']) + self.assertEqual(calls[-1][0][-1],'prose-sdk-'+('a'*32)+'-freeze') + self.assertEqual(calls[-1][2]['timeout'],10) + def test_wrong_ownership_label_never_removes_container(self): + with tempfile.TemporaryDirectory() as t: + out=Path(t).resolve();row=d.load_lock()['platforms']['linux-x64-gnu'];command=d.container_command(row,'freezer',Path('/source'),out,'freeze.sh',owner='a'*32);calls=[] + def fake(c,o,label,**kw): + calls.append(label) + if label=='freeze':raise ValueError('original') + return b'other-owner\n' + with patch.object(d,'process',side_effect=fake),self.assertRaisesRegex(ValueError,'original'):d.run(command,out,'freeze') + self.assertEqual(calls,['freeze','freeze-cleanup-inspect']);self.assertTrue((out/'freeze-cleanup-error.txt').is_file()) + def test_actual_process_output_is_bounded(self): + import sys + with tempfile.TemporaryDirectory() as t: + out=Path(t).resolve();(out/'home').mkdir() + with self.assertRaisesRegex(ValueError,'output limit'):d.process([sys.executable,'-c','print("x"*10000)'],out,'large',max_bytes=64) + self.assertLessEqual((out/'large.log').stat().st_size,64) + def test_low_space_refuses_before_any_command(self): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();source,driver,archive,lockpath,lock=self.setup_inputs(root) + with patch.object(d,'__file__',str(driver)),patch.object(d,'LOCK',lockpath),patch.object(d.sys,'platform','linux'),patch.object(d.platform,'machine',return_value='x86_64'),patch.object(d.shutil,'disk_usage') as disk,patch.object(d,'run') as run: + disk.return_value.free=1024 + with self.assertRaisesRegex(ValueError,'8GiB'):d.build(source,root/'out','linux-x64-gnu',archive) + run.assert_not_called();self.assertFalse((root/'out').exists()) + + +class NativeOriginControls(unittest.TestCase): + def fixture(self,root): + (root/'libgcc_s.so.1').write_bytes(b'fixture native bytes') + text='GNU GENERAL PUBLIC LICENSE\nGCC RUNTIME LIBRARY EXCEPTION' + (root/'license-0.txt').write_text(text) + lib={'path':'libgcc_s.so.1','sha256':d.sha(root/'libgcc_s.so.1'),'byteLength':len(b'fixture native bytes'), + 'supplierPath':'/usr/lib64/libgcc_s-actual.so.1','rpmFileDigestVerified':True} + package={'name':'libgcc','epoch':'0','version':'8.fixture','release':'test','architecture':'x86_64', + 'sourceRpm':'gcc-fixture.src.rpm','license':'fixture-not-production'} + return {'schema':'openprose.sdk-native-origin/1','library':lib,'package':package,'licenses':[ + {'packagePath':'/usr/share/licenses/libgcc/COPYING.RUNTIME','path':'license-0.txt', + 'sha256':d.sha(root/'license-0.txt'),'byteLength':(root/'license-0.txt').stat().st_size}]} + def test_missing_source_rpm_and_actual_license_fact_refused(self): + row=d.load_lock()['platforms']['linux-x64-gnu'] + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();origin=self.fixture(root);d.validate_origin(origin,root,row) + for mutation in ('sourceRpm','license','architecture'): + x=copy.deepcopy(origin);x['package'][mutation]='' + with self.assertRaises(ValueError):d.validate_origin(x,root,row) + x=copy.deepcopy(origin);x['licenses']=[] + with self.assertRaises(ValueError):d.validate_origin(x,root,row) + def test_mutated_library_or_license_refused(self): + row=d.load_lock()['platforms']['linux-x64-gnu'] + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();origin=self.fixture(root) + (root/'libgcc_s.so.1').write_bytes(b'changed') + with self.assertRaises(ValueError):d.validate_origin(origin,root,row) + origin=self.fixture(root);(root/'license-0.txt').write_text('changed') + with self.assertRaises(ValueError):d.validate_origin(origin,root,row) + def test_output_byte_and_file_caps_enforced(self): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();(root/'big').write_bytes(b'12345') + with patch.object(d,'MAX_RETAINED_BYTES',4),self.assertRaises(ValueError):d.check_output(root) + with patch.object(d,'MAX_FILES',0),self.assertRaises(ValueError):d.check_output(root) + + +if __name__ == '__main__': + unittest.main() diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index 6647e932..ca3e2fc4 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -131,5 +131,38 @@ def test_production_extracts_closed_sdk_payload(self): self.assertTrue(os.access(executable.parent / 'prose-agents-sdk', os.X_OK)) +class NativeSdkRoutingTests(unittest.TestCase): + def test_linux_requires_pin_and_never_uses_host_python(self): + with tempfile.TemporaryDirectory() as d, patch.object(rc.sys, 'platform', 'linux'), patch.object(rc, 'command') as command: + root = Path(d) + with self.assertRaisesRegex(ValueError, 'pinned full Python'): + rc.build_sdk(root, epoch='0', env={}, logs=root, agents_sdk_python=Path('/ambient/python')) + command.assert_not_called() + self.assertFalse((root / 'agents-sdk').exists()) + + def test_native_linux_uses_driver_and_exact_frozen_trio(self): + import build_agents_sdk_linux as driver + with tempfile.TemporaryDirectory() as d: + root = Path(d) + def frozen(source, stage, target, archive, *, epoch): + self.assertEqual((source, target, archive, epoch), (rc.ROOT, 'linux-arm64-gnu', root / 'pinned.tar.zst', 123)) + (stage / 'frozen').mkdir(parents=True) + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + (stage / 'frozen' / name).write_bytes(name.encode()) + with patch.object(rc.sys, 'platform', 'linux'), patch.object(rc.platform, 'machine', return_value='aarch64'), patch.object(driver, 'build', side_effect=frozen), patch.object(rc, 'command') as command: + actual = rc.build_sdk(root, epoch='123', env={}, logs=root, linux_python_archive=root / 'pinned.tar.zst') + command.assert_not_called() + self.assertEqual(len(list(actual.iterdir())), 3) + for p in actual.iterdir(): + self.assertEqual(p.read_bytes(), p.name.encode()) + + def test_macos_keeps_separate_pinned_interpreter(self): + with tempfile.TemporaryDirectory() as d, patch.object(rc.sys, 'platform', 'darwin'), patch.object(rc, 'command') as command: + root = Path(d) + rc.build_sdk(root, epoch='123', env={'PATH':'/bin'}, logs=root, agents_sdk_python=Path('/selected/python')) + self.assertEqual(command.call_args.args[0][0], Path('/selected/python')) + self.assertEqual(command.call_args.args[0][-1], '123') + + if __name__ == '__main__': unittest.main() diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 6ef9156f..06029d80 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1092,3 +1092,6 @@ IMP-098 second native Mac Intel probe diagnosis: root owns only `cli/ci/check_pu IMP-098 native GNU 2.34 builder wave: `/root/configuration_contract` owns `cli/ci/build_agents_sdk.py`, `cli/ci/test_build_agents_sdk.py`, new `cli/ci/sdk_native_inventory.py` and new `cli/ci/test_sdk_native_inventory.py`. Follow workspace plan `work/plans/IMP-098/linux-sdk-native-build.md`. Implement the explicit same-architecture, digest-bound libgcc supplier/native-input hooks, exact frozen membership/hash and required-symbol closure, ELF/machine/native origin evidence, matching supplier notices and existing receipt extension. Preserve the five-second version probe, conservative GNU 2.34 gate, all default/runtime/special argument dispatch, fresh owned output and ambient credential exclusion. Source metadata, wheel records and exact supplier package/source/license facts establish origins; unknown origins and mismatched bytes fail. Reuse the retained archive-only ELF analysis as reviewed evidence, not a copied oracle. Freeze a strict common native-input/receipt interface with root before dependent packaging work. Agents run lightweight controls only; no Git, Docker/images/heavy builds, providers or key access. Root owns architecture registration, integrated/native/full checks and commits. Full admission12 ended before this wave; prior receipts keep their original source scope. + + +IMP-098 native Linux driver integration: root owns new `cli/ci/build_agents_sdk_linux.py`, `cli/ci/test_build_agents_sdk_linux.py`, and `cli/ci/agents-sdk-linux-build.lock.json`, plus `cli/ci/build_kernel_rc.py`, `cli/ci/test_build_kernel_rc.py`, both native build/distribution workflows and their exact workflow validators/tests. Apply independently reviewed v3 driver with nested inventory custody; preserve bounded owned containers, fixed native architectures, exact pins and zero provider calls. Root also owns registering the new CI inputs/tests in the existing architecture/admission contracts. Native and full admission remain pending; no publication authority. From 0621b25e5a72d54786b4b093b611d9ebbe28b73c Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:41:33 -0400 Subject: [PATCH 45/64] Lease native SDK package custody and artifact SBOM integration --- cli/protocol/OWNERSHIP.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 06029d80..692aacd0 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1095,3 +1095,6 @@ IMP-098 native GNU 2.34 builder wave: `/root/configuration_contract` owns `cli/c IMP-098 native Linux driver integration: root owns new `cli/ci/build_agents_sdk_linux.py`, `cli/ci/test_build_agents_sdk_linux.py`, and `cli/ci/agents-sdk-linux-build.lock.json`, plus `cli/ci/build_kernel_rc.py`, `cli/ci/test_build_kernel_rc.py`, both native build/distribution workflows and their exact workflow validators/tests. Apply independently reviewed v3 driver with nested inventory custody; preserve bounded owned containers, fixed native architectures, exact pins and zero provider calls. Root also owns registering the new CI inputs/tests in the existing architecture/admission contracts. Native and full admission remain pending; no publication authority. + + +IMP-098 frozen native receipt consumer wave: `/root/sdk_assessment` owns only `cli/ci/package_local.py`, `cli/ci/test_package_local.py`, `cli/ci/kernel_rc_evidence.py`, `cli/ci/test_assemble_kernel_rc.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py`. Consume the configuration lane's pure `validate_linux_receipt` and concrete synthetic interface example. Keep the existing twelve manifest identity fields; require complete native additions on Linux and reject them on macOS. Verify actual builder CArchive/runtime rows and exact source snapshot before packaging, central receipt custody for every downstream consumer, and separate measured shipped native SBOM components from declared provider extension licenses and locked wheel candidates. No dependency lock schema changes or new selected-wheel claims. Exact component omission/substitution/duplicate/source/license poison tests; preserve existing Mac acceptance. No shared inspection-module edits without owner/root coordination, Git, heavy builds, Docker, providers or credential access. Root retains independent review, integration and full/native admission. From c53312e4c7cc9269f271e32e1dd90ea3d35cc08b Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:49:57 -0400 Subject: [PATCH 46/64] Verify frozen GNU SDK supplier symbols origins and license evidence --- cli/ci/build_agents_sdk.py | 85 ++++- cli/ci/sdk_native_inventory.py | 562 ++++++++++++++++++++++++++++ cli/ci/test_build_agents_sdk.py | 70 ++++ cli/ci/test_sdk_native_inventory.py | 270 +++++++++++++ 4 files changed, 975 insertions(+), 12 deletions(-) create mode 100644 cli/ci/sdk_native_inventory.py create mode 100644 cli/ci/test_sdk_native_inventory.py diff --git a/cli/ci/build_agents_sdk.py b/cli/ci/build_agents_sdk.py index 116e7d87..eb635e94 100644 --- a/cli/ci/build_agents_sdk.py +++ b/cli/ci/build_agents_sdk.py @@ -13,6 +13,7 @@ import subprocess import sys import tempfile +import sdk_native_inventory as native ROOT = Path(__file__).resolve().parents[2] LOCK = ROOT / 'harnesses/agents-sdk/requirements-build.txt' @@ -35,16 +36,8 @@ ' import sdk_tool_selftest\n' ' sdk_tool_selftest.main()\n' "elif '--packaged-library-test' in sys.argv:\n" - ' import json, pathlib, re\n' - ' versions=[]; count=0\n' - " for path in pathlib.Path(sys._MEIPASS).rglob('*'):\n" - ' if path.is_file() and not path.is_symlink() and path.stat().st_size<=268435456:\n' - " with path.open('rb') as stream:\n" - " if stream.read(4)!=b'\\x7fELF': continue\n" - ' data=stream.read()\n' - ' count+=1\n' - " versions.extend(tuple(int(x) for x in v.split(b'.')) for v in re.findall(rb'GLIBC_([0-9]+\\.[0-9]+(?:\\.[0-9]+)?)',data))\n" - " print(json.dumps({'schema':'openprose.sdk-packaged-libraries/1','elfCount':count,'requiredGlibcMaximum':'.'.join(str(x) for x in max(versions)) if versions else None,'modelCalls':0},sort_keys=True))\n" + ' import json, sdk_native_inventory\n' + ' print(json.dumps(sdk_native_inventory.inspect_tree(sys._MEIPASS),sort_keys=True))\n' 'else:\n' ' import runpy\n' " runpy.run_module('prose_sdk_runtime',run_name='__main__')\n" @@ -174,11 +167,38 @@ def run(command, *, env, cwd, log, timeout=900): require(completed.returncode == 0, 'SDK build command failed; inspect ' + str(log)) -def build(output, *, epoch=0, codesign_identity=None): +def inspect_archive(helper): + from PyInstaller.archive.readers import CArchiveReader + native.read_file(helper) + archive = CArchiveReader(str(helper)); rows = []; total_bytes = 0 + for name, entry in sorted(archive.toc.items()): + native.safe_path(name) + if entry[-1] != 'b': continue + native.require(type(entry[2]) is int and 0 <= entry[2] <= native.MAX_BYTES, 'Oversized native archive declaration') + total_bytes += entry[2] + native.require(total_bytes <= 768 * 1024 * 1024, 'Native archive extraction exceeds bound') + data = archive.extract(name) + native.require(len(data) == entry[2], 'Native archive size declaration mismatch') + native.require(len(data) <= native.MAX_BYTES, 'Oversized frozen archive member') + if data[:4] == b'\x7fELF': rows.append(native.elf_record(name, data)) + native.require(len(rows) <= native.MAX_FILES, 'Too many native archive members') + return rows + + +def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_native_origin=None): require(platform.python_version() == '3.10.20', 'SDK release builder requires Python 3.10.20') require(sys.platform in ('darwin', 'linux') and platform.machine() in ('arm64', 'aarch64', 'x86_64'), 'SDK builder requires native supported POSIX host') if codesign_identity is not None: require(sys.platform == 'darwin' and re.fullmatch(r'Developer ID Application: [^\r\n]+ \([A-Z0-9]{10}\)', codesign_identity), 'SDK signing requires a macOS Developer ID Application identity') + require((linux_libgcc is None) == (linux_native_origin is None), 'Linux native hooks must be supplied together') + require(sys.platform == 'linux' or linux_libgcc is None, 'Linux native hooks are Linux-only') + require(sys.platform != 'linux' or linux_libgcc is not None, 'Linux build requires explicit libgcc supplier and native origin') + native_input = None + if linux_libgcc is not None: + target = 'linux-arm64-gnu' if platform.machine() == 'aarch64' else 'linux-x64-gnu' + native_input, native_input_sha, supplier_texts = native.validate_input(linux_native_origin, linux_libgcc, ROOT, target) + supplier_row = native.elf_record('libgcc_s.so.1', native.read_file(linux_libgcc)) + require(supplier_row['machine'] == (183 if target == 'linux-arm64-gnu' else 62), 'Supplier machine differs from target') require(not output.exists() and not output.is_symlink(), 'SDK output must be fresh') require(epoch >= 0, 'Source date epoch must be non-negative') output.parent.mkdir(parents=True, exist_ok=True) @@ -186,6 +206,15 @@ def build(output, *, epoch=0, codesign_identity=None): 'SDK freeze requires at least 512 MiB free scratch space') packages = lock_packages() notices = installed_inventory(packages) + if native_input is not None: + origin = native_input['libgcc'] + supplier_header = json.dumps({'package': origin['package'], 'library': origin['library'], 'image': native_input['images']['supplier']}, sort_keys=True) + python_origin, _ = native.python_origin(native_input) + texts = ['Actual libgcc supplier provenance: ' + supplier_header, *supplier_texts, + 'Full Python native provenance: ' + json.dumps(python_origin, sort_keys=True)] + python_root = Path(native_input['pythonDistribution']['root']) + texts += [native.read_file(python_root / row['path'], 1024 * 1024).decode('utf-8') for row in python_origin['licenses']] + notices += ('\n' + '\n'.join(texts) + '\n').encode('utf-8') output.mkdir(parents=True) env = environment(output, os.environ); env['SOURCE_DATE_EPOCH'] = str(epoch) (output / NOTICES).write_bytes(notices) @@ -197,12 +226,21 @@ def build(output, *, epoch=0, codesign_identity=None): # Snapshot the concurrently maintained runtime before freezing. Receipt binds exact bytes. source_copy = output / 'prose_sdk_runtime.py'; source_copy.write_bytes(SOURCE.read_bytes()) source_sha = digest(source_copy) + native_source = Path(native.__file__) + (output / 'sdk_native_inventory.py').write_bytes(native.read_file(native_source)) + if native_input is not None: + require(source_sha == native_input['sourceSnapshot']['sources']['harnesses/agents-sdk/run.py'] and + digest(output / 'sdk_native_inventory.py') == native_input['sourceSnapshot']['sources']['cli/ci/sdk_native_inventory.py'], + 'Frozen source copies differ from bound source snapshot') args = [sys.executable, '-m', 'PyInstaller', '--clean', '--noconfirm', '--onefile', '--noupx', '--name', NAME, '--distpath', output / 'dist', '--workpath', output / 'work', '--specpath', output, '--paths', output, '--hidden-import', 'prose_sdk_runtime', '--collect-all', 'agents', '--collect-all', 'openai', '--collect-all', 'certifi', '--copy-metadata', 'openai-agents', '--copy-metadata', 'openai', '--add-data', str(output / NOTICES) + os.pathsep + '.', entry] + args.extend(['--hidden-import', 'sdk_native_inventory']) + if native_input is not None: + args.extend(['--add-binary', str(linux_libgcc) + os.pathsep + '.']) if codesign_identity is not None: args.extend(['--codesign-identity', codesign_identity]) for package in packages: @@ -210,6 +248,17 @@ def build(output, *, epoch=0, codesign_identity=None): run(args, env=env, cwd=output, log=output / 'freeze.log') helper = output / NAME; shutil.copyfile(output / 'dist' / NAME, helper); helper.chmod(0o755) require(0 < helper.stat().st_size <= MAX_BYTES, 'Frozen SDK helper exceeds size limit') + native_dependencies = None + if native_input is not None: + toc_sha = native.verify_analysis_toc(output / 'work' / NAME / 'Analysis-00.toc', linux_libgcc) + archive_rows = inspect_archive(helper) + native.verify_closure(archive_rows, target) + origin_rows, origins = native.assign_origins(archive_rows, native_input, packages, metadata.distribution) + native_dependencies = {'libraries': origin_rows, 'symbolClosureVerified': True, + 'libgccSelection': {'path': 'libgcc_s.so.1', 'sha256': supplier_row['sha256'], + 'byteLength': supplier_row['byteLength'], 'analysisTocSha256': toc_sha}, + 'origins': origins} + native.validate_native_dependencies(native_dependencies, target) # Disposable build caches duplicate the frozen payload and need not accompany receipts. for directory in ('dist', 'work', 'pyinstaller-cache'): shutil.rmtree(output / directory) @@ -230,6 +279,7 @@ def build(output, *, epoch=0, codesign_identity=None): linux_libraries = json.loads((output / 'libraries.json').read_text()) require(linux_libraries.get('elfCount', 0) > 0 and linux_libraries.get('requiredGlibcMaximum') is not None, 'Missing frozen Linux library inspection') + require(linux_libraries.get('libraries') == archive_rows, 'Frozen extracted native membership differs from archive') maximum = tuple(int(x) for x in linux_libraries['requiredGlibcMaximum'].split('.')) require(maximum <= (2, 34), 'Frozen SDK libraries require glibc newer than 2.34') if sys.platform == 'darwin': @@ -245,6 +295,14 @@ def build(output, *, epoch=0, codesign_identity=None): 'toolSelfTest': tool_self_test, 'linuxLibraries': linux_libraries, 'pythonExecutableSha256': digest(Path(sys.executable).resolve()), 'authority': 'local-build-and-imports-only', 'publicationAuthorized': False} + if native_input is not None: + final_input, final_sha, _ = native.validate_input(linux_native_origin, linux_libgcc, ROOT, target) + final_rows, final_origins = native.assign_origins(archive_rows, final_input, packages, metadata.distribution) + require(final_rows == origin_rows and final_origins == origins, 'Native origin/license bytes changed during build') + require(final_input == native_input and final_sha == native_input_sha, 'Native input changed during build') + receipt.update(linuxBuildInputSha256=native_input_sha, + linuxBuildSourceSnapshot=native_input['sourceSnapshot'], nativeDependencies=native_dependencies) + native.validate_linux_receipt(receipt) (output / RECEIPT).write_text(json.dumps(receipt, indent=2, sort_keys=True) + '\n') return receipt @@ -254,8 +312,11 @@ def build(output, *, epoch=0, codesign_identity=None): parser.add_argument('--out', type=Path, required=True) parser.add_argument('--codesign-identity', help='macOS Developer ID identity for every embedded binary; requires prepared signing keychain') parser.add_argument('--source-date-epoch', type=int, default=0) + parser.add_argument('--linux-libgcc', type=Path, help='Digest-verified same-architecture libgcc supplier') + parser.add_argument('--linux-native-origin', type=Path, help='Owned Linux native-input provenance record') args = parser.parse_args() try: - print(json.dumps(build(args.out.absolute(), epoch=args.source_date_epoch, codesign_identity=args.codesign_identity), sort_keys=True)) + print(json.dumps(build(args.out.absolute(), epoch=args.source_date_epoch, codesign_identity=args.codesign_identity, + linux_libgcc=args.linux_libgcc, linux_native_origin=args.linux_native_origin), sort_keys=True)) except (ValueError, OSError, subprocess.SubprocessError, metadata.PackageNotFoundError) as error: parser.exit(2, str(error) + '\n') diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py new file mode 100644 index 00000000..4ac69ac5 --- /dev/null +++ b/cli/ci/sdk_native_inventory.py @@ -0,0 +1,562 @@ +"""Bounded native inspection for the Linux package-owned SDK helper. + +ELF64 section, symbol and GNU version layouts follow the System V ABI and LSB +symbol-version specification. This module uses only the standard library inside +its frozen inspection entrypoint. The builder separately inspects the archive. +""" +from __future__ import annotations +import ast +import base64 +import hashlib +import json +from pathlib import Path, PurePosixPath +import re +import struct + +MAX_BYTES = 256 * 1024 * 1024 +MAX_FILES = 128 +MAX_METADATA = 8 * 1024 * 1024 +SOURCE_PATHS = ('harnesses/agents-sdk/run.py', 'harnesses/agents-sdk/requirements-build.txt', + 'harnesses/agents-sdk/test_run.py', 'cli/ci/build_agents_sdk.py', + 'cli/ci/sdk_native_inventory.py') + + +def require(value, message): + if not value: + raise ValueError(message) + + +def load_json(data): + def closed_pairs(pairs): + result = {} + for name, value in pairs: + require(name not in result, 'Duplicate native metadata key') + result[name] = value + return result + return json.loads(data, object_pairs_hook=closed_pairs) + + +def sha(data): + return hashlib.sha256(data).hexdigest() + + +def keys(value, expected, label): + require(isinstance(value, dict) and set(value) == set(expected), 'Unsupported ' + label + ' shape') + + +def safe_path(value): + require(isinstance(value, str) and value and '\\' not in value and '\x00' not in value, + 'Invalid native member path') + path = PurePosixPath(value) + require(not path.is_absolute() and '..' not in path.parts and str(path) == value, + 'Unsafe native member path') + return value + + +def read_file(path, maximum=MAX_BYTES): + path = Path(path) + require(path.is_file() and not path.is_symlink(), 'Native input must be a regular file') + size = path.stat().st_size + require(0 < size <= maximum, 'Native input size is outside bounds') + with path.open('rb') as stream: + data = stream.read(maximum + 1) + require(len(data) <= maximum and len(data) == size, 'Native input changed while reading') + return data + + +def record(path, data): + return {'path': safe_path(path), 'sha256': sha(data), 'byteLength': len(data)} + + +def elf_record(path, data): + """Inspect ELF64 little-endian dynamic imports/exports, rejecting partial data.""" + require(64 <= len(data) <= MAX_BYTES and data[:7] == b'\x7fELF\x02\x01\x01', 'Unsupported ELF header') + machine = struct.unpack_from('= 0, 'Unterminated ELF string') + try: value = table[offset:end].decode('ascii') + except UnicodeDecodeError as error: raise ValueError('Non-ASCII ELF dynamic string') from error + require(all(32 <= ord(c) < 127 for c in value), 'Invalid ELF dynamic string') + return value + def linked(section): + require(0 < section[6] < count and sections[section[6]][1] == 3, 'Invalid ELF string table link') + return payload(sections[section[6]]) + def single(kind): + matches = [s for s in sections if s[1] == kind] + require(len(matches) <= 1, 'Duplicate ELF dynamic section') + return matches[0] if matches else None + dynamic = single(6); symbols = single(11); versions = single(0x6fffffff) + needs_section = single(0x6ffffffe); defs_section = single(0x6ffffffd) + needed = []; dynamic_counts = {} + if dynamic: + require(dynamic[9] == 16 and dynamic[5] % 16 == 0, 'Invalid ELF dynamic entries') + table = linked(dynamic); terminated = False + for offset in range(0, dynamic[5], 16): + tag, value = struct.unpack_from('= width, + 'Invalid ELF version auxiliary count') + acursor = cursor + aux; avisited = set(); first = None + for i in range(auxiliary_count): + awidth = 8 if definition else 16 + require(acursor not in avisited and acursor + awidth <= len(blob), 'Invalid ELF version auxiliary chain') + avisited.add(acursor) + if definition: + name_offset, next_aux = struct.unpack_from(' 1 and key not in need_indices, 'Duplicate ELF needed version index') + need_indices[key] = (library, name) + need_rows.append({'library': library, 'version': name, 'flags': flags}) + if i + 1 < auxiliary_count: + require(next_aux >= awidth, 'Short ELF auxiliary version chain') + acursor += next_aux + else: require(next_aux == 0, 'ELF auxiliary count mismatch') + if definition: + index &= 0x7fff + require(index > 0 and index not in def_indices, 'Duplicate ELF defined version index') + def_indices[index] = first + if following == 0: break + require(following >= width, 'Invalid ELF version next offset') + cursor += following + require(section[7] == total, 'ELF version section count mismatch') + return total + nneeds = chains(needs_section, False); ndefs = chains(defs_section, True) + for tag, actual in ((0x6fffffff, nneeds), (0x6ffffffd, ndefs)): + if tag in dynamic_counts: require(dynamic_counts[tag] == actual, 'ELF dynamic version count mismatch') + imported = []; defined = [] + if symbols: + require(symbols[9] == 24 and symbols[5] % 24 == 0, 'Invalid ELF symbol table') + sym_count = symbols[5] // 24; table = linked(symbols); blob = payload(symbols) + require(sym_count <= 1000000, 'Oversized ELF dynamic symbol table') + vblob = payload(versions) if versions else None + if versions: + require(versions[6] == sections.index(symbols) and versions[5] == sym_count * 2, + 'ELF version and symbol tables disagree') + for i in range(sym_count): + name_offset, info, other, section_index, _, _ = struct.unpack_from('> 4 not in (1, 2): continue + name = string(table, name_offset) + if not name: continue + version_index = struct.unpack_from('> 4 == 2}) + elif other & 3 in (0, 3): + require(section_index != 0xffff, 'Unsupported extended ELF symbol index') + require(version_key <= 1 or version_key in def_indices, 'Unknown ELF defined symbol version') + defined.append({'symbol': name, 'version': def_indices.get(version_key), + 'defaultVersion': not bool(version_index & 0x8000)}) + else: require(not versions and not needs_section and not defs_section, 'ELF versions without symbols') + # Keep the established conservative raw-version gate in addition to actual imports. + glibc = [tuple(int(x) for x in v.split(b'.')) for v in re.findall(rb'GLIBC_([0-9]+\.[0-9]+(?:\.[0-9]+)?)', data)] + return {**record(path, data), 'machine': machine, 'needed': sorted(needed), + 'versionNeeds': sorted(need_rows, key=lambda r: (r['library'], r['version'], r['flags'])), + 'importedSymbols': sorted(imported, key=lambda r: (r['symbol'], r['library'] or '', r['version'] or '', r['weak'])), + 'definedSymbols': sorted(defined, key=lambda r: (r['symbol'], r['version'] or '', r['defaultVersion'])), + 'maximumRequiredGlibc': '.'.join(map(str, max(glibc))) if glibc else None} + + +def verify_closure(libraries, target): + machine = {'linux-arm64-gnu': 183, 'linux-x64-gnu': 62}.get(target) + require(machine is not None and 0 < len(libraries) <= MAX_FILES, 'Invalid native target or library count') + require(len({r['path'] for r in libraries}) == len(libraries), 'Duplicate native member') + require(all(r['machine'] == machine for r in libraries), 'Native library machine differs from target') + suppliers = [r for r in libraries if r['path'] == 'libgcc_s.so.1'] + require(len(suppliers) == 1, 'Frozen helper must contain exactly one libgcc supplier') + supplier = suppliers[0] + exports = {(r['symbol'], r['version']) for r in supplier['definedSymbols']} + supplied_versions = {r['version'] for r in supplier['definedSymbols'] if r['version'] is not None} + for library in libraries: + maximum = library['maximumRequiredGlibc'] + require(maximum is None or tuple(map(int, maximum.split('.'))) <= (2, 34), + 'Frozen SDK libraries require glibc newer than 2.34') + for need in library['versionNeeds']: + if need['library'] == 'libgcc_s.so.1' and not need['flags'] & 2: + require(need['version'] in supplied_versions, 'libgcc does not supply a required version') + for symbol in library['importedSymbols']: + if symbol['library'] == 'libgcc_s.so.1' and not symbol['weak']: + require((symbol['symbol'], symbol['version']) in exports, 'libgcc does not supply a required strong symbol') + return True + + +def inspect_tree(root): + root = Path(root); rows = [] + for path in sorted(root.rglob('*')): + require(not path.is_symlink(), 'Unexpected symlink in frozen native inspection') + if not path.is_file(): continue + with path.open('rb') as stream: magic = stream.read(4) + if magic != b'\x7fELF': continue + rows.append(elf_record(path.relative_to(root).as_posix(), read_file(path))) + require(len(rows) <= MAX_FILES, 'Too many frozen native libraries') + maximum = max((tuple(map(int, r['maximumRequiredGlibc'].split('.'))) for r in rows + if r['maximumRequiredGlibc'] is not None), default=None) + return {'schema': 'openprose.sdk-packaged-libraries/1', 'elfCount': len(rows), + 'requiredGlibcMaximum': '.'.join(map(str, maximum)) if maximum else None, + 'libraries': rows, 'modelCalls': 0} + + +def verify_analysis_toc(path, supplier): + data = read_file(path, MAX_METADATA) + try: value = ast.literal_eval(data.decode('utf-8')) + except (ValueError, SyntaxError, RecursionError) as error: raise ValueError('Invalid PyInstaller analysis TOC') from error + found = [] + def visit(node, depth=0): + require(depth <= 64, 'PyInstaller TOC nesting exceeds bound') + if isinstance(node, (list, tuple)): + if len(node) == 3 and node[0] == 'libgcc_s.so.1' and node[2] == 'BINARY': found.append(node) + else: + for child in node: visit(child, depth + 1) + visit(value) + require(len(found) == 1, 'PyInstaller must select exactly one libgcc binary') + require(Path(found[0][1]).resolve() == Path(supplier).resolve() and sha(read_file(found[0][1])) == sha(read_file(supplier)), + 'PyInstaller selected an ambient libgcc binary') + return sha(data) + + +def validate_input(path, supplier, source_root, target): + data = read_file(path, MAX_METADATA) + value = load_json(data) + keys(value, ('schema', 'target', 'images', 'pythonArchive', 'sourceSnapshot', 'libgcc', + 'driverSha256', 'lockSha256', 'pythonDistribution'), 'Linux native input') + require(value['schema'] == 'openprose.sdk-linux-native-input/1' and value['target'] == target, + 'Native input target/schema mismatch') + def hash_value(digest): + require(isinstance(digest, str) and re.fullmatch('[0-9a-f]{64}', digest), 'Invalid native input digest') + snapshot = value['sourceSnapshot']; keys(snapshot, ('sources', 'driverSha256', 'lockSha256'), 'source snapshot') + keys(snapshot['sources'], SOURCE_PATHS, 'source snapshot files') + for relative, expected in snapshot['sources'].items(): + hash_value(expected) + require(sha(read_file(Path(source_root) / relative)) == expected, 'Linux build source snapshot changed') + for name in ('driverSha256', 'lockSha256'): + hash_value(value[name]); require(snapshot[name] == value[name], 'Native driver/lock binding differs') + keys(value['images'], ('supplier', 'freezer'), 'native images') + arch = 'aarch64' if target == 'linux-arm64-gnu' else 'x86_64' + for role, floor in (('supplier', '2_28'), ('freezer', '2_34')): + image = value['images'][role]; keys(image, ('image', 'configSha256', 'metadataUrl'), 'native image') + require(re.fullmatch(r'quay\.io/pypa/manylinux_' + floor + '_' + arch + r'@sha256:[0-9a-f]{64}', image['image']), + 'Native image must be same-architecture digest-pinned PyPA image') + hash_value(image['configSha256']) + require(isinstance(image['metadataUrl'], str) and image['metadataUrl'].startswith('https://'), 'Missing native image metadata URL') + archive = value['pythonArchive'] + keys(archive, ('url', 'sha256', 'byteLength', 'targetTriple', 'metadataUrl'), 'Python archive') + hash_value(archive['sha256']) + require(type(archive['byteLength']) is int and archive['byteLength'] > 0, 'Invalid Python archive size') + triple = arch + '-unknown-linux-gnu' + require(archive['targetTriple'] == triple and isinstance(archive['url'], str) and + re.fullmatch(r'https://github\.com/astral-sh/python-build-standalone/releases/download/[0-9]+/cpython-3\.10\.20%2B[0-9]+-' + + triple + r'-pgo%2Blto-full\.tar\.zst', archive['url']), 'Expected pinned full shared Python archive') + require(isinstance(archive['metadataUrl'], str) and archive['metadataUrl'].startswith('https://'), 'Missing Python archive metadata URL') + origin = value['libgcc']; keys(origin, ('schema', 'package', 'library', 'licenses'), 'libgcc origin') + require(origin['schema'] == 'openprose.sdk-native-origin/1', 'Unsupported libgcc origin schema') + package = origin['package']; keys(package, ('name', 'epoch', 'version', 'release', 'architecture', 'sourceRpm', 'license'), 'supplier package') + require(package['name'] == 'libgcc' and package['architecture'] == arch and + all(isinstance(v, str) and v and not any(ord(c) < 32 for c in v) for v in package.values()) and + re.fullmatch(r'[0-9]+', package['epoch']) and package['sourceRpm'].endswith('.src.rpm'), 'Invalid supplier package provenance') + library = origin['library']; keys(library, ('path', 'sha256', 'byteLength', 'supplierPath', 'rpmFileDigestVerified'), 'supplier library') + require(library['path'] == 'libgcc_s.so.1' and library['rpmFileDigestVerified'] is True and + isinstance(library['supplierPath'], str) and library['supplierPath'].startswith('/usr/lib64/') and + '..' not in PurePosixPath(library['supplierPath']).parts, 'Invalid supplier library origin') + actual = read_file(supplier) + require(library['sha256'] == sha(actual) and type(library['byteLength']) is int and library['byteLength'] == len(actual), + 'Explicit libgcc bytes differ from supplier receipt') + licenses = origin['licenses']; require(isinstance(licenses, list) and 0 < len(licenses) <= 32, 'Missing supplier license evidence') + texts = []; seen = set() + for item in licenses: + keys(item, ('packagePath', 'path', 'sha256', 'byteLength'), 'supplier license') + safe_path(item['path']); require(item['path'] not in seen, 'Duplicate supplier license'); seen.add(item['path']) + require(isinstance(item['packagePath'], str) and item['packagePath'].startswith('/usr/share/licenses/'), 'Supplier license is not RPM-owned') + text = read_file(Path(supplier).parent / item['path'], 1024 * 1024) + require(item['sha256'] == sha(text) and type(item['byteLength']) is int and item['byteLength'] == len(text), 'Supplier license digest mismatch') + texts.append(text.decode('utf-8')) + joined = '\n'.join(texts) + require('GNU GENERAL PUBLIC LICENSE' in joined and 'Version 3' in joined and + 'GCC RUNTIME LIBRARY EXCEPTION' in joined and 'Version 3.1' in joined, + 'Supplier GPL and runtime exception texts are required') + distribution = value['pythonDistribution']; keys(distribution, ('metadataPath', 'root', 'metadataSha256'), 'Python distribution') + root = Path(distribution['root']); metadata_path = Path(distribution['metadataPath']) + require(root.is_absolute() and metadata_path.is_absolute() and not root.is_symlink() and + metadata_path == root / 'PYTHON.json', 'Python metadata must belong to supplied full distribution') + hash_value(distribution['metadataSha256']) + require(sha(read_file(metadata_path, MAX_METADATA)) == distribution['metadataSha256'], 'Python metadata digest mismatch') + return value, sha(data), texts + + +def python_origin(value): + distribution = value['pythonDistribution']; root = Path(distribution['root']) + data = read_file(distribution['metadataPath'], MAX_METADATA); info = load_json(data) + require(info.get('version') == 8, 'Unsupported full Python metadata version') + require(info.get('python_version') == '3.10.20' and info.get('target_triple') == value['pythonArchive']['targetTriple'] and + info.get('libpython_link_mode') == 'shared', 'Python metadata is not the target full shared distribution') + license_paths = {info.get('license_path')} if info.get('license_path') else set() + require(isinstance(info.get('licenses'), list) and info['licenses'] and license_paths, + 'Python distribution license metadata is absent') + build_info = info.get('build_info'); require(isinstance(build_info, dict), 'Missing Python native build metadata') + core = build_info.get('core'); require(isinstance(core, dict) and isinstance(core.get('shared_lib'), str), 'Python core shared library is absent') + static = [] + extensions = build_info.get('extensions'); require(isinstance(extensions, dict), 'Missing Python extension metadata') + for name, variants in sorted(extensions.items()): + require(isinstance(variants, list), 'Invalid Python extension variants') + for variant in variants: + require(isinstance(variant, dict), 'Invalid Python extension metadata') + declared = variant.get('licenses', []) + paths = variant.get('license_paths', []) + legacy_paths = variant.get('license_path', []) + if isinstance(legacy_paths, str): legacy_paths = [legacy_paths] + require(isinstance(legacy_paths, list), 'Invalid Python extension license paths') + paths = [*paths, *legacy_paths] + require(isinstance(declared, list) and isinstance(paths, list), 'Invalid Python extension license metadata') + if declared: + require(paths, 'Declared Python native license lacks corresponding text') + license_paths.update(paths) + static.append({'name': name, 'licenses': declared, 'licensePaths': sorted(paths)}) + licenses = [] + for relative in sorted(license_paths): + safe_path(relative); text = read_file(root / relative, 1024 * 1024) + text.decode('utf-8') + licenses.append(record(relative, text)) + # Actual full-provider member bytes establish ownership, not a basename guess. + owned = {} + for path in sorted(root.rglob('*')): + if path.is_symlink() or not path.is_file(): continue + with path.open('rb') as stream: magic = stream.read(4) + if magic == b'\x7fELF': owned[sha(read_file(path))] = path.relative_to(root).as_posix() + core_path = (root / safe_path(core['shared_lib'])).resolve() + require(core_path.is_relative_to(root.resolve()) and sha(read_file(core_path)) in owned, 'Python shared core bytes are absent') + # These are full-provider declarations, not proof that each extension shipped. + return {'archive': value['pythonArchive'], 'metadataSha256': sha(data), 'licenses': licenses, + 'declaredExtensionLicenses': static}, owned + + +def wheel_origins(packages, distributions): + origins = []; owned = {} + for package in packages: + dist = distributions(package['name']) + require(dist.version == package['version'], 'Installed wheel differs from lock') + files = dist.files or [] + record_files = [entry for entry in files if str(entry).endswith('.dist-info/RECORD')] + require(len(record_files) == 1, 'Wheel RECORD is absent or ambiguous') + record_bytes = read_file(dist.locate_file(record_files[0]), MAX_METADATA) + licenses = [] + for entry in files: + path = Path(dist.locate_file(entry)) + if path.is_symlink() or not path.is_file(): continue + is_license = '.dist-info/' in str(entry) and ('license' in str(entry).lower() or 'copying' in str(entry).lower()) + with path.open('rb') as stream: native = stream.read(4) == b'\x7fELF' + if not native and not is_license: continue + data = read_file(path, 1024 * 1024 if is_license else MAX_BYTES) + require(entry.hash is not None and entry.hash.mode == 'sha256' and + base64.urlsafe_b64encode(hashlib.sha256(data).digest()).rstrip(b'=').decode() == entry.hash.value and + entry.size == len(data), 'Installed native/license bytes differ from wheel RECORD') + if native: + digest = sha(data) + require(digest not in owned, 'Ambiguous installed native wheel origin') + owned[digest] = package['name'] + if is_license: + data.decode('utf-8'); licenses.append(record(str(entry), data)) + if any(name == package['name'] for name in owned.values()): + require(licenses, 'Native wheel lacks corresponding license texts') + origins.append({'name': package['name'], 'version': package['version'], 'wheelCandidateSha256s': package['wheelSha256'], + 'recordSha256': sha(record_bytes), 'licenses': sorted(licenses, key=lambda r: r['path'])}) + return sorted(origins, key=lambda r: r['name']), owned + + +def assign_origins(libraries, value, packages, distributions): + python, python_members = python_origin(value) + wheels, wheel_members = wheel_origins(packages, distributions) + result = [] + for row in libraries: + if row['path'] == 'libgcc_s.so.1': + require(row['sha256'] == value['libgcc']['library']['sha256'], 'Frozen libgcc differs from selected supplier') + origin = 'supplier' + elif row['sha256'] in python_members: origin = 'python' + elif row['sha256'] in wheel_members: origin = 'wheel:' + wheel_members[row['sha256']] + else: raise ValueError('Frozen native member has unknown origin: ' + row['path']) + result.append({**row, 'origin': origin}) + return result, {'supplier': {**value['libgcc'], 'image': value['images']['supplier']}, 'python': python, 'wheels': wheels} + + +def validate_native_dependencies(value, target): + """Pure closed receipt validation; artifact consumers also compare actual ELF bytes. + +This verifies the receipt's internal proof and origin references. It does not +replace CArchive/extracted-byte custody or establish missing supplier facts. +""" + keys(value, ('libraries', 'symbolClosureVerified', 'libgccSelection', 'origins'), 'native dependencies') + require(value['symbolClosureVerified'] is True, 'Native symbol closure was not established') + libraries = value['libraries']; require(isinstance(libraries, list), 'Invalid native libraries') + require(0 < len(libraries) <= MAX_FILES and all(isinstance(row, dict) and isinstance(row.get('path'), str) for row in libraries), 'Invalid native library rows') + require(libraries == sorted(libraries, key=lambda row: row['path']), 'Native libraries must be sorted') + def digest(value): + require(isinstance(value, str) and re.fullmatch('[0-9a-f]{64}', value), 'Invalid native receipt digest') + def file_row(row, maximum=1024 * 1024): + keys(row, ('path', 'sha256', 'byteLength'), 'native license file') + safe_path(row['path']); digest(row['sha256']) + require(type(row['byteLength']) is int and 0 < row['byteLength'] <= maximum, 'Invalid native receipt size') + origins = value['origins']; keys(origins, ('supplier', 'python', 'wheels'), 'native origins') + supplier = origins['supplier']; keys(supplier, ('schema', 'package', 'library', 'licenses', 'image'), 'supplier origin') + require(supplier['schema'] == 'openprose.sdk-native-origin/1', 'Invalid supplier origin schema') + keys(supplier['package'], ('name', 'epoch', 'version', 'release', 'architecture', 'sourceRpm', 'license'), 'supplier package') + arch = {'linux-x64-gnu': 'x86_64', 'linux-arm64-gnu': 'aarch64'}.get(target) + require(arch is not None, 'Unsupported native receipt target') + require(supplier['package']['architecture'] == arch and supplier['package']['name'] == 'libgcc' and all(isinstance(x, str) and x for x in supplier['package'].values()), + 'Invalid supplier package metadata') + keys(supplier['library'], ('path', 'sha256', 'byteLength', 'supplierPath', 'rpmFileDigestVerified'), 'supplier file') + require(supplier['library']['rpmFileDigestVerified'] is True, 'Supplier RPM file digest was not verified') + file_row({k: supplier['library'][k] for k in ('path', 'sha256', 'byteLength')}, MAX_BYTES) + keys(supplier['image'], ('image', 'configSha256', 'metadataUrl'), 'supplier image') + digest(supplier['image']['configSha256']) + require(re.fullmatch(r'quay\.io/pypa/manylinux_2_28_' + arch + r'@sha256:[0-9a-f]{64}', supplier['image']['image']), 'Supplier image target/floor mismatch') + require(supplier['library']['path'] == 'libgcc_s.so.1' and isinstance(supplier['library']['supplierPath'], str) and supplier['library']['supplierPath'].startswith('/usr/lib64/'), 'Supplier library origin mismatch') + require(re.fullmatch(r'[0-9]+', supplier['package']['epoch']) and supplier['package']['sourceRpm'].endswith('.src.rpm'), 'Supplier RPM identity is incomplete') + require(isinstance(supplier['licenses'], list) and supplier['licenses'], 'Missing supplier licenses') + for row in supplier['licenses']: + keys(row, ('path', 'sha256', 'byteLength', 'packagePath'), 'supplier license') + file_row({k: row[k] for k in ('path', 'sha256', 'byteLength')}) + python = origins['python']; keys(python, ('archive', 'metadataSha256', 'licenses', 'declaredExtensionLicenses'), 'Python origin') + keys(python['archive'], ('url', 'sha256', 'byteLength', 'targetTriple', 'metadataUrl'), 'Python origin archive') + digest(python['metadataSha256']); digest(python['archive']['sha256']) + require(python['archive']['targetTriple'] == arch + '-unknown-linux-gnu' and + re.fullmatch(r'https://github\.com/astral-sh/python-build-standalone/releases/download/[0-9]+/cpython-3\.10\.20%2B[0-9]+-' + arch + r'-unknown-linux-gnu-pgo%2Blto-full\.tar\.zst', python['archive']['url']), 'Python archive target/type mismatch') + require(type(python['archive']['byteLength']) is int and python['archive']['byteLength'] > 0, 'Invalid Python archive size') + require(isinstance(python['licenses'], list) and python['licenses'], 'Missing Python license evidence') + for row in python['licenses']: file_row(row) + require(isinstance(python['declaredExtensionLicenses'], list), 'Invalid Python static dependencies') + for row in python['declaredExtensionLicenses']: + keys(row, ('name', 'licenses', 'licensePaths'), 'Python static dependency') + require(isinstance(row['name'], str) and row['name'] and isinstance(row['licenses'], list) and row['licenses'] and + isinstance(row['licensePaths'], list) and row['licensePaths'] and + all(path in {r['path'] for r in python['licenses']} for path in row['licensePaths']), + 'Unbound Python static dependency licenses') + wheels = origins['wheels']; require(isinstance(wheels, list) and wheels == sorted(wheels, key=lambda r: r['name']), 'Unsorted native wheel origins') + names = set() + for row in wheels: + keys(row, ('name', 'version', 'wheelCandidateSha256s', 'recordSha256', 'licenses'), 'native wheel origin') + require(isinstance(row['name'], str) and re.fullmatch('[a-z0-9]+(?:-[a-z0-9]+)*', row['name']) and row['name'] not in names, + 'Duplicate/invalid wheel origin name'); names.add(row['name']) + require(isinstance(row['version'], str) and row['version'], 'Invalid wheel origin version') + digest(row['recordSha256']) + require(isinstance(row['wheelCandidateSha256s'], list) and row['wheelCandidateSha256s'] and row['wheelCandidateSha256s'] == sorted(set(row['wheelCandidateSha256s'])), 'Invalid locked wheel candidates') + for candidate in row['wheelCandidateSha256s']: digest(candidate) + require(isinstance(row['licenses'], list), 'Invalid wheel licenses') + for license_row in row['licenses']: file_row(license_row) + for row in libraries: + keys(row, ('path', 'sha256', 'byteLength', 'machine', 'needed', 'versionNeeds', 'importedSymbols', + 'definedSymbols', 'maximumRequiredGlibc', 'origin'), 'native library') + file_row({k: row[k] for k in ('path', 'sha256', 'byteLength')}, MAX_BYTES) + require(type(row['machine']) is int and isinstance(row['needed'], list) and + all(isinstance(x, str) and x for x in row['needed']), 'Invalid ELF receipt machine/needed libraries') + require(row['maximumRequiredGlibc'] is None or (isinstance(row['maximumRequiredGlibc'], str) and + re.fullmatch(r'[0-9]+\.[0-9]+(?:\.[0-9]+)?', row['maximumRequiredGlibc'])), 'Invalid GLIBC requirement') + require(all(isinstance(row[field], list) and len(row[field]) <= 100000 for field in ('needed', 'versionNeeds', 'importedSymbols', 'definedSymbols')), 'Invalid ELF receipt arrays') + for need in row['versionNeeds']: + keys(need, ('library', 'version', 'flags'), 'native version need') + require(need['library'] in row['needed'] and isinstance(need['version'], str) and need['version'] and + type(need['flags']) is int and 0 <= need['flags'] <= 65535, 'Invalid native version need') + for symbol in row['importedSymbols']: + keys(symbol, ('symbol', 'library', 'version', 'weak'), 'native imported symbol') + require(isinstance(symbol['symbol'], str) and symbol['symbol'] and type(symbol['weak']) is bool and + ((symbol['library'] is None and symbol['version'] is None) or + (symbol['library'] in row['needed'] and isinstance(symbol['version'], str) and symbol['version'])), 'Invalid native imported symbol') + for symbol in row['definedSymbols']: + keys(symbol, ('symbol', 'version', 'defaultVersion'), 'native defined symbol') + require(isinstance(symbol['symbol'], str) and symbol['symbol'] and type(symbol['defaultVersion']) is bool and + (symbol['version'] is None or (isinstance(symbol['version'], str) and symbol['version'])), 'Invalid native defined symbol') + reference = row['origin'] + require(reference in ('python', 'supplier') or (isinstance(reference, str) and reference.startswith('wheel:') and reference[6:] in names), 'Unknown native origin reference') + if reference.startswith('wheel:'): + require(next(r for r in wheels if r['name'] == reference[6:])['licenses'], 'Native wheel has no license evidence') + selection = value['libgccSelection']; keys(selection, ('path', 'sha256', 'byteLength', 'analysisTocSha256'), 'libgcc selection') + digest(selection['analysisTocSha256']) + actual_supplier = next((r for r in libraries if r['path'] == 'libgcc_s.so.1'), None) + require(actual_supplier is not None and actual_supplier['origin'] == 'supplier' and + all(selection[k] == actual_supplier[k] == supplier['library'][k] for k in ('path', 'sha256', 'byteLength')), + 'libgcc selection differs from supplier bytes') + verify_closure(libraries, target) + require(len(json.dumps(value, sort_keys=True).encode('utf-8')) <= 2 * 1024 * 1024, 'Native receipt exceeds size bound') + return value + + +def validate_linux_receipt(receipt): + """Validate Linux additions against the enclosing unchanged SDK receipt. + +Consumers separately hash the exact helper/notices bytes. The driver's pinned +input and before/after snapshots bind build custody; this pure check requires +all source, dependency and extracted-library claims to agree internally. +""" + require(receipt.get('platform') == 'linux', 'Linux native receipt expected') + target = {'aarch64': 'linux-arm64-gnu', 'x86_64': 'linux-x64-gnu'}.get(receipt.get('architecture')) + require(target is not None, 'Unsupported Linux receipt architecture') + require(isinstance(receipt.get('linuxBuildInputSha256'), str) and + re.fullmatch('[0-9a-f]{64}', receipt['linuxBuildInputSha256']), 'Missing Linux build input binding') + snapshot = receipt.get('linuxBuildSourceSnapshot') + keys(snapshot, ('sources', 'driverSha256', 'lockSha256'), 'Linux receipt source snapshot') + keys(snapshot['sources'], SOURCE_PATHS, 'Linux receipt source files') + require(all(isinstance(digest, str) and re.fullmatch('[0-9a-f]{64}', digest) + for digest in [*snapshot['sources'].values(), snapshot['driverSha256'], snapshot['lockSha256']]), + 'Invalid Linux receipt source digest') + sources = receipt.get('sources'); keys(sources, ('harnesses/agents-sdk/run.py', 'harnesses/agents-sdk/requirements-build.txt'), 'SDK receipt sources') + require(all(snapshot['sources'][name] == digest for name, digest in sources.items()), 'SDK source scopes disagree') + deps = validate_native_dependencies(receipt.get('nativeDependencies'), target) + packages = receipt.get('dependencies'); require(isinstance(packages, list), 'SDK dependencies are absent') + expected = {row['name']: (row['version'], row['wheelSha256']) for row in packages} + require(len(expected) == len(packages), 'Duplicate SDK dependencies') + actual = {row['name']: (row['version'], row['wheelCandidateSha256s']) for row in deps['origins']['wheels']} + require(actual == expected, 'Native wheel origins differ from locked SDK candidates') + runtime = receipt.get('linuxLibraries') + keys(runtime, ('schema', 'elfCount', 'requiredGlibcMaximum', 'libraries', 'modelCalls'), 'extracted Linux inspection') + raw = [{key: value for key, value in row.items() if key != 'origin'} for row in deps['libraries']] + require(runtime['schema'] == 'openprose.sdk-packaged-libraries/1' and type(runtime['elfCount']) is int and + runtime['elfCount'] == len(raw) and type(runtime['modelCalls']) is int and runtime['modelCalls'] == 0 and + runtime['libraries'] == raw, 'Extracted Linux inspection differs from native receipt') + maximum = max((tuple(map(int, row['maximumRequiredGlibc'].split('.'))) for row in raw + if row['maximumRequiredGlibc'] is not None), default=None) + require(maximum is not None and runtime['requiredGlibcMaximum'] == '.'.join(map(str, maximum)), + 'Extracted GLIBC aggregate differs from native receipt') + return deps diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py index d8a4ba0c..232cca54 100644 --- a/cli/ci/test_build_agents_sdk.py +++ b/cli/ci/test_build_agents_sdk.py @@ -149,3 +149,73 @@ def test_sdk_signing_rejects_non_developer_id_before_freezing(self): with self.assertRaisesRegex(ValueError, 'Developer ID'): sdk.build(Path(temp) / 'out', codesign_identity=identity) self.assertFalse((Path(temp) / 'out').exists()) + + def test_linux_hooks_are_paired_platform_scoped_and_required_before_output(self): + with tempfile.TemporaryDirectory() as temp: + for host, supplier, origin, reason in (('darwin', Path('/fixture'), Path('/origin'), 'Linux-only'), + ('linux', None, None, 'requires explicit'), ('linux', Path('/fixture'), None, 'supplied together')): + with self.subTest(host=host, reason=reason), mock.patch.object(sdk.sys, 'platform', host), \ + mock.patch.object(sdk.platform, 'python_version', return_value='3.10.20'), \ + mock.patch.object(sdk.platform, 'machine', return_value='x86_64'): + output = Path(temp) / 'out' + with self.assertRaisesRegex(ValueError, reason): + sdk.build(output, linux_libgcc=supplier, linux_native_origin=origin) + self.assertFalse(output.exists()) + + def test_linux_build_binds_toc_archive_extracted_bytes_and_source_snapshot(self): + from test_sdk_native_inventory import input_fixture + with tempfile.TemporaryDirectory() as temp: + root = Path(temp) + value, origin_path, supplier, source = input_fixture(root) + for relative in sdk.native.SOURCE_PATHS: + (source / relative).write_bytes((sdk.ROOT / relative).read_bytes()) + value['sourceSnapshot']['sources'] = {name: sdk.digest(source / name) for name in sdk.native.SOURCE_PATHS} + origin_path.write_text(json.dumps(value)) + rows = [sdk.native.elf_record('libgcc_s.so.1', supplier.read_bytes())] + report = {'schema': 'openprose.sdk-packaged-libraries/1', 'elfCount': 1, + 'requiredGlibcMaximum': '2.17', 'libraries': rows, 'modelCalls': 0} + executed = [] + def simulated_run(command, *, env, cwd, log, timeout=900): + executed.append((command, timeout)) + self.assertNotIn('OPENAI_API_KEY', env) + if '-m' in command: + self.assertIn('--add-binary', command) + index = command.index('--add-binary') + self.assertEqual(command[index + 1], str(supplier) + sdk.os.pathsep + '.') + self.assertEqual((cwd / 'sdk_native_inventory.py').read_bytes(), Path(sdk.native.__file__).read_bytes()) + (cwd / 'dist').mkdir(); (cwd / 'dist' / sdk.NAME).write_bytes(b'nonexecuted fixture helper') + work = cwd / 'work' / sdk.NAME; work.mkdir(parents=True) + (work / 'Analysis-00.toc').write_text(repr(([], [('libgcc_s.so.1', str(supplier), 'BINARY')]))) + log.write_text('simulated freeze') + elif '--packaged-self-test' in command: + log.write_text(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':'0.22.2','openai':'3.13.0','certificates':True,'modelCalls':0})) + elif '--packaged-tool-self-test' in command: + log.write_text(json.dumps({'schema':'openprose.sdk-packaged-tools-self-test/1','shellEffects':True,'boundedOutput':True,'shellCancellation':True,'mockedPublicRetrieval':True,'incompleteHttpRejected':True,'modelCalls':0,'networkUsed':False})) + elif '--version' in command: log.write_text('prose-agents-sdk 0.1.0\n') + elif '--packaged-library-test' in command: log.write_text(json.dumps(report)) + else: self.fail('Unexpected simulated command') + with mock.patch.object(sdk.sys, 'platform', 'linux'), \ + mock.patch.object(sdk.platform, 'machine', return_value='x86_64'), \ + mock.patch.object(sdk.platform, 'python_version', return_value='3.10.20'), \ + mock.patch.object(sdk, 'ROOT', source), mock.patch.object(sdk, 'SOURCE', source / 'harnesses/agents-sdk/run.py'), \ + mock.patch.object(sdk, 'LOCK', source / 'harnesses/agents-sdk/requirements-build.txt'), \ + mock.patch.object(sdk, 'lock_packages', return_value=[]), \ + mock.patch.object(sdk, 'installed_inventory', return_value=b'fixture notices'), \ + mock.patch.object(sdk.metadata, 'version', return_value='6.22.3'), \ + mock.patch.object(sdk, 'inspect_archive', return_value=rows), \ + mock.patch.object(sdk, 'run', side_effect=simulated_run), \ + mock.patch.object(sdk.shutil, 'disk_usage', return_value=mock.Mock(free=1024**3)): + output = root / 'out' + receipt = sdk.build(output, linux_libgcc=supplier, linux_native_origin=origin_path) + self.assertEqual(receipt['linuxBuildInputSha256'], sdk.digest(origin_path)) + self.assertEqual(receipt['linuxBuildSourceSnapshot'], value['sourceSnapshot']) + self.assertTrue(receipt['nativeDependencies']['symbolClosureVerified']) + self.assertEqual(receipt['nativeDependencies']['libraries'][0]['origin'], 'supplier') + self.assertIn(b'GCC RUNTIME LIBRARY EXCEPTION', (output / sdk.NOTICES).read_bytes()) + self.assertIn(b'gcc-8.5.0-fixture.src.rpm', (output / sdk.NOTICES).read_bytes()) + self.assertFalse((output / 'work').exists()) + self.assertEqual([timeout for command, timeout in executed if '--version' in command], [5]) + report['libraries'] = [] + with self.assertRaisesRegex(ValueError, 'membership differs'): + sdk.build(root / 'mismatch', linux_libgcc=supplier, linux_native_origin=origin_path) + self.assertFalse((root / 'mismatch' / sdk.RECEIPT).exists()) diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py new file mode 100644 index 00000000..087b5e61 --- /dev/null +++ b/cli/ci/test_sdk_native_inventory.py @@ -0,0 +1,270 @@ +import ast +import base64 +import copy +import hashlib +import json +from pathlib import Path +import struct +import sys +import tempfile +from types import SimpleNamespace +import unittest +from unittest import mock + +import sdk_native_inventory as native + + +def elf(*, imported=False, machine=62, version='GCC_3.0', symbol='_Unwind_Resume', glibc='2.17'): + strings = b'\0libgcc_s.so.1\0' + version.encode() + b'\0' + symbol.encode() + b'\0' + version_offset = 15; symbol_offset = version_offset + len(version) + 1 + symbols = bytes(24) + struct.pack(' Date: Wed, 7 Oct 2026 00:50:20 -0400 Subject: [PATCH 47/64] Qualify Linux SDK in pinned minimal runtime before packaging --- .github/workflows/cli-distribution-check.yml | 16 ++- .github/workflows/cli-kernel-rc.yml | 17 ++- cli/ci/agents-sdk-linux-runtime.lock.json | 25 ++++ cli/ci/build_kernel_rc.py | 7 + cli/ci/check_workflows.py | 20 ++- cli/ci/prepare_agents_sdk_linux.py | 77 +++++++++++ cli/ci/run_local.py | 2 + cli/ci/test_build_kernel_rc.py | 13 +- cli/ci/test_check_workflows.py | 14 ++ cli/ci/test_verify_agents_sdk_linux.py | 136 +++++++++++++++++++ cli/ci/verify_agents_sdk_linux.py | 78 +++++++++++ cli/protocol/OWNERSHIP.md | 3 + 12 files changed, 403 insertions(+), 5 deletions(-) create mode 100644 cli/ci/agents-sdk-linux-runtime.lock.json create mode 100644 cli/ci/prepare_agents_sdk_linux.py create mode 100644 cli/ci/test_verify_agents_sdk_linux.py create mode 100644 cli/ci/verify_agents_sdk_linux.py diff --git a/.github/workflows/cli-distribution-check.yml b/.github/workflows/cli-distribution-check.yml index 8067664a..c327f84d 100644 --- a/.github/workflows/cli-distribution-check.yml +++ b/.github/workflows/cli-distribution-check.yml @@ -60,6 +60,11 @@ jobs: python3 -m pip install --require-hashes --only-binary=:all: -r cli/ci/requirements-test.txt cd cli/bun bun install --frozen-lockfile --ignore-scripts + - name: Prepare the pinned full Linux SDK Python archive + if: runner.os == 'Linux' + env: + SDK_PLATFORM: ${{ runner.arch == 'ARM64' && 'linux-arm64-gnu' || 'linux-x64-gnu' }} + run: timeout --kill-after=1s 130s python3 cli/ci/prepare_agents_sdk_linux.py --platform "$SDK_PLATFORM" --out "$RUNNER_TEMP/agents-sdk-python-full.tar.zst" - name: Prepare the separately locked Agents SDK build interpreter run: | uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python" @@ -122,7 +127,7 @@ jobs: # qualification and reviewed publication remain separate gates. env: RC_VERSION: '0.15.0-rc.4' - run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" + run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" --agents-sdk-linux-python-archive "$RUNNER_TEMP/agents-sdk-python-full.tar.zst" - name: Fetch pinned genuine previous release inputs env: UPGRADE_PLATFORM: ${{ matrix.os == 'ubuntu-22.04' && 'linux-x64-gnu' || matrix.os == 'ubuntu-22.04-arm' && 'linux-arm64-gnu' || matrix.os == 'macos-15-intel' && 'darwin-x64' || 'darwin-arm64' }} @@ -166,6 +171,15 @@ jobs: ${{ runner.temp }}/kernel-rc/agents-sdk ${{ runner.temp }}/kernel-rc/package ${{ runner.temp }}/kernel-rc/logs + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*.json + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*.log + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*cleanup-error.txt + ${{ runner.temp }}/kernel-rc/agents-sdk-native/supplier + ${{ runner.temp }}/kernel-rc/agents-sdk-native/frozen + ${{ runner.temp }}/kernel-rc/agents-sdk-native/python-full/python/PYTHON.json + ${{ runner.temp }}/kernel-rc/agents-sdk-native/python-full/python/licenses + ${{ runner.temp }}/kernel-rc/agents-sdk-runtime/runtime-report.json + ${{ runner.temp }}/kernel-rc/agents-sdk-runtime/job/*.log ${{ runner.temp }}/kernel-rc/build-report.json ${{ runner.temp }}/kernel-rc/homebrew ${{ runner.temp }}/kernel-rc/installed-qualification diff --git a/.github/workflows/cli-kernel-rc.yml b/.github/workflows/cli-kernel-rc.yml index a19bb040..158619f1 100644 --- a/.github/workflows/cli-kernel-rc.yml +++ b/.github/workflows/cli-kernel-rc.yml @@ -67,6 +67,11 @@ jobs: python3 -m pip install --require-hashes --only-binary=:all: -r cli/ci/requirements-test.txt cd cli/bun bun install --frozen-lockfile --ignore-scripts + - name: Prepare the pinned full Linux SDK Python archive + if: runner.os == 'Linux' + env: + SDK_PLATFORM: ${{ runner.arch == 'ARM64' && 'linux-arm64-gnu' || 'linux-x64-gnu' }} + run: timeout --kill-after=1s 130s python3 cli/ci/prepare_agents_sdk_linux.py --platform "$SDK_PLATFORM" --out "$RUNNER_TEMP/agents-sdk-python-full.tar.zst" - name: Prepare the separately locked Agents SDK build interpreter run: | uv venv --python 3.10.20 --seed "$RUNNER_TEMP/agents-sdk-python" @@ -86,7 +91,7 @@ jobs: - name: Build and verify fresh standalone and npm installations env: RC_VERSION: ${{ inputs.version || '0.15.0-rc.4' }} - run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" + run: python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" --agents-sdk-linux-python-archive "$RUNNER_TEMP/agents-sdk-python-full.tar.zst" - name: Qualify actual SDK installations and genuine upgrades env: RC_VERSION: ${{ inputs.version || '0.15.0-rc.4' }} @@ -116,6 +121,16 @@ jobs: path: | ${{ runner.temp }}/kernel-rc/package ${{ runner.temp }}/kernel-rc/logs + ${{ runner.temp }}/kernel-rc/agents-sdk + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*.json + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*.log + ${{ runner.temp }}/kernel-rc/agents-sdk-native/*cleanup-error.txt + ${{ runner.temp }}/kernel-rc/agents-sdk-native/supplier + ${{ runner.temp }}/kernel-rc/agents-sdk-native/frozen + ${{ runner.temp }}/kernel-rc/agents-sdk-native/python-full/python/PYTHON.json + ${{ runner.temp }}/kernel-rc/agents-sdk-native/python-full/python/licenses + ${{ runner.temp }}/kernel-rc/agents-sdk-runtime/runtime-report.json + ${{ runner.temp }}/kernel-rc/agents-sdk-runtime/job/*.log ${{ runner.temp }}/kernel-rc/build-report.json ${{ runner.temp }}/kernel-rc/homebrew ${{ runner.temp }}/kernel-rc/installed-qualification diff --git a/cli/ci/agents-sdk-linux-runtime.lock.json b/cli/ci/agents-sdk-linux-runtime.lock.json new file mode 100644 index 00000000..32770245 --- /dev/null +++ b/cli/ci/agents-sdk-linux-runtime.lock.json @@ -0,0 +1,25 @@ +{ + "platforms": { + "linux-arm64-gnu": { + "dockerPlatform": "linux/arm64", + "machine": "aarch64", + "runtime": { + "configSha256": "81cb9b7fe37a4f8d3d51778736005c0f7d8619d6c3a0d9059935a5d88f368a86", + "image": "quay.io/almalinuxorg/9-minimal@sha256:7bc235d6597d8bcffd03a4e0ff120e44b733838cd3c16fa3a5ec4a49edecdaa5", + "indexDigest": "sha256:ae819e0517abfec9c71a3c37e78a281d18fba2341212fd96796d9fab489c797b", + "metadataUrl": "https://quay.io/api/v1/repository/almalinuxorg/9-minimal/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + } + }, + "linux-x64-gnu": { + "dockerPlatform": "linux/amd64", + "machine": "x86_64", + "runtime": { + "configSha256": "a9e621adbc87609e9f19bcffac9441b7089f242dc4fca3f6ff691ad8faad1e3f", + "image": "quay.io/almalinuxorg/9-minimal@sha256:dd354c60b7e383d1655699b897176f34e0210a2c5a43c3def6e766cb6d94c24e", + "indexDigest": "sha256:ae819e0517abfec9c71a3c37e78a281d18fba2341212fd96796d9fab489c797b", + "metadataUrl": "https://quay.io/api/v1/repository/almalinuxorg/9-minimal/tag/?limit=1&onlyActiveTags=true&specificTag=latest" + } + } + }, + "schema": "openprose.sdk-linux-runtime-inputs/1" +} diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index 04729a45..55563c10 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -137,6 +137,8 @@ def build_sdk(output, *, epoch, env, logs, agents_sdk_python=None, linux_python_ from build_agents_sdk_linux import build as build_linux stage = output / 'agents-sdk-native' build_linux(ROOT, stage, targets[platform.machine()], linux_python_archive, epoch=int(epoch)) + from verify_agents_sdk_linux import verify as verify_linux + verify_linux(stage / 'frozen', output / 'agents-sdk-runtime', targets[platform.machine()]) sdk_output.mkdir() for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): shutil.copy2(stage / 'frozen' / name, sdk_output / name) @@ -219,6 +221,11 @@ def check(binary, runner, label, node=None): 'publicationAuthorized': False, 'checks': [{'name': name, 'status': 'passed'} for name in ['built-bun', 'built-rust', 'installed-bun', 'installed-rust', 'installed-npm']], 'evidence': {str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} for directory in (logs, package) for p in sorted(directory.rglob('*')) if p.is_file()}} + if sys.platform.startswith('linux'): + native_evidence = output / 'agents-sdk-runtime' + report['evidence'].update({str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} + for p in sorted(native_evidence.rglob('*')) + if p.is_file() and not p.is_symlink() and 'payload' not in p.relative_to(native_evidence).parts}) (output / 'build-report.json').write_text(json.dumps(report, indent=2, sort_keys=True) + '\n') return report diff --git a/cli/ci/check_workflows.py b/cli/ci/check_workflows.py index 5ec7208d..e2ba7f9d 100644 --- a/cli/ci/check_workflows.py +++ b/cli/ci/check_workflows.py @@ -317,7 +317,7 @@ def require(condition: bool, message: str) -> None: ), "cli-kernel-rc.yml": ( "Build and verify fresh standalone and npm installations", - 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3"', + 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" --agents-sdk-linux-python-archive "$RUNNER_TEMP/agents-sdk-python-full.tar.zst"', ), }[name] require( @@ -328,6 +328,22 @@ def require(condition: bool, message: str) -> None: by_name[required_step]["run"] == command, "required qualification command drift", ) + if name in ("cli-kernel-rc.yml", "cli-distribution-check.yml"): + prefetch = by_name.get("Prepare the pinned full Linux SDK Python archive", {}) + require(prefetch.get("if") == "runner.os == 'Linux'" and + prefetch.get("env") == {"SDK_PLATFORM": "${{ runner.arch == 'ARM64' && 'linux-arm64-gnu' || 'linux-x64-gnu' }}"} and + prefetch.get("run") == 'timeout --kill-after=1s 130s python3 cli/ci/prepare_agents_sdk_linux.py --platform "$SDK_PLATFORM" --out "$RUNNER_TEMP/agents-sdk-python-full.tar.zst"', + "native Linux SDK preparation requires pinned archive and external transfer watchdog") + require(steps.index(prefetch) < steps.index(by_name["Prepare the separately locked Agents SDK build interpreter"]), + "prepare pinned full archive before native SDK construction") + retention = next(step for step in steps if step.get("uses", "").startswith("actions/upload-artifact@")) + native_retained = retention.get("with", {}).get("path", "").splitlines() + require(all("${{ runner.temp }}/kernel-rc/" + path in native_retained for path in + ("agents-sdk", "agents-sdk-native/*.json", "agents-sdk-native/*.log", + "agents-sdk-native/supplier", "agents-sdk-native/frozen", + "agents-sdk-native/python-full/python/PYTHON.json", "agents-sdk-native/python-full/python/licenses", + "agents-sdk-runtime/runtime-report.json", "agents-sdk-runtime/job/*.log")), + "retain actual native SDK construction/runtime facts and failures") if name == "cli-distribution-check.yml": sdk_build = by_name["Build and verify production SDK standalone and npm installations"] sdk_homebrew = by_name["Rehearse Homebrew against the production SDK archives"] @@ -335,7 +351,7 @@ def require(condition: bool, message: str) -> None: "production SDK installations must be qualified unconditionally") require(sdk_build.get("env") == {"RC_VERSION": "0.15.0-rc.4"} and sdk_build.get("run") == - 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3"', + 'python3 cli/ci/build_kernel_rc.py --version "$RC_VERSION" --out "$RUNNER_TEMP/kernel-rc" --agents-sdk-python "$RUNNER_TEMP/agents-sdk-python/bin/python3" --agents-sdk-linux-python-archive "$RUNNER_TEMP/agents-sdk-python-full.tar.zst"', "production SDK must use the native source-bound RC builder") require(sdk_homebrew.get("env") == {"RC_VERSION": "0.15.0-rc.4", "EXPECTED_SOURCE": "${{ github.sha }}"} and sdk_homebrew.get("run") == diff --git a/cli/ci/prepare_agents_sdk_linux.py b/cli/ci/prepare_agents_sdk_linux.py new file mode 100644 index 00000000..b9adf611 --- /dev/null +++ b/cli/ci/prepare_agents_sdk_linux.py @@ -0,0 +1,77 @@ +"""Prefetch the exact native full Python archive before offline SDK freezing. + +The transfer deadline is cooperative. Native workflows additionally enforce a +130-second GNU timeout with forced termination one second later. +""" +import argparse,hashlib,json,os,platform,time,sys +from pathlib import Path +from urllib.parse import urlsplit +from urllib.request import build_opener,ProxyHandler,HTTPRedirectHandler,Request +import build_agents_sdk_linux as driver + +HOSTS={'github.com','release-assets.githubusercontent.com','objects.githubusercontent.com'} +def checked_url(url): + u=urlsplit(url) + driver.require(u.scheme=='https' and u.hostname in HOSTS and u.port in (None,443) + and u.username is None and u.password is None and not u.fragment, + 'Unapproved Python artifact URL') + return url +class PinnedRedirects(HTTPRedirectHandler): + def redirect_request(self,req,fp,code,msg,headers,newurl): + checked_url(newurl) + return super().redirect_request(req,fp,code,msg,headers,newurl) + +def _fetch(target,destination,*,opener=None,clock=time.monotonic,timeout=120,phase): + phase[0]='selection' + row=driver.load_lock()['platforms'].get(target);driver.require(row is not None,'Unsupported native target') + driver.require(os.name=='posix' and platform.system()=='Linux' and platform.machine()==row['machine'],'Native Linux target required') + p=driver.checked_path(destination);driver.require(p.parent.is_dir() and not p.exists(),'Fresh owned archive destination required') + partial=driver.checked_path(p.with_name(p.name+'.part'));driver.require(not partial.exists(),'Fresh owned partial required') + r=row['pythonArchive'];checked_url(r['url']);deadline=clock()+timeout + # Explicit opener excludes proxy discovery, cookie jars and auth handlers. + opener=opener or build_opener(ProxyHandler({}),PinnedRedirects()) + h=hashlib.sha256();size=0 + phase[0]='opening' + with partial.open('xb') as out: + with opener.open(Request(r['url'],headers={'User-Agent':'OpenProse-pinned-build-input/1','Accept-Encoding':'identity'}),timeout=min(10,timeout)) as response: + checked_url(response.geturl());driver.require(response.status==200,'Python artifact HTTP response failed') + encoding=response.headers.get('Content-Encoding','identity');driver.require(encoding=='identity','Encoded artifact response refused') + length=response.headers.get('Content-Length') + driver.require(length is None or (length.isdigit() and int(length)==r['byteLength']),'Python artifact HTTP size differs from pin') + phase[0]='reading' + while True: + driver.require(clock() tuple[Gate, ...]: "test_npm_identity", "test_npm_same_name", "test_build_kernel_rc", + "test_build_agents_sdk", "test_build_agents_sdk_linux", + "test_verify_agents_sdk_linux", "test_sdk_native_inventory", "test_assemble_kernel_rc", "test_kernel_rc_evidence", diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index ca3e2fc4..5e60651b 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -142,6 +142,7 @@ def test_linux_requires_pin_and_never_uses_host_python(self): def test_native_linux_uses_driver_and_exact_frozen_trio(self): import build_agents_sdk_linux as driver + import verify_agents_sdk_linux as verifier with tempfile.TemporaryDirectory() as d: root = Path(d) def frozen(source, stage, target, archive, *, epoch): @@ -149,13 +150,23 @@ def frozen(source, stage, target, archive, *, epoch): (stage / 'frozen').mkdir(parents=True) for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): (stage / 'frozen' / name).write_bytes(name.encode()) - with patch.object(rc.sys, 'platform', 'linux'), patch.object(rc.platform, 'machine', return_value='aarch64'), patch.object(driver, 'build', side_effect=frozen), patch.object(rc, 'command') as command: + with patch.object(rc.sys, 'platform', 'linux'), patch.object(rc.platform, 'machine', return_value='aarch64'), patch.object(driver, 'build', side_effect=frozen), patch.object(verifier, 'verify') as verify, patch.object(rc, 'command') as command: actual = rc.build_sdk(root, epoch='123', env={}, logs=root, linux_python_archive=root / 'pinned.tar.zst') command.assert_not_called() + verify.assert_called_once_with(root / 'agents-sdk-native/frozen', root / 'agents-sdk-runtime', 'linux-arm64-gnu') self.assertEqual(len(list(actual.iterdir())), 3) for p in actual.iterdir(): self.assertEqual(p.read_bytes(), p.name.encode()) + def test_failed_clean_runtime_never_admits_sdk_payload(self): + import build_agents_sdk_linux as driver + import verify_agents_sdk_linux as verifier + with tempfile.TemporaryDirectory() as d, patch.object(rc.sys, 'platform', 'linux'), patch.object(rc.platform, 'machine', return_value='x86_64'), patch.object(driver, 'build'), patch.object(verifier, 'verify', side_effect=ValueError('runtime rejected')), patch.object(rc, 'command') as command: + root = Path(d) + with self.assertRaisesRegex(ValueError, 'runtime rejected'): + rc.build_sdk(root, epoch='0', env={}, logs=root, linux_python_archive=root / 'pinned') + command.assert_not_called(); self.assertFalse((root / 'agents-sdk').exists()) + def test_macos_keeps_separate_pinned_interpreter(self): with tempfile.TemporaryDirectory() as d, patch.object(rc.sys, 'platform', 'darwin'), patch.object(rc, 'command') as command: root = Path(d) diff --git a/cli/ci/test_check_workflows.py b/cli/ci/test_check_workflows.py index f10fd568..9e936cd1 100644 --- a/cli/ci/test_check_workflows.py +++ b/cli/ci/test_check_workflows.py @@ -286,6 +286,20 @@ def test_production_sdk_setup_and_runtime_cannot_be_bypassed(self): self.changed(name, lambda w, j, i=index, a=before, b=after: j["steps"][i].update( run=j["steps"][i]["run"].replace(a, b))) + def test_linux_sdk_pins_deadlines_and_failure_evidence_cannot_be_removed(self): + for name in ("cli-distribution-check.yml", "cli-kernel-rc.yml"): + steps = self.workflows[name]["jobs"][JOBS[name]]["steps"] + index = next(i for i, step in enumerate(steps) if step.get("name") == "Prepare the pinned full Linux SDK Python archive") + self.changed(name, lambda w, j, i=index: j["steps"].pop(i)) + self.changed(name, lambda w, j, i=index: j["steps"][i].update({"if": "false"})) + for before, after in (("timeout --kill-after=1s 130s ", ""), ("prepare_agents_sdk_linux.py", "echo"), + ("--platform", "--unpinned-platform")): + self.changed(name, lambda w, j, i=index, a=before, b=after: j["steps"][i].update(run=j["steps"][i]["run"].replace(a, b))) + retention = next(i for i, step in enumerate(steps) if step.get("uses", "").startswith("actions/upload-artifact@")) + for path in ("agents-sdk-native/supplier", "agents-sdk-native/frozen", "agents-sdk-native/*.log", + "agents-sdk-native/python-full/python/PYTHON.json", "agents-sdk-runtime/runtime-report.json"): + self.changed(name, lambda w, j, i=retention, p=path: j["steps"][i]["with"].update(path=j["steps"][i]["with"]["path"].replace("${{ runner.temp }}/kernel-rc/" + p, "omitted"))) + def test_distribution_production_sdk_custody_is_required(self): name = "cli-distribution-check.yml" steps = self.workflows[name]["jobs"][JOBS[name]]["steps"] diff --git a/cli/ci/test_verify_agents_sdk_linux.py b/cli/ci/test_verify_agents_sdk_linux.py new file mode 100644 index 00000000..46ee4b61 --- /dev/null +++ b/cli/ci/test_verify_agents_sdk_linux.py @@ -0,0 +1,136 @@ +import copy,hashlib,io,json,tempfile,unittest,sys,time +from urllib.error import HTTPError +from pathlib import Path +from unittest.mock import patch +import prepare_agents_sdk_linux as fetcher +import verify_agents_sdk_linux as verifier +import build_agents_sdk_linux as d + +class Response(io.BytesIO): + def __init__(self,data,url,headers=None,status=200):super().__init__(data);self.url=url;self.headers=headers or {};self.status=status + def geturl(self):return self.url +class Opener: + def __init__(self,response):self.response=response;self.requests=[] + def open(self,request,**kwargs):self.requests.append((request,kwargs));return self.response +class FetchControls(unittest.TestCase): + def fixture(self,data=b'pinned test bytes'): + lock=copy.deepcopy(d.load_lock());r=lock['platforms']['linux-x64-gnu']['pythonArchive'];r.update(byteLength=len(data),sha256=hashlib.sha256(data).hexdigest()) + return lock,r + def transfer(self,data=b'pinned test bytes',kind=None): + lock,r=self.fixture();url=r['url'] if kind!='redirect' else 'http://untrusted.invalid/artifact';headers={} + if kind=='length':headers['Content-Length']='999' + if kind=='encoding':headers['Content-Encoding']='gzip' + opener=Opener(Response(data,url,headers)) + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();out=root/'python-full.tar.zst' + with patch.object(d,'load_lock',return_value=lock),patch.object(fetcher.platform,'system',return_value='Linux'),patch.object(fetcher.platform,'machine',return_value='x86_64'): + if kind: + with self.assertRaises(ValueError):fetcher.fetch('linux-x64-gnu',out,opener=opener) + self.assertFalse(out.exists());self.assertLessEqual(out.with_name(out.name+'.part').stat().st_size,r['byteLength']) + else: + report=fetcher.fetch('linux-x64-gnu',out,opener=opener);self.assertEqual(out.read_bytes(),data);self.assertEqual(report['sha256'],r['sha256']) + return opener + def test_pinned_bounded_transfer(self):self.assertEqual(self.transfer().requests[0][1]['timeout'],10) + def test_bad_hash_or_length_and_encoding_refused(self): + for kind,data in [('hash',b'changed'),('overflow',b'x'*100),('length',b'pinned test bytes'),('encoding',b'pinned test bytes'),('redirect',b'pinned test bytes')]: + with self.subTest(kind=kind):self.transfer(data,kind) + def test_deadline_and_existing_output_refused(self): + lock,r=self.fixture() + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();out=root/'python';opener=Opener(Response(b'pinned test bytes',r['url'])) + with patch.object(d,'load_lock',return_value=lock),patch.object(fetcher.platform,'system',return_value='Linux'),patch.object(fetcher.platform,'machine',return_value='x86_64'): + with self.assertRaises(ValueError):fetcher.fetch('linux-x64-gnu',out,opener=opener,clock=iter([0,121]).__next__) + out.write_bytes(b'owned existing') + with self.assertRaises(ValueError):fetcher.fetch('linux-x64-gnu',out,opener=opener) + self.assertEqual(out.read_bytes(),b'owned existing') + def test_signed_http_error_diagnostic_redacts_url(self): + lock,r=self.fixture();signed='https://release-assets.githubusercontent.com/file?secret=signed-sentinel' + class FailedOpener: + def open(self,*a,**kw):raise HTTPError(signed,403,'signed-sentinel',{},None) + with tempfile.TemporaryDirectory() as t,patch.object(d,'load_lock',return_value=lock),patch.object(fetcher.platform,'system',return_value='Linux'),patch.object(fetcher.platform,'machine',return_value='x86_64'): + with self.assertRaises(fetcher.ArtifactPreparationError) as e:fetcher.fetch('linux-x64-gnu',Path(t).resolve()/'archive',opener=FailedOpener()) + self.assertEqual(str(e.exception),'Python artifact preparation failed during opening (HTTPError)') + self.assertNotIn('signed-sentinel',str(e.exception));self.assertNotIn('https://',str(e.exception)) + def test_actual_delayed_read_is_terminated_by_external_bounded_runner(self): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();(root/'home').mkdir();effect=root/'read-completed' + code="""import io,time,pathlib,hashlib +import sys;sys.path.insert(0, MODULE_DIRECTORY) +import prepare_agents_sdk_linux as f +import build_agents_sdk_linux as d +x=d.load_lock();r=x['platforms']['linux-x64-gnu']['pythonArchive'];r['byteLength']=4;r['sha256']=hashlib.sha256(b'test').hexdigest() +d.load_lock=lambda:x;f.platform.system=lambda:'Linux';f.platform.machine=lambda:'x86_64' +class Slow(io.BytesIO): + status=200;headers={} + def geturl(self):return r['url'] + def read(self,n): + if not getattr(self,'started',False): + self.started=True;pathlib.Path(STARTED).write_text('inside delayed read');time.sleep(1) + return super().read(n) +class Opener: + def open(self,*a,**kw):return Slow(b'test') +f.fetch('linux-x64-gnu',pathlib.Path(DEST),opener=Opener()) +pathlib.Path(EFFECT).write_text('completed') +""".replace('MODULE_DIRECTORY',repr(str(Path(fetcher.__file__).parent))).replace('DEST',repr(str(root/'archive'))).replace('EFFECT',repr(str(effect))).replace('STARTED',repr(str(root/'read-started'))) + with self.assertRaisesRegex(ValueError,'deadline'):d.process([sys.executable,'-c',code],root,'slow-read',timeout=.5) + self.assertTrue((root/'read-started').is_file());time.sleep(1.1);self.assertFalse(effect.exists());self.assertFalse((root/'archive').exists()) + def test_https_redirect_authority_is_closed(self): + for url in ('http://github.com/file','https://evil.invalid/file','https://user:password@github.com/file','https://github.com:444/file'): + with self.assertRaises(ValueError):fetcher.checked_url(url) + self.assertEqual(fetcher.checked_url('https://release-assets.githubusercontent.com/file?signed=fixture'),'https://release-assets.githubusercontent.com/file?signed=fixture') + +class RuntimeControls(unittest.TestCase): + def fixture(self,root): + build=root/'build';build.mkdir();header=bytearray(20);header[:6]=b'\x7fELF\x02\x01';header[18:20]=(62).to_bytes(2,'little');(build/verifier.NAMES[0]).write_bytes(header) + (build/verifier.NAMES[2]).write_text('fixture notices') + receipt={'platform':'linux','architecture':'x86_64','helper':{'sha256':d.sha(build/verifier.NAMES[0]),'byteLength':20}, + 'notices':{'sha256':d.sha(build/verifier.NAMES[2])},'nativeDependencies':{'symbolClosureVerified':True}, + 'selfTest':{'modelCalls':0},'toolSelfTest':{'modelCalls':0,'networkUsed':False},'linuxLibraries':{'requiredGlibcMaximum':'2.34'}} + (build/verifier.NAMES[1]).write_text(json.dumps(receipt));return build,receipt + def exercise(self,kind=None): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();build,receipt=self.fixture(root);out=root/'runtime';calls=[];row=verifier.load_lock()['platforms']['linux-x64-gnu'] + def fake(command,output,label,**kw): + calls.append((command,label,kw)) + if kind=='mutation' and label=='pull-runtime':(build/verifier.NAMES[2]).write_text('changed') + if label=='inspect-runtime':return json.dumps({'Id':'sha256:'+row['runtime']['configSha256'],'Os':'linux','Architecture':'arm64' if kind=='arch' else 'amd64','RepoDigests':[row['runtime']['image']]}).encode() + if label=='base':return (('glibc=2.35' if kind=='floor' else 'glibc=2.34')+'\narchitecture=x86_64\nnoBuildTools=true\n').encode() + if label=='version':return b'prose-agents-sdk 0.1.0\n' + if label in ('imports','tools','libraries'):return json.dumps(receipt[{'imports':'selfTest','tools':'toolSelfTest','libraries':'linuxLibraries'}[label]]).encode() + return b'' + with patch.object(verifier.sys,'platform','linux'),patch.object(verifier.platform,'machine',return_value='x86_64'),patch.object(verifier.shutil,'disk_usage') as disk,patch.object(d,'run',side_effect=fake),patch.object(verifier.inventory,'validate_linux_receipt') as validate: + disk.return_value.free=8*1024**3 + if kind: + with self.assertRaises(ValueError):verifier.verify(build,out,'linux-x64-gnu') + self.assertFalse((out/'runtime-report.json').exists()) + else: + report=verifier.verify(build,out,'linux-x64-gnu');self.assertFalse(report['cpuFloorQualified']);self.assertFalse(report['publicationAuthorized']) + self.assertEqual(report['inputs']['payload'][verifier.NAMES[0]],receipt['helper']['sha256']) + self.assertEqual(report['networkUsageScope'],'runtime-probes-only');self.assertTrue(report['preparationNetworkEnabled']) + validate.assert_called_once_with(receipt) + return calls + def test_exact_native_runtime_orchestration_and_five_second_version(self): + calls=self.exercise();self.assertEqual([x[1] for x in calls],['pull-runtime','inspect-runtime','base','version','imports','tools','libraries']) + for command,label,kwargs in calls[2:]: + self.assertIn('--network=none',command);self.assertEqual(command.count('--mount'),2) + self.assertIn('--read-only',command);self.assertEqual(kwargs['timeout'],25 if label=='version' else 50) + source_mount=next(c for c in command if c.startswith('type=bind,') and ',dst=/source,readonly' in c) + job_mount=next(c for c in command if c.startswith('type=bind,') and c.endswith(',dst=/job')) + self.assertIn('/runtime/payload,dst=/source,readonly',source_mount) + self.assertIn('/runtime/job,dst=/job',job_mount) + self.assertNotIn('/runtime,dst=/job',job_mount) + def test_wrong_architecture_floor_and_payload_mutation_refused(self): + for kind in ('arch','floor','mutation'): + with self.subTest(kind=kind):self.exercise(kind) + def test_invalid_native_receipt_is_rejected_before_docker_or_output(self): + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();build,receipt=self.fixture(root) + with patch.object(verifier.sys,'platform','linux'),patch.object(verifier.platform,'machine',return_value='x86_64'),patch.object(verifier.shutil,'disk_usage') as disk,patch.object(d,'run') as run: + disk.return_value.free=8*1024**3 + with self.assertRaises(ValueError):verifier.verify(build,root/'runtime','linux-x64-gnu') + run.assert_not_called();self.assertFalse((root/'runtime').exists()) + def test_runtime_not_build_image_and_no_cpu_claim(self): + self.assertIn('python3',verifier.BASE);self.assertIn('gcc',verifier.BASE) + for row in verifier.load_lock()['platforms'].values():self.assertIn('almalinuxorg/9-minimal@sha256:',row['runtime']['image']) + +if __name__=='__main__':unittest.main() diff --git a/cli/ci/verify_agents_sdk_linux.py b/cli/ci/verify_agents_sdk_linux.py new file mode 100644 index 00000000..01a9238e --- /dev/null +++ b/cli/ci/verify_agents_sdk_linux.py @@ -0,0 +1,78 @@ +"""Verify the exact frozen SDK in a native minimal GNU 2.34 runtime.""" +import argparse,hashlib,json,os,platform,re,shutil,sys +from pathlib import Path +import build_agents_sdk_linux as driver +import sdk_native_inventory as inventory +LOCK=Path(__file__).with_name('agents-sdk-linux-runtime.lock.json') +NAMES=('prose-agents-sdk','agents-sdk-build.json','AGENTS-SDK-NOTICES.txt') +PROBES=(('version','--version',5),('imports','--packaged-self-test',30), + ('tools','--packaged-tool-self-test',30),('libraries','--packaged-library-test',30)) +BASE='''test "$(getconf GNU_LIBC_VERSION)" = 'glibc 2.34' +test "$(uname -m)" = '{machine}' +for tool in python python3 pip pip3 gcc g++ cc clang make rustc cargo bun node; do + if command -v "$tool" >/dev/null 2>&1; then echo "Unexpected build/interpreter tool: $tool" >&2; exit 1; fi +done +printf 'glibc=2.34\\narchitecture={machine}\\nnoBuildTools=true\\n' +''' +def load_lock(): + x=json.loads(LOCK.read_text());driver.require(set(x)=={'schema','platforms'} and x['schema']=='openprose.sdk-linux-runtime-inputs/1','Closed runtime lock required') + driver.require(set(x['platforms'])=={'linux-x64-gnu','linux-arm64-gnu'},'Closed runtime targets') + for target,machine,arch in [('linux-x64-gnu','x86_64','amd64'),('linux-arm64-gnu','aarch64','arm64')]: + r=x['platforms'][target];v=r['runtime'] + driver.require(set(r)=={'machine','dockerPlatform','runtime'} and r['machine']==machine and r['dockerPlatform']=='linux/'+arch, + 'Runtime native target mismatch') + driver.require(set(v)=={'image','configSha256','metadataUrl','indexDigest'} and re.fullmatch('quay.io/almalinuxorg/9-minimal@sha256:[0-9a-f]{64}',v['image']) + and re.fullmatch('[0-9a-f]{64}',v['configSha256']) and re.fullmatch('sha256:[0-9a-f]{64}',v['indexDigest']), 'Pinned minimal runtime required') + return x + +def snapshot(build): + files={name:driver.checked_path(build/name) for name in NAMES} + driver.require(all(p.is_file() and 0=2*1024**3,'Runtime verification needs2GiB free') + initial=snapshot(build);receipt=json.loads((build/NAMES[1]).read_text()) + inventory.validate_linux_receipt(receipt) + driver.require(receipt['helper']['sha256']==initial['payload'][NAMES[0]] and receipt['helper']['byteLength']==(build/NAMES[0]).stat().st_size + and receipt['platform']=='linux' and receipt['architecture']==row['machine'] and receipt['notices']['sha256']==initial['payload'][NAMES[2]] + and receipt.get('nativeDependencies',{}).get('symbolClosureVerified') is True,'Frozen native receipt custody mismatch') + with (build/NAMES[0]).open('rb') as stream:header=stream.read(20) + driver.require(header[:6]==b'\x7fELF\x02\x01' and int.from_bytes(header[18:20],'little')==({'x86_64':62,'aarch64':183}[row['machine']]),'Actual helper ELF architecture differs') + out.mkdir();job=out/'job';job.mkdir();(job/'home').mkdir();(job/'docker-config').mkdir();payload=out/'payload';payload.mkdir() + for name in NAMES:shutil.copyfile(build/name,payload/name);(payload/name).chmod(0o555 if name==NAMES[0] else 0o444) + def stable(): + driver.require(snapshot(build)==initial,'Input trio/runtime driver/lock/lifecycle changed') + driver.require({name:driver.sha(driver.checked_path(payload/name)) for name in NAMES}==initial['payload'],'Copied frozen trio changed') + def run(command,label,**kwargs): + stable() + try:return driver.run(command,job,label,**kwargs) + finally:stable() + stable();image=row['runtime']['image'];prefix=['docker','--config',str(job/'docker-config')] + run(prefix+['pull','--platform',row['dockerPlatform'],image],'pull-runtime') + info=json.loads(run(prefix+['image','inspect','--format','{{json .}}',image],'inspect-runtime',timeout=15,max_bytes=65536));driver.validate_image(info,row,'runtime') + results={};all_probes=[('base',BASE.format(machine=row['machine']),30)]+[(name,'timeout --kill-after=1s '+str(seconds)+'s /source/prose-agents-sdk '+flag+'\n',seconds) for name,flag,seconds in PROBES] + for label,script,seconds in all_probes: + (job/(label+'.sh')).write_text(script) + c=driver.container_command(row,'runtime',payload,job,label+'.sh',network='none') + raw=run(c,label,timeout=seconds+20,max_bytes=2*1024**2) + if label=='base':driver.require(raw.decode()=='glibc=2.34\narchitecture='+row['machine']+'\nnoBuildTools=true\n','Actual runtime facts mismatch') + elif label=='version':driver.require(raw.decode().strip()=='prose-agents-sdk 0.1.0','Cold version probe differs') + else: + result=json.loads(raw);expected=receipt[{'imports':'selfTest','tools':'toolSelfTest','libraries':'linuxLibraries'}[label]] + driver.require(result==expected,'Runtime '+label+' differs from exact construction receipt');results[label]=result + if label=='libraries':driver.require(tuple(int(v) for v in result['requiredGlibcMaximum'].split('.'))<=(2,34),'Actual runtime requires newer GNU libc') + stable();report={'schema':'openprose.sdk-linux-clean-runtime/1','platform':target,'runtime':row['runtime'],'inputs':initial, + 'results':results,'version':'prose-agents-sdk 0.1.0','modelCalls':0,'networkUsed':False, + 'networkUsageScope':'runtime-probes-only','preparationNetworkEnabled':True,'cpuFloorQualified':False, + 'qualification':'native-clean-glibc-2.34-only','publicationAuthorized':False} + (out/'runtime-report.json').write_text(json.dumps(report,indent=2,sort_keys=True)+'\n');return report + +if __name__=='__main__': + p=argparse.ArgumentParser(description=__doc__);p.add_argument('--sdk-build',type=Path,required=True);p.add_argument('--out',type=Path,required=True);p.add_argument('--platform',required=True) + a=p.parse_args();print(json.dumps(verify(a.sdk_build,a.out,a.platform),sort_keys=True)) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 692aacd0..bc293fba 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1098,3 +1098,6 @@ IMP-098 native Linux driver integration: root owns new `cli/ci/build_agents_sdk_ IMP-098 frozen native receipt consumer wave: `/root/sdk_assessment` owns only `cli/ci/package_local.py`, `cli/ci/test_package_local.py`, `cli/ci/kernel_rc_evidence.py`, `cli/ci/test_assemble_kernel_rc.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py`. Consume the configuration lane's pure `validate_linux_receipt` and concrete synthetic interface example. Keep the existing twelve manifest identity fields; require complete native additions on Linux and reject them on macOS. Verify actual builder CArchive/runtime rows and exact source snapshot before packaging, central receipt custody for every downstream consumer, and separate measured shipped native SBOM components from declared provider extension licenses and locked wheel candidates. No dependency lock schema changes or new selected-wheel claims. Exact component omission/substitution/duplicate/source/license poison tests; preserve existing Mac acceptance. No shared inspection-module edits without owner/root coordination, Git, heavy builds, Docker, providers or credential access. Root retains independent review, integration and full/native admission. + + +IMP-098 reviewed clean Linux runtime/preparation integration: root owns new `cli/ci/prepare_agents_sdk_linux.py`, `cli/ci/verify_agents_sdk_linux.py`, `cli/ci/test_verify_agents_sdk_linux.py`, and `cli/ci/agents-sdk-linux-runtime.lock.json`. Apply independently reviewed scratch v2; mount payload read-only outside the writable job mount, sanitize network exceptions, enforce exact archive hash/length/native target under the workflow's external GNU timeout, validate the full closed Linux receipt before runtime verification, and bind the executed inventory module. Actual minimal GNU 2.34 probes retain five/thirty-second deadlines, separate preparation network use and CPU qualification nonclaims. Root owns native build invocation/evidence integration. No Docker or provider execution by agents; all native/full checks remain pending. From 9f9575b80828c48a9e958a907c43e3a3b97d0f86 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 00:51:04 -0400 Subject: [PATCH 48/64] Bind Linux SDK native inventory to package custody and SBOM --- cli/benchmarks/installed/benchmark.py | 26 ++++++- cli/benchmarks/installed/test_benchmark.py | 39 ++++++++++ cli/ci/kernel_rc_evidence.py | 61 ++++++++++++++++ cli/ci/package_local.py | 17 ++++- cli/ci/test_assemble_kernel_rc.py | 83 ++++++++++++++++++++++ cli/ci/test_package_local.py | 44 ++++++++++++ 6 files changed, 268 insertions(+), 2 deletions(-) diff --git a/cli/benchmarks/installed/benchmark.py b/cli/benchmarks/installed/benchmark.py index c94983ae..01ae0778 100644 --- a/cli/benchmarks/installed/benchmark.py +++ b/cli/benchmarks/installed/benchmark.py @@ -1288,7 +1288,7 @@ def verify_package_output( artifacts, dependency_digest, ) - if release['mode'] == 'release': + if release['mode'] in ('release', 'kernel-rc'): import sys ci_path = str(Path(__file__).resolve().parents[2] / 'ci') if ci_path not in sys.path: @@ -1318,6 +1318,30 @@ def verify_package_output( matches = [p for p in sbom['components'] if p.get('name') == 'prose-agents-sdk'] if len(matches) != 1 or matches[0].get('hashes') != [{'alg': 'SHA-256', 'content': sdk['sha256']}]: fail('IDENTITY_DIVERGENCE', 'SBOM lacks exact frozen SDK helper binding') + native_receipts = [] + for name, artifact in artifacts.items(): + if artifact['kind'] not in ('standalone-archive', 'npm-platform'): + continue + members = decode_archive_members(encoded[name], name) + native_receipts.extend(json_no_duplicates(data, 'SDK receipt') for path, (data, mode) in members.items() + if path.endswith('/agents-sdk-build.json')) + try: + if len(native_receipts) != 3 or any(row != native_receipts[0] for row in native_receipts[1:]): + raise ValueError('SDK native receipts differ between payloads') + custody.validate_sdk_native_sbom(sbom['components'], native_receipts[0]) + except (ValueError, KeyError, TypeError) as error: + fail('IDENTITY_DIVERGENCE', str(error)) + else: + # Native components cannot appear without an artifact-bound production receipt. + import sys + ci_path = str(Path(__file__).resolve().parents[2] / 'ci') + if ci_path not in sys.path: + sys.path.insert(0, ci_path) + import kernel_rc_evidence as custody + try: + custody.validate_sdk_native_sbom(json_no_duplicates(encoded['sbom.cdx.json'], 'sbom.cdx.json')['components']) + except (ValueError, KeyError, TypeError) as error: + fail('IDENTITY_DIVERGENCE', str(error)) return { "root": package_output, "purpose": purpose, diff --git a/cli/benchmarks/installed/test_benchmark.py b/cli/benchmarks/installed/test_benchmark.py index 9edb5080..20a1e994 100644 --- a/cli/benchmarks/installed/test_benchmark.py +++ b/cli/benchmarks/installed/test_benchmark.py @@ -432,6 +432,16 @@ def refresh_evidence(output: Path) -> None: sbom_path = output / 'sbom.cdx.json'; sbom = json.loads(sbom_path.read_text()) sbom['components'].append({'type': 'file', 'name': 'prose-agents-sdk', 'hashes': [{'alg': 'SHA-256', 'content': sdk['sha256']}]}) + import kernel_rc_evidence as custody + native_receipts = [] + for artifact in release['artifacts']: + if artifact['kind'] == 'standalone-archive': + with tarfile.open(output / artifact['path']) as archive: + native_receipts = [json.loads(archive.extractfile(m).read()) for m in archive.getmembers() + if m.name.endswith('/agents-sdk-build.json')] + break + if native_receipts: + sbom['components'].extend(custody.sdk_native_sbom_components(native_receipts[0])) sbom_path.write_text(json.dumps(sbom)) provenance_path = output / 'provenance.json'; provenance = json.loads(provenance_path.read_text()) definition = provenance['predicate']['buildDefinition'] @@ -2014,5 +2024,34 @@ def test_bun_runtime_evidence_is_platform_exact_and_closed(self) -> None: self.assertEqual(rejected.exception.code, "IDENTITY_DIVERGENCE") +class NativeSdkSbomCustodyTests(unittest.TestCase): + def test_artifact_native_sbom_poison_is_rejected_after_rehash(self): + with tempfile.TemporaryDirectory() as directory: + output = make_release_package_output(Path(directory), platform_value='linux-x64-gnu') + BENCHMARK.verify_package_output(output, expected_platform='linux-x64-gnu', purpose='release-invariants') + sbom_path = output / 'sbom.cdx.json' + original = json.loads(sbom_path.read_text()) + native = [row for row in original['components'] if any(p.get('value') == 'packaged-sdk-native-file' + for p in row.get('properties', []))] + self.assertEqual(len(native), 1) + for kind in ('omission', 'duplicate', 'license'): + sbom = copy.deepcopy(original) + if kind == 'omission': sbom['components'].remove(native[0]) + elif kind == 'duplicate': sbom['components'].append(native[0]) + else: + origin = next(row for row in sbom['components'] if any(p.get('value') == 'packaged-sdk-native-origin' + for p in row.get('properties', []))) + origin['properties'][-1]['value'] = '{}' + sbom_path.write_text(json.dumps(sbom)) + # Rehash the altered artifact without regenerating its poisoned SBOM. + paths = sorted(path for path in output.iterdir() if path.name != 'SHA256SUMS') + (output / 'SHA256SUMS').write_text( + ''.join(f'{digest(path.read_bytes())} {path.name}\n' for path in paths), 'utf-8') + with self.subTest(kind=kind), self.assertRaises(BENCHMARK.BenchmarkError) as rejected: + BENCHMARK.verify_package_output(output, expected_platform='linux-x64-gnu', purpose='release-invariants') + self.assertEqual(rejected.exception.code, 'IDENTITY_DIVERGENCE') + self.assertEqual(rejected.exception.message, 'Native SDK SBOM differs from bound receipt') + + if __name__ == "__main__": unittest.main() diff --git a/cli/ci/kernel_rc_evidence.py b/cli/ci/kernel_rc_evidence.py index adba8745..5028d09c 100644 --- a/cli/ci/kernel_rc_evidence.py +++ b/cli/ci/kernel_rc_evidence.py @@ -33,6 +33,65 @@ def asset_name(platform, relative, artifacts): 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} +LINUX_RECEIPT_KEYS = {'linuxBuildInputSha256', 'linuxBuildSourceSnapshot', 'nativeDependencies'} +NATIVE_SBOM_KINDS = {'packaged-sdk-native-file', 'packaged-sdk-native-origin'} + + +def validate_sdk_native_receipt(receipt): + """Pure artifact-consumer validation; never imports a freezer or executes payloads.""" + if receipt.get('platform') == 'linux': + import sdk_native_inventory + return sdk_native_inventory.validate_linux_receipt(receipt) + require(not LINUX_RECEIPT_KEYS.intersection(receipt), 'Non-Linux receipt contains Linux native additions') + return None + + +def sdk_native_sbom_components(receipt): + """Project shipped ELF bytes and their referenced origins, never lock candidates as downloads.""" + import json + native = validate_sdk_native_receipt(receipt) + if native is None: + return [] + canonical = lambda value: json.dumps(value, sort_keys=True, separators=(',', ':')) + components = [] + references = set() + for row in native['libraries']: + references.add(row['origin']) + components.append({'type': 'file', 'bom-ref': 'openprose:sdk-native:file:' + row['sha256'] + ':' + + hashlib.sha256(row['path'].encode()).hexdigest(), 'name': row['path'], + 'hashes': [{'alg': 'SHA-256', 'content': row['sha256']}], + 'properties': [{'name': 'openprose:kind', 'value': 'packaged-sdk-native-file'}, + {'name': 'openprose:native-record', 'value': canonical(row)}]}) + for reference in sorted(references): + if reference == 'supplier': + origin = native['origins']['supplier']; name = origin['package']['name'] + version = origin['package']['version'] + elif reference == 'python': + # Provider declarations describe the full distribution, not shipped extensions. + origin = {key: value for key, value in native['origins']['python'].items() + if key != 'declaredExtensionLicenses'} + name = 'CPython'; version = receipt['python'] + else: + origin = next(row for row in native['origins']['wheels'] if row['name'] == reference[6:]) + name = origin['name']; version = origin['version'] + components.append({'type': 'library', 'bom-ref': 'openprose:sdk-native:origin:' + reference, + 'name': name, 'version': version, + 'properties': [{'name': 'openprose:kind', 'value': 'packaged-sdk-native-origin'}, + {'name': 'openprose:origin-reference', 'value': reference}, + {'name': 'openprose:origin-evidence', 'value': canonical(origin)}]}) + return components + + +def validate_sdk_native_sbom(components, receipt=None): + """Require the complete, unique artifact-specific projection, including origin/license bindings.""" + import json + observed = [row for row in components if isinstance(row, dict) and (str(row.get('bom-ref', '')).startswith('openprose:sdk-native:') or any( + isinstance(p, dict) and p.get('name') == 'openprose:kind' and p.get('value') in NATIVE_SBOM_KINDS + for p in row.get('properties', [])))] + expected = sdk_native_sbom_components(receipt) if receipt is not None else [] + encode = lambda rows: sorted(json.dumps(row, sort_keys=True, separators=(',', ':')) for row in rows) + require(encode(observed) == encode(expected), 'Native SDK SBOM differs from bound receipt') + def validate_sdk_members(manifest, members, prefix, platform): """Bind the frozen helper and its source/dependency receipt without executing it.""" @@ -87,11 +146,13 @@ def validate_sdk_members(manifest, members, prefix, platform): 'SDK dependency receipt is malformed') seen.add(package['name']) if os_name == 'linux': + validate_sdk_native_receipt(receipt) libraries = receipt.get('linuxLibraries', {}) maximum = libraries.get('requiredGlibcMaximum') if isinstance(libraries, dict) else None require(isinstance(maximum, str) and re.fullmatch(r'[0-9]+\.[0-9]+(?:\.[0-9]+)?', maximum) and tuple(map(int, maximum.split('.'))) + (0,) * (3 - len(maximum.split('.'))) <= (2, 34, 0), 'SDK Linux glibc floor differs') else: + validate_sdk_native_receipt(receipt) require(receipt.get('linuxLibraries') == 'not-applicable', 'SDK platform library receipt differs') return sdk diff --git a/cli/ci/package_local.py b/cli/ci/package_local.py index 466b1b07..5d49c818 100644 --- a/cli/ci/package_local.py +++ b/cli/ci/package_local.py @@ -24,6 +24,7 @@ import time import uuid from typing import Any, Iterable +import kernel_rc_evidence as sdk_custody ROOT = Path(__file__).resolve().parents[2] @@ -2471,6 +2472,15 @@ def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, ("harnesses/agents-sdk/run.py", "harnesses/agents-sdk/requirements-build.txt")} if receipt.get("sources") != expected_sources: raise PackageError("SDK build sources differ from current package source") + try: + sdk_custody.validate_sdk_native_receipt(receipt) + if expected_os == 'linux': + snapshot = receipt['linuxBuildSourceSnapshot'] + actual_sources = {path: sha256_file(ROOT / path) for path in snapshot['sources']} + if actual_sources != snapshot['sources'] or snapshot['driverSha256'] != sha256_file(ROOT / 'cli/ci/build_agents_sdk_linux.py') or snapshot['lockSha256'] != sha256_file(ROOT / 'cli/ci/agents-sdk-linux-build.lock.json'): + raise PackageError('SDK native build source snapshot differs from current source') + except (ValueError, KeyError, TypeError) as error: + raise PackageError('SDK native receipt differs: ' + str(error)) from error helper, length, digest = snapshot_binary(directory / SDK_NAME, snapshots / SDK_NAME, "Agents SDK") if receipt.get("helper") != {"path": SDK_NAME, "sha256": digest, "byteLength": length}: raise PackageError("SDK helper differs from build receipt") @@ -2498,6 +2508,8 @@ def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, if tool_test.returncode != 0 or tool_test.stderr or strict_json_object(tool_test.stdout, "SDK tool self-test") != expected_tools or receipt.get('toolSelfTest') != expected_tools: raise PackageError("Packaged SDK tool self-test differs") if expected_os == "linux": + # The source-bound builder inspects the CArchive before recording it. + # Independently compare actual frozen extraction here without a freezer dependency. inspected = run_bounded([str(helper), "--packaged-library-test"], cwd=snapshots, environment=environment, timeout_seconds=30, label="packaged SDK libraries") if inspected.returncode != 0 or inspected.stderr: @@ -2906,7 +2918,10 @@ def build( "hashes": [{"alg": "SHA-256", "content": sdk_record["sha256"]}], "properties": [{"name": "openprose:kind", "value": "packaged-agents-sdk-helper"}]}] if isinstance(sdk_record, dict) else []) - + dependency_sbom_components(dependency_report), + + dependency_sbom_components(dependency_report) + + (sdk_custody.sdk_native_sbom_components( + strict_json_object(next(data for name, data, _ in sdk_members if name == SDK_RECEIPT), 'SDK receipt')) + if sdk_members else []), "properties": [ { "name": "openprose:dependency-inventory", diff --git a/cli/ci/test_assemble_kernel_rc.py b/cli/ci/test_assemble_kernel_rc.py index af84b28a..2e457753 100644 --- a/cli/ci/test_assemble_kernel_rc.py +++ b/cli/ci/test_assemble_kernel_rc.py @@ -11,6 +11,27 @@ import kernel_rc_evidence as custody +def linux_receipt_fixture(platform='linux-x64-gnu'): + """Synthetic ELF/origin proof assembled locally; no native build/download claim.""" + import sdk_native_inventory as native + from test_sdk_native_inventory import input_fixture, elf + with tempfile.TemporaryDirectory() as directory: + value, path, library, source = input_fixture(Path(directory), platform) + machine = 62 if platform == 'linux-x64-gnu' else 183 + raw = native.elf_record('libgcc_s.so.1', elf(machine=machine)) + rows, origins = native.assign_origins([raw], value, [], lambda _: None) + origins['wheels'] = [{'name': 'fixture', 'version': '1.0.0', 'wheelCandidateSha256s': ['e' * 64], + 'recordSha256': 'a' * 64, 'licenses': []}] + snapshot = value['sourceSnapshot'] + snapshot['sources']['harnesses/agents-sdk/run.py'] = 'f' * 64 + snapshot['sources']['harnesses/agents-sdk/requirements-build.txt'] = 'e' * 64 + return {'linuxBuildInputSha256': native.sha(('synthetic-native-input-' + platform).encode()), 'linuxBuildSourceSnapshot': snapshot, + 'nativeDependencies': {'libraries': rows, 'symbolClosureVerified': True, 'origins': origins, + 'libgccSelection': {**native.record('libgcc_s.so.1', library.read_bytes()), 'analysisTocSha256': 'a' * 64}}, + 'linuxLibraries': {'schema': 'openprose.sdk-packaged-libraries/1', 'elfCount': 1, + 'requiredGlibcMaximum': raw['maximumRequiredGlibc'], 'libraries': [raw], 'modelCalls': 0}} + + def sdk_fixture(platform): """Hermetic bytes with production-shaped custody; never executable release evidence.""" helper = ('fixture-sdk-' + platform).encode(); notices = b'fixture-notices' @@ -31,6 +52,8 @@ def sdk_fixture(platform): 'modelCalls': 0, 'publicationAuthorized': False, 'authority': 'hermetic-test-fixture-not-release-evidence', 'linuxLibraries': {'requiredGlibcMaximum': '2.34'} if os_name == 'linux' else 'not-applicable'} + if os_name == 'linux': + receipt.update(linux_receipt_fixture(platform)) encoded = json.dumps(receipt).encode(); sdk['receiptSha256'] = sha(encoded) return sdk, [('prose-agents-sdk', helper, 0o755), ('agents-sdk-build.json', encoded, 0o644), ('AGENTS-SDK-NOTICES.txt', notices, 0o644)] @@ -155,5 +178,65 @@ def test_live_evidence_must_bind_exact_both_binary_bytes(self): a.assemble(self.roots, self.root/'changed-live', self.evidence, path) +class NativeReceiptConsumerTests(unittest.TestCase): + def receipt(self, platform='linux-x64-gnu'): + sdk, members = sdk_fixture(platform) + return json.loads(next(data for name, data, mode in members if name == 'agents-sdk-build.json')) + + def test_complete_native_projection_and_nonshipped_declarations(self): + receipt = self.receipt() + components = custody.sdk_native_sbom_components(receipt) + self.assertEqual([r['name'] for r in components], ['libgcc_s.so.1', 'libgcc']) + self.assertFalse(any('fixture' == r['name'] for r in components)) + custody.validate_sdk_native_sbom(components, receipt) + self.assertEqual(custody.sdk_native_sbom_components(self.receipt('darwin-arm64')), []) + + def test_receipt_omission_source_origin_and_runtime_poison(self): + import copy + mutations = [lambda r: r.pop('nativeDependencies'), + lambda r: r['linuxBuildSourceSnapshot']['sources'].pop('cli/ci/sdk_native_inventory.py'), + lambda r: r['sources'].update({'harnesses/agents-sdk/run.py': 'b' * 64}), + lambda r: r['nativeDependencies']['libraries'][0].update(origin='wheel:missing'), + lambda r: r['nativeDependencies']['origins']['supplier']['library'].update(sha256='b' * 64), + lambda r: r['linuxLibraries'].update(elfCount=2)] + for mutate in mutations: + receipt = copy.deepcopy(self.receipt()); mutate(receipt) + with self.subTest(mutate=mutate), self.assertRaises((ValueError, KeyError, TypeError)): + custody.validate_sdk_native_receipt(receipt) + mac = self.receipt('darwin-arm64'); mac['nativeDependencies'] = {} + with self.assertRaises(ValueError): custody.validate_sdk_native_receipt(mac) + + def test_python_and_wheel_origins_do_not_promote_declarations_or_candidates(self): + import copy + receipt = self.receipt(); native = receipt['nativeDependencies'] + python = native['origins']['python'] + python['declaredExtensionLicenses'] = [{'name': 'unshipped-extension', 'licenses': ['fixture'], 'licensePaths': ['LICENSE']}] + wheel = native['origins']['wheels'][0] + wheel['licenses'] = [{'path': 'fixture.dist-info/licenses/LICENSE', 'sha256': 'b' * 64, 'byteLength': 3}] + for path, origin, digest in [('libpython.so', 'python', 'c' * 64), ('fixture.so', 'wheel:fixture', 'd' * 64)]: + row = copy.deepcopy(native['libraries'][0]); row.update(path=path, origin=origin, sha256=digest) + native['libraries'].append(row) + native['libraries'].sort(key=lambda row: row['path']) + receipt['linuxLibraries']['libraries'] = [{key: value for key, value in row.items() if key != 'origin'} for row in native['libraries']] + receipt['linuxLibraries']['elfCount'] = 3 + components = custody.sdk_native_sbom_components(receipt) + origins = [row for row in components if row['type'] == 'library'] + self.assertEqual({row['name'] for row in origins}, {'CPython', 'libgcc', 'fixture'}) + self.assertTrue(all('hashes' not in row for row in origins)) + self.assertNotIn('unshipped-extension', json.dumps(components)) + self.assertIn('wheelCandidateSha256s', json.dumps(components)) + + def test_sbom_omission_duplicate_substitution_and_license_poison(self): + import copy + receipt = self.receipt(); original = custody.sdk_native_sbom_components(receipt) + poisoned = [original[:-1], original + [original[0]], []] + changed = copy.deepcopy(original); changed[0]['hashes'][0]['content'] = 'b' * 64; poisoned.append(changed) + changed = copy.deepcopy(original); changed[-1]['properties'][-1]['value'] = '{}'; poisoned.append(changed) + for components in poisoned: + with self.subTest(components=components), self.assertRaises(ValueError): + custody.validate_sdk_native_sbom(components, receipt) + with self.assertRaises(ValueError): custody.validate_sdk_native_sbom(original) + + if __name__ == '__main__': unittest.main() diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index 7098ac11..e9eb88b4 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -2327,6 +2327,16 @@ def write_sdk_candidate(self): 'boundedOutput': True, 'shellCancellation': True, 'mockedPublicRetrieval': True, 'incompleteHttpRejected': True, 'modelCalls': 0, 'networkUsed': False} libraries = {"schema": "openprose.sdk-packaged-libraries/1", "elfCount": 1, "requiredGlibcMaximum": "2.34", "modelCalls": 0} + native_fields = None + if platform.system() == 'Linux': + from test_assemble_kernel_rc import linux_receipt_fixture + target = 'linux-arm64-gnu' if platform.machine() == 'aarch64' else 'linux-x64-gnu' + native_fields = linux_receipt_fixture(target) + libraries = native_fields['linuxLibraries'] + snapshot = native_fields['linuxBuildSourceSnapshot'] + snapshot['sources'] = {p: PACKAGE_LOCAL.sha256_file(ROOT / p) for p in snapshot['sources']} + snapshot['driverSha256'] = PACKAGE_LOCAL.sha256_file(ROOT / 'cli/ci/build_agents_sdk_linux.py') + snapshot['lockSha256'] = PACKAGE_LOCAL.sha256_file(ROOT / 'cli/ci/agents-sdk-linux-build.lock.json') c_source = directory / "fixture.c" c_source.write_text("#include \n#include \nint main(int argc,char **argv){if(argc!=2)return 2; if(strcmp(argv[1],\"--packaged-self-test\")==0) puts(" + json.dumps(json.dumps(expected)) + "); else if(strcmp(argv[1],\"--packaged-tool-self-test\")==0) puts(" + json.dumps(json.dumps(tools)) + "); else if(strcmp(argv[1],\"--packaged-library-test\")==0) puts(" + json.dumps(json.dumps(libraries)) + "); else if(strcmp(argv[1],\"--version\")==0) puts(\"prose-agents-sdk 0.1.0\"); else return 2; return 0;}") compiler = shutil.which("cc"); self.assertIsNotNone(compiler) @@ -2343,6 +2353,9 @@ def write_sdk_candidate(self): "notices": {"path": PACKAGE_LOCAL.SDK_NOTICES, "sha256": hashlib.sha256(notices).hexdigest(), "byteLength": len(notices)}, "selfTest": expected, "toolSelfTest": tools, "linuxLibraries": libraries if platform.system() == "Linux" else "not-applicable", "authority": "hermetic-test-fixture-not-release-evidence"} + if native_fields is not None: + record.update(native_fields) + record['dependencies'] = [{'name': 'fixture', 'version': '1.0.0', 'wheelSha256': ['e' * 64]}] (directory / PACKAGE_LOCAL.SDK_RECEIPT).write_text(json.dumps(record)) return directory @@ -4902,6 +4915,37 @@ def test_sdk_payload_is_sibling_in_standalone_and_npm(self): self.assertEqual(member.mode, mode) self.assertEqual(source.extractfile(member).read(), data) + def test_linux_source_and_runtime_rows_are_independently_checked(self): + from test_assemble_kernel_rc import sdk_fixture + for poison in ('source', 'runtime', None): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as directory: + root = Path(directory); sdk = root / 'sdk'; sdk.mkdir(); snapshots = root / 'snapshots'; snapshots.mkdir() + identity, members = sdk_fixture('linux-x64-gnu') + receipt = json.loads(next(data for name, data, mode in members if name == PACKAGE_LOCAL.SDK_RECEIPT)) + receipt['sources'] = {path: PACKAGE_LOCAL.sha256_file(ROOT / path) for path in receipt['sources']} + snapshot = receipt['linuxBuildSourceSnapshot'] + snapshot['sources'] = {path: PACKAGE_LOCAL.sha256_file(ROOT / path) for path in snapshot['sources']} + snapshot['driverSha256'] = PACKAGE_LOCAL.sha256_file(ROOT / 'cli/ci/build_agents_sdk_linux.py') + snapshot['lockSha256'] = PACKAGE_LOCAL.sha256_file(ROOT / 'cli/ci/agents-sdk-linux-build.lock.json') + if poison == 'source': snapshot['sources']['cli/ci/build_agents_sdk.py'] = 'b' * 64 + for name, data, mode in members: + (sdk / name).write_bytes(json.dumps(receipt).encode() if name == PACKAGE_LOCAL.SDK_RECEIPT else data) + (sdk / name).chmod(mode) + def execute(argv, **kwargs): + value = receipt[{'--packaged-self-test': 'selfTest', '--packaged-tool-self-test': 'toolSelfTest', + '--packaged-library-test': 'linuxLibraries'}[argv[-1]]] + value = json.loads(json.dumps(value)) + if poison == 'runtime' and argv[-1] == '--packaged-library-test': value['libraries'][0]['sha256'] = 'b' * 64 + return subprocess.CompletedProcess(argv, 0, json.dumps(value).encode(), b'') + with mock.patch.object(PACKAGE_LOCAL, 'run_bounded', side_effect=execute) as run: + if poison: + with self.assertRaises(PACKAGE_LOCAL.PackageError): + PACKAGE_LOCAL.snapshot_sdk(sdk, snapshots, 'linux-x64-gnu', 'kernel-rc', None) + if poison == 'source': run.assert_not_called() + else: + payload, record = PACKAGE_LOCAL.snapshot_sdk(sdk, snapshots, 'linux-x64-gnu', 'kernel-rc', None) + self.assertEqual(len(payload), 3); self.assertEqual(len(record), 12) + if __name__ == "__main__": unittest.main() From 57796a619dfc30cb473bb66adc7f05a0a799e535 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 01:22:04 -0400 Subject: [PATCH 49/64] Fix repository-root packaging test sibling imports --- cli/ci/test_package_local.py | 1 + 1 file changed, 1 insertion(+) diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index e9eb88b4..3055781a 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -22,6 +22,7 @@ ROOT = Path(__file__).resolve().parents[2] CLI = ROOT / "cli" SCRIPT = CLI / "ci" / "package_local.py" +sys.path.insert(0, str(SCRIPT.parent)) # Full fixture packaging compresses both debug executables and a second copy of # Bun into npm archives. Allow CPU-intensive construction on native CI. PACKAGING_TIMEOUT_SECONDS = 180 From a71977d608c669582b35a574bd93550c4ed6e31f Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 01:53:23 -0400 Subject: [PATCH 50/64] Lease strict RPM file provenance correction --- cli/protocol/OWNERSHIP.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index bc293fba..290412f5 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1101,3 +1101,6 @@ IMP-098 frozen native receipt consumer wave: `/root/sdk_assessment` owns only `c IMP-098 reviewed clean Linux runtime/preparation integration: root owns new `cli/ci/prepare_agents_sdk_linux.py`, `cli/ci/verify_agents_sdk_linux.py`, `cli/ci/test_verify_agents_sdk_linux.py`, and `cli/ci/agents-sdk-linux-runtime.lock.json`. Apply independently reviewed scratch v2; mount payload read-only outside the writable job mount, sanitize network exceptions, enforce exact archive hash/length/native target under the workflow's external GNU timeout, validate the full closed Linux receipt before runtime verification, and bind the executed inventory module. Actual minimal GNU 2.34 probes retain five/thirty-second deadlines, separate preparation network use and CPU qualification nonclaims. Root owns native build invocation/evidence integration. No Docker or provider execution by agents; all native/full checks remain pending. + + +IMP-098 strict RPM pathname/provenance repair: `/root/sdk_assessment` owns only `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` for the explicitly reviewed supplier selection correction. `/root/configuration_contract` owns only `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` for the matching origin schema /2 closed validators and shared fixtures. Preserve canonical supplierPath and exact SHA256 verification; additionally require rpmPayloadPath, integer rpmFileDigestAlgorithm=8 and rpmFileDigestSha256 equal to the measured library digest. Supplier selection must prove a unique declared regular RPM row, same actual inode/device, exact package ownership and architecture, no leaf symlink, no ambiguous aliases, and bounded queries/diagnostics. Reject unequal bytes and unexpected algorithms; never bless a failed receipt. Root owns source integration, Git, mandatory full admission and actual native matrix replay. No Git, Docker, heavy builds, providers or credential access by agents. Prior source577 receipts retain their scope; the actual Linux failure cause is not established by the old nondiagnostic assertion. From e50551d08c377174a23cd0899cb8339dcc26cb3b Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 02:01:50 -0400 Subject: [PATCH 51/64] Verify exact RPM payload aliases and SHA256 provenance --- cli/ci/build_agents_sdk_linux.py | 251 +++++++++++++++++++++++--- cli/ci/sdk_native_inventory.py | 51 ++++-- cli/ci/test_build_agents_sdk_linux.py | 224 ++++++++++++++++++++++- cli/ci/test_sdk_native_inventory.py | 51 +++++- 4 files changed, 532 insertions(+), 45 deletions(-) diff --git a/cli/ci/build_agents_sdk_linux.py b/cli/ci/build_agents_sdk_linux.py index 625e0558..9693be32 100644 --- a/cli/ci/build_agents_sdk_linux.py +++ b/cli/ci/build_agents_sdk_linux.py @@ -18,6 +18,7 @@ import uuid import shutil import sys +import sdk_native_inventory as native_inventory LOCK = Path(__file__).with_name('agents-sdk-linux-build.lock.json') SOURCES = ('harnesses/agents-sdk/run.py', 'harnesses/agents-sdk/requirements-build.txt', @@ -96,7 +97,7 @@ def verify_archive(path, row): def validate_origin(origin, root, row): require(isinstance(origin,dict) and set(origin)=={'schema','package','library','licenses'} - and origin['schema']=='openprose.sdk-native-origin/1','Closed native origin required') + and origin['schema']=='openprose.sdk-native-origin/2','Closed native origin required') package=origin['package'] require(set(package)=={'name','epoch','version','release','architecture','sourceRpm','license'} and all(isinstance(v,str) and v and len(v)<=1024 for v in package.values()) @@ -104,11 +105,9 @@ def validate_origin(origin, root, row): and re.fullmatch('[A-Za-z0-9_.+~-]+[.]src[.]rpm',package['sourceRpm']), 'Actual supplying RPM and source RPM facts required') lib=origin['library'];path=checked_path(root/'libgcc_s.so.1') - require(set(lib)=={'path','sha256','byteLength','supplierPath','rpmFileDigestVerified'} - and lib['path']=='libgcc_s.so.1' and lib['supplierPath'].startswith('/usr/lib64/') - and lib['rpmFileDigestVerified'] is True and sha(path)==lib['sha256'] - and type(lib['byteLength']) is int and path.stat().st_size==lib['byteLength'], - 'Supplier origin does not bind exact libgcc bytes') + native_inventory.validate_supplier_library(lib) + require(sha(path)==lib["sha256"] and path.stat().st_size==lib["byteLength"], + "Supplier origin does not bind exact libgcc bytes") licenses=origin['licenses'];require(isinstance(licenses,list) and 00,'supplier-timeout') + return value + +def rpm(args,deadline): + end=time.monotonic()+min(5,remaining(deadline)) + env={'PATH':'/usr/bin:/bin','LANG':'C','LC_ALL':'C'} + p=subprocess.Popen(['rpm',*args],stdout=subprocess.PIPE,stderr=subprocess.STDOUT, + env=env,start_new_session=True) + data=bytearray() + try: + with selectors.DefaultSelector() as selector: + selector.register(p.stdout,selectors.EVENT_READ) + while selector.get_map(): + require(time.monotonic()4096:encoded=json.dumps({'phase':'supplier','code':code}) + return encoded + +def main(): + deadline=time.monotonic()+30 + DIAG['phase']='target' + target=pathlib.Path('/job/target.json') + require(target.stat().st_size<=65536,'supplier-target-size') + arch=json.loads(target.read_text())['machine'] + require(arch in ('x86_64','aarch64'),'supplier-target-architecture') + DIAG['architecture']=arch + p=pathlib.Path('/usr/lib64/libgcc_s.so.1').resolve(strict=True) + require(canonical_path(str(p)),'supplier-canonical-path') + DIAG.update(phase='rpm-package',canonicalPath=str(p)) + package,rows=parse_table(rpm(['-q','--qf',HEADER+TABLE,'libgcc'],deadline),arch) + info=p.lstat();row=select_payload(rows,str(p),info) + DIAG['phase']='rpm-owner' + verify_owner(rpm(['-qf','--qf',HEADER,row['path']],deadline),package,arch) + DIAG['phase']='payload' + b,read_info=read_regular(p,MAX_LIBRARY_BYTES,deadline) + require(stable(read_info)==stable(info),'supplier-identity-change') + check_binding(str(p),row,info) + require(b[:4]==b'\x7fELF','supplier-not-elf') + digest=verify_digest(b,row) + DIAG['phase']='licenses' + licenses=[];license_bytes=[];seen=set();total=0 + for item in rows: + name=item['path'] + if not direct_path(name,('/usr/share/licenses',)) and not name.startswith('/usr/share/licenses/'):continue + require(text(name) and str(pathlib.PurePosixPath(name))==name and '..' not in pathlib.PurePosixPath(name).parts,'supplier-license-path') + require(name not in seen,'supplier-license-duplicate');seen.add(name) + q=pathlib.Path(name) + if not stat.S_ISREG(item['mode']):continue + data,_=read_regular(q,MAX_LICENSE_BYTES,deadline) + total+=len(data) + require(len(licenses)<32 and total<=MAX_LICENSE_TOTAL,'supplier-license-limit') + licenses.append({'packagePath':name,'path':'license-'+str(len(licenses))+'.txt','sha256':hashlib.sha256(data).hexdigest(),'byteLength':len(data)}) + license_bytes.append(data) + texts=b'\n'.join(license_bytes) + require(b'GCC RUNTIME LIBRARY EXCEPTION' in texts and b'GNU GENERAL PUBLIC LICENSE' in texts,'supplier-license-texts') + DIAG['phase']='publish' + verify_owner(rpm(['-q','--qf',HEADER,'libgcc'],deadline),package,arch) + check_binding(str(p),row,info) + remaining(deadline) + out=pathlib.Path('/job/supplier');out.mkdir() + (out/'libgcc_s.so.1').write_bytes(b) + copied=(out/'libgcc_s.so.1').read_bytes() + require(len(copied)==len(b) and hashlib.sha256(copied).hexdigest()==digest,'supplier-copy-digest') + for item,data in zip(licenses,license_bytes):(out/item['path']).write_bytes(data) + record={'schema':'openprose.sdk-native-origin/2','package':package,'library':{ + 'path':'libgcc_s.so.1','sha256':digest,'byteLength':len(b),'supplierPath':str(p), + 'rpmPayloadPath':row['path'],'rpmFileDigestAlgorithm':8,'rpmFileDigestSha256':row['digest'], + 'rpmFileDigestVerified':True},'licenses':licenses} + remaining(deadline) + (out/'origin.json').write_text(json.dumps(record,sort_keys=True,indent=2)+'\n') + +if __name__=='__main__': + try:main() + except Exception as error: + code=error.code if isinstance(error,SupplierFailure) else 'supplier-internal-failure' + try:print(diagnostic(code),file=sys.stderr) + except Exception:pass + sys.exit(1) ''' PREPARE = r'''/opt/python/cp310-cp310/bin/python - <<'PIN' import hashlib,json,pathlib diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index 4ac69ac5..530ca851 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -260,6 +260,41 @@ def visit(node, depth=0): return sha(data) +def validate_supplier_library(library): + """Validate portable /2 RPM facts; the supplier proves identity and ownership. + + Directory aliases are recorded only after the pinned supplier process has + proved a unique regular RPM payload row sharing the canonical inode/device. + A portable receipt cannot itself repeat those original filesystem checks. + """ + keys(library, ('path', 'sha256', 'byteLength', 'supplierPath', 'rpmFileDigestVerified', + 'rpmPayloadPath', 'rpmFileDigestAlgorithm', 'rpmFileDigestSha256'), 'supplier library') + require(library['path'] == 'libgcc_s.so.1' and library['rpmFileDigestVerified'] is True, + 'Invalid supplier library identity or RPM verification') + require(isinstance(library['sha256'], str) and re.fullmatch('[0-9a-f]{64}', library['sha256']), + 'Invalid supplier library SHA256') + require(type(library['byteLength']) is int and 0 < library['byteLength'] <= 8 * 1024 * 1024, + 'Supplier library size is outside bounds') + require(type(library['rpmFileDigestAlgorithm']) is int and library['rpmFileDigestAlgorithm'] == 8, + 'Supplier RPM digest algorithm must be integer SHA256 (8)') + require(isinstance(library['rpmFileDigestSha256'], str) and + re.fullmatch('[0-9a-f]{64}', library['rpmFileDigestSha256']) and + library['rpmFileDigestSha256'] == library['sha256'], 'Supplier RPM digest differs from measured SHA256') + def direct_path(value, parents): + require(isinstance(value, str) and 0 < len(value) <= 1024 and + all(32 <= ord(c) < 127 for c in value) and '\\' not in value, + 'Invalid supplier RPM path') + path = PurePosixPath(value) + require(path.is_absolute() and str(path) == value and str(path.parent) in parents and + re.fullmatch(r'libgcc_s(?:-[A-Za-z0-9._+\-]+)?[.]so[.]1', path.name), + 'Supplier RPM path must be a direct canonical libgcc leaf') + return path + canonical = direct_path(library['supplierPath'], {'/usr/lib64'}) + payload = direct_path(library['rpmPayloadPath'], {'/usr/lib64', '/lib64'}) + require(payload.name == canonical.name, 'Supplier RPM alias basename differs from canonical library') + return library + + def validate_input(path, supplier, source_root, target): data = read_file(path, MAX_METADATA) value = load_json(data) @@ -294,16 +329,13 @@ def hash_value(digest): triple + r'-pgo%2Blto-full\.tar\.zst', archive['url']), 'Expected pinned full shared Python archive') require(isinstance(archive['metadataUrl'], str) and archive['metadataUrl'].startswith('https://'), 'Missing Python archive metadata URL') origin = value['libgcc']; keys(origin, ('schema', 'package', 'library', 'licenses'), 'libgcc origin') - require(origin['schema'] == 'openprose.sdk-native-origin/1', 'Unsupported libgcc origin schema') + require(origin['schema'] == 'openprose.sdk-native-origin/2', 'Unsupported libgcc origin schema') package = origin['package']; keys(package, ('name', 'epoch', 'version', 'release', 'architecture', 'sourceRpm', 'license'), 'supplier package') require(package['name'] == 'libgcc' and package['architecture'] == arch and all(isinstance(v, str) and v and not any(ord(c) < 32 for c in v) for v in package.values()) and re.fullmatch(r'[0-9]+', package['epoch']) and package['sourceRpm'].endswith('.src.rpm'), 'Invalid supplier package provenance') - library = origin['library']; keys(library, ('path', 'sha256', 'byteLength', 'supplierPath', 'rpmFileDigestVerified'), 'supplier library') - require(library['path'] == 'libgcc_s.so.1' and library['rpmFileDigestVerified'] is True and - isinstance(library['supplierPath'], str) and library['supplierPath'].startswith('/usr/lib64/') and - '..' not in PurePosixPath(library['supplierPath']).parts, 'Invalid supplier library origin') - actual = read_file(supplier) + library = validate_supplier_library(origin['library']) + actual = read_file(supplier, 8 * 1024 * 1024) require(library['sha256'] == sha(actual) and type(library['byteLength']) is int and library['byteLength'] == len(actual), 'Explicit libgcc bytes differ from supplier receipt') licenses = origin['licenses']; require(isinstance(licenses, list) and 0 < len(licenses) <= 32, 'Missing supplier license evidence') @@ -441,19 +473,16 @@ def file_row(row, maximum=1024 * 1024): require(type(row['byteLength']) is int and 0 < row['byteLength'] <= maximum, 'Invalid native receipt size') origins = value['origins']; keys(origins, ('supplier', 'python', 'wheels'), 'native origins') supplier = origins['supplier']; keys(supplier, ('schema', 'package', 'library', 'licenses', 'image'), 'supplier origin') - require(supplier['schema'] == 'openprose.sdk-native-origin/1', 'Invalid supplier origin schema') + require(supplier['schema'] == 'openprose.sdk-native-origin/2', 'Invalid supplier origin schema') keys(supplier['package'], ('name', 'epoch', 'version', 'release', 'architecture', 'sourceRpm', 'license'), 'supplier package') arch = {'linux-x64-gnu': 'x86_64', 'linux-arm64-gnu': 'aarch64'}.get(target) require(arch is not None, 'Unsupported native receipt target') require(supplier['package']['architecture'] == arch and supplier['package']['name'] == 'libgcc' and all(isinstance(x, str) and x for x in supplier['package'].values()), 'Invalid supplier package metadata') - keys(supplier['library'], ('path', 'sha256', 'byteLength', 'supplierPath', 'rpmFileDigestVerified'), 'supplier file') - require(supplier['library']['rpmFileDigestVerified'] is True, 'Supplier RPM file digest was not verified') - file_row({k: supplier['library'][k] for k in ('path', 'sha256', 'byteLength')}, MAX_BYTES) + validate_supplier_library(supplier['library']) keys(supplier['image'], ('image', 'configSha256', 'metadataUrl'), 'supplier image') digest(supplier['image']['configSha256']) require(re.fullmatch(r'quay\.io/pypa/manylinux_2_28_' + arch + r'@sha256:[0-9a-f]{64}', supplier['image']['image']), 'Supplier image target/floor mismatch') - require(supplier['library']['path'] == 'libgcc_s.so.1' and isinstance(supplier['library']['supplierPath'], str) and supplier['library']['supplierPath'].startswith('/usr/lib64/'), 'Supplier library origin mismatch') require(re.fullmatch(r'[0-9]+', supplier['package']['epoch']) and supplier['package']['sourceRpm'].endswith('.src.rpm'), 'Supplier RPM identity is incomplete') require(isinstance(supplier['licenses'], list) and supplier['licenses'], 'Missing supplier licenses') for row in supplier['licenses']: diff --git a/cli/ci/test_build_agents_sdk_linux.py b/cli/ci/test_build_agents_sdk_linux.py index e32cf895..ee743b33 100644 --- a/cli/ci/test_build_agents_sdk_linux.py +++ b/cli/ci/test_build_agents_sdk_linux.py @@ -53,7 +53,6 @@ def test_required_selection_and_custody_hooks(self): self.assertIn('--linux-libgcc /job/supplier/libgcc_s.so.1',d.FREEZE) self.assertIn('--linux-native-origin /job/native-input.json',d.FREEZE) self.assertNotIn('exclude',d.FREEZE);self.assertNotIn('LD_LIBRARY_PATH',d.FREEZE) - self.assertIn("digests.get(str(p))==digest",d.SUPPLIER_PY) self.assertIn('GCC RUNTIME LIBRARY EXCEPTION',d.SUPPLIER_PY) self.assertIn("x['libpython_link_mode']=='shared'",d.PREPARE) def test_preparation_network_is_explicit_and_freeze_is_not(self): @@ -155,10 +154,12 @@ def fixture(self,root): text='GNU GENERAL PUBLIC LICENSE\nGCC RUNTIME LIBRARY EXCEPTION' (root/'license-0.txt').write_text(text) lib={'path':'libgcc_s.so.1','sha256':d.sha(root/'libgcc_s.so.1'),'byteLength':len(b'fixture native bytes'), - 'supplierPath':'/usr/lib64/libgcc_s-actual.so.1','rpmFileDigestVerified':True} + 'supplierPath':'/usr/lib64/libgcc_s-actual.so.1','rpmFileDigestVerified':True, + 'rpmPayloadPath':'/lib64/libgcc_s-actual.so.1','rpmFileDigestAlgorithm':8, + 'rpmFileDigestSha256':d.sha(root/'libgcc_s.so.1')} package={'name':'libgcc','epoch':'0','version':'8.fixture','release':'test','architecture':'x86_64', 'sourceRpm':'gcc-fixture.src.rpm','license':'fixture-not-production'} - return {'schema':'openprose.sdk-native-origin/1','library':lib,'package':package,'licenses':[ + return {'schema':'openprose.sdk-native-origin/2','library':lib,'package':package,'licenses':[ {'packagePath':'/usr/share/licenses/libgcc/COPYING.RUNTIME','path':'license-0.txt', 'sha256':d.sha(root/'license-0.txt'),'byteLength':(root/'license-0.txt').stat().st_size}]} def test_missing_source_rpm_and_actual_license_fact_refused(self): @@ -185,5 +186,222 @@ def test_output_byte_and_file_caps_enforced(self): with patch.object(d,'MAX_FILES',0),self.assertRaises(ValueError):d.check_output(root) +class ExecutedSupplierControls(unittest.TestCase): + def setUp(self): + self.s={'__name__':'supplier_controls'} + exec(compile(d.SUPPLIER_PY,'supplier.py','exec'),self.s) + self.canonical='/usr/lib64/libgcc_s-fixture.so.1' + self.alias='/lib64/libgcc_s-fixture.so.1' + self.temp=tempfile.TemporaryDirectory();self.addCleanup(self.temp.cleanup) + self.root=Path(self.temp.name) + self.file=self.root/'usr/lib64/libgcc_s-fixture.so.1' + self.file.parent.mkdir(parents=True);self.file.write_bytes(b'\x7fELFfixture-bytes') + (self.root/'lib64').symlink_to(self.root/'usr/lib64',target_is_directory=True) + self.data=self.file.read_bytes();self.digest=self.s['hashlib'].sha256(self.data).hexdigest() + self.header='libgcc\t0\t8.fixture\t1\tx86_64\tgcc-fixture.src.rpm\tGPLv3+\t8\n' + self.row={'path':self.alias,'digest':self.digest,'mode':self.file.stat().st_mode} + def lstat(self,name):return (self.root/name.lstrip('/')).lstat() + def select(self,rows=None,info=None): + return self.s['select_payload'](rows if rows is not None else [self.row],self.canonical,info or self.file.stat(),self.lstat) + def failure(self,code,fn,*args): + with self.assertRaisesRegex(self.s['SupplierFailure'],'^'+code+'$'):fn(*args) + def table(self,row=None): + row=row or self.row + return self.header+row['path']+'\t'+row['digest']+'\t'+str(row['mode'])+'\n' + def test_canonical_and_alias_regular_payloads_accept_on_both_architectures(self): + for arch in ('x86_64','aarch64'): + for path in (self.canonical,self.alias): + with self.subTest(arch=arch,path=path): + self.header=self.header.replace('x86_64',arch) + row={**self.row,'path':path} + package,rows=self.s['parse_table'](self.table(row),arch) + actual=self.select(rows) + self.s['verify_owner'](self.header,package,arch) + self.assertEqual(self.s['verify_digest'](self.data,actual),self.digest) + def test_duplicate_rows_and_two_aliases_are_ambiguous_even_with_equal_digests(self): + for rows in ([self.row,self.row],[self.row,{**self.row,'path':self.canonical}]): + with self.subTest(rows=rows):self.failure('rpm-payload-ambiguity',self.select,rows) + _,rows=self.s['parse_table'](self.table()+self.table().splitlines()[1]+'\n','x86_64') + self.assertEqual(len(rows),2) + self.failure('rpm-payload-ambiguity',self.select,rows) + def test_same_basename_and_digest_at_distinct_file_reject(self): + other=self.root/'other';other.write_bytes(self.data) + self.failure('rpm-payload-ambiguity',self.s['select_payload'],[self.row],self.canonical,self.file.stat(),lambda _:other.stat()) + def test_same_inode_number_on_other_device_reject(self): + from types import SimpleNamespace + found=self.file.stat();other=SimpleNamespace(st_dev=found.st_dev+1,st_ino=found.st_ino,st_mode=found.st_mode) + self.failure('rpm-payload-ambiguity',self.s['select_payload'],[self.row],self.canonical,found,lambda _:other) + def test_terminal_symlink_and_rpm_nonregular_mode_reject(self): + link=self.root/'leaf-link';link.symlink_to(self.file) + self.failure('rpm-payload-nonregular',self.s['select_payload'],[self.row],self.canonical,self.file.stat(),lambda _:link.lstat()) + self.failure('rpm-mode-nonregular',self.select,[{**self.row,'mode':0o120777}]) + self.failure('supplier-canonical-file',self.select,[self.row],link.lstat()) + def test_zero_matching_rows_and_malformed_matching_digest_reject(self): + self.failure('rpm-payload-ambiguity',self.select,[]) + for digest in ('','a'*63,'A'*64,'g'*64): + with self.subTest(digest=digest):self.failure('rpm-payload-digest-format',self.select,[{**self.row,'digest':digest}]) + def test_alias_with_actual_byte_mismatch_still_rejects_digest(self): + row=self.select() + self.failure('rpm-payload-digest-mismatch',self.s['verify_digest'],self.data+b'changed',row) + diag=json.loads(self.s['diagnostic']('rpm-payload-digest-mismatch')) + self.assertEqual(diag['expectedSha256'],self.digest) + self.assertNotEqual(diag['measuredSha256'],self.digest) + def test_unexpected_missing_or_noninteger_rpm_algorithm_rejects(self): + for value in ('(none)','1','2','9','True','8.0',''): + with self.subTest(value=value): + bad=self.header.rstrip('\n').rsplit('\t',1)[0]+'\t'+value + code='rpm-package-ambiguous' if not value else 'rpm-digest-algorithm' + self.failure(code,self.s['parse_header'],bad,'x86_64') + def test_wrong_or_multiple_package_records_reject(self): + self.failure('rpm-package-identity',self.s['parse_header'],self.header.replace('libgcc','other').rstrip(),'x86_64') + self.failure('rpm-package-identity',self.s['parse_header'],self.header.rstrip(),'aarch64') + self.failure('rpm-malformed-row',self.s['parse_table'],self.table()+self.header,'x86_64') + def test_exact_owner_all_package_facts_and_uniqueness_required(self): + package=self.s['parse_header'](self.header.rstrip(),'x86_64') + for i,value in enumerate(('other','1','different','2','aarch64','other.src.rpm','other-license')): + raw=self.header.rstrip().split('\t');raw[i]=value + code='rpm-package-identity' if i in (0,4) else 'rpm-owner-mismatch' + with self.subTest(field=i):self.failure(code,self.s['verify_owner'],'\t'.join(raw)+'\n',package,'x86_64') + for output in ('',self.header+self.header):self.failure('rpm-owner-ambiguity',self.s['verify_owner'],output,package,'x86_64') + def test_row_and_metadata_limits_are_enforced(self): + for output,code in ((self.header+('x\t\t1\n'*129),'rpm-row-limit'), + (self.header+('x'*1025+'\t\t1\n'),'rpm-malformed-row'), + (self.header+'x\ty\t-1\n','rpm-malformed-row'), + (self.header+'x\ty\t999999\n','rpm-malformed-row')): + with self.subTest(code=code):self.failure(code,self.s['parse_table'],output,'x86_64') + def test_read_limits_nonregular_file_and_stale_identity_reject(self): + import time + self.failure('supplier-file-size',self.s['read_regular'],self.file,4,time.monotonic()+1) + self.failure('supplier-timeout',self.s['read_regular'],self.file,100,time.monotonic()-1) + link=self.root/'leaf-link';link.symlink_to(self.file) + self.failure('supplier-nonregular-file',self.s['read_regular'],link,100,time.monotonic()+1) + old=self.file.stat();self.file.write_bytes(self.data+b'changed') + self.failure('supplier-identity-change',self.s['check_binding'],self.canonical,self.row,old,self.lstat) + def test_actual_rpm_process_output_timeout_and_invalid_utf8_are_bounded(self): + import subprocess,sys,time + original=subprocess.Popen + cases=(('print("x"*300000)','rpm-output-limit'),('import time;time.sleep(5)','rpm-timeout'), + ('import sys;sys.stdout.buffer.write(bytes([255]))','rpm-malformed-output'), + ('import sys;print("untrusted raw secret",file=sys.stderr);sys.exit(2)','rpm-query-failed')) + for code,expected in cases: + with self.subTest(expected=expected): + def launch(command,**kw):return original([sys.executable,'-c',code],**kw) + with patch.object(subprocess,'Popen',side_effect=launch): + self.failure(expected,self.s['rpm'],['unused'],time.monotonic()+.5) + diagnostic=self.s['diagnostic'](expected) + self.assertLessEqual(len(diagnostic.encode()),4096);self.assertNotIn('untrusted',diagnostic) + def test_payload_paths_have_exact_grammar(self): + for path in ('/lib64//libgcc_s-fixture.so.1','/lib64/./libgcc_s-fixture.so.1', + '/lib64/../lib64/libgcc_s-fixture.so.1','/tmp/libgcc_s-fixture.so.1', + '/lib64/other.so.1','/lib64/libgcc_s-fixture.so.1\n'): + with self.subTest(path=path):self.assertFalse(self.s['payload_path'](path,self.canonical)) + self.assertFalse(self.s['canonical_path']('/usr/lib64/libgcc_s-8~fixture.so.1')) + def test_diagnostic_is_closed_sanitized_and_bounded(self): + self.s['DIAG'].update(rawStderr='untrusted-secret',canonicalPath='bad\npath',rowCount=3) + value=json.loads(self.s['diagnostic']('rpm-query-failed')) + self.assertNotIn('rawStderr',value);self.assertNotIn('canonicalPath',value) + self.assertEqual(value['rowCount'],3) + for key in ('canonicalPath','payloadPath','ownerVersion','ownerRelease','architecture'): + self.s['DIAG'][key]='x'*1024 + encoded=self.s['diagnostic']('rpm-query-failed') + self.assertLessEqual(len(encoded.encode()),4096) + self.assertEqual(json.loads(encoded),{'phase':'supplier','code':'rpm-query-failed'}) + def run_main(self,arch='x86_64',path=None,owner=None,mutation=None,license_count=1): + import pathlib + original=type(self.file) + job=self.root/'job';job.mkdir(exist_ok=True) + (job/'target.json').write_text(json.dumps({'machine':arch})) + license_path='/usr/share/licenses/libgcc/COPYING.RUNTIME' + license_file=self.root/license_path.lstrip('/');license_file.parent.mkdir(parents=True,exist_ok=True) + license_file.write_bytes(b'GNU GENERAL PUBLIC LICENSE\nGCC RUNTIME LIBRARY EXCEPTION') + requested=self.root/'usr/lib64/libgcc_s.so.1';requested.symlink_to(self.file.name) + header=self.header.replace('x86_64',arch) + row={**self.row,'path':path or self.alias} + table=header+row['path']+'\t'+row['digest']+'\t'+str(row['mode'])+'\n' + for i in range(license_count): + name=license_path if i==0 else license_path+str(i) + extra=self.root/name.lstrip('/');extra.write_bytes(license_file.read_bytes()) + table+=name+'\t\t'+str(extra.stat().st_mode)+'\n' + calls=[] + def query(args,deadline): + calls.append(args) + if args[0]=='-qf': + self.assertEqual(args[-1],row['path']) + if mutation=='bytes':self.file.write_bytes(self.data+b'changed') + return owner or header + return table if self.s['TABLE'] in args[2] else header.replace('8.fixture','changed') if mutation=='package' else header + def map_path(value): + p=original(value) + return self.root/str(p).lstrip('/') if str(p).startswith(('/job','/usr/','/lib64')) else p + class CanonicalFile: + def __str__(inner):return self.canonical + def __fspath__(inner):return str(self.file) + def lstat(inner):return self.file.lstat() + resolve=original.resolve + def resolve_path(value,**kw):return CanonicalFile() if value==requested else resolve(value,**kw) + with patch.object(original,'resolve',resolve_path),patch.object(pathlib,'Path',side_effect=map_path),patch.dict(self.s,{'rpm':query}): + self.s['main']() + return json.loads((job/'supplier/origin.json').read_text()),calls + def test_actual_main_publishes_alias_and_canonical_exact_origin_on_both_targets(self): + for arch in ('x86_64','aarch64'): + for path in (self.alias,self.canonical): + with self.subTest(arch=arch,path=path): + # Fresh instance gives each publication a new output directory. + fresh=ExecutedSupplierControls();fresh.setUp() + try: + origin,calls=fresh.run_main(arch,path) + self.assertEqual(origin['schema'],'openprose.sdk-native-origin/2') + lib=origin['library'];self.assertEqual(len(lib),8) + self.assertEqual(lib['rpmPayloadPath'],path);self.assertEqual(lib['rpmFileDigestAlgorithm'],8) + self.assertEqual(lib['rpmFileDigestSha256'],lib['sha256']) + copied=fresh.root/'job/supplier/libgcc_s.so.1' + self.assertEqual(copied.read_bytes(),fresh.data) + self.assertEqual(len(calls),3) + finally:fresh.temp.cleanup() + def test_main_wrong_owner_and_changed_payload_fail_before_publication(self): + for owner,mutation,expected in ((self.header.replace('8.fixture','other'),None,'rpm-owner-mismatch'), + (None,'bytes','supplier-identity-change'), + (None,'package','rpm-owner-mismatch')): + fresh=ExecutedSupplierControls();fresh.setUp() + try: + fresh.failure(expected,fresh.run_main,'x86_64',None,owner,mutation) + self.assertFalse((fresh.root/'job/supplier/origin.json').exists()) + finally:fresh.temp.cleanup() + def test_main_license_aggregate_and_file_limits_fail_before_publication(self): + for key in ('MAX_LICENSE_BYTES','MAX_LICENSE_TOTAL'): + fresh=ExecutedSupplierControls();fresh.setUp() + try: + with patch.dict(fresh.s,{key:4}): + expected='supplier-file-size' if key=='MAX_LICENSE_BYTES' else 'supplier-license-limit' + fresh.failure(expected,fresh.run_main) + self.assertFalse((fresh.root/'job/supplier/origin.json').exists()) + finally:fresh.temp.cleanup() + def test_main_license_count_limit_rejects_without_receipt(self): + self.failure('supplier-license-limit',self.run_main,'x86_64',None,None,None,33) + self.assertFalse((self.root/'job/supplier/origin.json').exists()) + + +class NativeOriginV2Controls(unittest.TestCase): + fixture=NativeOriginControls.fixture + def test_closed_algorithm_digest_and_path_poison_refused(self): + row=d.load_lock()['platforms']['linux-x64-gnu'] + with tempfile.TemporaryDirectory() as t: + root=Path(t).resolve();origin=self.fixture(root) + for key,value in (('rpmFileDigestAlgorithm',True),('rpmFileDigestAlgorithm','8'),('rpmFileDigestAlgorithm',9), + ('rpmFileDigestSha256','0'*64),('rpmPayloadPath','/tmp/libgcc_s-actual.so.1'), + ('rpmPayloadPath','/lib64//libgcc_s-actual.so.1'),('rpmPayloadPath','/lib64/other.so.1'), + ('supplierPath','/usr/lib64/../libgcc_s-actual.so.1'),('rpmFileDigestVerified',False)): + x=copy.deepcopy(origin);x['library'][key]=value + with self.subTest(key=key,value=value),self.assertRaises(ValueError):d.validate_origin(x,root,row) + for key in ('rpmPayloadPath','rpmFileDigestAlgorithm','rpmFileDigestSha256'): + x=copy.deepcopy(origin);del x['library'][key] + with self.subTest(missing=key),self.assertRaises(ValueError):d.validate_origin(x,root,row) + x=copy.deepcopy(origin);x['schema']='openprose.sdk-native-origin/1' + with self.assertRaises(ValueError):d.validate_origin(x,root,row) + for parent in ('/lib64','/usr/lib64'): + x=copy.deepcopy(origin);x['library']['rpmPayloadPath']=parent+'/libgcc_s-actual.so.1' + d.validate_origin(x,root,row) + + if __name__ == '__main__': unittest.main() diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index 087b5e61..37af5e04 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -58,10 +58,12 @@ def input_fixture(root, target='linux-x64-gnu'): metadata = python / 'PYTHON.json'; metadata.write_text(json.dumps(info)) image = lambda floor: {'image': 'quay.io/pypa/manylinux_' + floor + '_' + arch + '@sha256:' + 'a' * 64, 'configSha256': 'b' * 64, 'metadataUrl': 'https://quay.io/fixture'} - origin = {'schema': 'openprose.sdk-native-origin/1', + origin = {'schema': 'openprose.sdk-native-origin/2', 'package': {'name': 'libgcc', 'epoch': '0', 'version': '8.5.0', 'release': 'fixture', 'architecture': arch, 'sourceRpm': 'gcc-8.5.0-fixture.src.rpm', 'license': 'GPLv3+ and GPLv3+ with exceptions'}, - 'library': {**native.record('libgcc_s.so.1', library.read_bytes()), 'supplierPath': '/usr/lib64/libgcc_s-8.so.1', 'rpmFileDigestVerified': True}, + 'library': {**native.record('libgcc_s.so.1', library.read_bytes()), 'supplierPath': '/usr/lib64/libgcc_s-8.so.1', 'rpmFileDigestVerified': True, + 'rpmPayloadPath': '/lib64/libgcc_s-8.so.1', 'rpmFileDigestAlgorithm': 8, + 'rpmFileDigestSha256': native.sha(library.read_bytes())}, 'licenses': [{**native.record('license-0.txt', license_text), 'packagePath': '/usr/share/licenses/libgcc/COPYING'}]} # Canonical encoded URLs exactly match the approved v3 driver lock's full archive pins. version = '20260807' @@ -158,6 +160,51 @@ def test_valid_v3_encoded_full_archive_input_both_architectures(self): self.assertEqual(actual, value); self.assertEqual(digest, native.sha(path.read_bytes())) self.assertIn('GCC RUNTIME LIBRARY EXCEPTION', texts[0]) + def test_origin_v2_aliases_and_closed_rpm_facts_in_inputs_and_receipts(self): + for target in ('linux-x64-gnu', 'linux-arm64-gnu'): + with self.subTest(target=target), tempfile.TemporaryDirectory() as temp: + value, path, library, source = input_fixture(Path(temp), target) + machine = 62 if target == 'linux-x64-gnu' else 183 + rows, origins = native.assign_origins( + [native.elf_record('libgcc_s.so.1', elf(machine=machine))], value, [], lambda _: None) + receipt = {'libraries': rows, 'symbolClosureVerified': True, + 'libgccSelection': {**native.record('libgcc_s.so.1', library.read_bytes()), + 'analysisTocSha256': 'a' * 64}, 'origins': origins} + for parent in ('/lib64', '/usr/lib64'): + actual = copy.deepcopy(value) + actual['libgcc']['library']['rpmPayloadPath'] = parent + '/libgcc_s-8.so.1' + path.write_text(json.dumps(actual)) + self.assertEqual(native.validate_input(path, library, source, target)[0], actual) + portable = copy.deepcopy(receipt) + portable['origins']['supplier']['library'] = actual['libgcc']['library'] + self.assertEqual(native.validate_native_dependencies(portable, target), portable) + mutations = [lambda o: o.update(schema='openprose.sdk-native-origin/1')] + for field in ('rpmPayloadPath', 'rpmFileDigestAlgorithm', 'rpmFileDigestSha256'): + mutations.append(lambda o, field=field: o['library'].pop(field)) + mutations.append(lambda o: o['library'].update(extra=True)) + for algorithm in (True, '8', 1, 2, 9): + mutations.append(lambda o, algorithm=algorithm: o['library'].update(rpmFileDigestAlgorithm=algorithm)) + for digest in ('A' * 64, 'b' * 64, None): + mutations.append(lambda o, digest=digest: o['library'].update(rpmFileDigestSha256=digest)) + mutations.append(lambda o: o['library'].update(rpmFileDigestVerified=False)) + for field in ('supplierPath', 'rpmPayloadPath'): + bad_paths = ('libgcc_s-8.so.1', '/etc/libgcc_s-8.so.1', + '/usr/lib64/../lib64/libgcc_s-8.so.1', '/usr/lib64/./libgcc_s-8.so.1', + '/usr//lib64/libgcc_s-8.so.1', '/usr/lib64/nested/libgcc_s-8.so.1', + '/usr/lib64/libgcc_s-9.so.1' if field == 'rpmPayloadPath' else '/lib64/libgcc_s-8.so.1', + '/usr/lib64/libgcc_s-8.so.1\n', '/usr/lib64/not-libgcc.so.1', + '/usr/lib64/libgcc_s-8.so.1/', '/usr/lib64/' + 'x' * 1025, + '/usr/lib64/libgcc_s-8~.so.1') + for bad in bad_paths: + mutations.append(lambda o, field=field, bad=bad: o['library'].update({field: bad})) + for index, mutate in enumerate(mutations): + with self.subTest(target=target, poison=index): + poisoned = copy.deepcopy(value); mutate(poisoned['libgcc']) + path.write_text(json.dumps(poisoned)) + with self.assertRaises(ValueError): native.validate_input(path, library, source, target) + portable = copy.deepcopy(receipt); mutate(portable['origins']['supplier']) + with self.assertRaises(ValueError): native.validate_native_dependencies(portable, target) + def test_source_module_mutation_encoded_pin_architecture_and_closed_shapes(self): with tempfile.TemporaryDirectory() as temp: value, path, library, source = input_fixture(Path(temp)) From 0f0bdf7a809a6a675f53b1c8576dabe34d750ef0 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 03:32:07 -0400 Subject: [PATCH 52/64] Decode pinned Linux SDK archive before container extraction --- cli/ci/build_agents_sdk_linux.py | 157 ++++++++++++++++++++--- cli/ci/test_build_agents_sdk_linux.py | 177 ++++++++++++++++++++++++++ cli/protocol/OWNERSHIP.md | 5 + 3 files changed, 321 insertions(+), 18 deletions(-) diff --git a/cli/ci/build_agents_sdk_linux.py b/cli/ci/build_agents_sdk_linux.py index 9693be32..ff31db22 100644 --- a/cli/ci/build_agents_sdk_linux.py +++ b/cli/ci/build_agents_sdk_linux.py @@ -366,19 +366,38 @@ def main(): except Exception:pass sys.exit(1) ''' -PREPARE = r'''/opt/python/cp310-cp310/bin/python - <<'PIN' -import hashlib,json,pathlib -r=json.loads(pathlib.Path('/job/target.json').read_text())['pythonArchive'] -p=pathlib.Path('/job/python-full.tar.zst') -assert p.is_file() and not p.is_symlink() and p.stat().st_size==r['byteLength'] -h=hashlib.sha256() -with p.open('rb') as f: - for b in iter(lambda:f.read(1048576),b''):h.update(b) -assert h.hexdigest()==r['sha256'], 'Copied Python archive changed before tar' -PIN +TRANSPORT_PIN = r'''import hashlib,json,pathlib,re + +def verify_transport(root): + root=pathlib.Path(root) + def read(name,limit): + p=root/name + assert p.is_file() and not p.is_symlink() and p.stat().st_size<=limit, 'Transport metadata path invalid' + return json.loads(p.read_text()) + r=read('target.json',65536)['pythonArchive'] + x=read('python-transport.json',16384) + assert set(x)=={'schema','archiveSha256','archiveByteLength','tarSha256','tarByteLength','decoder'} + assert x['schema']=='openprose.sdk-python-transport/1' + assert x['archiveSha256']==r['sha256'] and type(x['archiveByteLength']) is int and x['archiveByteLength']==r['byteLength'] + assert re.fullmatch('[0-9a-f]{64}',x['tarSha256']) and type(x['tarByteLength']) is int and 0=next_audit: + audit();check_output(output) + require(shutil.disk_usage(output).free>=512*1024**2,'Owned scratch reserve exhausted') + next_audit=time.monotonic()+1 + for key,_ in events.select(min(.1,max(0,deadline-time.monotonic()))): + data=os.read(key.fileobj.fileno(),65536) + if not data: events.unregister(key.fileobj);continue + # Flush both streams before aggregate accounting; never write a chunk + # that would exceed the existing entire-job retained-byte quota. + tar.flush();log.flush() + require(check_output(output)+len(data)<=MAX_RETAINED_BYTES,'Owned output byte limit exceeded') + if key.data=='tar':tar.write(data) + else: + total+=len(data) + require(total<=MAX_LOG_BYTES,'Decoder diagnostic limit exceeded') + log.write(data) + p.wait(timeout=max(.01,deadline-time.monotonic())) + require(p.returncode==0,'Inspect retained decode-python.log') + tar.flush();log.flush();audit();check_output(output) + require(time.monotonic()0,'Empty decoded Python archive') + + +def prepare_transport(output, row, deadline, audit): + audit();verify_archive(output/'python-full.tar.zst',row) + executable=shutil.which('zstd') + require(executable is not None,'Host zstd prerequisite missing') + executable=Path(executable).resolve() + require(executable.is_file() and executable.stat().st_size<=32*1024**2,'Host zstd prerequisite invalid') + identity=sha(executable) + remaining=deadline-time.monotonic();require(remaining>0,'Preparation deadline exceeded') + raw=process([str(executable),'--version'],output,'decoder-version',timeout=min(5,remaining),max_bytes=4096,deadline=deadline) + version=raw.decode('utf-8').strip() + require(version and len(version)<=4096 and not any(ord(c)<32 for c in version),'Host zstd version invalid') + require(sha(executable)==identity,'Host zstd identity changed') + audit();verify_archive(output/'python-full.tar.zst',row) + stream_transport([str(executable),'--decompress','--stdout','--quiet','-M128MB',str(output/'python-full.tar.zst')],output,deadline,audit) + audit();verify_archive(output/'python-full.tar.zst',row) + require(sha(executable)==identity,'Host zstd identity changed') + plain=checked_path(output/'python-full.tar') + descriptor={'schema':'openprose.sdk-python-transport/1','archiveSha256':row['pythonArchive']['sha256'], + 'archiveByteLength':row['pythonArchive']['byteLength'],'tarSha256':sha(plain), + 'tarByteLength':plain.stat().st_size,'decoder':{'path':str(executable),'sha256':identity,'version':version}} + require(time.monotonic() Date: Wed, 7 Oct 2026 04:55:11 -0400 Subject: [PATCH 53/64] Carry complete Mac SDK payload through packaging and installation --- cli/README.md | 3 + cli/benchmarks/installed/benchmark.py | 318 ++++++++++++++++++--- cli/benchmarks/installed/test_benchmark.py | 117 +++++++- cli/ci/README.md | 14 + cli/ci/assemble_kernel_rc.py | 14 +- cli/ci/build_agents_sdk.py | 164 ++++++++++- cli/ci/build_kernel_rc.py | 106 ++++++- cli/ci/homebrew_rehearsal.py | 114 ++++++-- cli/ci/kernel_rc_evidence.py | 197 ++++++++++++- cli/ci/package_local.py | 92 +++++- cli/ci/publication.py | 127 ++++++-- cli/ci/qualify_installed_sdk.py | 144 ++++++++-- cli/ci/rehearse_release.py | 88 +++++- cli/ci/sdk_native_inventory.py | 318 +++++++++++++++++++++ cli/ci/sign_macos.py | 179 ++++++++++-- cli/ci/test_assemble_kernel_rc.py | 67 ++++- cli/ci/test_build_agents_sdk.py | 147 ++++++++++ cli/ci/test_build_kernel_rc.py | 88 +++++- cli/ci/test_homebrew_rehearsal.py | 128 ++++++++- cli/ci/test_kernel_rc_evidence.py | 217 ++++++++++++++ cli/ci/test_package_local.py | 123 +++++++- cli/ci/test_publication.py | 49 +++- cli/ci/test_qualify_installed_sdk.py | 147 +++++++++- cli/ci/test_rehearse_release.py | 61 ++++ cli/ci/test_sdk_native_inventory.py | 175 ++++++++++++ cli/ci/test_sign_macos.py | 103 ++++++- cli/conformance/runner/run.py | 122 +++++++- cli/conformance/runner/test_runner.py | 70 +++++ cli/protocol/OWNERSHIP.md | 19 ++ docs/cli-distribution.md | 19 ++ docs/cli-publication.md | 17 ++ docs/packaged-sdk-design.md | 12 +- 32 files changed, 3334 insertions(+), 225 deletions(-) diff --git a/cli/README.md b/cli/README.md index da9844c6..d4c5316c 100644 --- a/cli/README.md +++ b/cli/README.md @@ -254,6 +254,9 @@ The local built-in selects the packaged Agents SDK and `gpt-6.1-sol`, using `OPENAI_API_KEY` from the process environment. A complete installation needs no separately provisioned Python or PATH helper. Other harnesses remain explicit alternatives, saved in `~/.prose/cli.toml` or selected for one command. +Keep the installed files together when moving a standalone installation. Mac +packages include a support directory beside the CLI and SDK helper; moving only +the executable leaves an incomplete installation. The explicitly selected OpenProse-billed local route remains unavailable and fails closed. No adapter silently changes harness, credential group or billing owner. Harness- internal account/provider routing remains harness-managed. See diff --git a/cli/benchmarks/installed/benchmark.py b/cli/benchmarks/installed/benchmark.py index 01ae0778..468b10d2 100644 --- a/cli/benchmarks/installed/benchmark.py +++ b/cli/benchmarks/installed/benchmark.py @@ -1295,8 +1295,7 @@ def verify_package_output( sys.path.insert(0, ci_path) import kernel_rc_evidence as custody try: - sdk = custody.validate_sdk_archives(release, lambda name: { - path: data for path, (data, mode) in decode_archive_members(encoded[name], name).items()}) + sdk = custody.validate_sdk_archives(release, lambda name: sdk_modules()[0].decode_sdk_archive(encoded[name], release, label=name)) except (ValueError, KeyError, TypeError) as error: fail('IDENTITY_DIVERGENCE', str(error)) if 'agentsSdkPython' not in dependency['inventories']: @@ -1322,7 +1321,7 @@ def verify_package_output( for name, artifact in artifacts.items(): if artifact['kind'] not in ('standalone-archive', 'npm-platform'): continue - members = decode_archive_members(encoded[name], name) + members = decode_archive_members(encoded[name], name, manifest=release) native_receipts.extend(json_no_duplicates(data, 'SDK receipt') for path, (data, mode) in members.items() if path.endswith('/agents-sdk-build.json')) try: @@ -1453,9 +1452,42 @@ def decode_exact_gzip_tar(archive_bytes: bytes, label: str) -> bytes: return decoded +class SdkMembers(dict): + """Regular projection retained only after complete typed SDK validation.""" + def __init__(self, table, manifest): + super().__init__(table['files']) + self.table, self.manifest = table, manifest + + +def sdk_modules(): + ci = str(Path(__file__).resolve().parents[2] / 'ci') + if ci not in sys.path: + sys.path.insert(0, ci) + import publication + import kernel_rc_evidence + import sdk_native_inventory + return publication, kernel_rc_evidence, sdk_native_inventory + + +def sdk_support_regular_paths(members): + if not isinstance(members, SdkMembers): + return set() + _, custody, _ = sdk_modules() + prefix = custody.sdk_archive_prefix(members.table) + return {name for name in members if name.startswith(prefix + 'prose-agents-sdk-runtime/')} + + def decode_archive_members( - archive_bytes: bytes, label: str + archive_bytes: bytes, label: str, *, manifest=None ) -> dict[str, tuple[bytes, int]]: + if manifest is not None and isinstance(manifest.get('agentsSdk'), dict): + pub, custody, _ = sdk_modules() + try: + table = pub.decode_sdk_archive(archive_bytes, manifest, label=label) + custody.validate_sdk_archive_table(manifest, table) + except (ValueError, KeyError, TypeError) as error: + fail('IDENTITY_DIVERGENCE', str(error)) + return SdkMembers(table, manifest) decoded = decode_exact_gzip_tar(archive_bytes, label) try: archive = tarfile.open(fileobj=io.BytesIO(decoded), mode="r:") @@ -1516,6 +1548,25 @@ def extract_members( "INSTALL_ROOT_UNSAFE", f"owned extraction destination already exists: {destination}", ) + if isinstance(members, SdkMembers): + _, custody, _ = sdk_modules() + table, manifest = members.table, members.manifest + custody.validate_sdk_archive_table(manifest, table) + prefix = custody.sdk_archive_prefix(table) + destination.mkdir(mode=0o700) + sibling = destination.joinpath(*PurePosixPath(prefix.rstrip('/')).parts) + sibling.mkdir(parents=True, mode=0o755) + for name in sorted(members): + if name in {prefix + n for n in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')} or name.startswith(prefix + 'prose-agents-sdk-runtime/'): + continue + data, mode = members[name] + target = destination.joinpath(*safe_member_name(name).parts) + target.parent.mkdir(parents=True, exist_ok=True) + with target.open('xb') as output: + output.write(data) + target.chmod(mode) + custody.materialize_sdk_members(manifest, table, sibling) + return destination.mkdir(mode=0o700) for name in sorted(members): data, mode = members[name] @@ -1619,7 +1670,7 @@ def validate_npm_packages( platform_name = f"openprose-prose-cli-{platform_value}-{version}.tgz" meta_members = decode_archive_members(context["encoded"][meta_name], meta_name) platform_members = decode_archive_members( - context["encoded"][platform_name], platform_name + context["encoded"][platform_name], platform_name, manifest=release ) if set(meta_members) != { "package/package.json", @@ -1641,6 +1692,7 @@ def validate_npm_packages( expected_platform_members.add("package/bin/openprose-windows-process-host.exe") if isinstance(release.get('agentsSdk'), dict): expected_platform_members.update('package/bin/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) + expected_platform_members.update(sdk_support_regular_paths(platform_members)) if set(platform_members) != expected_platform_members: fail( "MEMBERSHIP_MALFORMED", @@ -1792,6 +1844,7 @@ def validate_npm_packages( "platformManifest": platform_manifest, "platformManifestBytes": platform_manifest_bytes, "binary": packaged_binary, + "members": platform_members, "sidecar": platform_members.get( "package/bin/openprose-windows-process-host.exe", (None, 0) )[0], @@ -2186,8 +2239,35 @@ def read_installed_regular(path: Path, expected: os.stat_result) -> tuple[int, s return length, observed_digest.hexdigest() +def sdk_tree_context(members, prefix=None): + if not isinstance(members, SdkMembers) or not members.manifest['platform'].startswith('darwin-'): + return None + _, custody, _ = sdk_modules() + archive_prefix = custody.sdk_archive_prefix(members.table) + encoded = members[archive_prefix + 'agents-sdk-build.json'][0] + return {'receiptSha256': sha256_bytes(encoded), 'prefix': (archive_prefix.rstrip('/') if prefix is None else prefix), + 'encoded': encoded.decode('utf-8')} + + +def sdk_directory_identity(payload, resolved): + rows = [row for row in payload['entries'] if row['path'] == resolved or row['path'].startswith(resolved + '/')] + return {'resolvedDirectorySha256': sha256_bytes(canonical_json(rows)), 'resolvedEntryCount': len(rows)} + + +def sdk_report_receipt(context, platform=None): + encoded = context['encoded'].encode('utf-8') + if not 0 < len(encoded) <= 2 * 1024 * 1024 or sha256_bytes(encoded) != context['receiptSha256']: + fail('REPORT_MALFORMED', 'SDK evidence receipt digest/length differs') + receipt = require_object(json_no_duplicates(encoded, 'installed SDK evidence'), 'installed SDK evidence') + if receipt.get('schema') != 'openprose.agents-sdk-build/1' or receipt.get('platform') != 'darwin': + fail('REPORT_MALFORMED', 'SDK evidence is not a Mac build receipt') + if platform is not None and (not isinstance(platform, str) or receipt.get('architecture') != {'darwin-arm64': 'arm64', 'darwin-x64': 'x86_64'}.get(platform)): + fail('REPORT_MALFORMED', 'SDK receipt architecture differs from report platform') + return receipt + + def capture_installed_tree( - root: Path, allowed_symlinks: Mapping[Path, Path] | None = None + root: Path, allowed_symlinks: Mapping[Path, Path] | None = None, *, sdk_context=None ) -> dict[str, Any]: try: root_metadata = root.lstat() @@ -2196,6 +2276,25 @@ def capture_installed_tree( fail("INSTALL_UNSAFE", f"cannot inspect installed tree root: {error}") if not stat.S_ISDIR(root_metadata.st_mode) or stat.S_ISLNK(root_metadata.st_mode): fail("INSTALL_UNSAFE", "installed tree root must be a non-symlink directory") + sdk_aliases = {} + sdk_receipt = None + if sdk_context is not None: + sdk_receipt = sdk_report_receipt(sdk_context) + prefix = safe_member_name(sdk_context['prefix']).as_posix() + sibling = root / prefix + _, _, inventory = sdk_modules() + inventory.read_macos_payload(sibling, sdk_receipt['payload'], sdk_receipt['architecture']) + actual_receipt = safe_read(sibling / 'agents-sdk-build.json', 2 * 1024 * 1024) + if actual_receipt != sdk_context['encoded'].encode('utf-8'): + fail('INSTALL_MUTATED', 'Installed SDK receipt differs from evidence') + for row in sdk_receipt['payload']['entries']: + if row['type'] == 'symlink': + sdk_aliases[(sibling / row['path']).absolute()] = row + allowed_symlinks = dict(allowed_symlinks or {}) + for path, row in sdk_aliases.items(): + if path in allowed_symlinks: + fail('INSTALL_UNSAFE', 'SDK alias conflicts with generic launcher link') + allowed_symlinks[path] = sibling / row['resolvedPath'] allowed: dict[Path, Path] = {} for raw_link, raw_target in (allowed_symlinks or {}).items(): link = raw_link.absolute() @@ -2317,26 +2416,36 @@ def capture_installed_tree( f"installed symlink identity changed: {relative}", ) target_metadata = resolved.lstat() - if not stat.S_ISREG(target_metadata.st_mode): - fail( - "INSTALL_UNSAFE", - f"installed symlink target is not regular: {relative}", + sdk_alias = sdk_aliases.get(path.absolute()) + if sdk_alias is not None and link_target != sdk_alias['target']: + fail('INSTALL_MUTATED', 'Installed SDK alias text changed') + if stat.S_ISDIR(target_metadata.st_mode) and sdk_alias is not None: + observed_members.extend(((path, observed), (resolved, target_metadata))) + entries.append({'path': relative, 'type': 'sdk-directory-symlink', + 'linkTarget': link_target, 'linkTextSha256': sha256_bytes(os.fsencode(link_target)), + 'resolvedPath': resolved_relative, + **sdk_directory_identity(sdk_receipt['payload'], sdk_alias['resolvedPath'])}) + else: + if not stat.S_ISREG(target_metadata.st_mode): + fail( + "INSTALL_UNSAFE", + f"installed symlink target is not regular: {relative}", + ) + target_length, target_sha = read_installed_regular( + resolved, target_metadata + ) + observed_members.extend(((path, observed), (resolved, target_metadata))) + entries.append( + { + "path": relative, + "type": "symlink", + "linkTarget": link_target, + "linkTextSha256": sha256_bytes(os.fsencode(link_target)), + "resolvedPath": resolved_relative, + "resolvedByteLength": target_length, + "resolvedSha256": target_sha, + } ) - target_length, target_sha = read_installed_regular( - resolved, target_metadata - ) - observed_members.extend(((path, observed), (resolved, target_metadata))) - entries.append( - { - "path": relative, - "type": "symlink", - "linkTarget": link_target, - "linkTextSha256": sha256_bytes(os.fsencode(link_target)), - "resolvedPath": resolved_relative, - "resolvedByteLength": target_length, - "resolvedSha256": target_sha, - } - ) seen_allowed.add(path.absolute()) else: fail( @@ -2391,19 +2500,73 @@ def capture_installed_tree( ) entries.sort(key=lambda entry: entry["path"]) payload = {"schema": INSTALLED_TREE_SCHEMA, "entries": entries} - return { + result = { "schema": INSTALLED_TREE_SCHEMA, "entryCount": len(entries), "directoryCount": sum(entry["type"] == "directory" for entry in entries), "regularFileCount": sum(entry["type"] == "regular" for entry in entries), - "symlinkCount": sum(entry["type"] == "symlink" for entry in entries), + "symlinkCount": sum(entry["type"] in {"symlink", "sdk-directory-symlink"} for entry in entries), "byteCount": total_bytes, "digestSha256": sha256_bytes(canonical_json(payload)), "entries": entries, } + if sdk_context is not None: + inventory.read_macos_payload(sibling, sdk_receipt['payload'], sdk_receipt['architecture']) + result['sdkEvidenceRef'] = {'receiptSha256': sdk_context['receiptSha256'], 'prefix': prefix} + return result + -def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: +def validate_sdk_tree_rows(identity, sdk_evidence): + reference = require_object(identity.get('sdkEvidenceRef'), 'SDK evidence reference') + require_exact_keys(reference, {'receiptSha256', 'prefix'}, 'SDK evidence reference') + digest = require_sha(reference['receiptSha256'], 'SDK evidence receipt digest') + prefix = safe_member_name(require_string(reference['prefix'], 'SDK evidence prefix')).as_posix() + if not isinstance(sdk_evidence, dict) or digest not in sdk_evidence: + fail('REPORT_MALFORMED', 'SDK tree lacks digest-bound evidence') + receipt = sdk_report_receipt({**reference, 'encoded': sdk_evidence[digest]}) + _, _, inventory = sdk_modules() + try: + inventory.validate_macos_payload_structure(receipt['payload'], receipt['architecture']) + except (ValueError, KeyError, TypeError) as error: + fail('REPORT_MALFORMED', str(error)) + entries = {row['path']: row for row in identity['entries']} + support = receipt['payload'] + expected = {} + by_path = {row['path']: row for row in support['entries']} + for row in support['entries']: + path = prefix + '/' + row['path'] + if row['type'] == 'directory': + expected[path] = {'path': path, 'type': 'directory', 'mode': row['mode']} + elif row['type'] == 'file': + expected[path] = {'path': path, 'type': 'regular', **{key: row[key] for key in ('mode', 'byteLength', 'sha256')}} + else: + target = by_path[row['resolvedPath']] + link = {'path': path, 'type': 'symlink', 'linkTarget': row['target'], + 'linkTextSha256': sha256_bytes(os.fsencode(row['target'])), + 'resolvedPath': prefix + '/' + row['resolvedPath']} + if target['type'] == 'directory': + link.update(type='sdk-directory-symlink', **sdk_directory_identity(support, row['resolvedPath'])) + else: + link.update(resolvedByteLength=target['byteLength'], resolvedSha256=target['sha256']) + expected[path] = link + root = prefix + '/' + support['root'] + if {path for path in entries if path == root or path.startswith(root + '/')} != set(expected): + fail('REPORT_MALFORMED', 'SDK installed support membership differs') + if any(entries[path] != row for path, row in expected.items()): + fail('REPORT_MALFORMED', 'SDK installed support identity differs') + for name, sha, length, mode in ( + ('prose-agents-sdk', receipt['helper']['sha256'], receipt['helper']['byteLength'], 0o755), + ('agents-sdk-build.json', digest, len(sdk_evidence[digest].encode('utf-8')), 0o644), + ('AGENTS-SDK-NOTICES.txt', receipt['notices']['sha256'], receipt['notices']['byteLength'], 0o644)): + path = prefix + '/' + name + expected_row = {'path': path, 'type': 'regular', 'sha256': sha, 'byteLength': length, 'mode': mode} + if entries.get(path) != expected_row: + fail('REPORT_MALFORMED', 'SDK installed sibling evidence differs') + return receipt + + +def validate_installed_tree_identity(value: Any, label: str, sdk_evidence=None) -> dict[str, Any]: identity = require_object(value, label) if ( set(identity) @@ -2416,7 +2579,7 @@ def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: "byteCount", "digestSha256", "entries", - } + } | ({"sdkEvidenceRef"} if "sdkEvidenceRef" in identity else set()) or identity.get("schema") != INSTALLED_TREE_SCHEMA ): fail("REPORT_MALFORMED", f"{label} has an unsupported shape") @@ -2444,7 +2607,7 @@ def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: require_sha(identity.get("digestSha256"), f"{label} digest") paths: list[str] = [] portable: set[str] = set() - counts = {"directory": 0, "regular": 0, "symlink": 0} + counts = {"directory": 0, "regular": 0, "symlink": 0, "sdk-directory-symlink": 0} byte_count = 0 for raw in entries: entry = require_object(raw, f"{label} entry") @@ -2476,6 +2639,17 @@ def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: fail("REPORT_MALFORMED", f"{label} regular entry length is invalid") require_sha(entry.get("sha256"), f"{label} regular entry digest") byte_count += length + elif kind == 'sdk-directory-symlink': + require_exact_keys(entry, {'path', 'type', 'linkTarget', 'linkTextSha256', 'resolvedPath', + 'resolvedDirectorySha256', 'resolvedEntryCount'}, 'SDK directory alias') + if 'sdkEvidenceRef' not in identity: + fail('REPORT_MALFORMED', 'SDK directory alias lacks payload evidence') + require_string(entry['linkTarget'], 'SDK directory alias target') + if entry['linkTextSha256'] != sha256_bytes(os.fsencode(entry['linkTarget'])): + fail('REPORT_MALFORMED', 'SDK directory alias text identity differs') + safe_member_name(entry['resolvedPath']) + require_sha(entry['resolvedDirectorySha256'], 'SDK directory identity') + require_positive_int(entry['resolvedEntryCount'], 'SDK directory entry count') elif kind == "symlink": if set(entry) != { "path", @@ -2524,7 +2698,7 @@ def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: if ( identity.get("directoryCount") != counts["directory"] or identity.get("regularFileCount") != counts["regular"] - or identity.get("symlinkCount") != counts["symlink"] + or identity.get("symlinkCount") != counts["symlink"] + counts["sdk-directory-symlink"] or identity.get("byteCount") != byte_count or identity.get("digestSha256") != sha256_bytes( @@ -2532,16 +2706,20 @@ def validate_installed_tree_identity(value: Any, label: str) -> dict[str, Any]: ) ): fail("REPORT_MALFORMED", f"{label} counts or digest differ from its entries") + if 'sdkEvidenceRef' in identity: + validate_sdk_tree_rows(identity, sdk_evidence) return identity def verify_installed_tree( root: Path, expected: Any, - allowed_symlinks: Mapping[Path, Path] | None = None, + allowed_symlinks: Mapping[Path, Path] | None = None, *, sdk_evidence=None, ) -> str: - validated = validate_installed_tree_identity(expected, "retained installed tree") - observed = capture_installed_tree(root, allowed_symlinks) + validated = validate_installed_tree_identity(expected, "retained installed tree", sdk_evidence) + reference = validated.get('sdkEvidenceRef') + context = {**reference, 'encoded': sdk_evidence[reference['receiptSha256']]} if reference else None + observed = capture_installed_tree(root, allowed_symlinks, sdk_context=context) if observed != validated: fail("INSTALL_MUTATED", f"retained installed tree identity differs: {root}") return observed["digestSha256"] @@ -2553,6 +2731,19 @@ def installed_tree_entries_by_path( return {entry["path"]: entry for entry in identity["entries"]} +def linux_sdk_sibling_paths(entries, prefix, platform): + """Admit only the existing onefile trio; source proof requires release context.""" + if not platform.startswith('linux-'): + return set() + rows = {prefix + '/' + name: mode for name, mode in + (('prose-agents-sdk', 0o755), ('agents-sdk-build.json', 0o644), ('AGENTS-SDK-NOTICES.txt', 0o644))} + present = set(rows) & set(entries) + if not present: return set() + if present != set(rows) or any(entries[path].get('type') != 'regular' or entries[path].get('mode') != mode for path, mode in rows.items()): + fail('REPORT_MALFORMED', 'Linux installed SDK requires its full regular canonical trio') + return set(rows) + + def validate_installation_tree_relationships( surface: str, identity: Mapping[str, Any], @@ -2562,6 +2753,12 @@ def validate_installation_tree_relationships( launcher: Mapping[str, Any], ) -> None: entries = installed_tree_entries_by_path(identity) + reference = identity.get('sdkEvidenceRef') + sdk_paths = set() + if reference: + prefix = reference['prefix'] + '/' + sdk_paths = {path for path in entries if path.startswith(prefix + 'prose-agents-sdk-runtime/') or path == prefix + 'prose-agents-sdk-runtime'} + sdk_paths.update(prefix + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) executable = "prose.exe" if platform_value.startswith("win32-") else "prose" if surface in {"direct-rust", "direct-bun"}: implementation = surface.removeprefix("direct-") @@ -2576,6 +2773,8 @@ def validate_installation_tree_relationships( } if platform_value.startswith("win32-"): expected_paths.add(f"{root_name}/openprose-windows-process-host.exe") + sdk_paths.update(linux_sdk_sibling_paths(entries, root_name, platform_value)) + expected_paths.update(sdk_paths) if set(entries) != expected_paths: fail("REPORT_MALFORMED", f"{surface} installed tree membership differs") if ( @@ -2620,6 +2819,8 @@ def validate_installation_tree_relationships( f"{platform_root}/bin", f"{platform_root}/bin/{executable}", } + sdk_paths.update(linux_sdk_sibling_paths(entries, platform_root + '/bin', platform_value)) + required_paths.update(sdk_paths) if set(entries) != required_paths: fail("REPORT_MALFORMED", "npm installed tree membership differs") directory_paths = { @@ -2633,7 +2834,7 @@ def validate_installation_tree_relationships( platform_root, f"{platform_root}/bin", } - regular_paths = required_paths - directory_paths - {"bin/prose"} + regular_paths = required_paths - directory_paths - {"bin/prose"} - sdk_paths if any(entries[path].get("type") != "directory" for path in directory_paths): fail("REPORT_MALFORMED", "npm installed tree directory type differs") if any(entries[path].get("type") != "regular" for path in regular_paths): @@ -2864,11 +3065,12 @@ def validate_package_payloads(context: Mapping[str, Any]) -> dict[str, Any]: f"openprose-prose-cli-{implementation}-{version}-{platform_value}.tar.gz" ) members = decode_archive_members( - context["encoded"][artifact_name], artifact_name + context["encoded"][artifact_name], artifact_name, manifest=release ) root_name, expected = expected_standalone_members(artifact_name, platform_value) if isinstance(context['release'].get('agentsSdk'), dict): expected.update(root_name + '/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) + expected.update(sdk_support_regular_paths(members)) if members.get(root_name + '/prose-agents-sdk', (b'', 0))[1] & 0o111 == 0: fail('ARCHIVE_UNSAFE', 'SDK helper is not executable') if set(members) != expected: @@ -2965,6 +3167,7 @@ def install_verified_package_set( "Windows measurement requires native Job Object containment before any process spawn", ) install = prepare_install_root(install_root) + sdk_evidence = {} installations: list[dict[str, Any]] = [] extracted = payloads["extracted"] for implementation in ("rust", "bun"): @@ -2980,7 +3183,10 @@ def install_verified_package_set( "INPUT_MUTATED", f"installed {implementation} binary differs from archive", ) - tree_identity = capture_installed_tree(destination) + sdk_context = sdk_tree_context(extracted[implementation]['members']) + if sdk_context: + sdk_evidence[sdk_context['receiptSha256']] = sdk_context['encoded'] + tree_identity = capture_installed_tree(destination, sdk_context=sdk_context) installations.append( { "surface": f"direct-{implementation}", @@ -3081,7 +3287,10 @@ def install_verified_package_set( fail("INSTALL_FAILED", "npm did not create the launcher command") command_identity = launcher_command_identity(command, launcher_source) allowed_links = {command: launcher_source} if command.is_symlink() else {} - npm_tree_identity = capture_installed_tree(npm_prefix, allowed_links) + sdk_context = sdk_tree_context(npm_package['members'], (platform_root / 'bin').relative_to(npm_prefix).as_posix()) + if sdk_context: + sdk_evidence[sdk_context['receiptSha256']] = sdk_context['encoded'] + npm_tree_identity = capture_installed_tree(npm_prefix, allowed_links, sdk_context=sdk_context) npm_report_argv = [ "$NPM", "install", @@ -3152,6 +3361,7 @@ def install_verified_package_set( **payloads, "install": install, "installations": installations, + "sdkEvidence": sdk_evidence, "npmTool": npm_tool, "nodeTool": node_tool, "npmPrefix": npm_prefix, @@ -3196,6 +3406,7 @@ def run_benchmark( release = context["release"] platform_value = context["platform"] version = release["version"] + sdk_evidence = {} installations: list[dict[str, Any]] = [] extracted: dict[str, dict[str, Any]] = {} for implementation in ("rust", "bun"): @@ -3204,11 +3415,12 @@ def run_benchmark( f"openprose-prose-cli-{implementation}-{version}-{platform_value}.tar.gz" ) members = decode_archive_members( - context["encoded"][artifact_name], artifact_name + context["encoded"][artifact_name], artifact_name, manifest=release ) root_name, expected = expected_standalone_members(artifact_name, platform_value) if isinstance(context['release'].get('agentsSdk'), dict): expected.update(root_name + '/' + name for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')) + expected.update(sdk_support_regular_paths(members)) if members.get(root_name + '/prose-agents-sdk', (b'', 0))[1] & 0o111 == 0: fail('ARCHIVE_UNSAFE', 'SDK helper is not executable') if set(members) != expected: @@ -3230,7 +3442,10 @@ def run_benchmark( f"{implementation} standalone binary is not executable", ) binary_bytes = safe_read(binary, MAX_MEMBER_BYTES) - tree_identity = capture_installed_tree(destination) + sdk_context = sdk_tree_context(members) + if sdk_context: + sdk_evidence[sdk_context['receiptSha256']] = sdk_context['encoded'] + tree_identity = capture_installed_tree(destination, sdk_context=sdk_context) installations.append( { "surface": f"direct-{implementation}", @@ -3334,7 +3549,10 @@ def run_benchmark( fail("INSTALL_FAILED", "npm did not create the launcher command") command_identity = launcher_command_identity(command, launcher_source) allowed_links = {command: launcher_source} if command.is_symlink() else {} - npm_tree_identity = capture_installed_tree(npm_prefix, allowed_links) + sdk_context = sdk_tree_context(npm_package['members'], (platform_root / 'bin').relative_to(npm_prefix).as_posix()) + if sdk_context: + sdk_evidence[sdk_context['receiptSha256']] = sdk_context['encoded'] + npm_tree_identity = capture_installed_tree(npm_prefix, allowed_links, sdk_context=sdk_context) npm_report_argv = [ "$NPM", "install", @@ -3602,6 +3820,10 @@ def run_benchmark( "runtimeNetworkIsolation": "not-enforced", }, } + if sdk_evidence: + report['sdkEvidence'] = sdk_evidence + if len(render_json(report)) > MAX_EVIDENCE_BYTES: + fail('REPORT_MALFORMED', 'Installed report exceeds existing evidence bound') remaining_before(deadline_monotonic, "before final installed-tree reauthentication") verify_retained_install_trees(install, report) remaining_before(deadline_monotonic, "after final installed-tree reauthentication") @@ -3625,13 +3847,19 @@ def analyse_report(report: Any) -> dict[str, Any]: "toolchain", "limitations", } - if set(value) != expected_top_level: + if set(value) != expected_top_level | ({"sdkEvidence"} if "sdkEvidence" in value else set()): fail( "REPORT_MALFORMED", "benchmark report has missing or unknown top-level fields", ) if value.get("schema") != REPORT_SCHEMA: fail("REPORT_MALFORMED", "unsupported installed benchmark report schema") + sdk_evidence = value.get('sdkEvidence') + if sdk_evidence is not None: + if not isinstance(sdk_evidence, dict) or len(sdk_evidence) != 1 or len(render_json(value)) > MAX_EVIDENCE_BYTES: + fail('REPORT_MALFORMED', 'SDK evidence dictionary shape/size differs') + for digest, encoded in sdk_evidence.items(): + sdk_report_receipt({'receiptSha256': require_sha(digest, 'SDK evidence key'), 'encoded': require_string(encoded, 'SDK encoded evidence')}, value.get('platform')) platform_value = value.get("platform") if platform_value not in SUPPORTED_PLATFORMS or str(platform_value).startswith( "win32-" @@ -3994,6 +4222,8 @@ def analyse_report(report: Any) -> dict[str, Any]: install_summary = [] if len(installations) != 3: fail("REPORT_MALFORMED", "benchmark must contain exactly three installations") + if sdk_evidence and any(record.get('treeIdentity', {}).get('sdkEvidenceRef', {}).get('receiptSha256') not in sdk_evidence for record in installations): + fail('REPORT_MALFORMED', 'SDK evidence is not referenced by every installed surface') npm_installer: dict[str, Any] | None = None for index, item in enumerate(installations): record = require_object(item, "installation record") @@ -4020,7 +4250,7 @@ def analyse_report(report: Any) -> dict[str, Any]: record.get("installedByteCount"), "installed byte count" ) tree_identity = validate_installed_tree_identity( - record.get("treeIdentity"), f"installation tree {surface}" + record.get("treeIdentity"), f"installation tree {surface}", sdk_evidence ) if tree_identity["byteCount"] != installed_bytes: fail( @@ -4267,7 +4497,7 @@ def verify_retained_install_trees(install_root: Path, report: Any) -> dict[str, verified[surface] = verify_installed_tree( tree_root, records[surface]["treeIdentity"], - allowed, + allowed, sdk_evidence=value.get("sdkEvidence"), ) return verified diff --git a/cli/benchmarks/installed/test_benchmark.py b/cli/benchmarks/installed/test_benchmark.py index 20a1e994..43ad4957 100644 --- a/cli/benchmarks/installed/test_benchmark.py +++ b/cli/benchmarks/installed/test_benchmark.py @@ -89,8 +89,14 @@ def write_tar(path: Path, members: list[tuple[str, bytes, int | str]]) -> None: if isinstance(mode_or_type, int): info.mode = mode_or_type archive.addfile(info, io.BytesIO(data)) + elif isinstance(mode_or_type, tuple): + info.type = tarfile.DIRTYPE + info.mode = mode_or_type[1] + info.size = 0 + archive.addfile(info) else: info.type = tarfile.SYMTYPE + info.mode = 0o777 info.linkname = mode_or_type info.size = 0 archive.addfile(info) @@ -717,8 +723,11 @@ def release_platform_manifest(members): ci_path = str(ROOT / 'cli/ci') if ci_path not in sys.path: sys.path.insert(0, ci_path) - from test_assemble_kernel_rc import sdk_fixture - sdk, sdk_members = sdk_fixture(platform_value) + from test_kernel_rc_evidence import sdk_fixture + sdk, sdk_table = sdk_fixture(platform_value) + sdk_members = [(n, d, m) for n, (d, m) in sdk_table["files"].items()] + sdk_members += [(n, b"", ("directory", m)) for n, m in sdk_table["directories"].items()] + sdk_members += [(n, b"", target) for n, target in sdk_table["symlinks"].items()] manifest['agentsSdk'] = sdk for artifact in manifest['artifacts']: if artifact['kind'] in ('standalone-archive', 'npm-platform'): @@ -799,6 +808,8 @@ def rewrite_tar(path: Path, mutate) -> None: members.append((info.name, extracted.read(), info.mode)) elif info.issym(): members.append((info.name, b"", info.linkname)) + elif info.isdir(): + members.append((info.name, b"", ("directory", info.mode))) mutate(members) write_tar(path, members) @@ -2053,5 +2064,107 @@ def test_artifact_native_sbom_poison_is_rejected_after_rehash(self): self.assertEqual(rejected.exception.message, 'Native SDK SBOM differs from bound receipt') + +class CompleteSdkInstalledTreeTests(unittest.TestCase): + def fixture(self, root): + BENCHMARK.sdk_modules() + from test_kernel_rc_evidence import sdk_fixture + from test_sdk_native_inventory import onedir_fixture + _, _, inventory = BENCHMARK.sdk_modules() + sdk, table = sdk_fixture('darwin-x64') + view = onedir_fixture() + sibling = root / 'installed'; sibling.mkdir() + inventory.materialize_macos_payload(sibling, view['payload'], view['files'], view['directories'], view['symlinks'], view['architecture']) + for name in ('agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + data, mode = table['files'][name] + (sibling / name).write_bytes(data); (sibling / name).chmod(mode) + encoded = table['files']['agents-sdk-build.json'][0].decode() + context = {'receiptSha256': sdk['receiptSha256'], 'prefix': 'installed', 'encoded': encoded} + return sibling, context, {sdk['receiptSha256']: encoded} + + def test_complete_alias_tree_capture_and_retained_reauthentication(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); sibling, context, evidence = self.fixture(root) + identity = BENCHMARK.capture_installed_tree(root, sdk_context=context) + aliases = [row for row in identity['entries'] if row['type'] == 'sdk-directory-symlink'] + self.assertEqual(len(aliases), 1) + BENCHMARK.validate_installed_tree_identity(identity, 'SDK fixture', evidence) + self.assertEqual(BENCHMARK.verify_installed_tree(root, identity, sdk_evidence=evidence), identity['digestSha256']) + (sibling / 'prose-agents-sdk-runtime/base_library.zip').write_bytes(b'mutation') + with self.assertRaises(ValueError): + BENCHMARK.verify_installed_tree(root, identity, sdk_evidence=evidence) + + def test_generic_directory_alias_stays_forbidden(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); target = root / 'directory'; target.mkdir(); link = root / 'link'; link.symlink_to(target) + with self.assertRaises(BENCHMARK.BenchmarkError): + BENCHMARK.capture_installed_tree(root, {link: target}) + + def test_payload_poison_with_rehashed_report_identity_is_refused(self): + import copy + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); _, context, evidence = self.fixture(root) + valid = BENCHMARK.capture_installed_tree(root, sdk_context=context) + for poison in ('missing-reference', 'wrong-evidence', 'directory-hash', 'extra-support', 'link-text', 'helper-length', 'helper-mode', 'receipt-length', 'notices-length', 'notices-mode'): + changed = copy.deepcopy(valid) + if poison == 'missing-reference': changed.pop('sdkEvidenceRef') + elif poison == 'wrong-evidence': changed['sdkEvidenceRef']['receiptSha256'] = '0' * 64 + elif poison == 'directory-hash': + next(row for row in changed['entries'] if row['type'] == 'sdk-directory-symlink')['resolvedDirectorySha256'] = '0' * 64 + elif poison == 'extra-support': + changed['entries'].append({'path': 'installed/prose-agents-sdk-runtime/foreign', 'type': 'directory', 'mode': 0o755}) + changed['entries'].sort(key=lambda r: r['path']); changed['entryCount'] += 1; changed['directoryCount'] += 1 + elif poison in ('helper-length', 'helper-mode', 'receipt-length', 'notices-length', 'notices-mode'): + name = 'prose-agents-sdk' if poison.startswith('helper') else ('agents-sdk-build.json' if poison.startswith('receipt') else 'AGENTS-SDK-NOTICES.txt') + row = next(row for row in changed['entries'] if row['path'] == 'installed/' + name) + if poison.endswith('length'): row['byteLength'] += 1; changed['byteCount'] += 1 + else: row['mode'] = 0o644 if name == 'prose-agents-sdk' else 0o755 + else: + row = next(row for row in changed['entries'] if row['type'] == 'sdk-directory-symlink') + row['linkTarget'] = '../escape'; row['linkTextSha256'] = BENCHMARK.sha256_bytes(b'../escape') + changed['digestSha256'] = BENCHMARK.sha256_bytes(BENCHMARK.canonical_json({'schema': BENCHMARK.INSTALLED_TREE_SCHEMA, 'entries': changed['entries']})) + with self.subTest(poison=poison), self.assertRaises(BENCHMARK.BenchmarkError): + BENCHMARK.validate_installed_tree_identity(changed, 'SDK fixture', evidence) + + def test_linux_onefile_trio_stays_closed_without_mac_support_reference(self): + import copy + BENCHMARK.sdk_modules() + from test_kernel_rc_evidence import sdk_fixture + platform = 'linux-x64-gnu'; version = '0.15.0-rc.4' + name = 'openprose-prose-cli-rust-' + version + '-' + platform + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); sibling = root / name; sibling.mkdir() + (sibling / 'examples').mkdir() + for path in ('LICENSE', 'README.txt', 'examples/hello.prose.md', 'prose'): + (sibling / path).write_bytes(b'fixture'); (sibling / path).chmod(0o755 if path == 'prose' else 0o644) + _, table = sdk_fixture(platform) + for path, (data, mode) in table['files'].items(): + (sibling / path).write_bytes(data); (sibling / path).chmod(mode) + identity = BENCHMARK.capture_installed_tree(root) + surfaces = {'direct-rust': {'binarySha256': BENCHMARK.sha256_bytes(b'fixture')}} + BENCHMARK.validate_installation_tree_relationships('direct-rust', identity, version, platform, surfaces, {}) + self.assertNotIn('sdkEvidenceRef', identity) + for poison in ('partial', 'mode', 'alias', 'support'): + changed = copy.deepcopy(identity) + if poison == 'partial': changed['entries'] = [r for r in changed['entries'] if not r['path'].endswith('/agents-sdk-build.json')] + elif poison == 'mode': next(r for r in changed['entries'] if r['path'].endswith('/prose-agents-sdk'))['mode'] = 0o644 + elif poison == 'alias': next(r for r in changed['entries'] if r['path'].endswith('/prose-agents-sdk'))['type'] = 'symlink' + else: changed['entries'].append({'path': name + '/prose-agents-sdk-runtime', 'type': 'directory', 'mode': 0o755}) + with self.subTest(poison=poison), self.assertRaises(BENCHMARK.BenchmarkError): + BENCHMARK.validate_installation_tree_relationships('direct-rust', changed, version, platform, surfaces, {}) + + def test_synthetic_report_dictionary_avoids_receipt_duplication(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); _, context, evidence = self.fixture(root) + identity = BENCHMARK.capture_installed_tree(root, sdk_context=context) + report = {'sdkEvidence': evidence, 'trees': [identity, identity, identity]} + encoded = BENCHMARK.render_json(report) + self.assertLess(len(encoded), BENCHMARK.MAX_EVIDENCE_BYTES) + with self.assertRaisesRegex(BENCHMARK.BenchmarkError, 'architecture differs'): + BENCHMARK.sdk_report_receipt(context, 'darwin-arm64') + self.assertEqual(len(evidence), 1) + self.assertEqual(len([t for t in report['trees'] if t['sdkEvidenceRef']['receiptSha256'] in evidence]), 3) + self.synthetic_size = len(encoded) + if __name__ == "__main__": unittest.main() diff --git a/cli/ci/README.md b/cli/ci/README.md index ea37a108..b8812c7e 100644 --- a/cli/ci/README.md +++ b/cli/ci/README.md @@ -45,6 +45,20 @@ Products transport opaque image/task bytes, spawn argument arrays without an outer shell or PTY, and do not infer OpenProse semantics or install ambient skills. Black-box tests supply behavioral evidence alongside static checks. +## Packaged SDK checks + +Mac SDK construction uses PyInstaller's directory layout. Admission binds every +physical support file, directory and framework alias, complete COLLECT membership, +actual Mach-O architecture/signature targets and final hashes. Archive consumers, +fresh installation, upgrade, Homebrew and relocation verify the complete tree. +Linux retains the single-file layout and pinned native supplier/ELF closure checks. + +Provider-free adapter-oracle fixtures substitute an inert helper. Their recorded +production source identity and fixture substitution remain distinct from actual +packaged-helper execution. Synthetic tree and poison controls establish rejection +behavior; native jobs must separately prove real payload size, cold startup and +installation. The helper's five-second version-probe deadline remains unchanged. + ## Remote checks and release [Source admission](../../.github/workflows/cli-ci.yml) runs the full command on diff --git a/cli/ci/assemble_kernel_rc.py b/cli/ci/assemble_kernel_rc.py index 92b39728..bc711834 100644 --- a/cli/ci/assemble_kernel_rc.py +++ b/cli/ci/assemble_kernel_rc.py @@ -52,16 +52,21 @@ def assemble(roots, output, evidence, live_smoke=None): inventory[name] = (actual, record) artifact_names = {a['path'] for a in manifest['artifacts']} pub.require(set(custody.CHECK_PATHS).issubset(report['evidence']), 'Required structured evidence is missing') - custody.validate_sdk_archives(manifest, lambda name: pub.archive_members(package / name)) + custody.validate_sdk_archives(manifest, lambda name: pub.read_sdk_archive(package / name, manifest)) for relative in custody.SDK_PROBES: pub.require(relative in report['evidence'], 'Missing installed SDK evidence') pub.require(pub.read_json(root / relative) == (custody.SDK_TOOL_TEST if 'sdk-tools-' in relative else custody.SDK_IMPORT_TEST), 'Installed SDK probe differs') native_hashes = {} launcher_hash = None for artifact in manifest['artifacts']: - members = pub.archive_members(package / artifact['path']) + if artifact['kind'] == 'npm-meta': + members = pub.archive_members(package / artifact['path']) + else: + table = pub.read_sdk_archive(package / artifact['path'], manifest) + custody.validate_sdk_archive_table(manifest, table) + members = {name: data for name, (data, _) in table['files'].items()} if artifact['kind'] == 'standalone-archive': - binaries = [data for name, data in members.items() if name.endswith('/prose')] + binaries = [members[custody.sdk_archive_prefix(table) + 'prose']] pub.require(len(binaries) == 1, 'Expected one standalone binary') native_hashes[(artifact['implementation'], platform)] = hashlib.sha256(binaries[0]).hexdigest() elif artifact['kind'] == 'npm-meta': @@ -69,6 +74,9 @@ def assemble(roots, output, evidence, live_smoke=None): launcher_hash = hashlib.sha256(members['package/bin/prose.js']).hexdigest() checks = {name: pub.read_json(root / ('logs/' + name + '.json')) for name in custody.CHECKS} custody.validate_native(report, manifest, checks, native_hashes, launcher_hash) + pub.require(custody.SDK_PAYLOAD_EVIDENCE in report['evidence'], 'Missing complete installed SDK payload evidence') + custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE), manifest, table) + custody.validate_sdk_producer_evidence(manifest, table, report['evidence'], lambda relative: (root / relative).read_bytes()) all_binary_hashes.update(native_hashes) # Retain every report-bound input, including the five structured native # observations. Publication independently repeats these bindings. diff --git a/cli/ci/build_agents_sdk.py b/cli/ci/build_agents_sdk.py index eb635e94..79a7213a 100644 --- a/cli/ci/build_agents_sdk.py +++ b/cli/ci/build_agents_sdk.py @@ -2,6 +2,7 @@ """Freeze the package-owned SDK helper without provider calls or ambient credentials.""" from __future__ import annotations import argparse +import ast import hashlib import importlib.metadata as metadata import json @@ -9,10 +10,12 @@ from pathlib import Path import platform import re +import selectors import shutil import subprocess import sys import tempfile +import time import sdk_native_inventory as native ROOT = Path(__file__).resolve().parents[2] @@ -185,6 +188,139 @@ def inspect_archive(helper): return rows + +def verify_collect_toc(path, view): + """Bind generated COLLECT member/type/alias declarations to final tree. + + Native signing changes source bytes; the final payload separately binds + signed output bytes. Retain this producer TOC as hashed build evidence. + """ + data = native.read_file(path, native.MAX_METADATA) + try: value = ast.literal_eval(data.decode('utf-8')) + except (ValueError, SyntaxError, UnicodeDecodeError, RecursionError) as error: + raise ValueError('Invalid SDK COLLECT declaration') from error + require(type(value) is tuple and len(value) == 1 and isinstance(value[0], list) and + 0 < len(value[0]) <= native.SDK_PAYLOAD_MAX_ENTRIES, 'Unsupported SDK COLLECT shape') + files = {}; links = {}; parents = {native.SDK_PAYLOAD_ROOT} + for row in value[0]: + require(type(row) is tuple and len(row) == 3 and all(isinstance(x, str) for x in row), 'Invalid SDK COLLECT member') + name, source, kind = row; native._payload_path(name) + if kind == 'EXECUTABLE': + require(name == NAME, 'Unexpected SDK COLLECT executable'); destination = name + else: + require(kind in ('BINARY', 'EXTENSION', 'DATA', 'SYMLINK'), 'Unsupported SDK COLLECT type') + destination = native.SDK_PAYLOAD_ROOT + '/' + name + require(destination not in files and destination not in links, 'Duplicate SDK COLLECT destination') + if kind == 'SYMLINK': + native._payload_target(source); links[destination] = source + else: + files[destination] = kind + if destination != NAME: + parent = Path(destination).parent + while str(parent) != '.': + parents.add(parent.as_posix()); parent = parent.parent + require(set(files) == set(view['files']) and links == view['symlinks'] and parents == set(view['directories']), + 'SDK COLLECT physical membership/aliases differ from final tree') + return native.sha(data) + + +def verify_frozen_module_closure(helper): + # Build-time only: do not import this parser from the frozen stdlib inventory. + from PyInstaller.archive.readers import CArchiveReader + native.read_file(helper) + archive = CArchiveReader(str(helper)); require(len(archive.toc) <= native.SDK_PAYLOAD_MAX_ENTRIES, + 'SDK executable archive exceeds member bound') + modules = []; nested_count = 0 + for name, row in archive.toc.items(): + native.safe_path(name) + require(not (name == 'PyInstaller' or name.startswith(('PyInstaller.', 'PyInstaller/'))), + 'Build-tool PyInstaller is present in consumer archive') + if row[-1] == 'z': + embedded = archive.open_embedded_archive(name) + require(len(embedded.toc) <= native.SDK_PAYLOAD_MAX_ENTRIES, 'SDK Python module archive exceeds member bound') + nested_count += len(embedded.toc) + for module in embedded.toc: + require(isinstance(module, str) and not (module == 'PyInstaller' or module.startswith('PyInstaller.')), + 'Build-tool PyInstaller is present in consumer modules') + modules.append(module) + require(nested_count > 0, 'SDK consumer Python archive is absent') + return {'archiveMembers': len(archive.toc), 'pythonModules': nested_count, + 'buildToolNamespaceAbsent': True} + + +def _capture_signature(command, *, env, cwd, timeout, output_limit=65536): + """Bound both output streams while the fixed native verification runs.""" + deadline = time.monotonic() + timeout + process = subprocess.Popen(command, env=env, cwd=cwd, stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + chunks = {'stdout': bytearray(), 'stderr': bytearray()}; counts = {'stdout': 0, 'stderr': 0} + timed_out = overflow = False; code = None + try: + with selectors.DefaultSelector() as selector: + selector.register(process.stdout, selectors.EVENT_READ, 'stdout') + selector.register(process.stderr, selectors.EVENT_READ, 'stderr') + while selector.get_map(): + remaining = deadline - time.monotonic() + if remaining <= 0: + timed_out = True; break + for key, _ in selector.select(min(remaining, 0.1)): + data = os.read(key.fileobj.fileno(), 8192) + if not data: + selector.unregister(key.fileobj); continue + name = key.data; counts[name] += len(data) + available = max(0, output_limit - len(chunks[name])); chunks[name].extend(data[:available]) + if counts[name] > output_limit: overflow = True; break + if overflow: break + if not timed_out and not overflow: + try: code = process.wait(timeout=max(0.001, deadline - time.monotonic())) + except subprocess.TimeoutExpired: timed_out = True + finally: + if process.poll() is None: + process.kill() + process.wait(timeout=1) + if code is None: code = process.returncode + process.stdout.close(); process.stderr.close() + return {'exitCode': code, 'stdout': bytes(chunks['stdout']), 'stderr': bytes(chunks['stderr']), + 'stdoutBytes': counts['stdout'], 'stderrBytes': counts['stderr'], + 'timedOut': timed_out, 'outputLimitExceeded': overflow} + + +def verify_code_signature(output, relative, log_name, *, env, deep=False): + native._payload_path(relative) + command = ['/usr/bin/codesign', '--verify', '--strict'] + if deep: command.append('--deep') + command.append(str(output / relative)) + result = _capture_signature(command, env=env, cwd=output, timeout=30) + proof = {'schema': 'openprose.sdk-code-signature-check/1', 'phase': 'code-signature', + 'path': relative, 'verification': 'strict-deep' if deep else 'strict', + **{k: result[k] for k in ('exitCode', 'stdoutBytes', 'stderrBytes', 'timedOut', 'outputLimitExceeded')}, + 'timeoutSeconds': 30, 'success': result['exitCode'] == 0 and + not result['timedOut'] and not result['outputLimitExceeded']} + proof['outputComplete'] = not result['timedOut'] and not result['outputLimitExceeded'] + raw = {} + for name in ('stdout', 'stderr'): + if result[name]: + path = output / (log_name + '.' + name); path.write_bytes(result[name]) + raw[name] = {'path': path.name, 'sha256': digest(path), 'byteLength': len(result[name])} + proof['rawOutput'] = raw + (output / log_name).write_text(json.dumps(proof, sort_keys=True) + '\n') + require(proof['success'], 'SDK code signature verification failed; inspect retained proof') + return proof + + +def verify_macos_signatures(output, payload, *, env): + # Paths are independently derived from actual final bytes by the full view + # validator. Verify every native regular file, including DATA/BINARY members. + view = native.read_macos_payload(output, payload, platform.machine()) + for index, path in enumerate(payload['codeSignaturePaths']): + require(native.macho_architectures(view['files'][path][0]) == (platform.machine(),), 'SDK signing target mismatch') + verify_code_signature(output, path, 'codesign-' + str(index).zfill(4) + '.log', env=env) + # Preserve the existing helper-level strict/deep verification as well. + verify_code_signature(output, NAME, 'codesign.log', env=env, deep=True) + require(native.read_macos_payload(output, payload, platform.machine()) == view, + 'SDK final signed tree changed during verification') + + def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_native_origin=None): require(platform.python_version() == '3.10.20', 'SDK release builder requires Python 3.10.20') require(sys.platform in ('darwin', 'linux') and platform.machine() in ('arm64', 'aarch64', 'x86_64'), 'SDK builder requires native supported POSIX host') @@ -226,18 +362,22 @@ def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_n # Snapshot the concurrently maintained runtime before freezing. Receipt binds exact bytes. source_copy = output / 'prose_sdk_runtime.py'; source_copy.write_bytes(SOURCE.read_bytes()) source_sha = digest(source_copy) + builder_sources = {name: digest(ROOT / name) for name in native.SDK_BUILDER_SOURCES} native_source = Path(native.__file__) (output / 'sdk_native_inventory.py').write_bytes(native.read_file(native_source)) if native_input is not None: require(source_sha == native_input['sourceSnapshot']['sources']['harnesses/agents-sdk/run.py'] and digest(output / 'sdk_native_inventory.py') == native_input['sourceSnapshot']['sources']['cli/ci/sdk_native_inventory.py'], 'Frozen source copies differ from bound source snapshot') - args = [sys.executable, '-m', 'PyInstaller', '--clean', '--noconfirm', '--onefile', '--noupx', + args = [sys.executable, '-m', 'PyInstaller', '--clean', '--noconfirm', + '--onedir' if sys.platform == 'darwin' else '--onefile', '--noupx', '--name', NAME, '--distpath', output / 'dist', '--workpath', output / 'work', '--specpath', output, '--paths', output, '--hidden-import', 'prose_sdk_runtime', '--collect-all', 'agents', '--collect-all', 'openai', '--collect-all', 'certifi', '--copy-metadata', 'openai-agents', '--copy-metadata', 'openai', '--add-data', str(output / NOTICES) + os.pathsep + '.', entry] + if sys.platform == 'darwin': + args.extend(['--contents-directory', native.SDK_PAYLOAD_ROOT, '--target-arch', platform.machine()]) args.extend(['--hidden-import', 'sdk_native_inventory']) if native_input is not None: args.extend(['--add-binary', str(linux_libgcc) + os.pathsep + '.']) @@ -246,7 +386,19 @@ def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_n for package in packages: args.extend(['--copy-metadata', package['name']]) run(args, env=env, cwd=output, log=output / 'freeze.log') - helper = output / NAME; shutil.copyfile(output / 'dist' / NAME, helper); helper.chmod(0o755) + helper = output / NAME; payload = None + if sys.platform == 'darwin': + dist_root = output / 'dist' / NAME + view = native._read_payload_view(dist_root) + collect_path = output / 'work' / NAME / 'COLLECT-00.toc' + collect_sha = verify_collect_toc(collect_path, view) + (output / 'collect.toc').write_bytes(native.read_file(collect_path, native.MAX_METADATA)) + payload = native.inventory_macos_payload(dist_root, platform.machine(), builder_sources, digest(entry), collect_sha) + native.materialize_macos_payload(output, payload, architecture=platform.machine(), **view) + verify_frozen_module_closure(helper) + verify_macos_signatures(output, payload, env=env) + else: + shutil.copyfile(output / 'dist' / NAME, helper); helper.chmod(0o755) require(0 < helper.stat().st_size <= MAX_BYTES, 'Frozen SDK helper exceeds size limit') native_dependencies = None if native_input is not None: @@ -283,7 +435,9 @@ def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_n maximum = tuple(int(x) for x in linux_libraries['requiredGlibcMaximum'].split('.')) require(maximum <= (2, 34), 'Frozen SDK libraries require glibc newer than 2.34') if sys.platform == 'darwin': - run(['/usr/bin/codesign', '--verify', '--deep', '--strict', helper], env=env, cwd=output, log=output / 'codesign.log', timeout=30) + native.read_macos_payload(output, payload, platform.machine()) + require(builder_sources == {name: digest(ROOT / name) for name in native.SDK_BUILDER_SOURCES}, + 'Mac SDK builder source changed during freeze') signing = 'developer-id-embedded' if codesign_identity is not None else 'ad-hoc-integrity-only' receipt = {'schema': 'openprose.agents-sdk-build/1', 'helper': {'path': NAME, 'sha256': digest(helper), 'byteLength': helper.stat().st_size}, 'python': platform.python_version(), 'pyinstaller': metadata.version('pyinstaller'), @@ -295,6 +449,10 @@ def build(output, *, epoch=0, codesign_identity=None, linux_libgcc=None, linux_n 'toolSelfTest': tool_self_test, 'linuxLibraries': linux_libraries, 'pythonExecutableSha256': digest(Path(sys.executable).resolve()), 'authority': 'local-build-and-imports-only', 'publicationAuthorized': False} + if payload is not None: + receipt['payload'] = payload + require(len(json.dumps(receipt, sort_keys=True).encode('utf-8')) <= native.SDK_PAYLOAD_MAX_METADATA, + 'Mac SDK complete receipt exceeds metadata bound') if native_input is not None: final_input, final_sha, _ = native.validate_input(linux_native_origin, linux_libgcc, ROOT, target) final_rows, final_origins = native.assign_origins(archive_rows, final_input, packages, metadata.distribution) diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index 55563c10..36ad5225 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 """Build and install an unsigned RC with published-kernel startup, without model calls.""" import argparse +import base64 import hashlib import json import os @@ -11,6 +12,9 @@ import subprocess import sys import tarfile +import publication as pub +import kernel_rc_evidence as sdk_custody +import sdk_native_inventory as sdk_inventory ROOT = Path(__file__).resolve().parents[2] RC = re.compile(r'(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)-rc\.(0|[1-9][0-9]*)') @@ -73,8 +77,76 @@ def prepare_macos_binary(binary, env, logs): cwd=ROOT, log=logs / 'rust-ad-hoc-verify.log', timeout=60) -def extract_binary(archive, output, *, require_sdk=False): +def sdk_producer_evidence(output, sdk_output): + """Retain producer COLLECT and actual signature execution sidecars.""" + receipt = json.loads(sdk_inventory.read_file(sdk_output / 'agents-sdk-build.json', 2 * 1024 * 1024), object_pairs_hook=pub.object_pairs) + if receipt.get('platform') != 'darwin': + return {} + payload = receipt['payload']; paths = [sdk_output / 'collect.toc'] + require(paths[0].is_file() and not paths[0].is_symlink() and 0 < paths[0].stat().st_size <= 2 * 1024 * 1024, 'SDK COLLECT evidence must be bounded regular bytes') + collect = sdk_inventory.read_file(paths[0], 2 * 1024 * 1024) + require(hashlib.sha256(collect).hexdigest() == payload['collectTocSha256'], 'SDK COLLECT evidence differs') + checks = [('codesign-' + str(i).zfill(4) + '.log', path, 'strict') + for i, path in enumerate(payload['codeSignaturePaths'])] + checks.append(('codesign.log', 'prose-agents-sdk', 'strict-deep')) + check_rows = []; raw_outputs = {} + for name, target, verification in checks: + path = sdk_output / name + require(path.is_file() and not path.is_symlink() and 0 < path.stat().st_size <= 128 * 1024, 'SDK signature sidecar missing or unbounded') + proof = json.loads(sdk_inventory.read_file(path, 128 * 1024), object_pairs_hook=pub.object_pairs) + require(isinstance(proof, dict) and set(proof) == {'schema', 'phase', 'path', 'verification', 'exitCode', 'stdoutBytes', 'stderrBytes', 'timedOut', 'outputLimitExceeded', 'timeoutSeconds', 'success', 'outputComplete', 'rawOutput'} and proof.get('phase') == 'code-signature' and proof.get('schema') == 'openprose.sdk-code-signature-check/1' and + proof.get('path') == target and proof.get('verification') == verification and + proof.get('exitCode') == 0 and type(proof.get('exitCode')) is int and + proof.get('timeoutSeconds') == 30 and proof.get('success') is True and + proof.get('outputComplete') is True and proof.get('timedOut') is False and + proof.get('outputLimitExceeded') is False, 'SDK signature sidecar is not a completed verification') + paths.append(path); check_rows.append({'log': name, 'proof': proof}) + raw = proof.get('rawOutput') + require(isinstance(raw, dict) and set(raw) <= {'stdout', 'stderr'} and + all(type(proof.get(stream + 'Bytes')) is int and 0 <= proof[stream + 'Bytes'] <= 65536 and + (stream in raw) == (proof[stream + 'Bytes'] > 0) for stream in ('stdout', 'stderr')), 'SDK raw signature output differs') + for stream, record in raw.items(): + require(isinstance(record, dict) and set(record) == {'path', 'sha256', 'byteLength'} and record['path'] == name + '.' + stream, 'SDK raw signature output path differs') + raw_path = sdk_output / record['path'] + require(raw_path.is_file() and not raw_path.is_symlink() and type(record['byteLength']) is int and + 0 < record['byteLength'] <= 65536 and raw_path.stat().st_size == record['byteLength'] and + proof.get(stream + 'Bytes') == record['byteLength'], 'SDK raw signature output custody differs') + paths.append(raw_path) + raw_data = sdk_inventory.read_file(raw_path, 65536) + require(len(raw_data) == record['byteLength'] and hashlib.sha256(raw_data).hexdigest() == record['sha256'], 'SDK raw signature output changed while bundling') + raw_outputs[raw_path.name] = {'sha256': record['sha256'], 'byteLength': record['byteLength'], + 'base64': base64.b64encode(raw_data).decode('ascii')} + aggregate = {'schema': 'openprose.sdk-code-signatures/1', 'architecture': receipt['architecture'], + 'payloadSha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'checks': check_rows, 'rawOutputs': raw_outputs} + encoded = json.dumps(aggregate, sort_keys=True).encode() + b'\n' + require(0 < len(encoded) <= 16 * 1024 * 1024, 'SDK signature aggregate exceeds evidence bound') + logs = output / 'logs'; logs.mkdir(exist_ok=True) + require(logs.is_dir() and not logs.is_symlink(), 'SDK producer logs directory must be real') + targets = [(logs / 'sdk-code-signatures.json', encoded), (logs / 'sdk-collect.toc', collect)] + require(all(not path.exists() and not path.is_symlink() for path, _ in targets), 'SDK producer aggregate paths must be fresh') + result = {} + for path, data in targets: + with path.open('xb') as stream: stream.write(data) + result[str(path.relative_to(output))] = {'sha256': digest(path), 'byteLength': len(data)} + return result + + +def extract_binary(archive, output, *, require_sdk=False, manifest=None): """Extract only one regular executable; reject unsafe archive metadata first.""" + if require_sdk: + require(isinstance(manifest, dict), 'SDK extraction requires bound production manifest') + table = pub.read_sdk_archive(archive, manifest) + sdk_custody.validate_sdk_archive_table(manifest, table) + prefix = sdk_custody.sdk_archive_prefix(table) + data, mode = table['files'][prefix + 'prose'] + require(mode & 0o111, 'Missing executable') + output.parent.mkdir(parents=True, exist_ok=False) + with output.open('xb') as target: + target.write(data) + output.chmod(0o755) + sdk_custody.materialize_sdk_members(manifest, table, output.parent) + return output with tarfile.open(archive) as source: members = source.getmembers() require(len(members) <= 100, 'Too many archive members') @@ -107,6 +179,12 @@ def extract_binary(archive, output, *, require_sdk=False): return output +def installed_sdk_identity(directory, manifest): + """Retain exact complete identity before and after actual installed probes.""" + from qualify_installed_sdk import installed_sdk_identity as inspect + return inspect(directory, manifest['agentsSdk']) + + def verified_artifacts(package): manifest = json.loads((package / 'release-manifest.json').read_text()) require(manifest.get('mode') == 'kernel-rc', 'Wrong package mode') @@ -188,15 +266,20 @@ def check(binary, runner, label, node=None): args += ['--readelf', executable_tool('readelf', env)] command(args, env=env, cwd=ROOT, log=logs / 'package.log') manifest = verified_artifacts(package) + sdk_records = [] for item in manifest['artifacts']: if item['kind'] == 'standalone-archive': runner = item['implementation'] - binary = extract_binary(package / item['path'], output / 'installed' / runner / 'prose', require_sdk=True) + binary = extract_binary(package / item['path'], output / 'installed' / runner / 'prose', require_sdk=True, manifest=manifest) + sdk_before = installed_sdk_identity(binary.parent, manifest) check(binary, runner, 'installed-' + runner) command([binary.parent / 'prose-agents-sdk', '--packaged-self-test'], env=env, cwd=output, log=logs / ('installed-sdk-' + runner + '.json'), timeout=30) command([binary.parent / 'prose-agents-sdk', '--packaged-tool-self-test'], env=env, cwd=output, log=logs / ('installed-sdk-tools-' + runner + '.json'), timeout=30) + sdk_after = installed_sdk_identity(binary.parent, manifest) + require(sdk_after == sdk_before, 'Installed SDK tree changed during probes') + sdk_records.append({'surface': 'installed-' + runner, 'before': sdk_before, 'after': sdk_after}) prefix = output / 'npm-prefix' from npm_alias_install import install as install_alias meta = next(package / a['path'] for a in manifest['artifacts'] if a['kind'] == 'npm-meta') @@ -205,15 +288,21 @@ def check(binary, runner, label, node=None): command=command, log=logs / 'npm-install.log') (logs / 'npm-alias-install.json').write_text(json.dumps(alias_install, sort_keys=True) + '\n') node = shutil.which('node', path=env.get('PATH')); require(node, 'Node is required for npm launcher') - check(prefix / 'bin/prose', 'bun', 'installed-npm', node) - installed_helpers = list((prefix / 'lib/node_modules').rglob('prose-agents-sdk')) - require(len(installed_helpers) == 1 and installed_helpers[0].is_file() and not installed_helpers[0].is_symlink(), + installed_helpers = [prefix / 'lib/node_modules/@openprose' / ('prose-cli-' + manifest['platform']) / 'bin/prose-agents-sdk'] + require(installed_helpers[0].is_file() and not installed_helpers[0].is_symlink(), 'npm installation requires one package-owned SDK helper') - require(digest(installed_helpers[0]) == manifest['agentsSdk']['sha256'], 'Installed npm SDK helper differs') + sdk_before = installed_sdk_identity(installed_helpers[0].parent, manifest) + check(prefix / 'bin/prose', 'bun', 'installed-npm', node) command([installed_helpers[0], '--packaged-self-test'], env=env, cwd=output, log=logs / 'installed-sdk-npm.json', timeout=30) command([installed_helpers[0], '--packaged-tool-self-test'], env=env, cwd=output, log=logs / 'installed-sdk-tools-npm.json', timeout=30) + sdk_after = installed_sdk_identity(installed_helpers[0].parent, manifest) + require(sdk_after == sdk_before, 'Installed npm SDK tree changed during probes') + sdk_records.append({'surface': 'installed-npm', 'before': sdk_before, 'after': sdk_after}) + sdk_encoded = json.dumps(sdk_records, sort_keys=True).encode('utf-8') + b'\n' + require(len(sdk_encoded) <= 16 * 1024 * 1024, 'Installed SDK payload record exceeds evidence bound') + (logs / 'installed-sdk-payloads.json').write_bytes(sdk_encoded) report = {'schema': 'openprose.kernel-rc-build/1', 'version': version, 'sourceRevision': revision, 'platform': manifest['platform'], 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'signing': 'unsigned', 'modelCalls': 0, 'kernelFetches': 0, @@ -221,6 +310,11 @@ def check(binary, runner, label, node=None): 'publicationAuthorized': False, 'checks': [{'name': name, 'status': 'passed'} for name in ['built-bun', 'built-rust', 'installed-bun', 'installed-rust', 'installed-npm']], 'evidence': {str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} for directory in (logs, package) for p in sorted(directory.rglob('*')) if p.is_file()}} + report['evidence'].update(sdk_producer_evidence(output, sdk_output)) + if manifest['platform'].startswith('darwin-'): + producer_archive = next(package / item['path'] for item in manifest['artifacts'] if item['kind'] == 'standalone-archive') + sdk_custody.validate_sdk_producer_evidence(manifest, pub.read_sdk_archive(producer_archive, manifest), + report['evidence'], lambda name: (output / name).read_bytes()) if sys.platform.startswith('linux'): native_evidence = output / 'agents-sdk-runtime' report['evidence'].update({str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 0b6d4fca..cd934ce9 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -104,25 +104,22 @@ def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, raise ValueError('Invalid archive byte limit') if archive.stat().st_size != item['byteLength'] or hashlib.sha256(archive.read_bytes()).hexdigest() != item.get('sha256'): raise ValueError('Native archive identity mismatch') - # Homebrew strips the package root and installs only the verified executable. - with tarfile.open(archive, 'r:gz') as contents: - members = contents.getmembers() - for member in members: - parts = Path(member.name).parts - if member.name.startswith('/') or '..' in parts or not (member.isfile() or member.isdir()): - raise ValueError('Unsafe archive member') - executables = [m for m in members if Path(m.name).name == 'prose' and m.isfile()] - if len(executables) != 1 or len(Path(executables[0].name).parts) != 2: + if 'agentsSdk' in manifest: + table = pub.read_sdk_archive(archive, manifest) + custody.validate_sdk_archive_table(manifest, table) + cli_path = custody.sdk_archive_prefix(table) + 'prose' + if cli_path not in table['files'] or len(Path(cli_path).parts) != 2: raise ValueError('Archive requires one prose executable under one package root') - sdk_record = manifest.get('agentsSdk') - if isinstance(sdk_record, dict): - for filename, digest_key in (('prose-agents-sdk', 'sha256'), ('agents-sdk-build.json', 'receiptSha256'), ('AGENTS-SDK-NOTICES.txt', 'noticesSha256')): - selected_members = [m for m in members if Path(m.name).name == filename and m.isfile()] - if len(selected_members) != 1 or Path(selected_members[0].name).parent != Path(executables[0].name).parent: - raise ValueError('Archive requires unique SDK helper/receipt/notices siblings') - member = selected_members[0] - if not 0 < member.size <= 256 * 1024 * 1024 or hashlib.sha256(contents.extractfile(member).read()).hexdigest() != sdk_record.get(digest_key): - raise ValueError('Archive SDK member differs from package identity') + else: + with tarfile.open(archive, 'r:gz') as contents: + members = contents.getmembers() + for member in members: + parts = Path(member.name).parts + if member.name.startswith('/') or '..' in parts or not (member.isfile() or member.isdir()): + raise ValueError('Unsafe archive member') + executables = [m for m in members if Path(m.name).name == 'prose' and m.isfile()] + if len(executables) != 1 or len(Path(executables[0].name).parts) != 2: + raise ValueError('Archive requires one prose executable under one package root') selected[implementation] = item if set(selected) != {'bun', 'rust'}: raise ValueError('Both native implementations are required') @@ -137,6 +134,7 @@ def formula(version: str, implementation: str, archive: Path, sha256: str) -> st f' url "{archive.as_uri()}"', f' sha256 "{sha256}"', '', ' def install', ' bin.install "prose"', ' bin.install "prose-agents-sdk" if File.exist?("prose-agents-sdk")', + ' bin.install "prose-agents-sdk-runtime" if File.directory?("prose-agents-sdk-runtime")', ' pkgshare.install "agents-sdk-build.json", "AGENTS-SDK-NOTICES.txt" if File.exist?("agents-sdk-build.json")', ' end', '', ' test do', f' assert_equal "prose {version} ({implementation})", shell_output("#{{bin}}/prose --version").strip', @@ -186,13 +184,24 @@ def verify_kernel_rc(root: Path, expected_source: str, expected_version: str) -> 'Native artifact is not bound to report evidence') hashes = {} for implementation, item in archives.items(): - members = pub.archive_members(root / 'package' / item['path']) - binaries = [data for name, data in members.items() if Path(name).name == 'prose'] + if 'agentsSdk' in manifest: + table = pub.read_sdk_archive(root / 'package' / item['path'], manifest) + custody.validate_sdk_archive_table(manifest, table) + members = {name: data for name, (data, _) in table['files'].items()} + else: + members = pub.archive_members(root / 'package' / item['path']) + binaries = ([members[custody.sdk_archive_prefix(table) + 'prose']] if 'agentsSdk' in manifest + else [data for name, data in members.items() if Path(name).name == 'prose']) custody.require(len(binaries) == 1, 'A unique native binary is required') hashes[(implementation, manifest['platform'])] = hashlib.sha256(binaries[0]).hexdigest() platform_artifact = next((a for a in artifacts if a.get('kind') == 'npm-platform'), None) custody.require(platform_artifact is not None, 'Missing npm native artifact') - npm_members = pub.archive_members(root / 'package' / platform_artifact['path']) + if 'agentsSdk' in manifest: + npm_table = pub.read_sdk_archive(root / 'package' / platform_artifact['path'], manifest) + custody.validate_sdk_archive_table(manifest, npm_table) + npm_members = {name: data for name, (data, _) in npm_table['files'].items()} + else: + npm_members = pub.archive_members(root / 'package' / platform_artifact['path']) custody.require('package/bin/prose' in npm_members and hashlib.sha256(npm_members['package/bin/prose']).hexdigest() == hashes[('bun', manifest['platform'])], 'npm native binary differs from standalone archive') @@ -203,6 +212,10 @@ def verify_kernel_rc(root: Path, expected_source: str, expected_version: str) -> launcher_hash = hashlib.sha256(members['package/bin/prose.js']).hexdigest() checks = {name: pub.read_json(root / 'logs' / (name + '.json')) for name in custody.CHECKS} custody.validate_native(report, manifest, checks, hashes, launcher_hash) + if 'agentsSdk' in manifest: + custody.require(custody.SDK_PAYLOAD_EVIDENCE in evidence, 'Missing complete installed SDK payload evidence') + custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE), manifest, npm_table) + custody.validate_sdk_producer_evidence(manifest, npm_table, evidence, lambda relative: (root / relative).read_bytes()) # Original build trees retain these files; uploaded artifact trees retain the # built probes instead. Both bind to the exact packaged executable hashes. binaries = root / 'binaries' @@ -220,6 +233,8 @@ def verify_kernel_rc(root: Path, expected_source: str, expected_version: str) -> 'nativeReportSha256': pub.digest(root / 'build-report.json'), 'archiveIdentities': [{key: item[key] for key in ('kind', 'implementation', 'platform', 'path', 'sha256', 'byteLength')} for item in artifacts]} + if 'agentsSdk' in manifest: + verified['sdkBuild'] = json.loads(npm_table['files'][custody.sdk_archive_prefix(npm_table) + 'agents-sdk-build.json'][0]) return verified, manifest, archives @@ -240,12 +255,19 @@ def run_kernel_rc(root: Path, output: Path, brew: str, expected_source: str, exp def qualify_installed_sdk(active: Path, manifest: dict[str, Any], verified: dict[str, Any], - command: Any, label: str, *, expected_timeout: str = '10m') -> None: + command: Any, label: str, *, expected_timeout: str = '10m') -> dict[str, Any]: """Probe the real installed default; synthetic-key dry runs never infer.""" helper = active.resolve(strict=True).parent / 'prose-agents-sdk' sdk = manifest.get('agentsSdk') if not isinstance(sdk, dict) or not helper.is_file() or helper.is_symlink() or pub.digest(helper) != sdk.get('sha256'): raise ValueError('Homebrew SDK helper differs from the verified package') + payload_view = None + build = verified.get('sdkBuild') + if manifest['platform'].startswith('darwin'): + import sdk_native_inventory as native + if not isinstance(build, dict) or not isinstance(build.get('payload'), dict): + raise ValueError('Homebrew Mac SDK requires a verified complete payload') + payload_view = native.read_macos_payload(helper.parent, build['payload'], build['architecture']) command(label + '-sdk-imports', [str(helper), '--packaged-self-test']) command(label + '-sdk-tools', [str(helper), '--packaged-tool-self-test']) def report(suffix: str, argv: list[str], code: int = 0, additions: dict[str, str] | None = None) -> dict[str, Any]: @@ -275,6 +297,13 @@ def report(suffix: str, argv: list[str], code: int = 0, additions: dict[str, str raise ValueError('Installed command did not discover the packaged SDK default') if dry.get('billingOwner') != 'user-provider' or dry.get('blockingError') is not None: raise ValueError('Installed SDK dry run changed billing or reported a blocker') + proof = {'stage': label, 'helperSha256': sdk['sha256'], 'receiptSha256': sdk['receiptSha256']} + if payload_view is not None: + if native.read_macos_payload(helper.parent, build['payload'], build['architecture']) != payload_view: + raise ValueError('Homebrew SDK support tree changed during qualification') + proof.update(layout=build['payload']['layout'], supportEntries=len(build['payload']['entries']), + totalRegularBytes=build['payload']['totalRegularBytes'], completePayloadVerified=True) + return proof def exercise(package: Path, manifest: dict[str, Any], archives: dict[str, Any], verified: dict[str, Any], @@ -289,6 +318,7 @@ def exercise(package: Path, manifest: dict[str, Any], archives: dict[str, Any], for name in ('cache', 'logs', 'tmp', 'trust', 'user-settings'): (output / name).mkdir() checks = [] + sdk_payload_checks = [] def command(label: str, argv: list[str], *, expected_failure: bool = False, expected_code: int = 0, additions: dict[str, str] | None = None) -> subprocess.CompletedProcess[str]: @@ -313,6 +343,9 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False, active_helper = prefix / 'bin/prose-agents-sdk' if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): raise ValueError('Rehearsal refuses to overwrite an existing SDK helper command') + active_support = prefix / 'bin/prose-agents-sdk-runtime' + if manifest.get('agentsSdk') and (active_support.exists() or active_support.is_symlink()): + raise ValueError('Rehearsal refuses to overwrite an existing SDK support directory') tap = output / 'tap' (tap / 'Formula').mkdir(parents=True) for implementation, item in archives.items(): @@ -344,15 +377,39 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False, if digest != verified['packageIdentity'][f'{implementation}BinarySha256']: raise ValueError('Homebrew executable differs from the verified candidate') if isinstance(manifest.get('agentsSdk'), dict): - qualify_installed_sdk(active, manifest, verified, command, 'fresh-' + implementation) + sdk_payload_checks.append(qualify_installed_sdk(active, manifest, verified, command, 'fresh-' + implementation)) command(f'unlink-{implementation}', [brew, 'unlink', name]) command('link-bun', [brew, 'link', f'{TAP}/prose-bun']) original = hashlib.sha256(active.read_bytes()).hexdigest() + def sdk_link_identity(): + if not manifest.get('agentsSdk'): + return None + helper = active.resolve(strict=True).parent / 'prose-agents-sdk' + if active_helper.resolve(strict=True) != helper or pub.digest(helper) != manifest['agentsSdk']['sha256']: + raise ValueError('Active Homebrew SDK helper linkage differs from selected keg') + identity = {'links': {}} + for name, path in (('prose', active), ('helper', active_helper), ('support', active_support)): + if not (path.exists() or path.is_symlink()): + identity['links'][name] = None + else: + metadata = path.lstat() + identity['links'][name] = {'device': metadata.st_dev, 'inode': metadata.st_ino, + 'mode': metadata.st_mode, 'mtimeNs': metadata.st_mtime_ns, + 'target': os.readlink(path) if path.is_symlink() else None} + if manifest['platform'].startswith('darwin'): + import sdk_native_inventory as native + build = verified['sdkBuild'] + native.read_macos_payload(helper.parent, build['payload'], build['architecture']) + identity['payloadSha256'] = hashlib.sha256(json.dumps(build['payload'], sort_keys=True).encode()).hexdigest() + return identity + sdk_conflict_before = sdk_link_identity() conflict = command('reject-overwrite', [brew, 'link', f'{TAP}/prose-rust'], expected_failure=True) if 'Could not symlink' not in conflict.stdout + conflict.stderr: raise ValueError('The rejected link did not report a symlink collision') if hashlib.sha256(active.read_bytes()).hexdigest() != original: raise ValueError('A failed link changed the active executable') + if sdk_link_identity() != sdk_conflict_before: + raise ValueError('A failed link changed the active SDK linkage or payload') command('switch-unlink-bun', [brew, 'unlink', f'{TAP}/prose-bun']) command('switch-link-rust', [brew, 'link', f'{TAP}/prose-rust']) if hashlib.sha256(active.read_bytes()).hexdigest() != verified['packageIdentity']['rustBinarySha256']: @@ -364,7 +421,7 @@ def require_empty_owned_installation(label: str) -> None: installed = command(label, [brew, 'list', '--formula', '--versions']).stdout if any(line.split()[0].split('/')[-1] in {'prose-bun', 'prose-rust'} for line in installed.splitlines() if line.split()): raise ValueError('Owned rehearsal kegs survived all-version uninstall') - if active.exists() or active.is_symlink() or active_helper.exists() or active_helper.is_symlink(): + if any(path.exists() or path.is_symlink() for path in (active, active_helper, active_support)): raise ValueError('Owned rehearsal commands survived all-version uninstall') command('uninstall-fresh-before-upgrade', [brew, 'uninstall', '--force', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) require_empty_owned_installation('remaining-kegs-before-upgrades') @@ -400,7 +457,7 @@ def preserve(label: str) -> None: command(selected + '-test-upgraded-inactive', [brew, 'test', f'{TAP}/prose-{other}']) if pub.digest(active) != verified['packageIdentity'][other + 'BinarySha256']: raise ValueError('Inactive implementation did not upgrade to exact candidate bytes') - qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-inactive', expected_timeout='9m') + sdk_payload_checks.append(qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-inactive', expected_timeout='9m')) preserve(selected + '-settings-after-inactive-upgrade') command(selected + '-unlink-upgraded-inactive', [brew, 'unlink', f'{TAP}/prose-{other}']) command(selected + '-upgrade-selected', [brew, 'upgrade', f'{TAP}/prose-{selected}']) @@ -409,7 +466,7 @@ def preserve(label: str) -> None: selected_prefix = Path(command(selected + '-selected-prefix', [brew, '--prefix', f'{TAP}/prose-{selected}']).stdout.strip()) if active.resolve() != (selected_prefix / 'bin/prose').resolve() or pub.digest(active) != verified['packageIdentity'][selected + 'BinarySha256']: raise ValueError('Upgrade changed the selected implementation or candidate identity') - qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-selected', expected_timeout='9m') + sdk_payload_checks.append(qualify_installed_sdk(active, manifest, verified, command, selected + '-upgraded-selected', expected_timeout='9m')) preserve(selected + '-settings-after-selected-upgrade') command(selected + '-uninstall-upgraded', [brew, 'uninstall', '--force', f'{TAP}/prose-bun', f'{TAP}/prose-rust']) require_empty_owned_installation(selected + '-remaining-kegs-after-upgrade') @@ -433,6 +490,8 @@ def preserve(label: str) -> None: raise ValueError('The rehearsal command did not uninstall cleanly') if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): raise ValueError('The rehearsal SDK helper command did not uninstall cleanly') + if manifest.get('agentsSdk') and (active_support.exists() or active_support.is_symlink()): + raise ValueError('The rehearsal SDK support directory did not uninstall cleanly') if previous: preserve('settings-after-final-cleanup') receipt = {'schema': 'openprose.homebrew-rehearsal/1', 'status': ('passed-native-rc-packaging-check' if verified['custodyKind'] == 'kernel-rc-native' else 'passed-development-packaging-check') if completed else 'failed', @@ -440,6 +499,9 @@ def preserve(label: str) -> None: 'source': verified.get('source', verified['packageIdentity'].get('sourceRevision', manifest.get('source', {}).get('revision'))), 'custodyKind': verified['custodyKind'], 'rehearsalManifestSha256': verified['manifestSha256'], 'checks': checks, 'modelCalls': 0, 'publicationAuthorized': False, 'releaseQualification': False, 'uninstallPassed': True} + if sdk_payload_checks: + receipt['sdkPayloadChecks'] = sdk_payload_checks + receipt['sdkLinkConflictPreserved'] = True receipt['upgradeQualification'] = 'passed-genuine-upgrade-both-selections' if previous else 'not-requested-fresh-install-only' receipt['kernelRetrieval'] = 'published-kernel-may-be-acquired-by-provider-free-dry-run' if manifest.get('agentsSdk') else 'not-measured' if previous: diff --git a/cli/ci/kernel_rc_evidence.py b/cli/ci/kernel_rc_evidence.py index 5028d09c..bc9bbc45 100644 --- a/cli/ci/kernel_rc_evidence.py +++ b/cli/ci/kernel_rc_evidence.py @@ -1,7 +1,9 @@ """Bind native kernel RC observations to the exact final package bytes.""" import hashlib import re -from pathlib import PurePosixPath +from pathlib import PurePosixPath, Path +import json +import os CHECKS = ('built-bun', 'built-rust', 'installed-bun', 'installed-rust', 'installed-npm') CHECK_PATHS = tuple('logs/' + name + '.json' for name in CHECKS) @@ -93,10 +95,13 @@ def validate_sdk_native_sbom(components, receipt=None): require(encode(observed) == encode(expected), 'Native SDK SBOM differs from bound receipt') -def validate_sdk_members(manifest, members, prefix, platform): +def validate_sdk_members(manifest, members, prefix, platform, *, historical=None): """Bind the frozen helper and its source/dependency receipt without executing it.""" import json import publication as pub + table = members if set(members) == {'files','directories','symlinks'} else None + if table is not None: + members = {name: value[0] for name,value in table['files'].items()} sdk = manifest.get('agentsSdk') keys = {'path', 'byteLength', 'sha256', 'receiptSha256', 'noticesSha256', 'python', 'pyinstaller', 'version', 'discovery', 'selfTest', 'toolSelfTest', 'dependencyLockSha256'} @@ -107,6 +112,8 @@ def validate_sdk_members(manifest, members, prefix, platform): and sdk['selfTest'] == SDK_IMPORT_TEST and sdk['toolSelfTest'] == SDK_TOOL_TEST, 'Packaged SDK policy or tool qualification differs') require(all(prefix + name in members for name in SDK_NAMES), 'Missing packaged SDK siblings') + if table is not None: + require(all(table['files'][prefix+name][1]==(0o755 if name==SDK_NAMES[0] else 0o644) for name in SDK_NAMES),'SDK sibling modes differ') helper, encoded, notices = (members[prefix + name] for name in SDK_NAMES) require(type(sdk['byteLength']) is int and 0 < sdk['byteLength'] == len(helper) and hashlib.sha256(helper).hexdigest() == sdk['sha256'] @@ -154,9 +161,83 @@ def validate_sdk_members(manifest, members, prefix, platform): else: validate_sdk_native_receipt(receipt) require(receipt.get('linuxLibraries') == 'not-applicable', 'SDK platform library receipt differs') + if historical is None: + require(table is not None and isinstance(receipt.get('payload'),dict),'Current Mac SDK requires complete onedir payload') + import sdk_native_inventory as native + scope = sdk_scoped_table(table,prefix) + native.validate_macos_payload(receipt['payload'],**scope,architecture=architecture) + else: + require(isinstance(historical,dict) and set(historical)=={'sourceRevision','archiveSha256','receiptSha256'} + and isinstance(historical['sourceRevision'],str) and re.fullmatch('[0-9a-f]{40}',historical['sourceRevision']) + and manifest.get('source',{}).get('revision')==historical['sourceRevision'] + and historical['receiptSha256']==sdk['receiptSha256'] + and re.fullmatch('[0-9a-f]{64}',historical['archiveSha256']) + and 'payload' not in receipt,'Exact historical onefile custody required') + if table is not None: + root=prefix+'prose-agents-sdk-runtime' + allowed = set() + if platform.startswith('darwin') and historical is None: + allowed={prefix+row['path'] for row in receipt['payload']['entries']} + require(set(table['symlinks']) <= allowed,'Undeclared non-SDK archive links forbidden') + require(all(not (name==root or name.startswith(root+'/')) or name in allowed for name in + set(table['files'])|set(table['directories'])|set(table['symlinks'])),'Unexpected SDK support member') return sdk +def sdk_archive_prefix(table): + require(isinstance(table,dict) and set(table)=={'files','directories','symlinks'},'Complete typed SDK archive table required') + candidates=[name for name in table['files'] if name.endswith('/prose') and + (len(PurePosixPath(name).parts)==2 or PurePosixPath(name).parts==('package','bin','prose'))] + require(len(candidates)==1,'Expected one packaged CLI for SDK sibling binding') + return candidates[0].rsplit('/',1)[0]+'/' + + +def sdk_scoped_table(table,prefix): + root=prefix+'prose-agents-sdk-runtime' + def selected(name):return name==prefix+SDK_NAMES[0] or name==root or name.startswith(root+'/') + return {kind:{name[len(prefix):]:value for name,value in table[kind].items() if selected(name)} + for kind in ('files','directories','symlinks')} + + +def validate_sdk_archive_table(manifest,table,*,historical=None): + return validate_sdk_members(manifest,table,sdk_archive_prefix(table),manifest['platform'],historical=historical) + + +def materialize_sdk_members(manifest,table,destination,*,historical=None): + """Materialize only a previously complete validated SDK, never generic archive links.""" + validate_sdk_archive_table(manifest,table,historical=historical) + prefix=sdk_archive_prefix(table);destination=Path(destination) + require(destination.is_dir() and destination.resolve()==destination.absolute() + and destination.stat().st_uid==os.getuid(),'Canonical owned SDK destination required') + for ancestor in (destination,*destination.parents):require(not ancestor.is_symlink(),'SDK destination ancestor is aliased') + for name in (*SDK_NAMES,'prose-agents-sdk-runtime'): + target=destination/name + require(not target.exists() and not target.is_symlink(),'SDK destination must be fresh') + scope=sdk_scoped_table(table,prefix) + if manifest['platform'].startswith('darwin') and historical is None: + import sdk_native_inventory as native + encoded=table['files'][prefix+SDK_NAMES[1]][0] + receipt=json.loads(encoded) + native.materialize_macos_payload(destination,receipt['payload'],**scope,architecture=receipt['architecture']) + else: + data,mode=scope['files'][SDK_NAMES[0]] + target=destination/SDK_NAMES[0] + require(not target.exists() and not target.is_symlink(),'SDK destination must be fresh') + with target.open('xb') as f:f.write(data) + target.chmod(mode) + for name in SDK_NAMES[1:]: + target=destination/name;require(not target.exists() and not target.is_symlink(),'SDK destination must be fresh') + data,mode=table['files'][prefix+name] + with target.open('xb') as f:f.write(data) + target.chmod(mode) + # Recheck actual SDK support bytes; receipt/notices are separate regular siblings. + for name in SDK_NAMES: + data,mode=table['files'][prefix+name];target=destination/name + require(target.is_file() and not target.is_symlink() and target.read_bytes()==data + and target.stat().st_mode&0o777==mode,'Materialized SDK bytes or modes differ') + return scope + + def validate_sdk_archives(manifest, archive_reader): """All three platform payloads must contain the identical bound helper.""" platform = manifest['platform'] @@ -165,6 +246,8 @@ def validate_sdk_archives(manifest, archive_reader): if artifact['kind'] not in ('standalone-archive', 'npm-platform'): continue members = archive_reader(artifact['path']) + if set(members)=={'files','directories','symlinks'}: + observed.append(validate_sdk_archive_table(manifest,members));continue cli_names = [name for name in members if name.endswith('/prose') or name.endswith('/prose.exe')] require(len(cli_names) == 1, 'Expected one packaged CLI for SDK sibling binding') prefix = cli_names[0].rsplit('/', 1)[0] + '/' @@ -221,6 +304,107 @@ def validate_native(report, manifest, checks, final_hashes, launcher_hash): 'Installed npm check lacks Node interpreter identity') + +SDK_PAYLOAD_EVIDENCE='logs/installed-sdk-payloads.json' + + +def validate_installed_sdk_payloads(records,manifest,table): + """Bind all three measured installed views to the complete packaged bytes.""" + prefix=sdk_archive_prefix(table) + validate_sdk_archive_table(manifest,table) + expected={name:{'sha256':hashlib.sha256(table['files'][prefix+name][0]).hexdigest(), + 'byteLength':len(table['files'][prefix+name][0])} for name in SDK_NAMES} + if manifest['platform'].startswith('darwin'): + payload=json.loads(table['files'][prefix+SDK_NAMES[1]][0])['payload'] + digest=hashlib.sha256(json.dumps(payload,sort_keys=True,separators=(',',':')).encode()).hexdigest() + expected['supportTree']={'payload':payload,'sha256':digest,'byteLength':payload['totalRegularBytes'],'entryCount':len(payload['entries'])} + require(isinstance(records,list) and len(records)==3,'Three installed SDK payload surfaces required') + for row,surface in zip(records,('installed-bun','installed-rust','installed-npm')): + require(isinstance(row,dict) and set(row)=={'surface','before','after'} and row['surface']==surface, + 'Installed SDK payload surface identity differs') + for key in ('before','after'): + identity=row[key] + require(isinstance(identity,dict) and set(identity)==set(expected),'Installed SDK payload identity coverage differs') + for name in SDK_NAMES: + record=identity[name] + require(isinstance(record,dict) and set(record)=={'sha256','byteLength'} and type(record['byteLength']) is int, + 'Installed SDK payload sibling record differs') + if 'supportTree' in expected: + record=identity['supportTree'] + require(isinstance(record,dict) and set(record)=={'payload','sha256','byteLength','entryCount'} + and type(record['byteLength']) is int and type(record['entryCount']) is int,'Installed SDK support record differs') + require(identity==expected,'Installed SDK payload differs from packaged complete tree') + return records + + +SDK_SIGNATURE_EVIDENCE='logs/sdk-code-signatures.json' +SDK_COLLECT_EVIDENCE='logs/sdk-collect.toc' + + +def validate_sdk_producer_evidence(manifest,table,evidence,reader): + """Authenticate retained producer records; records alone are not execution proof.""" + import base64 + validate_sdk_archive_table(manifest,table) + if not manifest['platform'].startswith('darwin'): + prefix=sdk_archive_prefix(table) + receipt=json.loads(table['files'][prefix+SDK_NAMES[1]][0]) + require('payload' not in receipt and not {SDK_SIGNATURE_EVIDENCE,SDK_COLLECT_EVIDENCE}.intersection(evidence), 'Linux must not claim Mac SDK producer evidence') + return None + require({SDK_SIGNATURE_EVIDENCE,SDK_COLLECT_EVIDENCE}.issubset(evidence),'Mac SDK producer evidence is missing') + prefix=sdk_archive_prefix(table) + payload=json.loads(table['files'][prefix+SDK_NAMES[1]][0])['payload'] + encoded_payload=json.dumps(payload,sort_keys=True,separators=(',',':')).encode() + def bound(relative,maximum): + row=evidence[relative] + require(isinstance(row,dict) and set(row)=={'sha256','byteLength'} and type(row['byteLength']) is int + and 00),'SDK producer raw output count differs') + if count==0:continue + record=raw[stream];name=log+'.'+stream + require(isinstance(record,dict) and set(record)=={'path','sha256','byteLength'} and record['path']==name + and type(record['byteLength']) is int and record['byteLength']==count and name not in seen, + 'SDK producer raw output reference differs') + seen.add(name);stored=bundle['rawOutputs'].get(name) + require(isinstance(stored,dict) and set(stored)=={'sha256','byteLength','base64'} and type(stored['byteLength']) is int + and stored['byteLength']==count and stored['sha256']==record['sha256'] and isinstance(stored['base64'],str) + and len(stored['base64'])<=4*((65536+2)//3),'SDK producer raw output custody differs') + try:data=base64.b64decode(stored['base64'],validate=True) + except ValueError as error:raise ValueError('SDK producer raw output encoding differs') from error + require(len(data)==count and base64.b64encode(data).decode()==stored['base64'] and hashlib.sha256(data).hexdigest()==record['sha256'], + 'SDK producer raw output bytes differ') + require(set(bundle['rawOutputs'])==seen,'SDK producer undeclared raw output is forbidden') + return bundle + def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): # Runtime import keeps the validator usable from publication.py's CLI without # a module initialization cycle. These helpers never execute package content. @@ -239,7 +423,7 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): require(manifest.get('embeddedDiagnosticImage') == preflight.get('embeddedDiagnosticImage') and manifest.get('kernelPolicy') == preflight.get('kernelPolicy'), 'Native manifest kernel policy differs from qualification') artifact_names = {a['path'] for a in manifest.get('artifacts', [])} evidence = report.get('evidence', {}) - require(isinstance(evidence, dict) and set(CHECK_PATHS + SDK_PROBES).union({'package/release-manifest.json'}).issubset(evidence), 'Required structured evidence is missing') + require(isinstance(evidence, dict) and set(CHECK_PATHS + SDK_PROBES).union({'package/release-manifest.json',SDK_PAYLOAD_EVIDENCE}).issubset(evidence), 'Required structured evidence is missing') seen = set() for relative, record in evidence.items(): name = asset_name(platform, relative, artifact_names) @@ -260,7 +444,12 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): and item['kind'] == kind and item['implementation'] == artifact['implementation'] and item['platform'] == (artifact['platform'] or 'all'), 'Native artifact differs from final reviewed inventory') require(manifest.get('platform') == platform, 'Native manifest identity mismatch') - validate_sdk_archives(manifest, lambda name: pub.archive_members(root / name)) + tables=[] + def sdk_archive(name): + table=pub.read_sdk_archive(root/name,manifest);tables.append(table);return table + validate_sdk_archives(manifest,sdk_archive) + validate_installed_sdk_payloads(pub.read_json(root/asset_name(platform,SDK_PAYLOAD_EVIDENCE,artifact_names),max_bytes=16*1024*1024),manifest,tables[0]) + validate_sdk_producer_evidence(manifest,tables[0],evidence,lambda relative:(root/asset_name(platform,relative,artifact_names)).read_bytes()) for relative in SDK_PROBES: probe = pub.read_json(root / asset_name(platform, relative, artifact_names)) require(probe == (SDK_TOOL_TEST if 'sdk-tools-' in relative else SDK_IMPORT_TEST), diff --git a/cli/ci/package_local.py b/cli/ci/package_local.py index 5d49c818..75f98059 100644 --- a/cli/ci/package_local.py +++ b/cli/ci/package_local.py @@ -813,8 +813,21 @@ def validated_archive_members( return sorted(result, key=lambda item: item[0]) -def tar_gz(path: Path, members: Iterable[tuple[str, bytes, int]], epoch: int) -> None: +def tar_gz(path: Path, members: Iterable[tuple[str, bytes, int]], epoch: int, *, sdk_table=None, sdk_prefix="") -> None: closed_members = validated_archive_members(members) + if sdk_table is not None: + # SDK table was snapshotted and fully validated; validate its receipt/table + # again before retaining nonregular metadata in an artifact. + encoded=sdk_table['files'][SDK_RECEIPT][0] + receipt=strict_json_object(encoded,'SDK receipt') + if receipt['platform']=='darwin': + import sdk_native_inventory as native + scoped={kind:{n:v for n,v in sdk_table[kind].items() if n not in (SDK_RECEIPT,SDK_NOTICES)} for kind in ('files','directories','symlinks')} + native.validate_macos_payload(receipt['payload'],**scoped,architecture=receipt['architecture']) + else: + if sdk_table['directories'] or sdk_table['symlinks']:raise PackageError('Non-Mac SDK cannot contain support aliases') + actual={n[len(sdk_prefix):]:(data,mode) for n,data,mode in closed_members if n.startswith(sdk_prefix) and (n[len(sdk_prefix):] in (SDK_NAME,SDK_RECEIPT,SDK_NOTICES) or n[len(sdk_prefix):]=='prose-agents-sdk-runtime' or n[len(sdk_prefix):].startswith('prose-agents-sdk-runtime/'))} + if actual!=sdk_table['files']:raise PackageError('SDK regular archive membership differs') with path.open("wb") as raw: with gzip.GzipFile( filename="", mode="wb", fileobj=raw, mtime=epoch, compresslevel=9 @@ -832,6 +845,21 @@ def tar_gz(path: Path, members: Iterable[tuple[str, bytes, int]], epoch: int) -> info.uname = "" info.gname = "" archive.addfile(info, io.BytesIO(data)) + if sdk_table is not None: + metadata=[] + for name,mode in sdk_table['directories'].items():metadata.append((name,'directory',mode,None)) + for name,target in sdk_table['symlinks'].items():metadata.append((name,'symlink',0o777,target)) + for name,kind,mode,target in sorted(metadata): + info=tarfile.TarInfo(sdk_prefix+name);info.mode=mode;info.mtime=epoch + info.uid=info.gid=0;info.uname=info.gname='' + info.type=tarfile.DIRTYPE if kind=='directory' else tarfile.SYMTYPE + if target is not None:info.linkname=target + archive.addfile(info) + + +def sdk_regular_members(sdk_members): + if isinstance(sdk_members,dict):return [(name,data,mode) for name,(data,mode) in sdk_members['files'].items()] + return sdk_members or [] def package_status(mode: str) -> tuple[str, str]: @@ -1415,7 +1443,7 @@ def standalone_readme( if mode in {"kernel-rc", "release"}: first_use = ( "First run with the packaged Agents SDK default:\n" - " Keep prose and its sibling prose-agents-sdk together. No Python installation is required.\n" + " Keep prose and its sibling prose-agents-sdk together, including prose-agents-sdk-runtime on macOS. No Python installation is required.\n" " Supply OPENAI_API_KEY in your environment; requests use your OpenAI API account.\n" " The fresh-install model default is gpt-6.1-sol. Saved explicit harness choices remain effective.\n" f" {prose_command} cli doctor\n" @@ -1750,7 +1778,7 @@ def standalone_archive( mode: str, hello_example: bytes, windows_host: bytes | None = None, - sdk_members: list[tuple[str, bytes, int]] | None = None, + sdk_members: dict | list[tuple[str, bytes, int]] | None = None, ) -> Path: root = f"openprose-prose-cli-{implementation}-{version}-{platform_identifier}" executable_name = ( @@ -1780,8 +1808,8 @@ def standalone_archive( ] if windows_host is not None: members.append((f"{root}/{WINDOWS_HOST_NAME}", windows_host, 0o755)) - members.extend((f"{root}/{name}", data, mode) for name, data, mode in (sdk_members or [])) - tar_gz(destination, members, epoch) + members.extend((f"{root}/{name}", data, mode) for name, data, mode in sdk_regular_members(sdk_members)) + tar_gz(destination, members, epoch, sdk_table=sdk_members if isinstance(sdk_members,dict) else None, sdk_prefix=root+"/") return destination @@ -1919,7 +1947,7 @@ def npm_packages( *, package_name: str = "@openprose/prose-cli", publication_platforms: str | None = None, - sdk_members: list[tuple[str, bytes, int]] | None = None, + sdk_members: dict | list[tuple[str, bytes, int]] | None = None, ) -> tuple[Path, Path]: if package_name not in {"@openprose/prose-cli", "@openprose/prose"}: raise PackageError("npm identity must be explicitly supported") @@ -2001,7 +2029,7 @@ def named_manifest(value: dict[str, Any]) -> dict[str, Any]: 0o644, ), (f"package/bin/{executable_name}", bun_binary, 0o755), - *((f"package/bin/{name}", data, mode) for name, data, mode in (sdk_members or [])), + *((f"package/bin/{name}", data, mode) for name, data, mode in sdk_regular_members(sdk_members)), *( [(f"package/bin/{WINDOWS_HOST_NAME}", windows_host, 0o755)] if windows_host is not None @@ -2010,6 +2038,7 @@ def named_manifest(value: dict[str, Any]) -> dict[str, Any]: ("package/LICENSE", LICENSE.read_bytes(), 0o644), ], epoch, + sdk_table=sdk_members if isinstance(sdk_members,dict) else None, sdk_prefix="package/bin/", ) return meta, platform_package @@ -2454,7 +2483,7 @@ def dependency_sbom_components(report: dict[str, Any]) -> list[dict[str, Any]]: def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, - mode: str, readelf: Path | None) -> tuple[list[tuple[str, bytes, int]], dict[str, Any]]: + mode: str, readelf: Path | None) -> tuple[dict, dict[str, Any]]: """Bind the package-owned helper to its native build receipt and exact source.""" if directory.is_symlink() or not directory.is_dir(): raise PackageError("SDK build directory must be a real directory") @@ -2481,14 +2510,40 @@ def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, raise PackageError('SDK native build source snapshot differs from current source') except (ValueError, KeyError, TypeError) as error: raise PackageError('SDK native receipt differs: ' + str(error)) from error - helper, length, digest = snapshot_binary(directory / SDK_NAME, snapshots / SDK_NAME, "Agents SDK") + mac_view = None + if expected_os == 'darwin': + import sdk_native_inventory as native + if not isinstance(receipt.get('payload'),dict):raise PackageError('Current Mac SDK requires complete onedir payload') + payload=receipt['payload'] + try: + if payload['builderSources'] != {name:sha256_file(ROOT/name) for name in ('cli/ci/build_agents_sdk.py','cli/ci/sdk_native_inventory.py')}: + raise ValueError('SDK payload builder source differs from current source') + import ast + module=ast.parse((ROOT/'cli/ci/build_agents_sdk.py').read_text()) + entry=next(ast.literal_eval(node.value) for node in module.body if isinstance(node,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ENTRY_SOURCE' for t in node.targets)) + if payload['entrySourceSha256']!=sha256_bytes(entry.encode()):raise ValueError('SDK entry source differs from current source') + entry_sidecar=read_static_asset(directory/'sdk-entry.py','SDK entry sidecar',1024*1024) + collect_sidecar=read_static_asset(directory/'collect.toc','SDK COLLECT sidecar',2*1024*1024) + if entry_sidecar!=entry.encode() or sha256_bytes(entry_sidecar)!=payload['entrySourceSha256']: + raise ValueError('SDK entry sidecar differs from current source') + if sha256_bytes(collect_sidecar)!=payload['collectTocSha256']: + raise ValueError('SDK COLLECT sidecar differs from payload') + mac_view=native.read_macos_payload(directory,payload,expected_arch) + except (ValueError,KeyError,TypeError) as error:raise PackageError('SDK complete payload differs: '+str(error)) from error + helper=snapshots/SDK_NAME;length=len(mac_view['files'][SDK_NAME][0]);digest=sha256_bytes(mac_view['files'][SDK_NAME][0]) + else: + helper, length, digest = snapshot_binary(directory / SDK_NAME, snapshots / SDK_NAME, "Agents SDK") if receipt.get("helper") != {"path": SDK_NAME, "sha256": digest, "byteLength": length}: raise PackageError("SDK helper differs from build receipt") notices = read_static_asset(directory / SDK_NOTICES, "SDK notices", 16 * 1024 * 1024) if receipt.get("notices") != {"path": SDK_NOTICES, "sha256": sha256_bytes(notices), "byteLength": len(notices)}: raise PackageError("SDK notices differ from build receipt") + if mac_view is not None: + try:native.materialize_macos_payload(snapshots,payload,**mac_view,architecture=expected_arch) + except (ValueError,KeyError,TypeError) as error:raise PackageError('SDK materialization differs: '+str(error)) from error if expected_os == "darwin" and mode in {"kernel-rc", "release", "alpha"}: - verify_darwin_code_signature(helper, "Agents SDK") + for name in receipt['payload']['codeSignaturePaths']: + verify_darwin_code_signature(snapshots/name, 'Agents SDK '+name) if readelf is not None: inspect_linux_glibc(helper, "Agents SDK", readelf) environment = {"PATH": "/usr/bin:/bin", "PYTHONDONTWRITEBYTECODE": "1"} @@ -2521,7 +2576,18 @@ def snapshot_sdk(directory: Path, snapshots: Path, platform_identifier: str, if version_tuple(maximum) > version_tuple(LINUX_MINIMUM_GLIBC): raise PackageError("Packaged SDK libraries require newer than the admitted glibc floor") helper_bytes = verified_snapshot_bytes(helper, length, digest, "Agents SDK") - return [(SDK_NAME, helper_bytes, 0o755), (SDK_RECEIPT, receipt_bytes, 0o644), (SDK_NOTICES, notices, 0o644)], { + table={'files':{SDK_NAME:(helper_bytes,0o755),SDK_RECEIPT:(receipt_bytes,0o644),SDK_NOTICES:(notices,0o644)},'directories':{},'symlinks':{}} + if mac_view is not None: + try: + if native.read_macos_payload(directory,payload,expected_arch)!=mac_view or native.read_macos_payload(snapshots,payload,expected_arch)!=mac_view: + raise ValueError('SDK support tree changed during package tests') + except (ValueError,KeyError,TypeError) as error:raise PackageError('SDK complete payload changed: '+str(error)) from error + if read_static_asset(directory/'sdk-entry.py','SDK entry sidecar',1024*1024)!=entry_sidecar or read_static_asset(directory/'collect.toc','SDK COLLECT sidecar',2*1024*1024)!=collect_sidecar: + raise PackageError('SDK source sidecars changed during package tests') + if payload['builderSources']!={name:sha256_file(ROOT/name) for name in payload['builderSources']} or receipt['sources']!={name:sha256_file(ROOT/name) for name in receipt['sources']}: + raise PackageError('SDK source files changed during package tests') + table['files'].update(mac_view['files']);table['directories']=mac_view['directories'];table['symlinks']=mac_view['symlinks'] + return table, { "path": SDK_NAME, "byteLength": length, "sha256": digest, "receiptSha256": sha256_bytes(receipt_bytes), "noticesSha256": sha256_bytes(notices), "python": "3.10.20", "pyinstaller": "6.22.3", "version": "0.1.0", @@ -2647,7 +2713,7 @@ def build( with tempfile.TemporaryDirectory( prefix=".openprose-package-", dir=args.out.parent ) as temporary: - temporary_root = Path(temporary) + temporary_root = Path(temporary).resolve() snapshots = temporary_root / "inputs" snapshots.mkdir(mode=0o700) executable_suffix = ".exe" if platform_identifier.startswith("win32-") else "" @@ -2920,7 +2986,7 @@ def build( if isinstance(sdk_record, dict) else []) + dependency_sbom_components(dependency_report) + (sdk_custody.sdk_native_sbom_components( - strict_json_object(next(data for name, data, _ in sdk_members if name == SDK_RECEIPT), 'SDK receipt')) + strict_json_object(next(data for name, data, _ in sdk_regular_members(sdk_members) if name == SDK_RECEIPT), 'SDK receipt')) if sdk_members else []), "properties": [ { diff --git a/cli/ci/publication.py b/cli/ci/publication.py index 81c29892..a67178b1 100644 --- a/cli/ci/publication.py +++ b/cli/ci/publication.py @@ -12,6 +12,7 @@ import time import zipfile import tarfile +import unicodedata from package_local import npm_payload_version REPOSITORY = 'openprose/prose-cli' @@ -34,8 +35,8 @@ def object_pairs(pairs): return result -def read_json(path): - require(path.is_file() and not path.is_symlink() and path.stat().st_size < 1024 * 1024, 'Invalid JSON file') +def read_json(path, *, max_bytes=1024*1024): + require(path.is_file() and not path.is_symlink() and path.stat().st_size <= max_bytes, 'Invalid JSON file') return json.loads(path.read_text(), object_pairs_hook=object_pairs) @@ -107,6 +108,83 @@ def archive_members(path): return result +def decode_sdk_archive(encoded,manifest,*,label='SDK archive',historical=None): + """Known SDK context only: retain all typed metadata and validate before extraction.""" + import io,zlib + from kernel_rc_evidence import validate_sdk_archive_table + require(isinstance(encoded,bytes) and 0 dict: +def extract_standalone(archive: Path, directory: Path, *, sdk: bool, manifest=None) -> dict: """Extract only verified regular payloads, never archive metadata or links.""" pub.require(not directory.exists() and not directory.is_symlink(), 'Installation must be fresh') - members = pub.archive_members(archive) - executables = [name for name in members if Path(name).name == 'prose'] - pub.require(len(executables) == 1 and len(Path(executables[0]).parts) == 2, 'Unique package-root CLI required') - prefix = executables[0].rsplit('/', 1)[0] + '/' - names = ('prose', *SDK_NAMES) if sdk else ('prose',) + table = None + if sdk: + pub.require(isinstance(manifest, dict), 'SDK installation requires bound production manifest') + table = pub.read_sdk_archive(archive, manifest) + kernel_rc_evidence.validate_sdk_archive_table(manifest, table) + members = {name: value[0] for name, value in table['files'].items()} + else: + # Explicit published RC3 extraction retains its original regular-only policy. + members = pub.archive_members(archive) + if sdk: + prefix = kernel_rc_evidence.sdk_archive_prefix(table) + else: + executables = [name for name in members if Path(name).name == 'prose'] + pub.require(len(executables) == 1 and len(Path(executables[0]).parts) == 2, 'Unique package-root CLI required') + prefix = executables[0].rsplit('/', 1)[0] + '/' + names = ('prose',) pub.require(all(prefix + name in members for name in names), 'Required executable siblings missing') directory.mkdir(parents=True) receipt = {} @@ -91,9 +105,88 @@ def extract_standalone(archive: Path, directory: Path, *, sdk: bool) -> dict: stream.write(data) path.chmod(0o755 if name in ('prose', 'prose-agents-sdk') else 0o644) receipt[name] = {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + if sdk: + kernel_rc_evidence.materialize_sdk_members(manifest, table, directory) + receipt.update(installed_sdk_identity(directory, manifest['agentsSdk'])) return receipt +def installed_sdk_identity(directory: Path, sdk_record: dict) -> dict: + """Authenticate the full installed Mac tree; Linux keeps its frozen trio.""" + identities = {} + for name, key in zip(SDK_NAMES, ('sha256', 'receiptSha256', 'noticesSha256')): + path = directory / name + regular(path) + pub.require(pub.digest(path) == sdk_record[key], 'Installed SDK sibling bytes differ') + identities[name] = {'sha256': sdk_record[key], 'byteLength': path.stat().st_size} + receipt_path = directory / SDK_NAMES[1] + pub.require(0 < receipt_path.stat().st_size <= 2 * 1024 * 1024, 'Installed SDK receipt exceeds bound') + receipt = json.loads(receipt_path.read_bytes(), object_pairs_hook=pub.object_pairs) + pub.require(isinstance(receipt, dict), 'Installed SDK receipt must be an object') + if receipt.get('platform') == 'darwin': + payload = receipt.get('payload') + sdk_inventory.read_macos_payload(directory, payload, receipt.get('architecture')) + identities['supportTree'] = {'payload': payload, + 'sha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'byteLength': payload['totalRegularBytes'], 'entryCount': len(payload['entries'])} + return identities + + +def validate_payload_evidence(records, evidence, platform): + """Validate captured receipt references; this does not re-execute native proof.""" + pub.require(isinstance(records, list) and 0 < len(records) <= 32, 'Installed payload record bound differs') + mac = isinstance(platform, str) and platform.startswith('darwin-') + pub.require(isinstance(evidence, dict) and (len(evidence) == 1 if mac else not evidence), 'SDK evidence dictionary differs') + receipts = {} + for digest, encoded in evidence.items(): + pub.require(isinstance(digest, str) and re.fullmatch('[0-9a-f]{64}', digest) + and isinstance(encoded, str), 'SDK evidence key/value differs') + raw = encoded.encode('utf-8') + pub.require(0 < len(raw) <= 2 * 1024 * 1024 and hashlib.sha256(raw).hexdigest() == digest, + 'SDK evidence receipt digest/length differs') + receipt = json.loads(raw, object_pairs_hook=pub.object_pairs) + pub.require(isinstance(receipt, dict) and receipt.get('schema') == 'openprose.agents-sdk-build/1' + and receipt.get('platform') == 'darwin', 'SDK evidence platform differs') + pub.require(receipt.get('architecture') == {'darwin-arm64': 'arm64', 'darwin-x64': 'x86_64'}.get(platform), 'SDK evidence architecture differs from qualification platform') + sdk_inventory.validate_macos_payload_structure(receipt['payload'], receipt['architecture']) + receipts[digest] = receipt + referenced = set() + for record in records: + pub.require(isinstance(record, dict) and set(record) == {'directory', 'members', 'before', 'after'} + and record['before'] == record['after'], 'Installed payload record shape/equality differs') + pub.require(isinstance(record['directory'], str) and record['directory'] + and not Path(record['directory']).is_absolute() and '..' not in Path(record['directory']).parts, + 'Installed payload directory differs') + identity = record['before'] + pub.require(isinstance(identity, dict) and set(identity) == set(SDK_NAMES) | ({'supportTree'} if mac else set()), + 'Installed SDK identity shape differs') + pub.require(isinstance(record['members'], dict) and set(record['members']) == set(identity) | {'prose'} + and all(record['members'][key] == value for key, value in identity.items()), + 'Installed member identities differ') + for name in ('prose', *SDK_NAMES): + row = record['members'][name] + pub.require(isinstance(row, dict) and set(row) == {'sha256', 'byteLength'} + and isinstance(row['sha256'], str) and re.fullmatch('[0-9a-f]{64}', row['sha256']) + and type(row['byteLength']) is int and 0 < row['byteLength'] <= 256 * 1024 * 1024, + 'Installed sibling identity differs') + if mac: + tree = identity['supportTree'] + pub.require(isinstance(tree, dict) and set(tree) == {'sha256', 'byteLength', 'entryCount', 'receiptSha256'}, + 'Installed support reference shape differs') + digest = tree['receiptSha256']; pub.require(isinstance(digest, str) and digest in receipts, 'Missing SDK receipt reference') + referenced.add(digest); receipt = receipts[digest]; payload = receipt['payload'] + expected = {'sha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'byteLength': payload['totalRegularBytes'], 'entryCount': len(payload['entries']), 'receiptSha256': digest} + pub.require(tree == expected and all(type(tree[key]) is int for key in ('byteLength', 'entryCount')), + 'Installed support identity differs from receipt') + pub.require(identity['agents-sdk-build.json'] == {'sha256': digest, 'byteLength': len(evidence[digest].encode())} + and identity['prose-agents-sdk'] == {key: receipt['helper'][key] for key in ('sha256', 'byteLength')} + and identity['AGENTS-SDK-NOTICES.txt'] == {key: receipt['notices'][key] for key in ('sha256', 'byteLength')}, + 'Installed SDK sibling receipt bindings differ') + pub.require(referenced == set(evidence), 'Unreferenced SDK evidence') + return records + + class Qualification: def __init__(self, output: Path, node: Path, npm: Path): pub.require(not output.exists() and not output.is_symlink(), 'Output must be fresh') @@ -104,7 +197,7 @@ def __init__(self, output: Path, node: Path, npm: Path): self.logs = output / 'logs'; self.logs.mkdir() self.tools = output / 'node-only'; self.tools.mkdir() (self.tools / 'node').symlink_to(node) - self.commands = []; self.checks = []; self.payloads = [] + self.commands = []; self.checks = []; self.payloads = []; self.sdk_evidence = {} def environment(self, root: Path, *, npm=False, explicit_config=False) -> dict[str, str]: # A boundary marker stops project configuration discovery at this @@ -193,22 +286,32 @@ def preferences(self, command, root, env, label, saved): def payload(self, directory, expected_binary, sdk_record): regular(directory / 'prose') pub.require(pub.digest(directory / 'prose') == expected_binary, 'Installed CLI bytes differ') - for name, key in zip(SDK_NAMES, ('sha256', 'receiptSha256', 'noticesSha256')): - regular(directory / name) - pub.require(pub.digest(directory / name) == sdk_record[key], 'Installed SDK sibling bytes differ') + before = installed_sdk_identity(directory, sdk_record) for argument, expected in (('--packaged-self-test', kernel_rc_evidence.SDK_IMPORT_TEST), ('--packaged-tool-self-test', kernel_rc_evidence.SDK_TOOL_TEST)): env = {'PATH': '', 'HOME': str(self.output), 'TMPDIR': str(self.output), 'LANG': 'C', 'LC_ALL': 'C'} record = self.execute([directory / 'prose-agents-sdk', argument], env=env, cwd=self.output, label=directory.parent.name + '-' + argument.lstrip('-'), timeout=45) pub.require(not record['stderr'] and json.loads(record['stdout']) == expected, 'Installed SDK self-test differs') - identities = {name: {'sha256': pub.digest(directory / name), 'byteLength': (directory / name).stat().st_size} for name in ('prose', *SDK_NAMES)} - pub.require(identities['prose']['sha256'] == expected_binary and all(identities[name]['sha256'] == sdk_record[key] for name, key in zip(SDK_NAMES, ('sha256', 'receiptSha256', 'noticesSha256'))), 'Installed bytes changed during probes') - self.payloads.append({'directory': str(directory.relative_to(self.output)), 'members': identities}) + after = installed_sdk_identity(directory, sdk_record) + pub.require(before == after and pub.digest(directory / 'prose') == expected_binary, + 'Installed bytes changed during probes') + if 'supportTree' in after: + encoded = (directory / 'agents-sdk-build.json').read_text('utf-8') + digest = sdk_record['receiptSha256'] + pub.require(digest not in self.sdk_evidence or self.sdk_evidence[digest] == encoded, 'Installed SDK receipt identity conflicts') + self.sdk_evidence[digest] = encoded + for identity in (before, after): + identity['supportTree'] = {key: value for key, value in identity['supportTree'].items() if key != 'payload'} + identity['supportTree']['receiptSha256'] = digest + identities = {'prose': {'sha256': expected_binary, 'byteLength': (directory / 'prose').stat().st_size}, **after} + self.payloads.append({'directory': str(directory.relative_to(self.output)), 'members': identities, + 'before': before, 'after': after}) + def qualify(candidate_root, previous_release, output, source, version, node, npm): """All writes and installations are beneath fresh output; prior bytes stay original.""" verified, manifest, archives = custody.verify_kernel_rc(candidate_root, source, version) - kernel_rc_evidence.validate_sdk_archives(manifest, lambda name: pub.archive_members(candidate_root / 'package' / name)) + kernel_rc_evidence.validate_sdk_archives(manifest, lambda name: pub.read_sdk_archive(candidate_root / 'package' / name, manifest)) previous = custody.verify_previous_release(previous_release, manifest['platform'], version) previous_plan = custody.read_previous_manifest(previous_release) pub.require(pub.digest(previous_release / 'manifest.json') == PREVIOUS_MANIFEST_SHA256, 'Previous manifest is not the pinned published release') @@ -224,7 +327,7 @@ def qualify(candidate_root, previous_release, output, source, version, node, npm root = output / ('standalone-' + runner); root.mkdir() env = q.environment(root, explicit_config=runner == 'rust') fresh = root / 'fresh' - extract_standalone(candidate_root / 'package' / archives[runner]['path'], fresh, sdk=True) + extract_standalone(candidate_root / 'package' / archives[runner]['path'], fresh, sdk=True, manifest=manifest) q.payload(fresh, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) q.defaults([fresh / 'prose'], runner, version, source, root, env, runner + '-fresh') relocated = root / 'relocated'; fresh.rename(relocated) @@ -236,7 +339,7 @@ def qualify(candidate_root, previous_release, output, source, version, node, npm pub.require(pub.digest(upgrade / 'prose') == previous['binaryHashes'][runner], 'Installed previous bytes differ') q.identity([upgrade / 'prose'], runner, previous_plan['version'], previous_plan['source'], root, env, runner + '-previous') shutil.rmtree(upgrade) - extract_standalone(candidate_root / 'package' / archives[runner]['path'], upgrade, sdk=True) + extract_standalone(candidate_root / 'package' / archives[runner]['path'], upgrade, sdk=True, manifest=manifest) q.payload(upgrade, verified['packageIdentity'][runner + 'BinarySha256'], manifest['agentsSdk']) q.identity([upgrade / 'prose'], runner, version, source, root, env, runner + '-upgraded') q.preferences([upgrade / 'prose'], root, env, runner + '-upgraded', saved) @@ -297,7 +400,12 @@ def npm_payload(): for path, digest in input_hashes.items(): pub.require(pub.digest(Path(path)) == digest, 'Original qualification input changed') report = {'schema': 'openprose.installed-sdk-qualification/1', 'status': 'pass', 'sourceRevision': source, 'version': version, 'platform': manifest['platform'], 'candidate': verified, 'previous': {'version': previous_plan['version'], 'source': previous_plan['source'], 'manifestSha256': PREVIOUS_MANIFEST_SHA256}, 'inputSha256': input_hashes, 'node': {'path': str(node), 'sha256': pub.digest(node)}, 'npm': {'path': str(npm), 'sha256': pub.digest(npm)}, 'checks': q.checks, 'commands': q.commands, 'installedPayloads': q.payloads, 'npmInstallations': [*installs, install_record], 'modelCalls': 0, 'networkScope': 'loopback npm cache priming and public published-kernel retrieval only; no provider credentials', 'networkIsolation': 'not-enforced', 'publicationAuthorized': False, 'globalUserStateModified': False} - (output / 'qualification.json').write_text(json.dumps(report, indent=2, sort_keys=True) + '\n') + validate_payload_evidence(q.payloads, q.sdk_evidence, manifest['platform']) + if q.sdk_evidence: + report['sdkEvidence'] = q.sdk_evidence + encoded = json.dumps(report, indent=2, sort_keys=True).encode('utf-8') + b'\n' + pub.require(len(encoded) <= 16 * 1024 * 1024, 'Installed qualification report exceeds evidence bound') + (output / 'qualification.json').write_bytes(encoded) return report diff --git a/cli/ci/rehearse_release.py b/cli/ci/rehearse_release.py index 1c2a3b2e..92db11de 100644 --- a/cli/ci/rehearse_release.py +++ b/cli/ci/rehearse_release.py @@ -802,6 +802,79 @@ def _npm_conformance_context(install: Path, raw_report: Mapping[str, Any]) -> di return {'prefix':str(prefix.resolve()),'platform':platform,'files':files} +def _sdk_conformance_contexts(install, raw_report, release, module): + """Bind real SDK trees before deliberately substituting the fixture oracle.""" + if not isinstance(release, dict) or 'agentsSdk' not in release: + if 'sdkEvidence' in raw_report: + raise RehearsalError('SDK installation evidence lacks an explicit release identity') + return {} + sdk = require_object(release['agentsSdk'], 'release SDK identity') + import qualify_installed_sdk as installed_sdk + import kernel_rc_evidence as sdk_custody + import sdk_native_inventory as inventory + records = raw_report.get('installations', []) + contexts = {} + for label, tree_root in (('direct-rust', install / 'rust-standalone'), + ('direct-bun', install / 'bun-standalone'), + ('npm-launcher', install / 'npm-prefix')): + matches = [r for r in records if r.get('surface') == label] + if len(matches) != 1: + raise RehearsalError('SDK conformance lacks three benchmark installation trees') + tree = matches[0]['treeIdentity']; allowed = {} + directory = tree_root + if label != 'npm-launcher': + native = _installed_path(install, raw_report['measurementPlan']['executablePaths'][label], label) + directory = native.parent + try: directory.resolve(strict=True).relative_to(tree_root.resolve(strict=True)) + except (OSError, ValueError) as error: raise RehearsalError('SDK native source escapes benchmark tree') from error + if label == 'npm-launcher': + command, meta, child, _ = module.npm_layout(tree_root, raw_report['platform']) + if command.is_symlink(): allowed[command] = meta / 'bin/prose.js' + directory = child / 'bin' + module.verify_installed_tree(tree_root, tree, allowed, sdk_evidence=raw_report.get('sdkEvidence')) + installed_sdk.installed_sdk_identity(directory, sdk) + files = {name: (module.safe_read(directory / name, 2 * 1024 * 1024 if name == 'agents-sdk-build.json' else (8 * 1024 * 1024 if name == 'AGENTS-SDK-NOTICES.txt' else 256 * 1024 * 1024)), + stat.S_IMODE((directory / name).stat().st_mode)) + for name in ('prose', 'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')} + receipt = _json_no_duplicates(files['agents-sdk-build.json'][0], 'SDK source receipt') + table = {'files': {'source/' + n: v for n, v in files.items()}, 'directories': {}, 'symlinks': {}} + if release['platform'].startswith('darwin-'): + view = inventory.read_macos_payload(directory, receipt['payload'], receipt['architecture']) + for kind, rows in view.items(): table[kind].update({'source/' + n: v for n, v in rows.items()}) + sdk_custody.validate_sdk_archive_table(release, table) + contexts[label] = {'directory': str(directory.resolve()), 'platform': release['platform'], 'agentsSdk': sdk} + return contexts + + +def _sdk_report_binding(raw_report, release): + if not isinstance(release, dict) or 'agentsSdk' not in release: + return None + sdk = require_object(release['agentsSdk'], 'release SDK identity') + receipt_sha = require_sha(sdk.get('receiptSha256'), 'SDK receipt') + payload_sha = 'not-applicable' + if release['platform'].startswith('darwin-'): + evidence = require_object(raw_report.get('sdkEvidence'), 'SDK receipt evidence') + if set(evidence) != {receipt_sha} or not isinstance(evidence[receipt_sha], str): + raise RehearsalError('SDK receipt evidence differs from release identity') + encoded = evidence[receipt_sha].encode('utf-8') + if not 0 < len(encoded) <= 2 * 1024 * 1024 or digest(encoded) != receipt_sha: + raise RehearsalError('SDK receipt evidence bytes differ') + receipt = _json_no_duplicates(encoded, 'SDK receipt evidence') + payload_sha = digest(json.dumps(receipt['payload'], ensure_ascii=False, sort_keys=True, separators=(',', ':')).encode('utf-8')) + return {'evidence': {receipt_sha: {'platform': release['platform'], 'agentsSdk': sdk, 'payloadSha256': payload_sha}}, + 'candidateReferences': {label: receipt_sha for label in ('direct-rust', 'direct-bun', 'npm-launcher')}} + + +def _validate_sdk_conformance_binding(value, raw_report, release): + expected_sdk = _sdk_report_binding(raw_report, release) + observed_sdk = value.get('sdkSourceCustody') + if expected_sdk is None: + if observed_sdk is not None or 'sdkEvidence' in raw_report: + raise RehearsalError('SDK conformance report lacks explicit release context') + elif not isinstance(observed_sdk, dict) or any(observed_sdk.get(k) != v for k, v in expected_sdk.items()): + raise RehearsalError('SDK conformance source differs from authenticated release') + + def _run_mechanical_conformance( runner: Any, raw_report: Mapping[str, Any], @@ -810,6 +883,7 @@ def _run_mechanical_conformance( environment: Mapping[str, str], deadline: float, executor: Callable[[Sequence[str], Path, Mapping[str, str]], Any] | None, + *, release=None, benchmark_module=None, ) -> bytes: candidates, node_path, _node_digest = _conformance_inputs(install, raw_report) argv = [ @@ -825,6 +899,9 @@ def _run_mechanical_conformance( argv.extend(("--candidate-interpreter", "npm-launcher", str(node_path))) npm_context=_npm_conformance_context(install,raw_report) argv.extend(('--candidate-npm-context','npm-launcher',json.dumps(npm_context,sort_keys=True,separators=(',',':')))) + sdk_contexts = _sdk_conformance_contexts(install, raw_report, release, benchmark_module) + for label, context in sdk_contexts.items(): + argv.extend(('--candidate-sdk-source', label, json.dumps(context, sort_keys=True, separators=(',', ':')))) remaining = _deadline_remaining(deadline, "before Phase 7 conformance") execute = executor if execute is None: @@ -839,6 +916,8 @@ def _run_mechanical_conformance( if _npm_conformance_context(install,raw_report)!=npm_context: raise RehearsalError('npm conformance closure changed during execution') _conformance_inputs(install,raw_report) + if _sdk_conformance_contexts(install, raw_report, release, benchmark_module) != sdk_contexts: + raise RehearsalError('Production SDK source changed during fixture conformance') if completed.returncode != 0: diagnostic = (completed.stderr or completed.stdout)[:4096] raise RehearsalError( @@ -857,6 +936,7 @@ def validate_mechanical_conformance( raw_report: Mapping[str, Any], *, require_current_admission: bool = True, + release=None, ) -> dict[str, Any]: value = require_object(report, "mechanical conformance report") try: @@ -923,6 +1003,7 @@ def validate_mechanical_conformance( raise RehearsalError( "mechanical conformance result is not the exact admitted pass" ) + _validate_sdk_conformance_binding(value, raw_report, release) surfaces = require_object(raw_report.get("surfaces"), "installed surfaces") rust_surface = require_object(surfaces.get("direct-rust"), "Rust surface") bun_surface = require_object(surfaces.get("direct-bun"), "Bun surface") @@ -1090,6 +1171,7 @@ def deadline_executor( current_case_ids = _current_conformance_case_ids( runner, require_frozen_count=True ) + conformance_release = module.verify_package_output(package, expected_platform=raw_report['platform'])['release'] conformance_path = Path(temporary) / "installed-mechanical-conformance.json" encoded_conformance = _run_mechanical_conformance( runner, @@ -1098,13 +1180,13 @@ def deadline_executor( conformance_path, environment, deadline, - conformance_executor, + conformance_executor, release=conformance_release, benchmark_module=module, ) conformance_report = _json_no_duplicates( encoded_conformance, "mechanical conformance report" ) conformance_binding = validate_mechanical_conformance( - conformance_report, encoded_conformance, runner, raw_report + conformance_report, encoded_conformance, runner, raw_report, release=conformance_release ) if conformance_binding["caseIds"] != current_case_ids: raise RehearsalError( @@ -1381,6 +1463,7 @@ def _verify_current_package_and_install( raise RehearsalError( "current npm launcher identity differs from benchmark evidence" ) + _sdk_conformance_contexts(install, raw, context['release'], module) try: package_after = package.lstat() install_after = install.lstat() @@ -1572,6 +1655,7 @@ def verify_rehearsal( runner, raw, require_current_admission=False, + release=module.verify_package_output(root.path / "package", expected_platform=raw["platform"])["release"], ) try: tree_digests = module.verify_retained_install_trees( diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index 530ca851..ca3195a1 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -6,12 +6,16 @@ """ from __future__ import annotations import ast +from collections import deque import base64 import hashlib import json +import os +import stat from pathlib import Path, PurePosixPath import re import struct +import unicodedata MAX_BYTES = 256 * 1024 * 1024 MAX_FILES = 128 @@ -589,3 +593,317 @@ def validate_linux_receipt(receipt): require(maximum is not None and runtime['requiredGlibcMaximum'] == '.'.join(map(str, maximum)), 'Extracted GLIBC aggregate differs from native receipt') return deps + + +# This allowance is restricted to the package-owned Mac SDK support tree. Generic +# publication archives keep their separate 128-member and no-link policies. +SDK_PAYLOAD_ROOT = 'prose-agents-sdk-runtime' +SDK_HELPER = 'prose-agents-sdk' +SDK_PAYLOAD_MAX_ENTRIES = 8192 +SDK_PAYLOAD_MAX_BYTES = 512 * 1024 * 1024 +SDK_PAYLOAD_MAX_METADATA = 2 * 1024 * 1024 +SDK_BUILDER_SOURCES = ('cli/ci/build_agents_sdk.py', 'cli/ci/sdk_native_inventory.py') + + +def _payload_path(value): + safe_path(value) + require(len(value.encode('utf-8')) <= 1024 and + ':' not in value and all(unicodedata.category(c) not in ('Cc', 'Cf', 'Cs') for c in value), 'Invalid SDK payload path') + return value + + +def _payload_target(value): + require(isinstance(value, str) and 0 < len(value.encode('utf-8')) <= 1024 and + not value.startswith('/') and '\\' not in value and + ':' not in value and all(unicodedata.category(c) not in ('Cc', 'Cf', 'Cs') for c in value), 'Invalid SDK alias target') + return value + + +def _resolve_payload_alias(path, entries): + """Resolve declared POSIX links with active frames and bounded work. + + Active expansions reject A->A/A immediately; completion frames allow a + legitimate alias to be reused later, such as Current/../Current/Python. + """ + pending = deque(PurePosixPath(path).parts); resolved = []; active = set(); steps = 0 + pending_bytes = sum(len(p.encode('utf-8')) for p in pending); work = pending_bytes + while pending: + steps += 1 + require(steps <= SDK_PAYLOAD_MAX_ENTRIES * 4, 'SDK alias resolution exceeds bound') + part = pending.popleft() + if isinstance(part, tuple): + active.remove(part[0]); continue + pending_bytes -= len(part.encode('utf-8')) + if part == '.': continue + if part == '..': + require(len(resolved) > 1, 'SDK alias escapes support root') + resolved.pop(); continue + candidate = '/'.join([*resolved, part]) + require(candidate in entries, 'SDK alias is dangling') + row = entries[candidate] + if row['type'] == 'symlink': + require(candidate not in active, 'SDK alias cycle') + require(len(active) < 128, 'SDK alias nesting exceeds bound') + target = PurePosixPath(row['target']).parts + size = sum(len(p.encode('utf-8')) for p in target); work += size + require(len(pending) + len(target) + 1 <= 4096 and pending_bytes + size <= 8192 and + work <= 1024 * 1024, 'SDK alias expansion work exceeds bound') + active.add(candidate); pending.appendleft((candidate,)) + pending.extendleft(reversed(target)); pending_bytes += size + continue + require(not any(isinstance(p, str) for p in pending) or row['type'] == 'directory', + 'SDK alias traverses a regular file') + resolved.append(part) + result = '/'.join(resolved) + require(result in entries and entries[result]['type'] in ('file', 'directory'), + 'SDK alias does not resolve to a real member') + return result + + +def macho_architectures(data): + """Inspect bounded thin/fat Mach-O headers and load-command layout. + + CPU values/layouts follow Apple mach-o/loader.h and fat.h. Ordinary resource + bytes return None; a recognized malformed native header is never data. + """ + require(isinstance(data, bytes) and len(data) <= MAX_BYTES, 'Oversized Mach-O inspection') + magic = data[:4] + thin = {b'\xcf\xfa\xed\xfe': ('<', 32), b'\xfe\xed\xfa\xcf': ('>', 32), + b'\xce\xfa\xed\xfe': ('<', 28), b'\xfe\xed\xfa\xce': ('>', 28)} + fat = {b'\xca\xfe\xba\xbe': ('>', False), b'\xbe\xba\xfe\xca': ('<', False), + b'\xca\xfe\xba\xbf': ('>', True), b'\xbf\xba\xfe\xca': ('<', True)} + if magic not in thin and magic not in fat: + return None + cpu_names = {0x1000007: 'x86_64', 0x100000c: 'arm64'} + if magic in thin: + endian, size = thin[magic] + require(size == 32 and len(data) >= size, 'Unsupported Mach-O header') + _, cpu, _, kind, count, commands_size, _, _ = struct.unpack_from(endian + 'IIIIIIII', data) + require(cpu in cpu_names and kind in (2, 6, 8), 'Unsupported Mach-O target or file type') + require(count <= 65536 and commands_size <= len(data) - size, 'Invalid Mach-O command table') + position = size + for _ in range(count): + require(position + 8 <= size + commands_size, 'Truncated Mach-O command') + _, length = struct.unpack_from(endian + 'II', data, position) + require(length >= 8 and length % 4 == 0 and position + length <= size + commands_size, + 'Invalid Mach-O command size') + position += length + require(position == size + commands_size, 'Mach-O command count differs') + return (cpu_names[cpu],) + endian, wide = fat[magic] + require(len(data) >= 8, 'Truncated fat Mach-O header') + count = struct.unpack_from(endian + 'I', data, 4)[0]; row_size = 32 if wide else 20 + require(0 < count <= 16 and 8 + count * row_size <= len(data), 'Invalid fat Mach-O count') + result = []; ranges = []; identities = set() + for index in range(count): + values = struct.unpack_from(endian + ('IIQQII' if wide else 'IIIII'), data, 8 + index * row_size) + cpu, subtype, offset, length, alignment = values[:5] + require(cpu in cpu_names and (cpu, subtype) not in identities, 'Duplicate or unsupported fat Mach-O target') + identities.add((cpu, subtype)) + require(alignment <= 31 and offset % (1 << alignment) == 0 and + offset >= 8 + count * row_size and length >= 32 and offset + length <= len(data), + 'Invalid fat Mach-O slice') + require(all(offset + length <= a or offset >= b for a, b in ranges), 'Overlapping fat Mach-O slices') + ranges.append((offset, offset + length)) + require(data[offset:offset + 4] in thin, 'Fat Mach-O slice must be thin') + slice_endian = thin[data[offset:offset + 4]][0] + require(struct.unpack_from(slice_endian + 'II', data, offset + 4) == (cpu, subtype), + 'Fat Mach-O subtype differs from slice') + actual = macho_architectures(data[offset:offset + length]) + require(actual == (cpu_names[cpu],), 'Fat Mach-O CPU differs from slice') + result.extend(actual) + return tuple(sorted(set(result))) + + +def _macos_payload_structure(payload, architecture): + """Shared closed metadata/alias validation without claiming native bytes.""" + keys(payload, ('layout', 'root', 'entries', 'totalRegularBytes', 'builderSources', + 'entrySourceSha256', 'collectTocSha256', 'codeSignaturePaths'), 'Mac SDK payload') + require(payload['layout'] == 'pyinstaller-onedir/1' and payload['root'] == SDK_PAYLOAD_ROOT and + architecture in ('arm64', 'x86_64'), 'Unsupported Mac SDK layout/target') + keys(payload['builderSources'], SDK_BUILDER_SOURCES, 'Mac SDK builder sources') + hashes = [*payload['builderSources'].values(), payload['entrySourceSha256'], payload['collectTocSha256']] + require(all(isinstance(h, str) and re.fullmatch('[0-9a-f]{64}', h) for h in hashes), 'Invalid Mac SDK source digest') + require(isinstance(payload['entries'], list) and 0 < len(payload['entries']) <= SDK_PAYLOAD_MAX_ENTRIES, + 'Invalid Mac SDK member count') + require(len(json.dumps(payload, sort_keys=True).encode('utf-8')) <= SDK_PAYLOAD_MAX_METADATA, + 'Mac SDK payload metadata exceeds bound') + entries = {}; portable_names = set(); expected_files = {SDK_HELPER}; expected_dirs = set(); expected_links = set(); total = 0 + for row in payload['entries']: + require(isinstance(row, dict), 'Invalid SDK payload entry') + kind = row.get('type') + shape = {'file': ('path', 'type', 'sha256', 'byteLength', 'mode'), + 'directory': ('path', 'type', 'mode'), 'symlink': ('path', 'type', 'target', 'resolvedPath')} + require(isinstance(kind, str) and kind in shape, 'Unsupported SDK payload member type'); keys(row, shape[kind], 'SDK payload entry') + path = _payload_path(row['path']) + require((path == SDK_PAYLOAD_ROOT or path.startswith(SDK_PAYLOAD_ROOT + '/')) and path not in entries, + 'Foreign or duplicate SDK payload member') + portable = unicodedata.normalize('NFC', unicodedata.normalize('NFC', path).casefold()) + require(portable not in portable_names, 'Portable SDK path collision') + portable_names.add(portable); entries[path] = row + if kind == 'file': + require(type(row['mode']) is int and row['mode'] in (0o644, 0o755) and + type(row['byteLength']) is int and 0 <= row['byteLength'] <= MAX_BYTES and + isinstance(row['sha256'], str) and re.fullmatch('[0-9a-f]{64}', row['sha256']), + 'Invalid SDK regular file metadata') + expected_files.add(path); total += row['byteLength'] + elif kind == 'directory': + require(type(row['mode']) is int and row['mode'] == 0o755, 'Invalid SDK directory mode'); expected_dirs.add(path) + else: + _payload_target(row['target']); _payload_path(row['resolvedPath']); expected_links.add(path) + require(list(entries) == sorted(entries), 'SDK payload entries are not sorted') + require(SDK_PAYLOAD_ROOT in entries and entries[SDK_PAYLOAD_ROOT]['type'] == 'directory', 'Missing SDK support root') + require(type(payload['totalRegularBytes']) is int and total == payload['totalRegularBytes'] and + total <= SDK_PAYLOAD_MAX_BYTES, 'SDK payload aggregate bytes differ/exceed bound') + for path, row in entries.items(): + if path != SDK_PAYLOAD_ROOT: + parent = str(PurePosixPath(path).parent) + require(parent in entries and entries[parent]['type'] == 'directory', 'SDK physical member beneath alias/missing directory') + if row['type'] == 'symlink': + require(_resolve_payload_alias(path, entries) == row['resolvedPath'], 'SDK alias resolved member differs') + signatures = payload['codeSignaturePaths'] + require(isinstance(signatures, list) and signatures and all(isinstance(p, str) for p in signatures) and + signatures == sorted(set(signatures)) and SDK_HELPER in signatures and + set(signatures) <= expected_files, 'Invalid SDK signature path declaration') + return entries, expected_files, expected_dirs, expected_links + + +def validate_macos_payload_structure(payload, architecture): + """Validate metadata only; native bytes/signatures remain unverified here.""" + _macos_payload_structure(payload, architecture) + return payload + + +def validate_macos_payload(payload, files, directories, symlinks, architecture): + """Validate COMPLETE SDK-scoped typed tables before any byte projection.""" + entries, expected_files, expected_dirs, expected_links = _macos_payload_structure(payload, architecture) + require(all(isinstance(table, dict) for table in (files, directories, symlinks)), 'Invalid SDK typed member view') + require(set(files) == expected_files and set(directories) == expected_dirs and set(symlinks) == expected_links, + 'SDK complete typed member inventory differs') + native_paths = [] + for path in sorted(files): + item = files[path] + require(type(item) is tuple and len(item) == 2 and isinstance(item[0], bytes) and + type(item[1]) is int and item[1] in (0o644, 0o755) and len(item[0]) <= MAX_BYTES, + 'Invalid SDK file view') + data, mode = item + if path == SDK_HELPER: + require(mode == 0o755 and data, 'Invalid SDK helper mode/bytes') + else: + row = entries[path] + require(mode == row['mode'] and len(data) == row['byteLength'] and sha(data) == row['sha256'], + 'SDK support file bytes/mode differ') + actual = macho_architectures(data) + if actual is not None: + require(actual == (architecture,), 'SDK Mach-O architecture differs from target') + native_paths.append(path) + elif path == SDK_HELPER: + require(False, 'SDK helper must be Mach-O') + require(payload['codeSignaturePaths'] == native_paths, 'SDK code signature path closure differs') + require(all(type(mode) is int and mode == entries[path]['mode'] for path, mode in directories.items()), + 'SDK directory table differs') + require(all(target == entries[path]['target'] for path, target in symlinks.items()), 'SDK literal alias target differs') + return payload + + +def _owned_payload_root(root): + root = Path(root) + require(root.is_dir() and not root.is_symlink() and root == root.resolve(), 'SDK sibling directory must be canonical') + require(root.stat().st_uid == os.getuid(), 'SDK output directory must be owned') + for parent in root.parents: + require(not parent.is_symlink(), 'SDK sibling directory has symlink ancestor') + return root + + +def _payload_file(path): + before = path.lstat() + require(stat.S_ISREG(before.st_mode) and before.st_nlink == 1 and 0 <= before.st_size <= MAX_BYTES, + 'SDK payload file is not bounded regular bytes') + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + with os.fdopen(descriptor, 'rb') as stream: + opened = os.fstat(stream.fileno()) + require((opened.st_dev, opened.st_ino) == (before.st_dev, before.st_ino), 'SDK payload identity changed before read') + data = stream.read(MAX_BYTES + 1) + after = path.lstat() + require(len(data) == before.st_size and + (before.st_dev, before.st_ino, before.st_size, before.st_mode, before.st_mtime_ns) == + (after.st_dev, after.st_ino, after.st_size, after.st_mode, after.st_mtime_ns), 'SDK payload changed while reading') + return data, stat.S_IMODE(before.st_mode) + + +def _read_payload_view(root): + root = _owned_payload_root(root); files = {SDK_HELPER: _payload_file(root / SDK_HELPER)}; directories = {}; links = {} + support = root / SDK_PAYLOAD_ROOT + regular_bytes = [0] + require(support.is_dir() and not support.is_symlink(), 'Missing/aliased SDK support directory') + def walk(path): + relative = path.relative_to(root).as_posix(); _payload_path(relative); before = path.lstat() + require(len(files) + len(directories) + len(links) <= SDK_PAYLOAD_MAX_ENTRIES + 1, 'SDK tree exceeds member bound') + if stat.S_ISLNK(before.st_mode): + links[relative] = os.readlink(path); _payload_target(links[relative]); return + if stat.S_ISREG(before.st_mode): + require(regular_bytes[0] + before.st_size <= SDK_PAYLOAD_MAX_BYTES, 'SDK physical tree exceeds aggregate bytes') + files[relative] = _payload_file(path); regular_bytes[0] += len(files[relative][0]); return + require(stat.S_ISDIR(before.st_mode), 'Unsupported physical SDK member') + directories[relative] = stat.S_IMODE(before.st_mode) + children = [] + for child in path.iterdir(): + children.append(child) + require(len(children) <= SDK_PAYLOAD_MAX_ENTRIES, 'SDK directory exceeds member bound') + children.sort() + for child in children: walk(child) + after = path.lstat() + require((before.st_dev, before.st_ino, before.st_mode, before.st_mtime_ns) == + (after.st_dev, after.st_ino, after.st_mode, after.st_mtime_ns), 'SDK tree changed while enumerating') + walk(support) + return {'files': files, 'directories': directories, 'symlinks': links} + + +def read_macos_payload(root, payload, architecture): + validate_macos_payload_structure(payload, architecture) + view = _read_payload_view(root) + validate_macos_payload(payload, architecture=architecture, **view) + return view + + +def materialize_macos_payload(destination, payload, files, directories, symlinks, architecture): + validate_macos_payload(payload, files, directories, symlinks, architecture) + destination = _owned_payload_root(destination) + require(not os.path.lexists(destination / SDK_HELPER) and not os.path.lexists(destination / SDK_PAYLOAD_ROOT), + 'SDK materialization requires absent SDK paths') + # Parent directories are physical, never links. Links are created last and no + # file is ever opened through a declared alias. Partial failures are retained. + for relative in sorted(directories, key=lambda p: (len(PurePosixPath(p).parts), p)): + path = destination / relative; path.mkdir(mode=0o755); path.chmod(directories[relative]) + for relative in sorted(files): + path = destination / relative + require(not any(parent.is_symlink() for parent in path.parents), 'SDK file parent became aliased') + with path.open('xb') as stream: stream.write(files[relative][0]) + path.chmod(files[relative][1]) + for relative in sorted(symlinks): + path = destination / relative + require(not any(parent.is_symlink() for parent in path.parents), 'SDK alias parent became aliased') + path.symlink_to(symlinks[relative]) + return read_macos_payload(destination, payload, architecture) + + + +def inventory_macos_payload(root, architecture, builder_sources, entry_source_sha256, collect_toc_sha256): + """Builder-only construction from actual final bytes; consumers validate it.""" + view = _read_payload_view(root); entries = [] + for path, (data, mode) in view['files'].items(): + if path != SDK_HELPER: + entries.append({'path': path, 'type': 'file', 'sha256': sha(data), 'byteLength': len(data), 'mode': mode}) + entries.extend({'path': p, 'type': 'directory', 'mode': mode} for p, mode in view['directories'].items()) + entries.extend({'path': p, 'type': 'symlink', 'target': target, 'resolvedPath': ''} for p, target in view['symlinks'].items()) + by_path = {r['path']: r for r in entries} + for row in entries: + if row['type'] == 'symlink': row['resolvedPath'] = _resolve_payload_alias(row['path'], by_path) + result = {'layout': 'pyinstaller-onedir/1', 'root': SDK_PAYLOAD_ROOT, + 'entries': sorted(entries, key=lambda r: r['path']), + 'totalRegularBytes': sum(len(data) for p, (data, _) in view['files'].items() if p != SDK_HELPER), + 'builderSources': builder_sources, 'entrySourceSha256': entry_source_sha256, + 'collectTocSha256': collect_toc_sha256, + 'codeSignaturePaths': sorted(p for p, (data, _) in view['files'].items() if macho_architectures(data) is not None)} + validate_macos_payload(result, architecture=architecture, **view) + return result diff --git a/cli/ci/sign_macos.py b/cli/ci/sign_macos.py index 120fa7e0..c446ece3 100644 --- a/cli/ci/sign_macos.py +++ b/cli/ci/sign_macos.py @@ -20,6 +20,9 @@ import plistlib import re import shutil +import stat +import copy +import unicodedata import subprocess import time import uuid @@ -44,6 +47,20 @@ def sha256(path: Path) -> str: return digest.hexdigest() +def signing_json(path): + path=regular_file(path) + if not 02*1024*1024:raise SigningError('Signing receipt exceeds bounds') + return json.loads(data,object_pairs_hook=pairs) + + def regular_file(path: Path) -> Path: if path.is_symlink() or not path.is_file(): raise SigningError("Inputs must be regular files, not symbolic links") @@ -101,6 +118,89 @@ def verify_entitlements(run, binary: Path, expected: dict) -> None: raise SigningError("Signed entitlements differ from release policy") +def sdk_source_sidecars(root,payload): + """Adjacent producer evidence; never executable support or notarization input.""" + result={} + for name,key,maximum in (('sdk-entry.py','entrySourceSha256',1024*1024),('collect.toc','collectTocSha256',2*1024*1024)): + path=regular_file(root/name) + if not 0maximum or hashlib.sha256(data).hexdigest()!=payload[key]:raise SigningError('SDK producer sidecar differs from build receipt') + result[name]=data + return result + + +def sdk_signature_paths(run,root,payload,identity,team_id): + for name in payload['codeSignaturePaths']: + target=regular_file(root/name) + run(['/usr/bin/codesign','--verify','--strict',str(target)]) + signature_details(run(['/usr/bin/codesign','--display','--verbose=4',str(target)]),identity,team_id) + verify_entitlements(run,target,{}) + + +def notarization_table(archive,receipt): + """Read the exact complete notarization ZIP without following aliases.""" + import sdk_native_inventory as native + table={'files':{},'directories':{},'symlinks':{}} + expected={record['file'] for record in receipt['binaries'].values()} + payload=receipt.get('agentsSdkPayload');sdk='agents-sdk' in receipt['binaries'] + if sdk: + if not isinstance(payload,dict) or receipt.get('agentsSdkArchitecture') not in ('arm64','x86_64'): + raise SigningError('Current signed SDK requires complete onedir payload') + native.validate_macos_payload_structure(payload,receipt['agentsSdkArchitecture']) + expected.update(row['path'] for row in payload['entries']) + elif payload is not None or 'agentsSdkArchitecture' in receipt: + raise SigningError('Unexpected SDK signing payload') + if archive.stat().st_size>512*1024*1024:raise SigningError('Notarization archive exceeds bounds') + with zipfile.ZipFile(archive) as zipped: + infos=zipped.infolist() + if len(infos)>8192+3 or len(infos)!=len(expected):raise SigningError('Notarization archive member count differs') + seen=set();total=0 + for info in infos: + name=info.filename[:-1] if info.is_dir() else info.filename + portable=unicodedata.normalize('NFC',unicodedata.normalize('NFC',name).casefold()) + if name not in expected or portable in seen:raise SigningError('Unexpected notarization member') + seen.add(portable) + mode=info.external_attr>>16;kind=stat.S_IFMT(mode);permissions=stat.S_IMODE(mode) + if info.flag_bits&1 or info.file_size<0 or info.file_size>256*1024*1024:raise SigningError('Unsafe notarization member') + total+=info.file_size + if total>512*1024*1024:raise SigningError('Notarization expanded bytes exceed bounds') + data=zipped.read(info) + if len(data)!=info.file_size:raise SigningError('Notarization member length differs') + if kind==stat.S_IFREG and not info.is_dir():table['files'][name]=(data,permissions) + elif kind==stat.S_IFDIR and info.is_dir() and not data:table['directories'][name]=permissions + elif kind==stat.S_IFLNK and not info.is_dir() and permissions==0o777: + if len(data)>1024:raise SigningError('Notarization alias exceeds bounds') + table['symlinks'][name]=data.decode('utf-8') + else:raise SigningError('Unsafe notarization member type') + for record in receipt['binaries'].values(): + value=table['files'].get(record['file']) + if value is None or value[1]!=0o755 or hashlib.sha256(value[0]).hexdigest()!=record['signedSha256']: + raise SigningError('Archive binary digest or mode mismatch') + if sdk: + scope={kind:{name:value for name,value in table[kind].items() if name not in ('prose-bun','prose-rust')} for kind in table} + native.validate_macos_payload(payload,**scope,architecture=receipt['agentsSdkArchitecture']) + return table + + +def write_notarization_archive(archive,binaries,output,sdk_view=None): + table={'files':{record['file']:((output/record['file']).read_bytes(),0o755) for record in binaries.values()},'directories':{},'symlinks':{}} + if sdk_view: + for kind in table:table[kind].update(sdk_view[kind]) + if sum(len(data) for data,mode in table['files'].values())+sum(len(t.encode()) for t in table['symlinks'].values())>512*1024*1024: + raise SigningError('Notarization expanded bytes exceed bounds') + with zipfile.ZipFile(archive,'w',compression=zipfile.ZIP_DEFLATED) as zipped: + for kind in ('files','directories','symlinks'): + for name,value in sorted(table[kind].items()): + info=zipfile.ZipInfo(name+'/' if kind=='directories' else name);info.create_system=3 + if kind=='files':data,mode=value;filetype=stat.S_IFREG + elif kind=='directories':data=b'';mode=value;filetype=stat.S_IFDIR + else:data=value.encode('utf-8');mode=0o777;filetype=stat.S_IFLNK + info.external_attr=(filetype|mode)<<16;info.compress_type=zipfile.ZIP_DEFLATED + zipped.writestr(info,data) + if archive.stat().st_size>512*1024*1024:raise SigningError('Notarization archive exceeds bounds') + + def verify_existing(output: Path, identity: str, team_id: str, notary_key: Path, notary_key_id: str, notary_issuer: str, timeout_seconds: int = 900, *, commands=None) -> dict: @@ -114,7 +214,7 @@ def verify_existing(output: Path, identity: str, team_id: str, notary_key = regular_file(notary_key) run = commands if commands is not None else Commands(timeout_seconds) try: - receipt = json.loads(regular_file(output / "receipt.json").read_text()) + receipt = signing_json(output / "receipt.json") if (receipt["schema"] != "openprose.macos-signing/1" or receipt["identity"] != identity or receipt["teamId"] != team_id or receipt["bunEntitlements"] != BUN_ENTITLEMENTS @@ -135,19 +235,13 @@ def verify_existing(output: Path, identity: str, team_id: str, archive_hash = sha256(archive) if archive_hash != receipt["notarization"]["sha256"]: raise SigningError("Notarization archive digest mismatch") - with zipfile.ZipFile(archive) as zipped: - if sorted(zipped.namelist()) != sorted(record["file"] for record in receipt["binaries"].values()) or len(zipped.infolist()) != len(receipt["binaries"]): - raise SigningError("Notarization archive must contain exactly both binaries") - for record in receipt["binaries"].values(): - info = zipped.getinfo(record["file"]) - if info.file_size != (output / record["file"]).stat().st_size: - raise SigningError("Archive binary size mismatch") - digest = hashlib.sha256() - with zipped.open(info) as archived: - for block in iter(lambda: archived.read(1024 * 1024), b""): - digest.update(block) - if digest.hexdigest() != record["signedSha256"]: - raise SigningError("Archive binary digest mismatch") + table=notarization_table(archive,receipt) + if 'agents-sdk' in receipt['binaries']: + import sdk_native_inventory as native + actual=native.read_macos_payload(output,receipt['agentsSdkPayload'],receipt['agentsSdkArchitecture']) + scope={kind:{name:value for name,value in table[kind].items() if name not in ('prose-bun','prose-rust')} for kind in table} + if actual!=scope:raise SigningError('Signed SDK tree differs from notarization archive') + sdk_signature_paths(run,output,receipt['agentsSdkPayload'],identity,team_id) submission_id = str(uuid.UUID(receipt["notarization"]["submissionId"])) log = json.loads(run(["/usr/bin/xcrun", "notarytool", "log", submission_id, "--key", str(notary_key), "--key-id", notary_key_id, @@ -182,14 +276,17 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, raise SigningError("Timeout must be between 30 and 3600 seconds") inputs = {"bun": regular_file(bun), "rust": regular_file(rust)} sdk_build = None + sdk_view = None + sdk_sidecars = None if agents_sdk is not None: inputs["agents-sdk"] = regular_file(agents_sdk) + if inputs["agents-sdk"].name!="prose-agents-sdk":raise SigningError("Canonical SDK helper name required") build_path = regular_file(agents_sdk.parent / 'agents-sdk-build.json') notices = regular_file(agents_sdk.parent / 'AGENTS-SDK-NOTICES.txt') if build_path.stat().st_size > 2 * 1024 * 1024 or notices.stat().st_size > 8 * 1024 * 1024: raise SigningError('SDK signing metadata exceeds bounds') try: - sdk_build = json.loads(build_path.read_text()) + sdk_build = signing_json(build_path) except ValueError as error: raise SigningError('SDK build receipt is malformed') from error if not isinstance(sdk_build, dict): @@ -200,6 +297,12 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, raise SigningError('SDK input differs from frozen build receipt') if sdk_build.get('notices') != {'path': notices.name, 'byteLength': notices.stat().st_size, 'sha256': sha256(notices)}: raise SigningError('SDK notices differ from frozen build receipt') + import sdk_native_inventory as native + try: + sdk_view=native.read_macos_payload(agents_sdk.parent,sdk_build['payload'],sdk_build['architecture']) + sdk_sidecars=sdk_source_sidecars(agents_sdk.parent,sdk_build['payload']) + except (KeyError,ValueError,OSError) as error: + raise SigningError('SDK signing requires complete valid onedir payload') from error notary_key = regular_file(notary_key) if keychain is not None: keychain = regular_file(keychain) @@ -213,11 +316,18 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, entitlements = output / "bun-entitlements.plist" entitlements.write_bytes(plistlib.dumps(BUN_ENTITLEMENTS)) binaries = {} + if sdk_view is not None: + native.materialize_macos_payload(output,sdk_build['payload'],**sdk_view,architecture=sdk_build['architecture']) + for name,data in sdk_sidecars.items(): + with (output/name).open('xb') as sidecar:sidecar.write(data) + (output/name).chmod(0o644) + sdk_signature_paths(run,output,sdk_build['payload'],identity,team_id) for name, source in inputs.items(): original_hash = sha256(source) target = output / f"prose-{name}" - shutil.copyfile(source, target) - target.chmod(0o755) + if name!="agents-sdk": + shutil.copyfile(source, target) + target.chmod(0o755) command = ["/usr/bin/codesign", "--force", "--sign", identity, "--options", "runtime", "--timestamp"] if keychain is not None: @@ -239,9 +349,15 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, binaries[name] = {"file": target.name, "inputSha256": original_hash, "signedSha256": sha256(target)} archive = output / "notarization.zip" - with zipfile.ZipFile(archive, "w", compression=zipfile.ZIP_DEFLATED) as zipped: - for record in binaries.values(): - zipped.write(output / record["file"], record["file"]) + signed_sdk_view=None + if sdk_view is not None: + signed_sdk_view=native.read_macos_payload(output,sdk_build['payload'],sdk_build['architecture']) + for kind in ('directories','symlinks'): + if signed_sdk_view[kind]!=sdk_view[kind]:raise SigningError('SDK layout changed during signing') + if any(value!=sdk_view['files'][name] for name,value in signed_sdk_view['files'].items() if name!='prose-agents-sdk'): + raise SigningError('SDK support bytes changed during signing') + sdk_signature_paths(run,output,sdk_build['payload'],identity,team_id) + write_notarization_archive(archive,binaries,output,signed_sdk_view) archive_hash = sha256(archive) auth = ["--key", str(notary_key), "--key-id", notary_key_id, "--issuer", notary_issuer] try: @@ -264,9 +380,13 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, ): raise SigningError("Signed artifacts changed during notarization") if sdk_build is not None: - if (json.loads(build_path.read_text()) != sdk_build + if (signing_json(build_path) != sdk_build or sha256(notices) != sdk_build['notices']['sha256']): raise SigningError('SDK build metadata changed during signing') + if sdk_source_sidecars(agents_sdk.parent,sdk_build['payload'])!=sdk_sidecars or sdk_source_sidecars(output,sdk_build['payload'])!=sdk_sidecars: + raise SigningError('SDK producer sidecars changed during signing') + if native.read_macos_payload(agents_sdk.parent,sdk_build['payload'],sdk_build['architecture'])!=sdk_view or native.read_macos_payload(output,sdk_build['payload'],sdk_build['architecture'])!=signed_sdk_view: + raise SigningError('SDK tree changed during notarization') # Store a bounded projection, not the raw service log, which can contain # local paths/account data. Native service logs remain available by ID. receipt = {"schema": "openprose.macos-signing/1", "teamId": team_id, @@ -275,7 +395,12 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, "notarization": {"submissionId": submission_id, "status": "Accepted", "archive": archive.name, "sha256": archive_hash}, "stapled": False} - (output / "receipt.json").write_text(json.dumps(receipt, indent=2) + "\n") + if sdk_build is not None: + receipt['agentsSdkPayload']=copy.deepcopy(sdk_build['payload']) + receipt['agentsSdkArchitecture']=sdk_build['architecture'] + notarization_table(archive,receipt) + encoded_receipt=json.dumps(receipt,indent=2)+"\n" + if len(encoded_receipt.encode())>2*1024*1024:raise SigningError("Signing receipt exceeds bounds") if sdk_build is not None: build = sdk_build signed = binaries['agents-sdk'] @@ -283,9 +408,15 @@ def sign(bun: Path, rust: Path, output: Path, identity: str, team_id: str, build['helper'] = {'path': 'prose-agents-sdk', 'byteLength': (output / signed['file']).stat().st_size, 'sha256': signed['signedSha256']} build['signing'] = 'developer-id-notarized' - build['signingReceiptSha256'] = sha256(output / 'receipt.json') - (output / 'agents-sdk-build.json').write_text(json.dumps(build, indent=2) + '\n') + build['signingReceiptSha256'] = hashlib.sha256(encoded_receipt.encode()).hexdigest() + encoded_build=json.dumps(build,indent=2)+'\n' + if len(encoded_build.encode())>2*1024*1024:raise SigningError('SDK signing receipt exceeds bounds') + (output/'agents-sdk-build.json').write_text(encoded_build) + (output/'agents-sdk-build.json').chmod(0o644) shutil.copyfile(notices, output / notices.name) + (output/notices.name).chmod(0o644) + (output/'receipt.json').write_text(encoded_receipt) + (output/'receipt.json').chmod(0o644) return receipt diff --git a/cli/ci/test_assemble_kernel_rc.py b/cli/ci/test_assemble_kernel_rc.py index 2e457753..00b0a97e 100644 --- a/cli/ci/test_assemble_kernel_rc.py +++ b/cli/ci/test_assemble_kernel_rc.py @@ -75,17 +75,14 @@ def setUp(self): output.mkdir(parents=True) self.roots.append(root) artifacts = [] - sdk, sdk_members = sdk_fixture(platform) + from test_kernel_rc_evidence import sdk_fixture as complete_sdk_fixture + sdk, sdk_members = complete_sdk_fixture(platform) for implementation in ('bun', 'rust'): name = implementation + '-' + platform + '.tgz' data = (implementation + platform).encode() - with tarfile.open(output / name, 'w:gz') as archive: - member = tarfile.TarInfo('root/prose') - member.size = len(data) - archive.addfile(member, io.BytesIO(data)) - for sdk_name, sdk_data, sdk_mode in sdk_members: - item = tarfile.TarInfo('root/' + sdk_name); item.size = len(sdk_data); item.mode = sdk_mode - archive.addfile(item, io.BytesIO(sdk_data)) + package.tar_gz(output / name, [('root/prose', data, 0o755), + *[('root/' + member, value, mode) for member, (value, mode) in sdk_members['files'].items()]], + 0, sdk_table=sdk_members, sdk_prefix='root/') artifacts.append(self.record(output / name, implementation, 'standalone-archive', platform)) runtime = {'minimumGlibc': '2.34', 'requiredGlibcMaximum': {'rust': '2.34', 'bun': '2.34'}, 'executionEvidence': 'ubuntu-22.04-only'} if platform.startswith('linux-') else 'not-applicable' meta, native = package.npm_packages(output, ('bun' + platform).encode(), self.version, platform, 0, self.source, self.image, runtime, 'kernel-rc', package.HELLO_EXAMPLE.read_bytes(), publication_platforms='posix-four', sdk_members=sdk_members) @@ -103,6 +100,22 @@ def setUp(self): (logs / (name + '.json')).write_text(json.dumps(check)) for relative in custody.SDK_PROBES: (root / relative).write_text(json.dumps(custody.SDK_TOOL_TEST if 'sdk-tools-' in relative else custody.SDK_IMPORT_TEST)) + identities = {name: {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + for name, (data, _) in sdk_members['files'].items() + if name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')} + sdk_receipt = json.loads(sdk_members['files']['agents-sdk-build.json'][0]) + if platform.startswith('darwin'): + payload = sdk_receipt['payload'] + identities['supportTree'] = {'payload': payload, + 'sha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'byteLength': payload['totalRegularBytes'], 'entryCount': len(payload['entries'])} + (logs / 'installed-sdk-payloads.json').write_text(json.dumps([ + {'surface': 'installed-' + surface, 'before': identities, 'after': identities} + for surface in ('bun', 'rust', 'npm')], sort_keys=True) + '\n') + if platform.startswith('darwin'): + from test_kernel_rc_evidence import sdk_producer_fixture + for relative, encoded in sdk_producer_fixture(sdk_members).items(): + (root / relative).write_bytes(encoded) evidence = {str(f.relative_to(root)): {'sha256': p.digest(f), 'byteLength': f.stat().st_size} for directory in (output, logs) for f in directory.iterdir()} report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, 'version': self.version, 'sourceRevision': self.source, 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'qualification': 'offline-install-only', 'publicationAuthorized': False, 'modelCalls': 0, 'kernelFetches': 0, 'checks': [{'name': n, 'status': 'passed'} for n in ('built-bun','built-rust','installed-bun','installed-rust','installed-npm')], 'evidence': evidence} (root / 'build-report.json').write_text(json.dumps(report)) @@ -153,6 +166,44 @@ def test_aggregate_checksums_bind_only_all_install_archives(self): a.assemble(list(reversed(self.roots)), reversed_output, self.evidence) self.assertEqual((reversed_output / 'SHA256SUMS').read_bytes(), expected) + def test_missing_complete_installed_payload_evidence_is_refused(self): + root = self.roots[0] + report_path = root / 'build-report.json' + report = json.loads(report_path.read_text()) + del report['evidence'][custody.SDK_PAYLOAD_EVIDENCE] + report_path.write_text(json.dumps(report)) + with self.assertRaisesRegex(ValueError, 'installed SDK payload evidence'): + a.assemble(self.roots, self.root / 'assembly', self.evidence) + self.assertFalse((self.root / 'assembly').exists()) + + def test_rehashed_installed_support_claim_must_match_packaged_tree(self): + root = next(root for root in self.roots if root.name == 'darwin-arm64') + path = root / custody.SDK_PAYLOAD_EVIDENCE + records = json.loads(path.read_text()) + records[0]['after']['supportTree']['entryCount'] += 1 + path.write_text(json.dumps(records)) + report_path = root / 'build-report.json' + report = json.loads(report_path.read_text()) + report['evidence'][custody.SDK_PAYLOAD_EVIDENCE] = {'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + with self.assertRaisesRegex(ValueError, 'packaged complete tree'): + a.assemble(self.roots, self.root / 'assembly', self.evidence) + self.assertFalse((self.root / 'assembly').exists()) + + def test_rehashed_failed_signature_proof_cannot_assemble(self): + root = next(root for root in self.roots if root.name == 'darwin-arm64') + path = root / custody.SDK_SIGNATURE_EVIDENCE + bundle = json.loads(path.read_text()) + bundle['checks'][0]['proof']['success'] = False + path.write_text(json.dumps(bundle)) + report_path = root / 'build-report.json' + report = json.loads(report_path.read_text()) + report['evidence'][custody.SDK_SIGNATURE_EVIDENCE] = {'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + with self.assertRaisesRegex(ValueError, 'signature verification'): + a.assemble(self.roots, self.root / 'assembly', self.evidence) + self.assertFalse((self.root / 'assembly').exists()) + def test_damaged_generated_package_refused(self): next((self.roots[0]/'package').glob('*.tgz')).write_bytes(b'changed') with self.assertRaisesRegex(ValueError, 'evidence bytes changed'): diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py index 232cca54..b6d00f61 100644 --- a/cli/ci/test_build_agents_sdk.py +++ b/cli/ci/test_build_agents_sdk.py @@ -5,6 +5,7 @@ import sys from pathlib import Path import tempfile +from types import ModuleType, SimpleNamespace import unittest from unittest import mock @@ -219,3 +220,149 @@ def simulated_run(command, *, env, cwd, log, timeout=900): with self.assertRaisesRegex(ValueError, 'membership differs'): sdk.build(root / 'mismatch', linux_libgcc=supplier, linux_native_origin=origin_path) self.assertFalse((root / 'mismatch' / sdk.RECEIPT).exists()) + + + +class MacOnedirBuilderTests(unittest.TestCase): + def simulated(self, root, poison=None): + from test_sdk_native_inventory import onedir_fixture + fixture = onedir_fixture('x86_64'); executed = [] + def run(command, *, env, cwd, log, timeout=900): + executed.append((list(command), timeout)); self.assertNotIn('OPENAI_API_KEY', env) + if '-m' in command: + self.assertIn('--onedir', command); self.assertNotIn('--onefile', command) + self.assertEqual(command[command.index('--contents-directory') + 1], sdk.native.SDK_PAYLOAD_ROOT) + self.assertEqual(command[command.index('--target-arch') + 1], 'x86_64') + dist = cwd / 'dist' / sdk.NAME; dist.mkdir(parents=True) + sdk.native.materialize_macos_payload(dist, **fixture) + toc = [] + for path, (_, _) in fixture['files'].items(): + name = path if path == sdk.NAME else path.removeprefix(sdk.native.SDK_PAYLOAD_ROOT + '/') + toc.append((name, '/synthetic/source/' + name, 'EXECUTABLE' if path == sdk.NAME else 'BINARY' if name.endswith('/Python') else 'DATA')) + toc.extend((path.removeprefix(sdk.native.SDK_PAYLOAD_ROOT + '/'), target, 'SYMLINK') for path, target in fixture['symlinks'].items()) + if poison == 'collect': toc[-1] = (toc[-1][0], 'foreign', 'SYMLINK') + work = cwd / 'work' / sdk.NAME; work.mkdir(parents=True) + (work / 'COLLECT-00.toc').write_text(repr((sorted(toc),))) + log.write_text('mocked freeze, no native execution') + elif '--packaged-self-test' in command: + log.write_text(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':'0.22.2','openai':'3.13.0','certificates':True,'modelCalls':0})) + elif '--packaged-tool-self-test' in command: + log.write_text(json.dumps({'schema':'openprose.sdk-packaged-tools-self-test/1','shellEffects':True,'boundedOutput':True,'shellCancellation':True,'mockedPublicRetrieval':True,'incompleteHttpRejected':True,'modelCalls':0,'networkUsed':False})) + elif '--version' in command: + log.write_text('prose-agents-sdk 0.1.0\n') + if poison == 'mutation': (cwd / sdk.native.SDK_PAYLOAD_ROOT / 'empty-data').write_bytes(b'mutation') + elif command[0] == '/usr/bin/codesign': + if poison == 'signature': raise ValueError('Mocked signature rejection') + log.write_text('mocked strict signature verification') + else: self.fail('Unexpected mocked command') + def signature(command, *, env, cwd, timeout, output_limit=65536): + executed.append((list(command), timeout)) + return {'exitCode': 1 if poison == 'signature' else 0, 'stdout': b'', 'stderr': b'', + 'stdoutBytes': 0, 'stderrBytes': 0, 'timedOut': False, 'outputLimitExceeded': False} + with mock.patch.object(sdk.sys, 'platform', 'darwin'), \ + mock.patch.object(sdk.platform, 'python_version', return_value='3.10.20'), \ + mock.patch.object(sdk.platform, 'machine', return_value='x86_64'), \ + mock.patch.object(sdk, 'lock_packages', return_value=[]), \ + mock.patch.object(sdk, 'installed_inventory', return_value=b'synthetic notices'), \ + mock.patch.object(sdk.metadata, 'version', return_value='6.22.3'), \ + mock.patch.object(sdk, 'verify_frozen_module_closure', return_value={'buildToolNamespaceAbsent': True}), \ + mock.patch.object(sdk, '_capture_signature', side_effect=signature), \ + mock.patch.object(sdk, 'run', side_effect=run), \ + mock.patch.object(sdk.shutil, 'disk_usage', return_value=mock.Mock(free=1024**3)): + return sdk.build(root / 'out'), executed + + def test_mac_builder_transports_complete_declared_tree_and_checks_every_code_member(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); receipt, executed = self.simulated(root); output = root / 'out' + payload = receipt['payload']; actual = sdk.native.read_macos_payload(output, payload, 'x86_64') + self.assertEqual(payload['builderSources'], {p: sdk.digest(sdk.ROOT / p) for p in sdk.native.SDK_BUILDER_SOURCES}) + self.assertEqual(payload['entrySourceSha256'], sdk.native.sha(sdk.ENTRY_SOURCE.encode())) + self.assertEqual(payload['collectTocSha256'], sdk.digest(output / 'collect.toc')) + self.assertEqual(len(actual['symlinks']), 2) + self.assertEqual([timeout for command, timeout in executed if '--version' in command], [5]) + code = [str(command[-1]) for command, _ in executed if command[0] == '/usr/bin/codesign' and '--deep' not in command] + self.assertEqual(code, [str(output / path) for path in payload['codeSignaturePaths']]) + self.assertFalse((output / 'work').exists()); self.assertFalse((output / 'dist').exists()) + for index, path in enumerate(payload['codeSignaturePaths']): + proof = json.loads((output / ('codesign-' + str(index).zfill(4) + '.log')).read_text()) + self.assertEqual((proof['path'], proof['exitCode'], proof['stdoutBytes'], proof['stderrBytes'], proof['success']), + (path, 0, 0, 0, True)) + self.assertTrue(proof['outputComplete']); self.assertEqual(proof['rawOutput'], {}) + + def test_collect_mismatch_signature_failure_and_post_probe_mutation_refuse_receipt(self): + for poison in ('collect', 'signature', 'mutation'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve() + with self.assertRaises(ValueError): self.simulated(root, poison) + self.assertFalse((root / 'out' / sdk.RECEIPT).exists()) + + def test_collect_requires_closed_unique_typed_members_and_real_directory_closure(self): + from test_sdk_native_inventory import onedir_fixture + value = onedir_fixture(); view = {k: value[k] for k in ('files','directories','symlinks')} + rows = [(p if p == sdk.NAME else p.removeprefix(sdk.native.SDK_PAYLOAD_ROOT + '/'), '/source', + 'EXECUTABLE' if p == sdk.NAME else 'DATA') for p in view['files']] + rows += [(p.removeprefix(sdk.native.SDK_PAYLOAD_ROOT + '/'), target, 'SYMLINK') for p,target in view['symlinks'].items()] + with tempfile.TemporaryDirectory() as raw: + toc = Path(raw) / 'collect.toc'; toc.write_text(repr((rows,))) + self.assertEqual(sdk.verify_collect_toc(toc, view), sdk.digest(toc)) + for changed in (rows[:-1], rows + [rows[0]], rows + [('foreign', '/source', 'DEPENDENCY')], + [('foreign-exe', '/source', 'EXECUTABLE'), *rows[1:]]): + toc.write_text(repr((changed,))) + with self.assertRaises(ValueError): sdk.verify_collect_toc(toc, view) + toc.write_text("__import__('os').system('never execute')") + with self.assertRaises(ValueError): sdk.verify_collect_toc(toc, view) + + def test_structured_signature_proof_retains_bounded_failure_output_and_silent_success(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve() + results = [dict(exitCode=0, stdout=b'', stderr=b'', stdoutBytes=0, stderrBytes=0, + timedOut=False, outputLimitExceeded=False), + dict(exitCode=3, stdout=b'', stderr=b'fixture signature refusal', stdoutBytes=0, + stderrBytes=25, timedOut=False, outputLimitExceeded=False)] + with mock.patch.object(sdk, '_capture_signature', side_effect=results): + success = sdk.verify_code_signature(root, sdk.NAME, 'success.log', env={}) + self.assertTrue(success['success']); self.assertGreater((root / 'success.log').stat().st_size, 0) + with self.assertRaisesRegex(ValueError, 'signature verification failed'): + sdk.verify_code_signature(root, sdk.NAME, 'failure.log', env={}) + failure = json.loads((root / 'failure.log').read_text()) + self.assertEqual(failure['exitCode'], 3); self.assertFalse(failure['success']) + self.assertEqual(failure['rawOutput']['stderr']['sha256'], sdk.digest(root / 'failure.log.stderr')) + + def test_signature_capture_bounds_real_pipe_bytes_without_starting_any_process(self): + # Fake Popen supplies disposable pipes. No code-signing/native child starts. + class FakeProcess: + def __init__(self, data): + read, write = os.pipe(); os.write(write, data); os.close(write); self.stdout = os.fdopen(read, 'rb') + read, write = os.pipe(); os.close(write); self.stderr = os.fdopen(read, 'rb'); self.returncode = 0 + def poll(self): return self.returncode + def wait(self, timeout=None): return self.returncode + for size, expected in ((4, False), (1024, True)): + process = FakeProcess(b'x' * size) + with mock.patch.object(sdk.subprocess, 'Popen', return_value=process) as spawn: + result = sdk._capture_signature(['/usr/bin/codesign', '--verify', '--strict', '/fixture'], + env={}, cwd=Path('/'), timeout=30, output_limit=16) + self.assertEqual(result['stdoutBytes'], size) + self.assertEqual(len(result['stdout']), min(size, 16)); self.assertEqual(result['outputLimitExceeded'], expected) + self.assertEqual(spawn.call_args.kwargs['env'], {}) + + def test_actual_archive_namespace_checker_rejects_build_tool_in_carchive_and_pyz(self): + # Exercise actual production parser logic with fake archive contents; + # importing/building/freezing PyInstaller or executing helper is unnecessary. + parent = ModuleType('PyInstaller'); package = ModuleType('PyInstaller.archive'); readers = ModuleType('PyInstaller.archive.readers') + names = ['prose_sdk_runtime', 'sdk_native_inventory']; archive_names = ['PYZ.pyz'] + class Reader: + def __init__(self, path): self.toc = {name: (0, 0, 0, 0, 'z' if name == 'PYZ.pyz' else 'm') for name in archive_names} + def open_embedded_archive(self, name): return SimpleNamespace(toc={name: (0, 0, 0) for name in names}) + readers.CArchiveReader = Reader + with tempfile.TemporaryDirectory() as raw: + helper = Path(raw) / 'parser-fixture'; helper.write_bytes(b'nonexecuted archive fixture') + with mock.patch.dict(sys.modules, {'PyInstaller': parent, 'PyInstaller.archive': package, 'PyInstaller.archive.readers': readers}): + self.assertTrue(sdk.verify_frozen_module_closure(helper)['buildToolNamespaceAbsent']) + names.append('PyInstaller.archive.readers') + with self.assertRaisesRegex(ValueError, 'consumer modules'): sdk.verify_frozen_module_closure(helper) + names.pop(); archive_names.append('PyInstaller.runtime') + with self.assertRaisesRegex(ValueError, 'consumer archive'): sdk.verify_frozen_module_closure(helper) + + +if __name__ == '__main__': + unittest.main() diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index 5e60651b..86503c1f 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -124,12 +124,94 @@ def test_production_extraction_requires_sdk_siblings_before_writing(self): def test_production_extracts_closed_sdk_payload(self): with tempfile.TemporaryDirectory() as d: root = Path(d); archive = root / 'source.tgz' - names = ('prose', 'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt') - self.archive(archive, [('release/' + name, 'file') for name in names]) - executable = rc.extract_binary(archive, root / 'install/prose', require_sdk=True) + from test_qualify_installed_sdk import sdk_archive + manifest, _ = sdk_archive(archive, prefix='release/') + names = ('prose', 'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt', 'prose-agents-sdk-runtime') + executable = rc.extract_binary(archive, root.resolve() / 'install/prose', require_sdk=True, manifest=manifest) self.assertEqual({p.name for p in executable.parent.iterdir()}, set(names)) self.assertTrue(os.access(executable.parent / 'prose-agents-sdk', os.X_OK)) + def test_mutated_support_is_rejected_before_installation(self): + from test_qualify_installed_sdk import sdk_archive, archive as write_archive + with tempfile.TemporaryDirectory() as d: + root = Path(d).resolve(); path = root / 'source.tgz' + manifest, table = sdk_archive(path, prefix='release/') + table['files']['release/prose-agents-sdk-runtime/empty-data'] = (b'poison', 0o644) + write_archive(path, table['files'], table) + with self.assertRaises(ValueError): + rc.extract_binary(path, root / 'install/prose', require_sdk=True, manifest=manifest) + self.assertFalse((root / 'install').exists()) + + +class CompleteSdkProducerEvidenceTests(unittest.TestCase): + def test_support_resource_names_cannot_select_cli_or_receipt(self): + from test_qualify_installed_sdk import sdk_archive_with_resource_names + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); path = root / 'source.tgz' + manifest, _ = sdk_archive_with_resource_names(path) + rc.extract_binary(path, root / 'installed/prose', require_sdk=True, manifest=manifest) + self.assertEqual((root / 'installed/prose').read_bytes(), b'cli') + self.assertEqual((root / 'installed/prose-agents-sdk-runtime/agents-sdk-build.json').read_bytes(), b'support resource agents-sdk-build.json') + + def test_producer_bundle_matches_shared_custody_and_retains_actual_mocked_outputs(self): + import hashlib + import kernel_rc_evidence as custody + from test_kernel_rc_evidence import sdk_fixture + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); sdk_dir = root / 'agents-sdk'; sdk_dir.mkdir() + sdk, scoped = sdk_fixture('darwin-x64') + receipt = json.loads(scoped['files']['agents-sdk-build.json'][0]); payload = receipt['payload'] + collect = b'nonexecuted producer declaration fixture' + payload['collectTocSha256'] = hashlib.sha256(collect).hexdigest() + (sdk_dir / 'collect.toc').write_bytes(collect) + encoded = json.dumps(receipt, sort_keys=True).encode(); scoped['files']['agents-sdk-build.json'] = (encoded, 0o644) + sdk['receiptSha256'] = hashlib.sha256(encoded).hexdigest(); (sdk_dir / 'agents-sdk-build.json').write_bytes(encoded) + checks = [('codesign-' + str(i).zfill(4) + '.log', path, 'strict') for i, path in enumerate(payload['codeSignaturePaths'])] + checks.append(('codesign.log', 'prose-agents-sdk', 'strict-deep')) + for name, target, verification in checks: + data = b'fixture diagnostic'; raw_name = name + '.stderr'; (sdk_dir / raw_name).write_bytes(data) + proof = {'schema': 'openprose.sdk-code-signature-check/1', 'phase': 'code-signature', 'path': target, + 'verification': verification, 'exitCode': 0, 'timeoutSeconds': 30, 'success': True, + 'outputComplete': True, 'timedOut': False, 'outputLimitExceeded': False, 'stdoutBytes': 0, 'stderrBytes': len(data), + 'rawOutput': {'stderr': {'path': raw_name, 'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)}}} + (sdk_dir / name).write_text(json.dumps(proof)) + evidence = rc.sdk_producer_evidence(root, sdk_dir) + table = {kind: {'source/' + n: v for n, v in rows.items()} for kind, rows in scoped.items()} + table['files']['source/prose'] = (b'fixture CLI', 0o755) + manifest = {'platform': 'darwin-x64', 'agentsSdk': sdk} + bundle = custody.validate_sdk_producer_evidence(manifest, table, evidence, lambda name: (root / name).read_bytes()) + self.assertEqual(len(bundle['checks']), len(checks)); self.assertEqual(len(bundle['rawOutputs']), len(checks)) + self.assertTrue(all((sdk_dir / name).is_file() for name, _, _ in checks)) + broken = dict(evidence); broken['logs/sdk-code-signatures.json'] = dict(broken['logs/sdk-code-signatures.json'], sha256='0'*64) + with self.assertRaises(ValueError): custody.validate_sdk_producer_evidence(manifest, table, broken, lambda name: (root / name).read_bytes()) + + def test_collect_and_silent_signature_proofs_are_retained_and_poison_refused(self): + import hashlib + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); sdk = root / 'agents-sdk'; sdk.mkdir() + (sdk / 'collect.toc').write_bytes(b'fixture producer TOC') + payload = {'collectTocSha256': rc.digest(sdk / 'collect.toc'), 'codeSignaturePaths': ['prose-agents-sdk']} + (sdk / 'agents-sdk-build.json').write_text(json.dumps({'platform': 'darwin', 'architecture': 'x86_64', 'payload': payload})) + for name, verification in (('codesign-0000.log', 'strict'), ('codesign.log', 'strict-deep')): + proof = {'schema': 'openprose.sdk-code-signature-check/1', 'phase': 'code-signature', 'path': 'prose-agents-sdk', + 'verification': verification, 'exitCode': 0, 'timeoutSeconds': 30, 'success': True, + 'outputComplete': True, 'timedOut': False, 'outputLimitExceeded': False, 'rawOutput': {}, 'stdoutBytes': 0, 'stderrBytes': 0} + (sdk / name).write_text(json.dumps(proof)) + evidence = rc.sdk_producer_evidence(root, sdk) + self.assertEqual(set(evidence), {'logs/sdk-collect.toc', 'logs/sdk-code-signatures.json'}) + for poison in ('collect', 'empty', 'failure', 'raw-path'): + with self.subTest(poison=poison): + original = (sdk / 'codesign.log').read_bytes(); toc = (sdk / 'collect.toc').read_bytes() + if poison == 'collect': (sdk / 'collect.toc').write_bytes(b'changed') + elif poison == 'empty': (sdk / 'codesign.log').write_bytes(b'') + else: + proof = json.loads(original) + if poison == 'failure': proof['success'] = False + else: proof['rawOutput'] = {'stderr': {'path': '../escape', 'sha256': '0'*64, 'byteLength': 1}} + (sdk / 'codesign.log').write_text(json.dumps(proof)) + with self.assertRaises(ValueError): rc.sdk_producer_evidence(root, sdk) + (sdk / 'codesign.log').write_bytes(original); (sdk / 'collect.toc').write_bytes(toc) + class NativeSdkRoutingTests(unittest.TestCase): def test_linux_requires_pin_and_never_uses_host_python(self): diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index 4d9467b3..37c759b7 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -43,25 +43,26 @@ def test_binds_both_native_archives_and_version(self): rendered = rehearsal.formula(manifest['version'], 'bun', self.package / 'bun.tar.gz', selected['bun']['sha256']) self.assertIn('bin.install "prose"', rendered) self.assertIn('bin.install "prose-agents-sdk"', rendered) + self.assertIn('bin.install "prose-agents-sdk-runtime"', rendered) self.assertIn((self.package / 'bun.tar.gz').as_uri(), rendered) self.assertNotIn('pkg.prose.md', rendered) def test_sdk_archive_identity_is_required_before_homebrew_mutation(self): - sdk = [('prose-agents-sdk', b'helper', 'sha256'), ('agents-sdk-build.json', b'{}', 'receiptSha256'), - ('AGENTS-SDK-NOTICES.txt', b'notices', 'noticesSha256')] - self.manifest['agentsSdk'] = {key: hashlib.sha256(data).hexdigest() for _, data, key in sdk} - with self.assertRaisesRegex(ValueError, 'SDK'): + from test_kernel_rc_evidence import sdk_fixture + import package_local as package + sdk, table = sdk_fixture('darwin-arm64') + self.manifest['agentsSdk'] = sdk + with self.assertRaises(ValueError): self.select() for item in self.manifest['artifacts']: path = self.package / item['path'] - with tarfile.open(path, 'w:gz') as archive: - for name, data in [('prose', item['implementation'].encode()), *[(n, d) for n, d, _ in sdk]]: - member = tarfile.TarInfo('package/' + name); member.size = len(data) - archive.addfile(member, io.BytesIO(data)) + package.tar_gz(path, [('package/prose', item['implementation'].encode(), 0o755), + *[('package/' + name, data, mode) for name, (data, mode) in table['files'].items()]], + 0, sdk_table=table, sdk_prefix='package/') item.update(byteLength=path.stat().st_size, sha256=hashlib.sha256(path.read_bytes()).hexdigest()) self.select() self.manifest['agentsSdk']['sha256'] = '0' * 64 - with self.assertRaisesRegex(ValueError, 'SDK member differs'): + with self.assertRaises(ValueError): self.select() def test_changed_bytes_and_size_are_rejected(self): @@ -322,18 +323,22 @@ class NativeExerciseTests(unittest.TestCase): """Exercise genuine install/upgrade transitions with an inert fake brew.""" def setUp(self): temp = tempfile.TemporaryDirectory(); self.addCleanup(temp.cleanup) - self.root = Path(temp.name); self.prefix = self.root / 'brew' + self.root = Path(temp.name).resolve(); self.prefix = self.root / 'brew' (self.prefix / 'bin').mkdir(parents=True) self.package = self.root / 'candidate'; self.package.mkdir() self.output = self.root / 'receipt' self.installed_tap = self.prefix / 'Homebrew/Library/Taps/openprose/homebrew-cli-rehearsal' self.active = self.prefix / 'bin/prose' self.manifest = {'version': '0.15.0-rc.4', 'platform': 'darwin-arm64', - 'agentsSdk': {'sha256': hashlib.sha256(b'sdk').hexdigest()}} + 'agentsSdk': {}} + from test_kernel_rc_evidence import sdk_fixture + sdk, self.sdk_table = sdk_fixture('darwin-arm64') + self.manifest['agentsSdk'] = sdk self.archives = {i: {'path': i + '.tar.gz', 'sha256': 'a' * 64} for i in ('bun', 'rust')} self.verified = {'custodyKind': 'kernel-rc-native', 'manifestSha256': 'b' * 64, 'source': 'c' * 40, 'packageIdentity': { i + 'BinarySha256': hashlib.sha256(('new-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} + self.verified['sdkBuild'] = json.loads(self.sdk_table['files']['agents-sdk-build.json'][0]) self.previous = {'version': '0.15.0-rc.3', 'root': self.root, 'manifestSha256': 'd' * 64, 'archives': self.archives, 'binaryHashes': {i: hashlib.sha256(('old-' + i).encode()).hexdigest() for i in ('bun', 'rust')}} @@ -373,20 +378,36 @@ def execute(self, argv, **kwargs): keg = self.prefix / 'Cellar' / implementation / version / 'bin'; keg.mkdir(parents=True, exist_ok=True) binary = keg / 'prose' binary.write_bytes((('new-' if version.endswith('.4') else 'old-') + implementation).encode()) - if version.endswith('.4'): (keg / 'prose-agents-sdk').write_bytes(b'sdk') + if version.endswith('.4'): + import sdk_native_inventory as native + view = self.sdk_table + files = {name: row for name, row in view['files'].items() + if name not in ('agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt')} + if (keg / 'prose-agents-sdk-runtime').exists(): + native.read_macos_payload(keg, self.verified['sdkBuild']['payload'], 'arm64') + else: + native.materialize_macos_payload(keg, self.verified['sdkBuild']['payload'], files, + view['directories'], view['symlinks'], 'arm64') self.kegs[implementation] = (keg.parent, version) self.retained_versions.setdefault(implementation, set()).add(version) self.assertFalse(self.active.exists() or self.active.is_symlink(), 'Install/upgrade must vacate shared command') self.active.symlink_to(binary) + for name in ('prose-agents-sdk', 'prose-agents-sdk-runtime'): + if (keg / name).exists(): (self.prefix / 'bin' / name).symlink_to(keg / name) if self.corrupt_settings and verb == 'upgrade': (Path(kwargs['env']['PROSE_CONFIG_DIR']) / 'cli.toml').write_text('timeout = "1m"\n') elif verb == 'unlink': - if self.active.is_symlink(): self.active.unlink() + for name in ('prose', 'prose-agents-sdk', 'prose-agents-sdk-runtime'): + link = self.prefix / 'bin' / name + if link.is_symlink(): link.unlink() elif verb == 'link': binary = self.kegs[argv[-1].split('-')[-1]][0] / 'bin/prose' if self.active.is_symlink(): if self.active.resolve() != binary.resolve(): out = 'Could not symlink'; code = 1 - else: self.active.symlink_to(binary) + else: + self.active.symlink_to(binary) + for name in ('prose-agents-sdk', 'prose-agents-sdk-runtime'): + if (binary.parent / name).exists(): (self.prefix / 'bin' / name).symlink_to(binary.parent / name) elif verb == 'uninstall': force = '--force' in argv for name in [arg for arg in argv[2:] if arg != '--force']: @@ -395,7 +416,10 @@ def execute(self, argv, **kwargs): if implementation in self.kegs: versions = self.retained_versions.get(implementation, {self.kegs[implementation][1]}) if force: self.retained_before_force.append((implementation, set(versions))) - if self.active.is_symlink() and self.active.resolve().parent.parent == self.kegs[implementation][0].resolve(): self.active.unlink() + if self.active.is_symlink() and self.active.resolve().parent.parent == self.kegs[implementation][0].resolve(): + for name in ('prose', 'prose-agents-sdk', 'prose-agents-sdk-runtime'): + link = self.prefix / 'bin' / name + if link.is_symlink(): link.unlink() remaining = set() if force else versions - {self.kegs[implementation][1]} if self.leave_retained_keg and force and len(versions) > 1: remaining = {'0.15.0-rc.3'} @@ -437,6 +461,8 @@ def test_both_genuine_upgrades_preserve_selection_settings_and_scrub_credentials receipt = self.exercise() self.assertEqual(receipt['upgradeQualification'], 'passed-genuine-upgrade-both-selections') self.assertEqual(receipt['modelCalls'], 0) + self.assertEqual(len(receipt['sdkPayloadChecks']), 6) + self.assertTrue(all(row['completePayloadVerified'] for row in receipt['sdkPayloadChecks'])) self.assertTrue(receipt['uninstallPassed']) self.assertEqual((self.output / 'user-settings/cli.toml').read_bytes(), b'# explicit upgrade preservation\ntimeout = "9m"\n') names = [c['name'] for c in receipt['checks']] @@ -558,3 +584,75 @@ def test_fresh_only_receipt_does_not_claim_genuine_upgrade(self): receipt = self.exercise(previous=False) self.assertEqual(receipt['upgradeQualification'], 'not-requested-fresh-install-only') self.assertFalse(any(argv[:2] == ['brew', 'upgrade'] for argv, _ in self.calls)) + + def test_existing_support_directory_is_not_replaced(self): + support = self.prefix / 'bin/prose-agents-sdk-runtime'; support.mkdir() + marker = support / 'caller.txt'; marker.write_bytes(b'caller-owned') + with self.assertRaisesRegex(ValueError, 'overwrite an existing SDK support directory'): + self.exercise() + self.assertEqual(marker.read_bytes(), b'caller-owned') + self.assertFalse(any(argv[:2] == ['brew', 'install'] for argv, _ in self.calls)) + + def test_missing_support_file_fails_before_helper_probe(self): + original = self.execute + def remove_support(argv, **kwargs): + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'install']: + helper = self.active.resolve().parent + name = next(name for name in self.sdk_table['files'] if name.startswith('prose-agents-sdk-runtime/')) + (helper / name).unlink() + return result + with patch.object(self, 'execute', side_effect=remove_support), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertFalse(any(Path(argv[0]).name == 'prose-agents-sdk' for argv, _ in self.calls)) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_support_mutation_during_probe_is_refused(self): + original = self.execute + def mutate_support(argv, **kwargs): + result = original(argv, **kwargs) + if '--dry-run' in argv: + helper = self.active.resolve().parent + name = next(name for name in self.sdk_table['files'] if name.startswith('prose-agents-sdk-runtime/')) + (helper / name).write_bytes(b'mutated') + return result + with patch.object(self, 'execute', side_effect=mutate_support), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_lingering_support_after_all_version_uninstall_is_refused(self): + original = self.execute + def leave_support(argv, **kwargs): + both = any(len(versions) > 1 for versions in self.retained_versions.values()) + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'uninstall'] and '--force' in argv and both: + (self.prefix / 'bin/prose-agents-sdk-runtime').mkdir(exist_ok=True) + return result + with patch.object(self, 'execute', side_effect=leave_support), self.assertRaisesRegex(ValueError, 'commands survived all-version uninstall'): + self.exercise() + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_rejected_link_must_preserve_helper_link_identity(self): + original = self.execute + def alter_link(argv, **kwargs): + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'link'] and result.returncode == 1: + link = self.prefix / 'bin/prose-agents-sdk' + link.unlink() + link.symlink_to(self.kegs['rust'][0] / 'bin/prose-agents-sdk') + return result + with patch.object(self, 'execute', side_effect=alter_link), self.assertRaisesRegex(ValueError, 'SDK helper linkage'): + self.exercise(previous=False) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_rejected_link_must_preserve_complete_selected_payload(self): + original = self.execute + def alter_tree(argv, **kwargs): + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'link'] and result.returncode == 1: + name = next(name for name in self.sdk_table['files'] if name.startswith('prose-agents-sdk-runtime/')) + (self.active.resolve().parent / name).write_bytes(b'changed-by-failed-link') + return result + with patch.object(self, 'execute', side_effect=alter_tree), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) diff --git a/cli/ci/test_kernel_rc_evidence.py b/cli/ci/test_kernel_rc_evidence.py index 7072a35d..db6aa669 100644 --- a/cli/ci/test_kernel_rc_evidence.py +++ b/cli/ci/test_kernel_rc_evidence.py @@ -7,6 +7,208 @@ import test_assemble_kernel_rc as assembly_tests +SDK_SYNTHETIC_COLLECT=b'Synthetic COLLECT declaration for nonexecuted consumer fixtures; no PyInstaller invocation.' + +def sdk_fixture(platform): + """Complete nonexecuted SDK table for consumer tests; never native release proof.""" + sdk, rows = assembly_tests.sdk_fixture(platform) + table={'files':{name:(data,mode) for name,data,mode in rows},'directories':{},'symlinks':{}} + if platform.startswith('darwin'): + import copy + from test_sdk_native_inventory import onedir_fixture + view=onedir_fixture('arm64' if platform=='darwin-arm64' else 'x86_64') + receipt=json.loads(table['files']['agents-sdk-build.json'][0]) + receipt['payload']=copy.deepcopy(view['payload']) + receipt['payload']['collectTocSha256']=hashlib.sha256(SDK_SYNTHETIC_COLLECT).hexdigest() + helper=view['files']['prose-agents-sdk'][0] + receipt['helper']={'path':'prose-agents-sdk','byteLength':len(helper),'sha256':hashlib.sha256(helper).hexdigest()} + encoded=json.dumps(receipt,sort_keys=True).encode() + sdk.update(receipt['helper']);sdk['receiptSha256']=hashlib.sha256(encoded).hexdigest() + table['files'].update(view['files']);table['files']['agents-sdk-build.json']=(encoded,0o644) + table['directories']=view['directories'];table['symlinks']=view['symlinks'] + return sdk,table + + +def sdk_producer_fixture(table,include_raw=False): + """Synthetic retained-record fixture ONLY; never signature/execution authority.""" + import base64 + prefix='' if 'prose-agents-sdk' in table['files'] else custody.sdk_archive_prefix(table) + receipt=json.loads(table['files'][prefix+'agents-sdk-build.json'][0]);payload=receipt['payload'] + assert payload['collectTocSha256']==hashlib.sha256(SDK_SYNTHETIC_COLLECT).hexdigest() + checks=[];outputs={} + expected=[('codesign-'+str(i).zfill(4)+'.log',path,'strict') for i,path in enumerate(payload['codeSignaturePaths'])] + expected.append(('codesign.log','prose-agents-sdk','strict-deep')) + for log,path,verification in expected: + raw={};count=0 + if include_raw: + data=b'Explicit synthetic signature stderr, no native execution.';name=log+'.stderr';count=len(data);digest=hashlib.sha256(data).hexdigest() + raw['stderr']={'path':name,'sha256':digest,'byteLength':count} + outputs[name]={'sha256':digest,'byteLength':count,'base64':base64.b64encode(data).decode()} + proof={'schema':'openprose.sdk-code-signature-check/1','phase':'code-signature','path':path,'verification':verification, + 'exitCode':0,'stdoutBytes':0,'stderrBytes':count,'timedOut':False,'outputLimitExceeded':False, + 'timeoutSeconds':30,'success':True,'outputComplete':True,'rawOutput':raw} + checks.append({'log':log,'proof':proof}) + bundle={'schema':'openprose.sdk-code-signatures/1','architecture':receipt['architecture'], + 'payloadSha256':hashlib.sha256(json.dumps(payload,sort_keys=True,separators=(',',':')).encode()).hexdigest(), + 'checks':checks,'rawOutputs':outputs} + return {custody.SDK_COLLECT_EVIDENCE:SDK_SYNTHETIC_COLLECT,custody.SDK_SIGNATURE_EVIDENCE:json.dumps(bundle,sort_keys=True).encode()} + + +def sdk_archive_fixture(table,prefix='root/',cli=b'fixture-cli',extra_files=None): + """Pure typed tar fixture encoder; not a release builder or executable proof.""" + import io,tarfile + stream=io.BytesIO() + with tarfile.open(fileobj=stream,mode='w:gz') as archive: + files={prefix+'prose':(cli,0o755),**{prefix+n:v for n,v in table['files'].items()},**(extra_files or {})} + for name,(data,mode) in files.items(): + info=tarfile.TarInfo(name);info.mode=mode;info.size=len(data) + archive.addfile(info,io.BytesIO(data)) + for name,mode in table['directories'].items(): + info=tarfile.TarInfo(prefix+name);info.type=tarfile.DIRTYPE;info.mode=mode;archive.addfile(info) + for name,target in table['symlinks'].items(): + info=tarfile.TarInfo(prefix+name);info.type=tarfile.SYMTYPE;info.mode=0o777;info.linkname=target;archive.addfile(info) + return stream.getvalue() + + + +class OnedirCustodyTests(unittest.TestCase): + def manifest(self,platform='darwin-x64'): + sdk,table=sdk_fixture(platform) + return {'platform':platform,'agentsSdk':sdk,'source':{'revision':'a'*40}},table + + def test_complete_tree_roundtrips_with_real_directory_and_file_aliases(self): + import tempfile + from pathlib import Path + for platform in ('darwin-arm64','darwin-x64','linux-x64-gnu'): + with self.subTest(platform=platform): + manifest,table=self.manifest(platform) + encoded=sdk_archive_fixture(table) + decoded=pub.decode_sdk_archive(encoded,manifest) + with tempfile.TemporaryDirectory() as temporary: + destination=Path(temporary).resolve() + observed=custody.materialize_sdk_members(manifest,decoded,destination) + self.assertEqual(observed,custody.sdk_scoped_table(decoded,'root/')) + if platform.startswith('darwin'): + self.assertTrue((destination/'prose-agents-sdk-runtime/Python.framework/Python').is_symlink()) + self.assertEqual((destination/'prose-agents-sdk-runtime/Python.framework/Python').read_bytes(),table['files']['prose-agents-sdk-runtime/Python.framework/Versions/3.10/Python'][0]) + + def test_rehashed_support_alias_mode_omission_and_extra_rows_are_rejected(self): + import copy + for poison in ('bytes','mode','omission','alias','extra-directory','extra-file','foreign-alias'): + manifest,table=self.manifest();bad=copy.deepcopy(table) + name='prose-agents-sdk-runtime/base_library.zip' + if poison=='bytes':bad['files'][name]=(b'poison',0o644) + elif poison=='mode':bad['files'][name]=(bad['files'][name][0],0o755) + elif poison=='omission':del bad['files'][name] + elif poison=='alias':bad['symlinks']['prose-agents-sdk-runtime/Python.framework/Python']='../../../outside' + elif poison=='extra-directory':bad['directories']['prose-agents-sdk-runtime/unlisted']=0o755 + elif poison=='extra-file':bad['files']['prose-agents-sdk-runtime/unlisted']=(b'x',0o644) + else:bad['symlinks']['unexpected']='prose-agents-sdk' + with self.subTest(poison=poison),self.assertRaises(ValueError): + pub.decode_sdk_archive(sdk_archive_fixture(bad),manifest) + + def test_foreign_macho_and_missing_signature_path_rejected(self): + from test_sdk_native_inventory import macho_fixture + for poison in ('cpu','signature'): + manifest,table=self.manifest();receipt=json.loads(table['files']['agents-sdk-build.json'][0]) + name='prose-agents-sdk-runtime/Python.framework/Versions/3.10/Python' + if poison=='cpu': + data=macho_fixture('arm64');table['files'][name]=(data,0o755) + row=next(r for r in receipt['payload']['entries'] if r['path']==name) + row.update(sha256=hashlib.sha256(data).hexdigest(),byteLength=len(data)) + else:receipt['payload']['codeSignaturePaths'].remove(name) + encoded=json.dumps(receipt).encode();table['files']['agents-sdk-build.json']=(encoded,0o644) + manifest['agentsSdk']['receiptSha256']=hashlib.sha256(encoded).hexdigest() + with self.subTest(poison=poison),self.assertRaises(ValueError):pub.decode_sdk_archive(sdk_archive_fixture(table),manifest) + + def test_missing_payload_fails_and_exact_historical_context_is_required(self): + manifest,table=self.manifest();old,rows=assembly_tests.sdk_fixture('darwin-x64') + # The root assembly fixture may now be current; remove the additive tree + # from this explicitly synthetic old source scope only. + if isinstance(rows,dict):rows=[(n,d,m) for n,(d,m) in rows['files'].items() if n in custody.SDK_NAMES] + oldtable={'files':{n:(d,m) for n,d,m in rows},'directories':{},'symlinks':{}} + receipt=json.loads(oldtable['files']['agents-sdk-build.json'][0]);receipt.pop('payload',None) + data=json.dumps(receipt).encode();oldtable['files']['agents-sdk-build.json']=(data,0o644);old['receiptSha256']=hashlib.sha256(data).hexdigest();manifest['agentsSdk']=old + encoded=sdk_archive_fixture(oldtable) + with self.assertRaisesRegex(ValueError,'complete onedir'):pub.decode_sdk_archive(encoded,manifest) + history={'sourceRevision':'a'*40,'archiveSha256':hashlib.sha256(encoded).hexdigest(),'receiptSha256':old['receiptSha256']} + pub.decode_sdk_archive(encoded,manifest,historical=history) + history['archiveSha256']='0'*64 + with self.assertRaisesRegex(ValueError,'Historical archive'):pub.decode_sdk_archive(encoded,manifest,historical=history) + + def test_existing_destination_refuses_before_any_sdk_tree_write(self): + import tempfile + from pathlib import Path + manifest,table=self.manifest();decoded=pub.decode_sdk_archive(sdk_archive_fixture(table),manifest) + with tempfile.TemporaryDirectory() as temporary: + destination=Path(temporary).resolve();(destination/'AGENTS-SDK-NOTICES.txt').write_bytes(b'old') + with self.assertRaisesRegex(ValueError,'fresh'):custody.materialize_sdk_members(manifest,decoded,destination) + self.assertEqual(sorted(p.name for p in destination.iterdir()),['AGENTS-SDK-NOTICES.txt']) + + def test_sdk_entry_allowance_never_enlarges_generic_reader(self): + import tempfile + from pathlib import Path + manifest,table=self.manifest() + encoded=sdk_archive_fixture(table) + with tempfile.TemporaryDirectory() as temporary: + path=Path(temporary)/'sdk.tgz';path.write_bytes(encoded) + with self.assertRaisesRegex(ValueError,'Links'):pub.archive_members(path) + with self.assertRaisesRegex(ValueError,'identity'):pub.decode_sdk_archive(encoded,{'platform':'darwin-x64'}) + extra={f'root/ordinary-{i}':(b'',0o644) for i in range(129)} + with self.assertRaisesRegex(ValueError,'generic member budget'):pub.decode_sdk_archive(sdk_archive_fixture(table,extra_files=extra),manifest) + + +class SdkProducerEvidenceTests(unittest.TestCase): + def fixture(self,raw=False): + sdk,table=sdk_fixture('darwin-x64');manifest={'platform':'darwin-x64','agentsSdk':sdk} + complete=pub.decode_sdk_archive(sdk_archive_fixture(table),manifest) + files=sdk_producer_fixture(table,raw);return manifest,complete,files + def verify(self,manifest,table,files): + evidence={name:{'sha256':hashlib.sha256(data).hexdigest(),'byteLength':len(data)} for name,data in files.items()} + return custody.validate_sdk_producer_evidence(manifest,table,evidence,files.__getitem__) + def test_silent_and_retained_raw_records_have_complete_strict_target_closure(self): + for raw in (False,True): + with self.subTest(raw=raw): + manifest,table,files=self.fixture(raw);bundle=self.verify(manifest,table,files) + self.assertEqual(len(bundle['checks']),3);self.assertEqual(bundle['checks'][-1]['proof']['verification'],'strict-deep') + def test_rehashed_signature_bundle_target_status_raw_and_digest_poison_rejected(self): + for poison in ('omit','duplicate','order','target','deep','timeout','success','complete','bool-exit','bool-count','payload','raw-bytes','raw-omit','raw-extra','raw-path','raw-size'): + manifest,table,files=self.fixture(True);bundle=json.loads(files[custody.SDK_SIGNATURE_EVIDENCE]);proof=bundle['checks'][0]['proof'] + if poison=='omit':bundle['checks'].pop() + elif poison=='duplicate':bundle['checks'][1]=bundle['checks'][0] + elif poison=='order':bundle['checks'].reverse() + elif poison=='target':proof['path']='unexpected' + elif poison=='deep':bundle['checks'][-1]['proof']['verification']='strict' + elif poison=='timeout':proof['timedOut']=True + elif poison=='success':proof['success']=False + elif poison=='complete':proof['outputComplete']=False + elif poison=='bool-exit':proof['exitCode']=False + elif poison=='bool-count':proof['stdoutBytes']=False + elif poison=='payload':bundle['payloadSha256']='0'*64 + elif poison=='raw-omit':bundle['rawOutputs'].clear() + elif poison=='raw-extra':bundle['rawOutputs']['undeclared']={} + elif poison=='raw-path':proof['rawOutput']['stderr']['path']='../unexpected' + elif poison=='raw-size':proof['stderrBytes']=65537 + else:next(iter(bundle['rawOutputs'].values()))['base64']='cG9pc29u' + files[custody.SDK_SIGNATURE_EVIDENCE]=json.dumps(bundle).encode() + with self.subTest(poison=poison),self.assertRaisesRegex(ValueError,'SDK producer'):self.verify(manifest,table,files) + def test_rehashed_collect_or_missing_bundle_rejected(self): + for poison in ('collect','missing'): + manifest,table,files=self.fixture() + if poison=='collect':files[custody.SDK_COLLECT_EVIDENCE]=b'poison' + else:del files[custody.SDK_SIGNATURE_EVIDENCE] + with self.subTest(poison=poison),self.assertRaisesRegex(ValueError,'SDK producer'):self.verify(manifest,table,files) + def test_linux_forbids_mac_producer_claims(self): + sdk,table=sdk_fixture('linux-x64-gnu');manifest={'platform':'linux-x64-gnu','agentsSdk':sdk} + complete=pub.decode_sdk_archive(sdk_archive_fixture(table),manifest) + self.assertIsNone(self.verify(manifest,complete,{})) + with self.assertRaisesRegex(ValueError,'Linux must not claim'):self.verify(manifest,complete,{custody.SDK_COLLECT_EVIDENCE:b'poison'}) + receipt=json.loads(table['files']['agents-sdk-build.json'][0]);receipt['payload']={'layout':'pyinstaller-onedir/1'} + encoded=json.dumps(receipt).encode();table['files']['agents-sdk-build.json']=(encoded,0o644);manifest['agentsSdk']['receiptSha256']=hashlib.sha256(encoded).hexdigest() + complete=pub.decode_sdk_archive(sdk_archive_fixture(table),manifest) + with self.assertRaisesRegex(ValueError,'Linux must not claim'):self.verify(manifest,complete,{}) + + class EvidenceTests(unittest.TestCase): setUp = assembly_tests.AssemblyTests.setUp record = assembly_tests.AssemblyTests.record @@ -81,6 +283,21 @@ def test_rehashed_checks_reject_fixed_startup_wrong_source_and_test_seams(self): with self.assertRaisesRegex(ValueError, 'exact release bytes'): self.verify(plan, output, hashes) + def test_rehashed_installed_payload_capture_cannot_omit_or_mutate_support(self): + import copy,shutil + for poison in ('omit','alias','order','after','count'): + with self.subTest(poison=poison): + plan,output,hashes=self.assembled() + rows=copy.deepcopy(pub.read_json(output/'darwin-arm64-logs-installed-sdk-payloads.json')) + if poison=='omit':rows[0]['before'].pop('supportTree') + elif poison=='alias':rows[0]['before']['supportTree']['payload']['entries'][-1]['path']='poison' + elif poison=='order':rows.reverse() + elif poison=='after':rows[0]['after']['prose-agents-sdk']['sha256']='0'*64 + else:rows[0]['before']['supportTree']['entryCount']=False + self.update_bound(plan,output,custody.SDK_PAYLOAD_EVIDENCE,rows) + with self.assertRaisesRegex(ValueError,'Installed SDK'):self.verify(plan,output,hashes) + shutil.rmtree(output) + def test_missing_retained_evidence_cannot_pass(self): plan, output, hashes = self.assembled() plan['artifacts'] = [a for a in plan['artifacts'] if a['name'] != 'darwin-arm64-logs-installed-npm.json'] diff --git a/cli/ci/test_package_local.py b/cli/ci/test_package_local.py index 3055781a..d5922928 100644 --- a/cli/ci/test_package_local.py +++ b/cli/ci/test_package_local.py @@ -1809,11 +1809,32 @@ def test_non_rust_version_receipt_does_not_inherit_rustup_selector(self) -> None self.assertNotIn("RUSTUP_TOOLCHAIN", run.call_args.kwargs["environment"]) +def macos_c_fixture_payload(directory,architecture): + """Actual supplied C-helper bytes + synthetic COLLECT; never PyInstaller proof.""" + import ast + import sdk_native_inventory as native + directory=directory.resolve() + support=directory/native.SDK_PAYLOAD_ROOT;support.mkdir(mode=0o755);support.chmod(0o755) + marker=support/'hermetic-c-fixture.txt' + marker.write_bytes(b'Hermetic compiled C helper fixture. Synthetic COLLECT only; no PyInstaller SDK qualification.');marker.chmod(0o644) + module=ast.parse((ROOT/'cli/ci/build_agents_sdk.py').read_text()) + entry=next(ast.literal_eval(node.value) for node in module.body if isinstance(node,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ENTRY_SOURCE' for t in node.targets)) + entry_path=directory/'sdk-entry.py';entry_path.write_text(entry) + # These declarations describe the actual hermetic fixture files, not a + # freezer invocation or SDK integration. The C helper was compiled/signed + # by write_sdk_candidate before this function captures its real bytes. + collect=repr(([(PACKAGE_LOCAL.SDK_NAME,str(directory/PACKAGE_LOCAL.SDK_NAME),'EXECUTABLE'), + ('hermetic-c-fixture.txt',str(marker),'DATA')],)).encode() + (directory/'collect.toc').write_bytes(collect) + builders={name:PACKAGE_LOCAL.sha256_file(ROOT/name) for name in ('cli/ci/build_agents_sdk.py','cli/ci/sdk_native_inventory.py')} + return native.inventory_macos_payload(directory,architecture,builders,hashlib.sha256(entry_path.read_bytes()).hexdigest(),hashlib.sha256(collect).hexdigest()) + + class LocalPackagingTests(unittest.TestCase): @classmethod def setUpClass(cls) -> None: cls.temporary = tempfile.TemporaryDirectory(prefix="openprose-package-test-") - cls.root = Path(cls.temporary.name) + cls.root = Path(cls.temporary.name).resolve() cls.source_revision = f"package-local-test-{uuid.uuid4().hex}" cls.rust_binary = ( cls.root @@ -2354,6 +2375,9 @@ def write_sdk_candidate(self): "notices": {"path": PACKAGE_LOCAL.SDK_NOTICES, "sha256": hashlib.sha256(notices).hexdigest(), "byteLength": len(notices)}, "selfTest": expected, "toolSelfTest": tools, "linuxLibraries": libraries if platform.system() == "Linux" else "not-applicable", "authority": "hermetic-test-fixture-not-release-evidence"} + if platform.system()=='Darwin': + record['payload']=macos_c_fixture_payload(directory,record['architecture']) + record['collectEvidenceAuthority']='synthetic-hermetic-C-fixture-not-PyInstaller' if native_fields is not None: record.update(native_fields) record['dependencies'] = [{'name': 'fixture', 'version': '1.0.0', 'wheelSha256': ['e' * 64]}] @@ -4948,5 +4972,102 @@ def execute(argv, **kwargs): self.assertEqual(len(payload), 3); self.assertEqual(len(record), 12) +class MacOnedirPackagingTests(unittest.TestCase): + def snapshot_fixture(self,root): + import ast + import sdk_native_inventory as native + from test_kernel_rc_evidence import sdk_fixture + identity,table=sdk_fixture('darwin-x64') + receipt=json.loads(table['files'][PACKAGE_LOCAL.SDK_RECEIPT][0]) + receipt['sources']={name:PACKAGE_LOCAL.sha256_file(ROOT/name) for name in receipt['sources']} + receipt['payload']['builderSources']={name:PACKAGE_LOCAL.sha256_file(ROOT/name) for name in receipt['payload']['builderSources']} + module=ast.parse((ROOT/'cli/ci/build_agents_sdk.py').read_text()) + entry=next(ast.literal_eval(node.value) for node in module.body if isinstance(node,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ENTRY_SOURCE' for t in node.targets)) + receipt['payload']['entrySourceSha256']=hashlib.sha256(entry.encode()).hexdigest() + collect=b'Synthetic package control COLLECT declaration; no freezer execution.' + receipt['payload']['collectTocSha256']=hashlib.sha256(collect).hexdigest() + sdk=root/'sdk';sdk.mkdir();snapshots=root/'snapshots';snapshots.mkdir() + (sdk/'sdk-entry.py').write_text(entry);(sdk/'collect.toc').write_bytes(collect) + scope={kind:{n:v for n,v in table[kind].items() if n not in (PACKAGE_LOCAL.SDK_RECEIPT,PACKAGE_LOCAL.SDK_NOTICES)} for kind in table} + native.materialize_macos_payload(sdk,receipt['payload'],**scope,architecture='x86_64') + (sdk/PACKAGE_LOCAL.SDK_RECEIPT).write_text(json.dumps(receipt));(sdk/PACKAGE_LOCAL.SDK_NOTICES).write_bytes(table['files'][PACKAGE_LOCAL.SDK_NOTICES][0]) + def execute(argv,**kwargs): + value=receipt['selfTest' if argv[-1]=='--packaged-self-test' else 'toolSelfTest'] + return subprocess.CompletedProcess(argv,0,json.dumps(value).encode(),b'') + return sdk,snapshots,receipt,execute + + def test_snapshot_retains_complete_support_and_verifies_every_native_file(self): + with tempfile.TemporaryDirectory() as temporary: + root=Path(temporary).resolve();sdk,snapshots,receipt,execute=self.snapshot_fixture(root) + with mock.patch.object(PACKAGE_LOCAL,'run_bounded',side_effect=execute) as run,mock.patch.object(PACKAGE_LOCAL,'verify_darwin_code_signature') as signatures: + table,identity=PACKAGE_LOCAL.snapshot_sdk(sdk,snapshots,'darwin-x64','kernel-rc',None) + self.assertEqual(len(identity),12) + self.assertEqual({str(call.args[0].relative_to(snapshots)) for call in signatures.call_args_list},set(receipt['payload']['codeSignaturePaths'])) + self.assertEqual(len(run.call_args_list),2) + self.assertTrue((snapshots/'prose-agents-sdk-runtime/Python.framework/Versions/Current').is_symlink()) + archive=root/'payload.tgz';members=[('root/prose',b'cli',0o755)]+[('root/'+n,d,m) for n,(d,m) in table['files'].items()] + PACKAGE_LOCAL.tar_gz(archive,members,0,sdk_table=table,sdk_prefix='root/') + import publication + decoded=publication.read_sdk_archive(archive,{'platform':'darwin-x64','agentsSdk':identity}) + self.assertEqual(decoded['symlinks'],{'root/'+n:t for n,t in table['symlinks'].items()}) + + def test_changed_support_is_rejected_before_selftests_and_copy(self): + with tempfile.TemporaryDirectory() as temporary: + root=Path(temporary).resolve();sdk,snapshots,receipt,execute=self.snapshot_fixture(root) + (sdk/'prose-agents-sdk-runtime/base_library.zip').write_bytes(b'poison') + with mock.patch.object(PACKAGE_LOCAL,'run_bounded') as run: + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError,'payload differs'):PACKAGE_LOCAL.snapshot_sdk(sdk,snapshots,'darwin-x64','kernel-rc',None) + run.assert_not_called();self.assertEqual(list(snapshots.iterdir()),[]) + + def test_poisoned_helper_and_notices_refuse_before_snapshot_writes(self): + for poison in ('helper','notices','entry','collect'): + with self.subTest(poison=poison),tempfile.TemporaryDirectory() as temporary: + root=Path(temporary).resolve();sdk,snapshots,receipt,execute=self.snapshot_fixture(root) + if poison in ('helper','notices'):receipt[poison]['sha256']='0'*64 + else:(sdk/('sdk-entry.py' if poison=='entry' else 'collect.toc')).write_bytes(b'poison') + (sdk/PACKAGE_LOCAL.SDK_RECEIPT).write_text(json.dumps(receipt)) + with mock.patch.object(PACKAGE_LOCAL,'run_bounded') as run: + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError,{'helper':'helper differs','notices':'notices differ','entry':'entry sidecar','collect':'COLLECT sidecar'}[poison]): + PACKAGE_LOCAL.snapshot_sdk(sdk,snapshots,'darwin-x64','kernel-rc',None) + run.assert_not_called();self.assertEqual(list(snapshots.iterdir()),[]) + + def test_native_C_fixture_payload_captures_supplied_bytes_and_truthful_sidecars(self): + import sdk_native_inventory as native + from test_sdk_native_inventory import macho_fixture + # Source-only factory control: header fixture is explicitly unexecuted; + # heavy LocalPackaging still supplies its real compiled/signed C helper. + with tempfile.TemporaryDirectory() as temporary: + root=Path(temporary).resolve();helper=root/PACKAGE_LOCAL.SDK_NAME + data=macho_fixture('x86_64',kind=2);helper.write_bytes(data);helper.chmod(0o755) + payload=macos_c_fixture_payload(root,'x86_64') + view=native.read_macos_payload(root,payload,'x86_64') + self.assertEqual(view['files'][PACKAGE_LOCAL.SDK_NAME],(data,0o755)) + self.assertEqual(payload['collectTocSha256'],hashlib.sha256((root/'collect.toc').read_bytes()).hexdigest()) + self.assertEqual(payload['entrySourceSha256'],hashlib.sha256((root/'sdk-entry.py').read_bytes()).hexdigest()) + self.assertEqual(payload['builderSources'],{name:PACKAGE_LOCAL.sha256_file(ROOT/name) for name in payload['builderSources']}) + self.assertIn(b'no PyInstaller SDK qualification',view['files']['prose-agents-sdk-runtime/hermetic-c-fixture.txt'][0]) + + def test_source_payload_change_during_selftests_fails(self): + with tempfile.TemporaryDirectory() as temporary: + root=Path(temporary).resolve();sdk,snapshots,receipt,execute=self.snapshot_fixture(root) + def changed(argv,**kwargs): + result=execute(argv,**kwargs) + (sdk/'prose-agents-sdk-runtime/base_library.zip').write_bytes(b'poison') + return result + with mock.patch.object(PACKAGE_LOCAL,'run_bounded',side_effect=changed),mock.patch.object(PACKAGE_LOCAL,'verify_darwin_code_signature'): + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError,'payload changed'):PACKAGE_LOCAL.snapshot_sdk(sdk,snapshots,'darwin-x64','kernel-rc',None) + + def test_archive_regular_projection_cannot_replace_or_add_sdk_member(self): + from test_kernel_rc_evidence import sdk_fixture + identity,table=sdk_fixture('darwin-x64') + for poison in ('replace','extra'): + with self.subTest(poison=poison),tempfile.TemporaryDirectory() as temporary: + path=Path(temporary)/'sdk.tgz';members=[('root/'+n,d,m) for n,(d,m) in table['files'].items()] + if poison=='replace':members=[(n,b'poison' if n.endswith('/base_library.zip') else d,m) for n,d,m in members] + else:members.append(('root/prose-agents-sdk-runtime/unlisted',b'poison',0o644)) + with self.assertRaisesRegex(PACKAGE_LOCAL.PackageError,'membership differs'):PACKAGE_LOCAL.tar_gz(path,members,0,sdk_table=table,sdk_prefix='root/') + self.assertFalse(path.exists()) + + if __name__ == "__main__": unittest.main() diff --git a/cli/ci/test_publication.py b/cli/ci/test_publication.py index 20e04b36..c7581495 100644 --- a/cli/ci/test_publication.py +++ b/cli/ci/test_publication.py @@ -8,7 +8,38 @@ import unittest from unittest.mock import patch import publication as p -from test_assemble_kernel_rc import sdk_fixture +from test_kernel_rc_evidence import sdk_fixture + + +class SdkRawArchiveTests(unittest.TestCase): + def manifest(self): + sdk,table=sdk_fixture('darwin-x64') + return {'platform':'darwin-x64','agentsSdk':sdk},table + + def test_negative_oversized_and_regressing_raw_sizes_are_rejected(self): + import gzip + from test_kernel_rc_evidence import sdk_archive_fixture + manifest,table=self.manifest() + original=gzip.decompress(sdk_archive_fixture(table)) + for value in (b'-0000000001',b'-0000002000',b'77777777777',b'02000000001'): + raw=bytearray(original);raw[124:136]=value+b'\x00' + with self.subTest(value=value),self.assertRaisesRegex(ValueError,'raw tar size'): + p.decode_sdk_archive(gzip.compress(raw),manifest) + + def test_truncated_raw_extent_and_trailing_gzip_or_tar_are_rejected(self): + import gzip + from test_kernel_rc_evidence import sdk_archive_fixture + manifest,table=self.manifest();encoded=sdk_archive_fixture(table) + raw=bytearray(gzip.decompress(encoded));raw[124:136]=b'00010000000\x00' + with self.assertRaisesRegex(ValueError,'raw tar offset'):p.decode_sdk_archive(gzip.compress(raw),manifest) + for bad in (encoded+b'opaque',encoded+gzip.compress(b'opaque'),gzip.compress(gzip.decompress(encoded)+b'opaque')): + with self.subTest(size=len(bad)),self.assertRaises(ValueError):p.decode_sdk_archive(bad,manifest) + + def test_unicode_normalization_collisions_rejected_before_projection(self): + from test_kernel_rc_evidence import sdk_archive_fixture + manifest,table=self.manifest() + extra={'root/caf\u00e9':(b'a',0o644),'root/cafe\u0301':(b'b',0o644)} + with self.assertRaisesRegex(ValueError,'Duplicate portable'):p.decode_sdk_archive(sdk_archive_fixture(table,extra_files=extra),manifest) class PublicationTests(unittest.TestCase): @@ -23,7 +54,7 @@ def setUp(self): sdk, sdk_members = sdk_fixture(platform) self.add(platform + '-release-manifest.json', json.dumps({'mode': 'release', 'platform': platform, 'version': self.plan['version'], 'source': {'revision': self.plan['source'], 'verification': 'matched-product-doctor'}, 'agentsSdk': sdk}).encode()) for implementation in ('bun', 'rust'): - self.tar(implementation + '-' + platform + '.tgz', {'root/prose': (implementation+platform).encode(), **{'root/' + n: d for n, d, m in sdk_members}}, 'standalone', platform, implementation) + self.tar(implementation + '-' + platform + '.tgz', {'root/prose': (implementation+platform).encode(), **{'root/' + n: d for n, (d, m) in sdk_members['files'].items()}}, 'standalone', platform, implementation, sdk_members, 'root/') if platform.startswith('darwin'): self.add(platform + '-receipt.json', b'{}') self.add(platform + '-notarization.zip', b'not a real signature') @@ -39,9 +70,9 @@ def setUp(self): meta.update(openproseSourceRevision='a'*40, openproseImage=self.image) members['package/bin/prose'] = ('bun'+platform).encode() _, sdk_members = sdk_fixture(platform) - members.update({'package/bin/' + n: d for n, d, m in sdk_members}) + members.update({'package/bin/' + n: d for n, (d, m) in sdk_members['files'].items()}) members['package/package.json'] = json.dumps(meta).encode() - self.tar(name.split('/')[1] + '.tgz', members, 'npm', platform, 'bun') + self.tar(name.split('/')[1] + '.tgz', members, 'npm', platform, 'bun', sdk_members if platform!='all' else None, 'package/bin/') self.path = self.root / 'plan.json' self.save() @@ -49,13 +80,21 @@ def add(self, name, data, kind='evidence', platform='all', implementation='share (self.root/name).write_bytes(data) self.plan['artifacts'].append({'name': name, 'sha256': hashlib.sha256(data).hexdigest(), 'size': len(data), 'kind': kind, 'platform': platform, 'implementation': implementation}) - def tar(self, name, members, kind, platform, implementation): + def tar(self, name, members, kind, platform, implementation, sdk=None, prefix=''): stream = io.BytesIO() with tarfile.open(fileobj=stream, mode='w:gz') as archive: for member, data in members.items(): info = tarfile.TarInfo(member) info.size = len(data) + info.mode = sdk['files'][member[len(prefix):]][1] if sdk and member.startswith(prefix) and member[len(prefix):] in sdk['files'] else 0o644 archive.addfile(info, io.BytesIO(data)) + if sdk: + for member, mode in sdk['directories'].items(): + info=tarfile.TarInfo(prefix+member);info.type=tarfile.DIRTYPE;info.mode=mode + archive.addfile(info) + for member, target in sdk['symlinks'].items(): + info=tarfile.TarInfo(prefix+member);info.type=tarfile.SYMTYPE;info.mode=0o777;info.linkname=target + archive.addfile(info) self.add(name, stream.getvalue(), kind, platform, implementation) def save(self): diff --git a/cli/ci/test_qualify_installed_sdk.py b/cli/ci/test_qualify_installed_sdk.py index ce344e55..2ca0ae55 100644 --- a/cli/ci/test_qualify_installed_sdk.py +++ b/cli/ci/test_qualify_installed_sdk.py @@ -15,17 +15,49 @@ import qualify_installed_sdk as installed -def archive(path, members): +def archive(path, members, table=None): with tarfile.open(path, 'w:gz') as target: for name, data in members.items(): - member = tarfile.TarInfo(name); member.size = len(data); member.mode = 0o755 + data, mode = data if isinstance(data, tuple) else (data, 0o755) + member = tarfile.TarInfo(name); member.size = len(data); member.mode = mode target.addfile(member, io.BytesIO(data)) + for name, mode in (table or {}).get('directories', {}).items(): + member = tarfile.TarInfo(name); member.type = tarfile.DIRTYPE; member.mode = mode; target.addfile(member) + for name, text in (table or {}).get('symlinks', {}).items(): + member = tarfile.TarInfo(name); member.type = tarfile.SYMTYPE; member.mode = 0o777; member.linkname = text; target.addfile(member) + + +def sdk_archive(path, extra=None, prefix='payload/'): + from test_kernel_rc_evidence import sdk_fixture + sdk, scoped = sdk_fixture('darwin-x64') + table = {kind: {prefix + name: value for name, value in rows.items()} for kind, rows in scoped.items()} + table['files'].update(extra or {prefix + 'prose': (b'cli', 0o755)}) + archive(path, table['files'], table) + return {'agentsSdk': sdk, 'platform': 'darwin-x64'}, table + + +def sdk_archive_with_resource_names(path): + manifest, table = sdk_archive(path) + payload = json.loads(table['files']['payload/agents-sdk-build.json'][0]) + for name in ('prose', 'agents-sdk-build.json'): + relative = 'prose-agents-sdk-runtime/' + name + data = b'support resource ' + name.encode() + table['files']['payload/' + relative] = (data, 0o644) + payload['payload']['entries'].append({'path': relative, 'type': 'file', 'mode': 0o644, + 'byteLength': len(data), 'sha256': hashlib.sha256(data).hexdigest()}) + payload['payload']['totalRegularBytes'] += len(data) + payload['payload']['entries'].sort(key=lambda row: row['path']) + encoded = json.dumps(payload, sort_keys=True).encode() + table['files']['payload/agents-sdk-build.json'] = (encoded, 0o644) + manifest['agentsSdk']['receiptSha256'] = hashlib.sha256(encoded).hexdigest() + archive(path, table['files'], table) + return manifest, table class InstallationQualificationTests(unittest.TestCase): def setUp(self): self.temporary = tempfile.TemporaryDirectory(); self.addCleanup(self.temporary.cleanup) - self.root = Path(self.temporary.name) + self.root = Path(self.temporary.name).resolve() self.node = self.root / 'node'; self.node.write_bytes(b'node-fixture'); self.node.chmod(0o755) self.npm = self.root / 'npm-cli.js'; self.npm.write_bytes(b'npm-fixture') @@ -74,25 +106,27 @@ def test_npx_preserves_exact_priming_registry_cache_key_and_remains_offline(self def test_standalone_extracts_bound_siblings_and_relocation_preserves_bytes(self): path = self.root / 'package.tgz' - data = {'payload/prose': b'cli', **{'payload/' + n: n.encode() for n in installed.SDK_NAMES}} - archive(path, data) - receipt = installed.extract_standalone(path, self.root / 'install', sdk=True) + manifest, _ = sdk_archive(path) + receipt = installed.extract_standalone(path, self.root / 'install', sdk=True, manifest=manifest) (self.root / 'install').rename(self.root / 'relocated') - self.assertEqual(set(receipt), {'prose', *installed.SDK_NAMES}) + self.assertEqual(set(receipt), {'prose', *installed.SDK_NAMES, 'supportTree'}) for name, record in receipt.items(): + if name == "supportTree": continue self.assertEqual(installed.pub.digest(self.root / 'relocated' / name), record['sha256']) with self.assertRaisesRegex(ValueError, 'fresh'): - installed.extract_standalone(path, self.root / 'relocated', sdk=True) + installed.extract_standalone(path, self.root / 'relocated', sdk=True, manifest=manifest) def test_archive_link_and_missing_helper_rejected_before_execution(self): path = self.root / 'bad.tgz' with tarfile.open(path, 'w:gz') as target: member = tarfile.TarInfo('payload/prose'); member.type = tarfile.SYMTYPE; member.linkname = '/user/bin/prose'; target.addfile(member) - with self.assertRaisesRegex(ValueError, 'Links'): - installed.extract_standalone(path, self.root / 'install', sdk=True) + from test_kernel_rc_evidence import sdk_fixture + manifest = {'platform': 'darwin-x64', 'agentsSdk': sdk_fixture('darwin-x64')[0]} + with self.assertRaises(ValueError): + installed.extract_standalone(path, self.root / 'install', sdk=True, manifest=manifest) archive(path, {'payload/prose': b'cli'}) - with self.assertRaisesRegex(ValueError, 'siblings'): - installed.extract_standalone(path, self.root / 'install', sdk=True) + with self.assertRaisesRegex(ValueError, 'SDK'): + installed.extract_standalone(path, self.root / 'install', sdk=True, manifest=manifest) self.assertFalse((self.root / 'install').exists()) def test_artifact_digest_and_regular_file_binding(self): @@ -104,19 +138,21 @@ def test_artifact_digest_and_regular_file_binding(self): installed.artifact(self.root, record) def test_npm_original_alias_cohort_and_native_binding(self): - source = '1' * 40; version = '0.15.0-rc.4'; platform = 'darwin-arm64'; binary = b'compiled-bun' + source = '1' * 40; version = '0.15.0-rc.4'; platform = 'darwin-x64'; binary = b'compiled-bun' payload_version = installed.package_local.npm_payload_version(version, platform) cohort = {'sourceRevision': source, 'version': version} meta = {'name': '@openprose/prose-cli', 'version': version, 'openproseCohort': cohort, 'optionalDependencies': {'@openprose/prose-cli-' + platform: 'npm:@openprose/prose-cli@' + payload_version}} native = {'name': '@openprose/prose-cli', 'version': payload_version, 'openproseCohort': cohort} archive(self.root / 'meta.tgz', {'package/package.json': json.dumps(meta).encode(), 'package/bin/prose.js': b'launcher'}) - archive(self.root / 'native.tgz', {'package/package.json': json.dumps(native).encode(), 'package/bin/prose': binary}) + sdk_manifest, native_table = sdk_archive(self.root / 'native.tgz', {'package/package.json': (json.dumps(native).encode(), 0o644), 'package/bin/prose': (binary, 0o755)}, prefix='package/bin/') manifest = {'source': {'revision': source}, 'version': version, 'artifacts': [{'kind': kind, 'platform': target, 'path': path.name, 'byteLength': path.stat().st_size, 'sha256': installed.pub.digest(path)} for kind, target, path in [('npm-meta', None, self.root / 'meta.tgz'), ('npm-platform', platform, self.root / 'native.tgz')]]} + manifest['agentsSdk'] = sdk_manifest['agentsSdk']; manifest['platform'] = 'darwin-x64' self.assertEqual(installed.npm_inputs(self.root, manifest, platform, expected_binary=hashlib.sha256(binary).hexdigest()), (self.root / 'meta.tgz', self.root / 'native.tgz')) with self.assertRaisesRegex(ValueError, 'differs from verified'): installed.npm_inputs(self.root, manifest, platform, expected_binary='0' * 64) native['scripts'] = {'postinstall': 'touch bad'} - archive(self.root / 'native.tgz', {'package/package.json': json.dumps(native).encode(), 'package/bin/prose': binary}) + native_table['files']['package/package.json'] = (json.dumps(native).encode(), 0o644) + archive(self.root / 'native.tgz', native_table['files'], native_table) manifest['artifacts'][1].update(byteLength=(self.root / 'native.tgz').stat().st_size, sha256=installed.pub.digest(self.root / 'native.tgz')) with self.assertRaisesRegex(ValueError, 'lifecycle'): installed.npm_inputs(self.root, manifest, platform) @@ -227,6 +263,87 @@ def test_candidate_custody_failure_prevents_output_and_execution(self): installed.qualify(self.root / 'candidate', self.root / 'previous', self.root / 'output', '1' * 40, '0.15.0-rc.4', self.node, self.npm) verifier.assert_called_once(); self.assertFalse((self.root / 'output').exists()) + def test_support_resource_named_prose_does_not_select_cli(self): + manifest, _ = sdk_archive_with_resource_names(self.root / 'complete.tgz') + directory = self.root / 'installed' + installed.extract_standalone(self.root / 'complete.tgz', directory, sdk=True, manifest=manifest) + self.assertEqual((directory / 'prose').read_bytes(), b'cli') + self.assertEqual((directory / 'prose-agents-sdk-runtime/prose').read_bytes(), b'support resource prose') + + def test_complete_tree_relocation_and_aliases_are_preserved(self): + manifest, _ = sdk_archive(self.root / 'complete.tgz') + directory = self.root / 'installed' + installed.extract_standalone(self.root / 'complete.tgz', directory, sdk=True, manifest=manifest) + before = installed.installed_sdk_identity(directory, manifest['agentsSdk']) + directory.rename(self.root / 'relocated-complete') + self.assertEqual(before, installed.installed_sdk_identity(self.root / 'relocated-complete', manifest['agentsSdk'])) + framework = self.root / 'relocated-complete/prose-agents-sdk-runtime/Python.framework' + self.assertTrue((framework / 'Versions/Current').is_symlink()) + self.assertEqual(os.readlink(framework / 'Versions/Current'), '3.10') + (framework / 'Versions/3.10/Python').write_bytes(b'changed') + with self.assertRaises(ValueError): + installed.installed_sdk_identity(self.root / 'relocated-complete', manifest['agentsSdk']) + + def test_missing_support_refused_before_any_helper_probe(self): + manifest, _ = sdk_archive(self.root / 'complete.tgz') + q = self.qualification(); directory = q.output / 'installed' + installed.extract_standalone(self.root / 'complete.tgz', directory, sdk=True, manifest=manifest) + import shutil + shutil.rmtree(directory / 'prose-agents-sdk-runtime') + with patch.object(q, 'execute') as execute: + with self.assertRaises(ValueError): + q.payload(directory, installed.pub.digest(directory / 'prose'), manifest['agentsSdk']) + execute.assert_not_called() + + def test_support_mutation_during_mocked_tool_probe_cannot_be_recorded(self): + manifest, _ = sdk_archive(self.root / 'complete.tgz') + q = self.qualification(); directory = q.output / 'installed' + installed.extract_standalone(self.root / 'complete.tgz', directory, sdk=True, manifest=manifest) + def mocked(argv, **kwargs): + if argv[-1] == '--packaged-tool-self-test': + (directory / 'prose-agents-sdk-runtime/empty-data').write_bytes(b'mutated') + expected = installed.kernel_rc_evidence.SDK_TOOL_TEST + else: expected = installed.kernel_rc_evidence.SDK_IMPORT_TEST + return {'stdout': json.dumps(expected), 'stderr': ''} + with patch.object(q, 'execute', side_effect=mocked): + with self.assertRaises(ValueError): + q.payload(directory, installed.pub.digest(directory / 'prose'), manifest['agentsSdk']) + self.assertEqual(q.payloads, []) + + def test_receipt_dictionary_is_once_bound_and_missing_unreferenced_poison_refused(self): + import copy + manifest, _ = sdk_archive(self.root / 'complete.tgz') + q = self.qualification(); directory = q.output / 'installed' + installed.extract_standalone(self.root / 'complete.tgz', directory, sdk=True, manifest=manifest) + def mocked(argv, **kwargs): + expected = installed.kernel_rc_evidence.SDK_TOOL_TEST if argv[-1] == '--packaged-tool-self-test' else installed.kernel_rc_evidence.SDK_IMPORT_TEST + return {'stdout': json.dumps(expected), 'stderr': ''} + with patch.object(q, 'execute', side_effect=mocked): + q.payload(directory, installed.pub.digest(directory / 'prose'), manifest['agentsSdk']) + installed.validate_payload_evidence(q.payloads, q.sdk_evidence, 'darwin-x64') + with self.assertRaisesRegex(ValueError, 'architecture differs'): + installed.validate_payload_evidence(q.payloads, q.sdk_evidence, 'darwin-arm64') + self.assertEqual(len(q.sdk_evidence), 1) + self.assertNotIn('payload', q.payloads[0]['members']['supportTree']) + for poison in ('missing', 'unreferenced', 'support-count', 'receipt-bytes', 'record-extra'): + records, evidence = copy.deepcopy(q.payloads), dict(q.sdk_evidence) + if poison == 'missing': evidence.clear() + elif poison == 'unreferenced': evidence['0' * 64] = next(iter(evidence.values())) + elif poison == 'support-count': records[0]['before']['supportTree']['entryCount'] = True + elif poison == 'receipt-bytes': evidence[next(iter(evidence))] += ' ' + else: records[0]['unknown'] = 'not-accepted' + with self.subTest(poison=poison), self.assertRaises(ValueError): + installed.validate_payload_evidence(records, evidence, 'darwin-x64') + digest = next(iter(q.sdk_evidence)) + raw = q.sdk_evidence[digest].replace('{', '{"schema":"duplicate",', 1) + rebound = hashlib.sha256(raw.encode()).hexdigest() + records = copy.deepcopy(q.payloads) + for identity in (records[0]['before'], records[0]['after'], records[0]['members']): + identity['supportTree']['receiptSha256'] = rebound + identity['agents-sdk-build.json'] = {'sha256': rebound, 'byteLength': len(raw.encode())} + with self.assertRaises(ValueError): + installed.validate_payload_evidence(records, {rebound: raw}, 'darwin-x64') + if __name__ == '__main__': unittest.main() diff --git a/cli/ci/test_rehearse_release.py b/cli/ci/test_rehearse_release.py index 95fa2cb4..c65f1677 100644 --- a/cli/ci/test_rehearse_release.py +++ b/cli/ci/test_rehearse_release.py @@ -1140,5 +1140,66 @@ def test_bounds_are_closed(self) -> None: ) +class CompleteSdkConformanceCustodyTests(unittest.TestCase): + def fixture(self, root, platform='darwin-x64'): + from test_kernel_rc_evidence import sdk_fixture + import kernel_rc_evidence as custody + module = rehearse_release.load_installed_benchmark(); module.sdk_modules() + sdk, scoped = sdk_fixture(platform) + manifest = {'platform': platform, 'agentsSdk': sdk} + raw = {'platform': platform, 'installations': [], 'measurementPlan': {'executablePaths': {'direct-rust': '$INSTALL_ROOT/rust-standalone/release/prose', 'direct-bun': '$INSTALL_ROOT/bun-standalone/release/prose'}}} + evidence = {sdk['receiptSha256']: scoped['files']['agents-sdk-build.json'][0].decode()} + if platform.startswith('darwin'): raw['sdkEvidence'] = evidence + for label, tree_root, prefix in ( + ('direct-rust', root / 'rust-standalone', 'release'), + ('direct-bun', root / 'bun-standalone', 'release'), + ('npm-launcher', root / 'npm-prefix', 'lib/node_modules/@openprose/prose-cli-' + platform + '/bin')): + directory = tree_root / prefix; directory.mkdir(parents=True) + table = {kind: {'source/' + n: v for n, v in rows.items()} for kind, rows in scoped.items()} + table['files']['source/prose'] = (b'nonexecuted CLI fixture', 0o755) + custody.materialize_sdk_members(manifest, table, directory) + (directory / 'prose').write_bytes(b'nonexecuted CLI fixture'); (directory / 'prose').chmod(0o755) + context = {'receiptSha256': sdk['receiptSha256'], 'prefix': prefix, 'encoded': evidence[sdk['receiptSha256']]} if platform.startswith('darwin') else None + tree = module.capture_installed_tree(tree_root, sdk_context=context) + raw['installations'].append({'surface': label, 'treeIdentity': tree}) + return module, manifest, raw + + def test_three_complete_source_trees_authenticate_on_both_platforms(self): + for platform in ('darwin-x64', 'linux-x64-gnu'): + with self.subTest(platform=platform), tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); module, release, raw = self.fixture(root, platform) + contexts = rehearse_release._sdk_conformance_contexts(root, raw, release, module) + self.assertEqual(set(contexts), {'direct-rust', 'direct-bun', 'npm-launcher'}) + self.assertTrue(all(c['agentsSdk'] == release['agentsSdk'] for c in contexts.values())) + expected = rehearse_release._sdk_report_binding(raw, release) + rehearse_release._validate_sdk_conformance_binding({'sdkSourceCustody': expected}, raw, release) + with self.assertRaises(rehearse_release.RehearsalError): rehearse_release._validate_sdk_conformance_binding({}, raw, release) + bad = json.loads(json.dumps(expected)); bad['candidateReferences']['direct-rust'] = '0' * 64 + with self.assertRaises(rehearse_release.RehearsalError): rehearse_release._validate_sdk_conformance_binding({'sdkSourceCustody': bad}, raw, release) + altered = json.loads(json.dumps(release)); altered['agentsSdk']['sha256'] = '0' * 64 + with self.assertRaises(ValueError): rehearse_release._sdk_conformance_contexts(root, raw, altered, module) + + def test_omitted_release_and_support_changes_refuse_instead_of_fixture_fallback(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); module, release, raw = self.fixture(root) + with self.assertRaises(rehearse_release.RehearsalError): rehearse_release._sdk_conformance_contexts(root, raw, None, module) + (root / 'rust-standalone/release/prose-agents-sdk-runtime/empty-data').write_bytes(b'changed') + with self.assertRaises(Exception): rehearse_release._sdk_conformance_contexts(root, raw, release, module) + + def test_source_rechecked_after_mocked_conformance_before_report_admission(self): + import time + from unittest.mock import patch + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); module, release, raw = self.fixture(root) + labels = [('direct-rust', 'rust', root / 'rust-standalone/prose'), ('direct-bun', 'bun', root / 'bun-standalone/prose'), ('npm-launcher', 'bun', root / 'npm-prefix/bin/prose')] + def execute(argv, cwd, env): + self.assertEqual(argv.count('--candidate-sdk-source'), 3) + (root / 'bun-standalone/release/prose-agents-sdk-runtime/empty-data').write_bytes(b'changed') + return subprocess.CompletedProcess(argv, 0, b'', b'') + with patch.object(rehearse_release, '_conformance_inputs', return_value=(labels, Path('/nonexecuted/node'), '0'*64)), patch.object(rehearse_release, '_npm_conformance_context', return_value={}): + with self.assertRaises(Exception): rehearse_release._run_mechanical_conformance(type('Runner', (), {'__file__': __file__}), raw, root, root / 'report.json', {}, time.monotonic() + 30, execute, release=release, benchmark_module=module) + self.assertFalse((root / 'report.json').exists()) + + if __name__ == "__main__": unittest.main(verbosity=2) diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index 37af5e04..d23e208a 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -40,6 +40,36 @@ def elf(*, imported=False, machine=62, version='GCC_3.0', symbol='_Unwind_Resume return bytes(blob) +def macho_fixture(architecture='x86_64', kind=6): + """Header-only, nonexecuted Mach-O parser fixture; no native success claim.""" + cpu = {'x86_64': 0x1000007, 'arm64': 0x100000c}[architecture] + return struct.pack('IIIIIII', 0xcafebabe, 1, 0x1000007, 3, 32, len(thin), 3) + bytes(4) + thin + self.assertEqual(native.macho_architectures(fat), ('x86_64',)) + for poisoned in (thin[:10], struct.pack('I', 0) + fat[16:]): + with self.assertRaises(ValueError): native.macho_architectures(poisoned) + + def test_growing_alias_cycle_and_repeat_reuse_are_bounded_without_false_rejection(self): + value = onedir_fixture(); root = native.SDK_PAYLOAD_ROOT; path = root + '/Python.framework/Python' + value['symlinks'][path] = 'Python/Python' + next(r for r in value['payload']['entries'] if r['path'] == path)['target'] = 'Python/Python' + with mock.patch.object(native, 'deque', wraps=native.deque) as queue: + with self.assertRaisesRegex(ValueError, 'alias cycle'): native.validate_macos_payload(**value) + self.assertEqual(queue.call_count, 1) + value = onedir_fixture(); target = 'Versions/Current/../Current/Python' + value['symlinks'][path] = target + next(r for r in value['payload']['entries'] if r['path'] == path)['target'] = target + native.validate_macos_payload(**value) + + def test_portable_file_directory_alias_unicode_collisions_and_drive_targets_refuse(self): + root = native.SDK_PAYLOAD_ROOT + for kind in ('file', 'directory', 'symlink'): + value = onedir_fixture(); row = {'path': root + '/EMPTY-DATA', 'type': kind} + if kind == 'file': row.update(sha256=native.sha(b''), byteLength=0, mode=0o644) + elif kind == 'directory': row.update(mode=0o755) + else: row.update(target='empty-data', resolvedPath=root + '/empty-data') + value['payload']['entries'].append(row); value['payload']['entries'].sort(key=lambda r: r['path']) + with self.subTest(kind=kind), self.assertRaisesRegex(ValueError, 'Portable SDK path collision'): + native.validate_macos_payload_structure(value['payload'], value['architecture']) + value = onedir_fixture() + for name in ('\u00e9', 'e\u0301'): + value['payload']['entries'].append({'path': root + '/' + name, 'type': 'directory', 'mode': 0o755}) + value['payload']['entries'].sort(key=lambda r: r['path']) + with self.assertRaisesRegex(ValueError, 'Portable SDK path collision'): + native.validate_macos_payload_structure(value['payload'], value['architecture']) + for target in ('C:relative', './C:/other', 'Versions/C:'): + value = onedir_fixture(); row = next(r for r in value['payload']['entries'] if r['type'] == 'symlink') + row['target'] = target + with self.assertRaisesRegex(ValueError, 'alias target'): + native.validate_macos_payload_structure(value['payload'], value['architecture']) + value = onedir_fixture(); value['payload']['entries'][0]['path'] = root + '/C:relative' + with self.assertRaisesRegex(ValueError, 'payload path'): + native.validate_macos_payload_structure(value['payload'], value['architecture']) + + def test_structure_api_reuses_closed_alias_rules_without_claiming_bytes(self): + value = onedir_fixture() + self.assertEqual(native.validate_macos_payload_structure(value['payload'], value['architecture']), value['payload']) + # Structure can validate a report declaration, but the full validator + # still rejects changed actual bytes or incomplete native coverage. + value['files'][native.SDK_HELPER] = (b'not Mach-O', 0o755) + native.validate_macos_payload_structure(value['payload'], value['architecture']) + with self.assertRaisesRegex(ValueError, 'must be Mach-O'): + native.validate_macos_payload(**value) + old_name = copy.deepcopy(value['payload']); old_name['sources'] = old_name.pop('builderSources') + with self.assertRaisesRegex(ValueError, 'payload.*shape'): + native.validate_macos_payload_structure(old_name, value['architecture']) + unknown_alias = copy.deepcopy(value['payload']) + next(r for r in unknown_alias['entries'] if r['type'] == 'symlink')['target'] = '/outside' + with self.assertRaises(ValueError): native.validate_macos_payload_structure(unknown_alias, value['architecture']) + + def test_sdk_only_member_aggregate_and_receipt_bounds(self): + value = onedir_fixture() + with mock.patch.object(native, 'SDK_PAYLOAD_MAX_ENTRIES', 2), self.assertRaisesRegex(ValueError, 'member count'): + native.validate_macos_payload(**value) + with mock.patch.object(native, 'SDK_PAYLOAD_MAX_BYTES', 2), self.assertRaisesRegex(ValueError, 'aggregate bytes'): + native.validate_macos_payload(**value) + with mock.patch.object(native, 'SDK_PAYLOAD_MAX_METADATA', 2), self.assertRaisesRegex(ValueError, 'metadata'): + native.validate_macos_payload(**value) + + if __name__ == '__main__': unittest.main() diff --git a/cli/ci/test_sign_macos.py b/cli/ci/test_sign_macos.py index 09687319..eddea04b 100644 --- a/cli/ci/test_sign_macos.py +++ b/cli/ci/test_sign_macos.py @@ -21,7 +21,7 @@ class SigningTests(unittest.TestCase): def setUp(self): self.temp = tempfile.TemporaryDirectory() self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) + self.root = Path(self.temp.name).resolve() for name in ("bun", "rust", "key"): (self.root / name).write_bytes(name.encode()) self.output = self.root / "signed" @@ -86,22 +86,113 @@ def test_sdk_missing_inner_signing_receipt_fails_before_native_commands(self): self.assertEqual(self.calls, []) self.assertFalse(self.output.exists()) - def test_sdk_helper_signed_and_notarized_with_both_runners(self): - helper = self.root / 'sdk'; helper.write_bytes(b'sdk') + def sdk_fixture(self): + from test_sdk_native_inventory import onedir_fixture + import sdk_native_inventory as native + view=onedir_fixture() + import ast + module=ast.parse((Path(signing.__file__).with_name('build_agents_sdk.py')).read_text()) + entry=next(ast.literal_eval(node.value) for node in module.body if isinstance(node,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ENTRY_SOURCE' for t in node.targets)) + (self.root/'sdk-entry.py').write_text(entry);collect=b'Explicit synthetic signing COLLECT; no freezer execution.' + (self.root/'collect.toc').write_bytes(collect) + view['payload']['entrySourceSha256']=hashlib.sha256(entry.encode()).hexdigest() + view['payload']['collectTocSha256']=hashlib.sha256(collect).hexdigest() + root=Path(signing.__file__).resolve().parents[2] + view['payload']['builderSources']={name:signing.sha256(root/name) for name in view['payload']['builderSources']} + native.materialize_macos_payload(self.root,view['payload'],view['files'],view['directories'],view['symlinks'],view['architecture']) + helper=self.root/'prose-agents-sdk' notices = self.root / 'AGENTS-SDK-NOTICES.txt'; notices.write_bytes(b'notices') - build = {'helper': {'path': 'prose-agents-sdk', 'byteLength': 3, 'sha256': signing.sha256(helper)}, + original=helper.read_bytes() + build = {'helper': {'path': 'prose-agents-sdk', 'byteLength': len(original), 'sha256': signing.sha256(helper)}, 'notices': {'path': notices.name, 'byteLength': 7, 'sha256': signing.sha256(notices)}, + 'architecture':view['architecture'],'payload':view['payload'], + 'schema':'openprose.agents-sdk-build/1','platform':'darwin','python':'3.10.20','pyinstaller':'6.22.3','modelCalls':0, + 'sources':{name:signing.sha256(root/name) for name in ('harnesses/agents-sdk/run.py','harnesses/agents-sdk/requirements-build.txt')}, + 'selfTest':__import__('kernel_rc_evidence').SDK_IMPORT_TEST,'toolSelfTest':__import__('kernel_rc_evidence').SDK_TOOL_TEST, 'embeddedSigning': {'identity': IDENTITY, 'verification': 'pyinstaller-inner-binaries-and-frozen-self-tests'}} (self.root / 'agents-sdk-build.json').write_text(json.dumps(build)) + return helper,view,original + + def test_sdk_helper_signed_and_notarized_with_both_runners(self): + helper,view,original=self.sdk_fixture() receipt = self.sign(agents_sdk=helper) bound = json.loads((self.output / 'agents-sdk-build.json').read_text()) self.assertEqual(bound['helper']['sha256'], receipt['binaries']['agents-sdk']['signedSha256']) self.assertEqual(bound['signingReceiptSha256'], signing.sha256(self.output / 'receipt.json')) self.assertEqual(set(receipt['binaries']), {'bun', 'rust', 'agents-sdk'}) - self.assertEqual(receipt['binaries']['agents-sdk']['signedSha256'], hashlib.sha256(b'sdk-signed').hexdigest()) + self.assertEqual(receipt['binaries']['agents-sdk']['signedSha256'], hashlib.sha256(original+b'-signed').hexdigest()) self.assertEqual(signing.verify_existing(self.output, **self.kwargs()), receipt) with zipfile.ZipFile(self.output / 'notarization.zip') as archive: - self.assertEqual(set(archive.namelist()), {'prose-bun', 'prose-rust', 'prose-agents-sdk'}) + self.assertEqual({n.rstrip('/') for n in archive.namelist()}, {'prose-bun', 'prose-rust', 'prose-agents-sdk'}|{row['path'] for row in view['payload']['entries']}) + self.assertEqual(archive.read('prose-agents-sdk-runtime/Python.framework/Python'),b'Versions/Current/Python') + signed={call[-1] for call in self.calls if '--sign' in call} + self.assertEqual(signed,{str(self.output/n) for n in ('prose-bun','prose-rust','prose-agents-sdk')}) + for name in view['payload']['codeSignaturePaths']: + self.assertTrue(any('--verify' in call and call[-1]==str(self.output/name) for call in self.calls)) + + def test_signed_output_source_sidecars_feed_package_snapshot_without_native_execution(self): + import package_local as package + helper,view,original=self.sdk_fixture();self.sign(agents_sdk=helper) + snapshots=self.root/'snapshots';snapshots.mkdir() + def probe(argv,**kwargs): + import kernel_rc_evidence as custody + value=custody.SDK_IMPORT_TEST if argv[-1]=='--packaged-self-test' else custody.SDK_TOOL_TEST + return subprocess.CompletedProcess(argv,0,json.dumps(value).encode(),b'') + with patch.object(package,'run_bounded',side_effect=probe),patch.object(package,'verify_darwin_code_signature'): + table,identity=package.snapshot_sdk(self.output,snapshots,'darwin-x64','kernel-rc',None) + self.assertEqual(len(identity),12);self.assertEqual(table['symlinks'],view['symlinks']) + for name in ('sdk-entry.py','collect.toc'): + self.assertEqual((self.output/name).read_bytes(),(self.root/name).read_bytes()) + with zipfile.ZipFile(self.output/'notarization.zip') as archive: + self.assertNotIn('sdk-entry.py',archive.namelist());self.assertNotIn('collect.toc',archive.namelist()) + + def test_producer_sidecar_mutation_during_notarization_issues_no_receipt(self): + helper,view,original=self.sdk_fixture() + def changed(args): + result=self.native(args) + if 'submit' in args:(self.output/'collect.toc').write_bytes(b'poison') + return result + with self.assertRaisesRegex(signing.SigningError,'sidecar'):self.sign(agents_sdk=helper,commands=changed) + self.assertFalse((self.output/'receipt.json').exists()) + self.assertFalse((self.output/'agents-sdk-build.json').exists()) + + def test_source_sidecar_tamper_refuses_before_any_native_command(self): + helper,view,original=self.sdk_fixture();(self.root/'collect.toc').write_bytes(b'poison') + with self.assertRaises(signing.SigningError):self.sign(agents_sdk=helper) + self.assertEqual(self.calls,[]);self.assertFalse(self.output.exists()) + + def test_sdk_support_tamper_fails_before_any_sign_or_notary_command(self): + helper,view,original=self.sdk_fixture() + (self.root/'prose-agents-sdk-runtime/base_library.zip').write_bytes(b'poison') + with self.assertRaisesRegex(signing.SigningError,'complete valid onedir'):self.sign(agents_sdk=helper) + self.assertEqual(self.calls,[]);self.assertFalse(self.output.exists()) + + def test_rehashed_notarization_support_alias_omission_and_modes_are_rejected(self): + helper,view,original=self.sdk_fixture();receipt=self.sign(agents_sdk=helper) + table=signing.notarization_table(self.output/'notarization.zip',receipt) + import copy + for poison in ('bytes','alias','omission','mode','directory-mode'): + with self.subTest(poison=poison): + bad=copy.deepcopy(table) + path='prose-agents-sdk-runtime/base_library.zip' + if poison=='bytes':bad['files'][path]=(b'poison',0o644) + elif poison=='mode':bad['files'][path]=(bad['files'][path][0],0o755) + elif poison=='omission':del bad['files'][path] + elif poison=='directory-mode':bad['directories']['prose-agents-sdk-runtime']=0o777 + else:bad['symlinks']['prose-agents-sdk-runtime/Python.framework/Python']='../../../outside' + archive=self.root/('poison-'+poison+'.zip') + signing.write_notarization_archive(archive,receipt['binaries'],self.output,bad) + modified=copy.deepcopy(receipt);modified['notarization']['sha256']=signing.sha256(archive) + with self.assertRaises(ValueError):signing.notarization_table(archive,modified) + + def test_sdk_support_mutation_during_notarization_issues_no_receipt(self): + helper,view,original=self.sdk_fixture() + def altered(args): + result=self.native(args) + if 'submit' in args:(self.output/'prose-agents-sdk-runtime/base_library.zip').write_bytes(b'poison') + return result + with self.assertRaisesRegex(ValueError,'support file'):self.sign(agents_sdk=helper,commands=altered) + self.assertFalse((self.output/'receipt.json').exists()) def test_non_macos_makes_no_native_calls(self): with patch.object(signing.platform, "system", return_value="Linux"): diff --git a/cli/conformance/runner/run.py b/cli/conformance/runner/run.py index a11ab1ac..699dbd25 100755 --- a/cli/conformance/runner/run.py +++ b/cli/conformance/runner/run.py @@ -105,6 +105,7 @@ class Product: execution_source_context: Path | None = None npm_context: dict[str, Any] | None = None execution_launch_path: Path | None = None + sdk_source_evidence: dict[str, Any] | None = None @property def expected_runner_name(self) -> str: @@ -412,6 +413,7 @@ def snapshot_product( interpreter, source_context, product.npm_context, + sdk_source_evidence=product.sdk_source_evidence, ) @@ -629,7 +631,7 @@ def make_report( differential_total = differential_passed + differential_failed total = candidate_total + differential_total failed = candidate_failed + differential_failed - return { + report = { "schema": "openprose.mechanical-conformance-report/1", "phase": phase, "caseIds": list(case_ids), @@ -662,6 +664,21 @@ def make_report( }, } + evidence = {} + references = {} + for product, _ in candidates: + if product.sdk_source_evidence is not None: + digest = product.sdk_source_evidence['agentsSdk']['receiptSha256'] + if digest in evidence and evidence[digest] != product.sdk_source_evidence: + raise ValueError('SDK source receipt has conflicting identities') + evidence[digest] = product.sdk_source_evidence + references[product.name] = digest + if evidence: + report['sdkSourceCustody'] = {'evidence': evidence, 'candidateReferences': references, + 'fixtureHelperSha256': sha256(INSTALLED_ADAPTER_HARNESS.read_bytes()), + 'helperSubstitution': 'provider-free-oracle', 'productionSdkExecuted': False} + return report + def _exact_keys(value: Any, expected: set[str], path: str) -> list[str]: if not isinstance(value, dict): @@ -801,7 +818,7 @@ def validate_report(report: Any) -> list[str]: "validations", "failures", "claims", - }, + } | ({"sdkSourceCustody"} if isinstance(report, dict) and "sdkSourceCustody" in report else set()), "$", ) if failures: @@ -991,6 +1008,39 @@ def validate_report(report: Any) -> list[str]: "$.failures must contain exactly one bounded record per failed validation" ) + if 'sdkSourceCustody' in report: + source = report['sdkSourceCustody'] + shape = _exact_keys(source, {'evidence', 'candidateReferences', 'fixtureHelperSha256', 'helperSubstitution', 'productionSdkExecuted'}, '$.sdkSourceCustody') + failures.extend(shape) + if not shape: + if source['helperSubstitution'] != 'provider-free-oracle' or source['productionSdkExecuted'] is not False or not isinstance(source['fixtureHelperSha256'], str) or not re.fullmatch('[0-9a-f]{64}', source['fixtureHelperSha256']): + failures.append('$.sdkSourceCustody overstates fixture execution') + evidence, references = source['evidence'], source['candidateReferences'] + if not isinstance(evidence, dict) or not evidence or len(evidence) > len(labels) or not isinstance(references, dict) or not references or not set(references) <= set(labels) or not all(isinstance(d, str) for d in references.values()) or set(references.values()) != set(evidence): + failures.append('$.sdkSourceCustody references are invalid') + else: + expected = {'path', 'byteLength', 'sha256', 'receiptSha256', 'noticesSha256', 'python', 'pyinstaller', 'version', 'discovery', 'selfTest', 'toolSelfTest', 'dependencyLockSha256'} + for digest, row in evidence.items(): + if _exact_keys(row, {'platform', 'agentsSdk', 'payloadSha256'}, '$.sdkSourceCustody evidence') or not isinstance(digest, str) or not re.fullmatch('[0-9a-f]{64}', digest): + failures.append('$.sdkSourceCustody evidence shape differs'); continue + sdk = row['agentsSdk'] + if not isinstance(sdk, dict) or set(sdk) != expected or sdk.get('receiptSha256') != digest: + failures.append('$.sdkSourceCustody receipt reference differs') + else: + ci = str(CLI / 'ci') + if ci not in sys.path: sys.path.insert(0, ci) + import kernel_rc_evidence as sdk_policy + if (sdk['path'] != 'prose-agents-sdk' or type(sdk['byteLength']) is not int or + not 0 < sdk['byteLength'] <= MAX_CANDIDATE_BYTES or + any(not isinstance(sdk[k], str) or not re.fullmatch('[0-9a-f]{64}', sdk[k]) + for k in ('sha256', 'receiptSha256', 'noticesSha256', 'dependencyLockSha256')) or + sdk['python'] != '3.10.20' or sdk['pyinstaller'] != '6.22.3' or sdk['version'] != '0.1.0' or + sdk['discovery'] != 'canonical-cli-sibling' or sdk['selfTest'] != sdk_policy.SDK_IMPORT_TEST or + sdk['toolSelfTest'] != sdk_policy.SDK_TOOL_TEST): + failures.append('$.sdkSourceCustody source SDK policy differs') + if not isinstance(row['platform'], str) or row['platform'] not in {'darwin-arm64', 'darwin-x64', 'linux-arm64-gnu', 'linux-x64-gnu'} or not isinstance(row['payloadSha256'], str) or (row['platform'].startswith('darwin-') and not re.fullmatch('[0-9a-f]{64}', row['payloadSha256'])) or (row['platform'].startswith('linux-') and row['payloadSha256'] != 'not-applicable'): + failures.append('$.sdkSourceCustody platform/payload differs') + claims = report["claims"] claim_failures = _exact_keys( claims, @@ -1695,6 +1745,60 @@ def attach_npm_contexts(products: list[Product], specifications: list[list[str]] return [replace(product,npm_context=contexts.get(product.name,product.npm_context)) for product in products] +def attach_sdk_sources(products, specifications): + """Authenticate an explicitly supplied production surface once, never execute it.""" + if not specifications: + return products + ci = str(CLI / 'ci') + if ci not in sys.path: + sys.path.insert(0, ci) + import publication as publication_custody + import kernel_rc_evidence as sdk_custody + import sdk_native_inventory as inventory + contexts = {} + names = {p.name for p in products} + for label, encoded in specifications: + if label not in names or label in contexts or len(encoded.encode()) > 2 * 1024 * 1024: + raise ValueError('Invalid or duplicate SDK source context') + context = json.loads(encoded, object_pairs_hook=publication_custody.object_pairs) + if not isinstance(context, dict) or set(context) != {'directory', 'platform', 'agentsSdk'}: + raise ValueError('Closed explicit SDK source context required') + if not isinstance(context['platform'], str) or context['platform'] not in publication_custody.PLATFORMS: + raise ValueError('Unsupported SDK source platform') + contexts[label] = context + result = [] + for product in products: + if product.name not in contexts: + result.append(product); continue + context = contexts[product.name] + if not isinstance(context['directory'], str): + raise ValueError('SDK source directory must be an absolute canonical directory') + directory = Path(context['directory']) + if not directory.is_absolute() or directory.resolve(strict=True) != directory or any(p.is_symlink() for p in (directory, *directory.parents)): + raise ValueError('SDK source directory must be an absolute canonical directory') + native = npm_context_paths(product.npm_context)['native'] if product.npm_context else product.executable.resolve(strict=True) + if directory / 'prose' != native: + raise ValueError('SDK source context differs from authenticated native candidate') + files = {} + for name in ('prose', 'prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + path = directory / name + limit = 2 * 1024 * 1024 if name == 'agents-sdk-build.json' else (8 * 1024 * 1024 if name == 'AGENTS-SDK-NOTICES.txt' else MAX_CANDIDATE_BYTES) + if path.is_symlink() or not path.is_file() or not 0 < path.stat().st_size <= limit: + raise ValueError('SDK source sibling is not regular and bounded') + files[name] = (path.read_bytes(), stat.S_IMODE(path.stat().st_mode)) + receipt = json.loads(files['agents-sdk-build.json'][0], object_pairs_hook=publication_custody.object_pairs) + table = {'files': {'source/' + n: v for n, v in files.items()}, 'directories': {}, 'symlinks': {}} + if context['platform'].startswith('darwin-'): + view = inventory.read_macos_payload(directory, receipt['payload'], receipt['architecture']) + for kind, values in view.items(): + table[kind].update({'source/' + n: v for n, v in values.items()}) + sdk_custody.validate_sdk_archive_table(context, table) + evidence = {'platform': context['platform'], 'agentsSdk': context['agentsSdk'], + 'payloadSha256': sha256(canonical_json(receipt['payload'])) if 'payload' in receipt else 'not-applicable'} + result.append(replace(product, sdk_source_evidence=evidence)) + return result + + def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: Path, environment_root: Path) -> tuple[Product, dict[str, Any], Path]: """Exercise real sibling discovery using an exact native candidate byte copy.""" @@ -1768,8 +1872,12 @@ def prepare_sdk_installation(product: Product, case: dict[str, Any], workspace: (workspace / '.sdk-compatibility-fixture.json').write_text(json.dumps(setup), encoding='utf-8') execution = Product(product.name, product.executable, product.runner_name, product.interpreter, (product.execution_executable or product.executable) if interpreted else target, product.execution_interpreter, - launcher_context, product.npm_context, target if interpreted else None) - fixture = {**fixture, 'nativePath': str(native), 'nativeSha256': sha256(source.read_bytes())} + launcher_context, product.npm_context, target if interpreted else None, product.sdk_source_evidence) + fixture = {**fixture, 'nativePath': str(native), 'nativeSha256': sha256(source.read_bytes()), + 'fixtureHelperSha256': sha256(INSTALLED_ADAPTER_HARNESS.read_bytes()), + 'helperSubstitution': 'provider-free-oracle', 'productionSdkExecuted': False} + if product.sdk_source_evidence is not None: + fixture['productionSdkSourceRef'] = product.sdk_source_evidence['agentsSdk']['receiptSha256'] if interpreted: fixture['npmContext']=product.npm_context fixture['npmCloneFiles']=[{'path':str(prefix/row['path']),'sha256':row['sha256'],'byteLength':row['byteLength']} for row in product.npm_context['files'].values()] @@ -1790,6 +1898,9 @@ def validate_sdk_effects(observation: Observation) -> list[str]: try: if sha256(Path(fixture['nativePath']).read_bytes()) != fixture['nativeSha256']: failures.append('SDK relocated candidate bytes changed during execution') + if (fixture.get('helperSubstitution') != 'provider-free-oracle' or fixture.get('productionSdkExecuted') is not False + or sha256((Path(fixture['nativePath']).parent / 'prose-agents-sdk').read_bytes()) != fixture['fixtureHelperSha256']): + failures.append('SDK fixture helper substitution bytes changed') if (workspace / '.sdk-wrong-helper-used').exists(): failures.append('SDK discovery used a foreign PATH helper') if fixture['noProbe'] and (workspace / '.sdk-harness-probed').exists(): @@ -2465,6 +2576,7 @@ def main(argv: list[str] | None = None) -> int: "the interpreter bytes are bound into the report" ), ) + parser.add_argument('--candidate-sdk-source',action='append',nargs=2,default=[],metavar=('LABEL','JSON'),help='Explicit production SDK source custody; fixtures deliberately substitute the provider-free oracle') parser.add_argument('--candidate-npm-context',action='append',nargs=2,default=[],metavar=('LABEL','JSON'),help='Explicit benchmark-bound npm native child closure for SDK installation fixtures') args = parser.parse_args(argv) if args.candidate and (args.rust is not None or args.bun is not None): @@ -2486,6 +2598,7 @@ def main(argv: list[str] | None = None) -> int: ) products = attach_candidate_interpreters(products, args.candidate_interpreter) products = attach_npm_contexts(products,args.candidate_npm_context) + products = attach_sdk_sources(products,args.candidate_sdk_source) except ValueError as error: parser.error(str(error)) verified_products: list[Product] = [] @@ -2502,6 +2615,7 @@ def main(argv: list[str] | None = None) -> int: else None ), npm_context=product.npm_context, + sdk_source_evidence=product.sdk_source_evidence, ) try: identity = capture_candidate_identity(verified) diff --git a/cli/conformance/runner/test_runner.py b/cli/conformance/runner/test_runner.py index 6ee0dad1..320826b0 100755 --- a/cli/conformance/runner/test_runner.py +++ b/cli/conformance/runner/test_runner.py @@ -1918,5 +1918,75 @@ def test_task_digest_is_shell_neutral(self) -> None: ) + +class SdkSourceCustodyTests(unittest.TestCase): + def source(self, root): + ci = str(runner.CLI / 'ci') + if ci not in sys.path: sys.path.insert(0, ci) + from test_kernel_rc_evidence import sdk_fixture + from test_sdk_native_inventory import onedir_fixture + import sdk_native_inventory as inventory + source = root / 'production'; source.mkdir() + sdk, table = sdk_fixture('darwin-x64'); view = onedir_fixture() + inventory.materialize_macos_payload(source, view['payload'], view['files'], view['directories'], view['symlinks'], view['architecture']) + for name in ('agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + data, mode = table['files'][name]; (source / name).write_bytes(data); (source / name).chmod(mode) + (source / 'prose').write_bytes(b'synthetic nonexecuted CLI'); (source / 'prose').chmod(0o755) + product = runner.Product('fixture', source / 'prose', 'bun') + context = {'directory': str(source), 'platform': 'darwin-x64', 'agentsSdk': sdk} + return product, context + + def test_source_authentication_is_once_separate_from_cli_only_oracle_substitution(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); product, context = self.source(root) + product = runner.attach_sdk_sources([product], [['fixture', json.dumps(context)]])[0] + identity = runner.capture_candidate_identity(product) + snapshot = runner.snapshot_product(product, identity, root / 'snapshot') + workspace = root / 'workspace'; workspace.mkdir(); environment = root / 'environment'; environment.mkdir() + case = json.loads((runner.CASES / 'adapters/sdk-production-04.json').read_text()) + execution, fixture, _ = runner.prepare_sdk_installation(snapshot, case, workspace, environment) + sibling = Path(fixture['nativePath']).parent + self.assertFalse((sibling / 'prose-agents-sdk-runtime').exists()) + self.assertEqual(fixture['productionSdkSourceRef'], context['agentsSdk']['receiptSha256']) + self.assertEqual(fixture['helperSubstitution'], 'provider-free-oracle') + self.assertIs(fixture['productionSdkExecuted'], False) + self.assertEqual((sibling / 'prose-agents-sdk').read_bytes(), runner.INSTALLED_ADAPTER_HARNESS.read_bytes()) + report = runner.make_report(phase=7, case_ids=[case['id']], candidates=[(execution, identity)], candidate_passed=1, candidate_failed=0, differential_passed=0, differential_failed=0, failures=[]) + self.assertEqual(len(report['sdkSourceCustody']['evidence']), 1) + self.assertEqual(runner.validate_report(report), []) + (sibling / 'prose-agents-sdk').write_bytes(b'poisoned oracle') + observation = runner.Observation(execution, case, 0, b'', b'', workspace=workspace, sdk_fixture=fixture) + self.assertIn('SDK fixture helper substitution bytes changed', runner.validate_sdk_effects(observation)) + + def test_source_mutation_and_foreign_native_directory_refused(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); product, context = self.source(root) + changed = dict(context, directory=str(root)) + with self.assertRaises(ValueError): runner.attach_sdk_sources([product], [['fixture', json.dumps(changed)]]) + (Path(context['directory']) / 'prose-agents-sdk-runtime/empty-data').write_bytes(b'changed') + with self.assertRaises(ValueError): runner.attach_sdk_sources([product], [['fixture', json.dumps(context)]]) + for poison in ({}, dict(context, platform=[]), dict(context, unknown='not-accepted')): + with self.assertRaises(ValueError): runner.attach_sdk_sources([product], [['fixture', json.dumps(poison)]]) + + def test_report_source_claims_and_receipt_references_fail_closed(self): + import copy + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); product, context = self.source(root) + product = runner.attach_sdk_sources([product], [['fixture', json.dumps(context)]])[0] + case = json.loads((runner.CASES / 'adapters/sdk-production-04.json').read_text()) + report = runner.make_report(phase=7, case_ids=[case['id']], candidates=[(product, runner.capture_candidate_identity(product))], candidate_passed=1, candidate_failed=0, differential_passed=0, differential_failed=0, failures=[]) + self.assertEqual(runner.validate_report(report), []) + for poison in ('claim', 'reference', 'shape', 'policy', 'bool-length', 'digest'): + changed = copy.deepcopy(report) + if poison == 'claim': changed['sdkSourceCustody']['productionSdkExecuted'] = True + elif poison == 'reference': changed['sdkSourceCustody']['candidateReferences']['fixture'] = [] + elif poison == 'shape': next(iter(changed['sdkSourceCustody']['evidence'].values()))['unknown'] = True + else: + sdk = next(iter(changed['sdkSourceCustody']['evidence'].values()))['agentsSdk'] + if poison == 'policy': sdk['discovery'] = 'PATH' + elif poison == 'bool-length': sdk['byteLength'] = True + else: sdk['dependencyLockSha256'] = 'not-a-digest' + with self.subTest(poison=poison): self.assertTrue(runner.validate_report(changed)) + if __name__ == "__main__": unittest.main(verbosity=2) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 50a5a225..afc4e653 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1109,3 +1109,22 @@ IMP-098 strict RPM pathname/provenance repair: `/root/sdk_assessment` owns only ## IMP-098 pinned Linux archive transport repair `/root/sdk_assessment` owns only `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` to repair the actual fourth-wave preparation failure: pinned freezer tar cannot execute zstd. Implement the independently reviewed scratch `linux-python-plain-tar-transport-proposal.md` (SHA256 `50905494d16da978a959403db6f252ccad33e7584f0db4a4a90d885b86d0b28d`): explicit host decoder prerequisite and identity, bounded binary stdout to fresh retained plain tar, closed private transport descriptor, compressed and derived byte custody before container extraction, and a shared existing 1,200-second preparation deadline. Preserve all image/archive/SDK locks, native schemas, twelve manifest identities, aggregate quotas, source guards, supplier `/2` validation, runtime floor and original evidence. No automatic tool installation, fallback, limit relaxation, mutable image or host library provisioning. Meaningful synthetic process/receiver tests only; no Git, Docker, real decompression, heavy builds, helper/provider or credential access by the agent. Root owns integration, mandatory fresh full admission and actual native replay. Current e505 Mac and full17 receipts retain their exact source scope. Homebrew qualification completed terminal zero with clean e505 source before this lease. + + +## IMP-098 Mac directory-payload cold-start wave + +Root authorizes the independently accepted `sdk-onedir-joint-custody-contract-1.md` (SHA256 `12173376cd02cd7f6548bbdc697dc1156010da5ca3860ac097150316b51d60ed`) and matching consumer map2 (`f85295e46052bf7ec92e9f51b056b069bd249180945426e7ac2e9a06aa60b3ae`). Both Mac targets use standard PyInstaller onedir with complete support-tree custody; Linux retains the existing onefile/native closure. Preserve actual five-second helper probes, all SDK/model/dependency/image pins, twelve manifest keys, and generic128/no-link policies. The explicit SDK-only8192-entry allowance is conjunctive with existing byte/receipt/publication/installation limits. Complete physical member tables, COLLECT-declared confined aliases, exact modes, final signed-byte inventories, actual Mach-O architecture/signature closure and source bindings must survive all package/install/relocation consumers. This design removes onefile extraction structurally; actual timeout cause and native improvement remain unproven. + +`/root/configuration_contract` owns `cli/ci/build_agents_sdk.py`, `cli/ci/test_build_agents_sdk.py`, `cli/ci/sdk_native_inventory.py`, and `cli/ci/test_sdk_native_inventory.py` for builder, retained COLLECT evidence and shared standard-library-only payload validation/copy/materialization API. Publish exact helper signatures and a concrete synthetic fixture before consumers depend on them. Preserve Linux supplier/ELF behavior. + +`/root/sdk_assessment` owns `cli/ci/package_local.py`, `cli/ci/test_package_local.py`, `cli/ci/kernel_rc_evidence.py`, `cli/ci/test_kernel_rc_evidence.py`, `cli/ci/publication.py`, `cli/ci/test_publication.py`, `cli/ci/sign_macos.py`, and `cli/ci/test_sign_macos.py` for complete typed SDK archive tables, package snapshots, shared central custody, signature/notarization transport and strict scoped decoders. Publish exact archive/table API before installed consumers depend on it. Preserve unrelated archives and historical evidence through explicit exact source/artifact replay, never missing-field fallback. + +`/root/operational_design` owns `cli/ci/build_kernel_rc.py`, `cli/ci/test_build_kernel_rc.py`, `cli/ci/qualify_installed_sdk.py`, `cli/ci/test_qualify_installed_sdk.py`, `cli/benchmarks/installed/benchmark.py`, and `cli/benchmarks/installed/test_benchmark.py` for complete validated extraction, fresh/relocated/genuine upgrade/npm/npx payload records and installed/rehearsal directory-alias custody. Consume frozen central APIs; no byte-only projections before complete validation. + +Root owns Git/index, integration, `cli/ci/assemble_kernel_rc.py` and its tests, required docs/workflow/admission adjustments, actual builds/signature/helper qualification, fresh mandatory full admission and native replay. Agents run only focused pure/synthetic controls; no Git, real builder/helper/qualification scripts, Cargo/npm/native/Docker/heavy builds, provider calls or credential access. Root's preservation clone is active on old completed artifacts: do not open its selected/protected inputs or launch role-named builder/qualifier subprocesses while it runs. Source paths are excluded from that frozen clone plan. Retain every historical artifact/report and failed-run identity; new-source evidence must be rebuilt/rebound. Full18 was not launched on prior0f0bdf7 and its scripts remain exact historical proposals until new source is frozen. + + +Mac payload integration lease extension: `/root/operational_design` additionally owns `cli/conformance/runner/run.py` and `cli/conformance/runner/test_runner.py` only for complete validated SDK installation copies across the seventeen existing production cases. Preserve opaque argv, fixtures, case counts, timeout and error comparisons. Root additionally owns `cli/ci/homebrew_rehearsal.py`, `cli/ci/test_homebrew_rehearsal.py`, `cli/README.md`, `cli/ci/README.md`, `docs/packaged-sdk-design.md`, `docs/cli-distribution.md`, and `docs/cli-publication.md` for conventional complete directory installation/qualification, unchanged link-conflict refusal and truthful user/developer instructions. Existing wave restrictions and exact source/native proof requirements remain. + + +Mac SDK conformance custody clarification and caller lease: the seventeen adapter-oracle cases substitute a provider-free fixture helper; they do not execute the production SDK or require copying its unused support tree. Authenticate the original production source once and retain source references separately from explicit fixture substitution. `/root/operational_design` additionally owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` for the exact `--candidate-sdk-source LABEL JSON` caller integration, benchmark receipt-reference custody and matching conformance report/source validation. Preserve all existing case counts, full admission, deadlines, byte bounds and historical validation. No native/helper/heavy/Git/provider execution by agents while preservation remains active. diff --git a/docs/cli-distribution.md b/docs/cli-distribution.md index 1264f13e..233e379a 100644 --- a/docs/cli-distribution.md +++ b/docs/cli-distribution.md @@ -64,6 +64,25 @@ respecting the organization policy against Actions-created PRs. Merging that checked PR selects the new Homebrew release. This avoids a separate cross-repository publishing token and preserves a reviewable update boundary. +## Packaged Agents SDK candidate + +The IMP-098 candidate supplies the Agents SDK helper with each native CLI package. +Its built-in selection is `agents-sdk`, `gpt-6.1-sol` and `openai-api-key`; +a supported first run requires an `OPENAI_API_KEY`. Saved alternatives and explicit +command overrides remain available. Use `prose cli config explain --json -- run +input.prose.md` to inspect the selected settings and their sources. + +On macOS, the helper requires its sibling `prose-agents-sdk-runtime` directory. +Standalone relocation must move the entire extracted package together. npm and +Homebrew install the helper and complete support directory with the selected CLI. +Linux retains the packaged single-file helper. End users do not provision Python +or a separate SDK executable. Missing prerequisites fail explicitly. + +These are candidate behaviors. Complete source admission, native installation and +upgrade checks on all supported platforms, and live fulfillment evidence must +qualify the final source before release. Existing public releases retain their +historical behavior and qualification. + ## Historical implementation record — September 16–17, 2026 The sections below retain earlier work, failures and decisions. Statements about diff --git a/docs/cli-publication.md b/docs/cli-publication.md index 300739f7..a3bbe53b 100644 --- a/docs/cli-publication.md +++ b/docs/cli-publication.md @@ -37,6 +37,23 @@ they are not prerequisites for the new release. Stable macOS signing remains separate. npm currently distributes Bun; Rust standalone downloads remain independently selectable. +## Packaged SDK custody + +Candidate SDK archives bind the helper, build receipt, notices and, on macOS, +the complete `prose-agents-sdk-runtime` support tree. Dedicated SDK admission +checks physical file hashes and modes, directory membership and bounded framework +aliases against the exact receipt. Generic archive admission retains its existing +member limit and refusal of links. Copying only the Mac helper is insufficient. + +The Mac builder records COLLECT membership and verifies signatures and architecture +for every actual Mach-O support file before recording final bytes. Signing and +notarization consumers bind the same complete payload. Unsigned candidate checks +establish their measured ad hoc signature behavior; they do not grant Developer ID +trust, notarization or publication authority. Linux keeps its native supplier and +ELF closure evidence. See [the SDK design](packaged-sdk-design.md) for the source, +member and byte bounds and [the release guide](../cli/release/README.md) for final +qualification and publication gates. + ## npm trusted publisher Configure the existing `@openprose/prose-cli` package with this exact identity: diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 5ed8c46a..b90bd29d 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -9,11 +9,15 @@ IMP-098, checked October 6, 2026; applies IMP-099. The user selects `~/.prose/cl - [OpenCode installation](https://opencode.ai/docs/): standalone, npm and Homebrew installation. - [Prime source](https://raw.githubusercontent.com/PrimeIntellect-ai/prime-agent/main/README.md): versioned installer and integrated Python tooling; a different runtime design. -The peers support package-owned platform runtimes and installation-independent preferences, without a universal Python layout. Retain the CLI's existing four native platform payloads. Freeze the existing Python SDK harness into a sibling `prose-agents-sdk` executable using pinned PyInstaller onefile; no user Python, source, provisioning or PATH changes. Archives contain sibling regular executables; npm uses `bin/`; Homebrew installs both to `bin`. Resolve the helper beside the canonical installed CLI executable. Explicit existing test/developer seams remain separate. +The peers support package-owned platform runtimes and installation-independent preferences, without a universal Python layout. Retain the CLI's existing four native platform payloads. Freeze the existing Python SDK harness with pinned PyInstaller. Both Mac targets use standard onedir: the regular sibling `prose-agents-sdk` executable and its complete `prose-agents-sdk-runtime/` support directory. Linux retains onefile and its existing native dependency closure. No user Python, provisioning or PATH helper is required. npm uses `bin/`; Homebrew installs the helper and Mac support directory together under `bin`. Resolve the helper beside the canonical installed CLI executable. Explicit existing test/developer seams remain separate. ## Choice and costs -[PyInstaller usage](https://pyinstaller.org/en/stable/usage.html) documents onefile extraction. [POSIX bundle guidance](https://pyinstaller.org/en/stable/common-issues-and-pitfalls.html) documents onedir symlinks and sanitizing dynamic-library variables for child tools. Onefile preserves the existing regular-file archive safety boundary; onedir or bundled interpreter trees would require broader packaging changes. Cost: larger payload/cold start, native build on every supported platform, temporary executable filesystem requirements, and possible leftover extraction directories after forced death. Do not claim forced-death temporary cleanup. Verify frozen-helper supervision including shell descendants. +[Pinned PyInstaller usage](https://pyinstaller.org/en/v6.22.3/usage.html) and [POSIX bundle guidance](https://pyinstaller.org/en/v6.22.3/common-issues-and-pitfalls.html) describe directory bundles, declared aliases and onefile extraction before Python entry. A retained Mac Intel npx installation timed out in the unchanged five-second version probe despite the fast Python version entry. That observation does not identify which startup phase consumed the time. Mac onedir removes unconditional extraction while preserving execution of the real frozen helper; fresh native evidence must demonstrate cold discovery. Linux onefile retains its temporary executable filesystem requirement and possible leftover extraction directories after forced death. Do not claim forced-death temporary cleanup. Verify frozen-helper supervision including shell descendants. + +Mac packaging binds every physical support file, directory and declared relative alias in the SDK receipt. Validate the whole confined graph before copying or extraction, write regular files before aliases, and recheck complete bytes, modes and targets after relocation and helper tests. Verify every actual Mach-O target and signature after final signing. COLLECT declarations establish producer membership and alias evidence; the final inventory binds signed bytes. The twelve distribution identity fields remain unchanged because the receipt digest binds this complete inventory. + +The SDK support inventory has an explicit 8,192-entry envelope. Generic archives retain their 128-member, no-link policy. The SDK allowance applies only to an authenticated SDK context with exact declared membership, and remains subject to 256 MiB per file, 512 MiB total support and whole publication payload, 2 MiB encoded receipt and 768 MiB expanded installed payload limits. These are simultaneous bounds; they do not establish that a candidate fits. The prior Mac onefile archive already contained 2,532 support declarations, making the generic member count unsuitable for the directory bundle. Build CPython 3.10.20, existing SDK0.22.2/OpenAI3.13.0, and a hash-locked transitive dependency graph with source identities, SBOM/notices and helper hashes in package evidence. Qualified release targets remain macOS/Linux ARM64/x64, with no expansion to unadmitted Windows execution. Verify actual packaged imports, HTTPS certificates, cold discovery, cancellation and tools on native target builders. Preserve existing glibc/platform admission policy. @@ -37,8 +41,8 @@ flag may imply enforcement the helper does not implement. Resolve `prose-agents-sdk` beside the canonical CLI executable, following the CLI's symlink to the installed payload before selecting its sibling. The helper -must be a regular executable; reject a sibling helper symlink. Relocating -both regular executables together preserves discovery. Never search PATH as a +must be a regular executable; reject a sibling helper symlink. Relocation keeps +the CLI, helper and complete Mac support directory together. Never search PATH as a production fallback or accidentally pair a CLI with another installation's helper. Existing explicit developer/test seams remain separate from production. Repair a missing/incompatible sibling by reinstalling the CLI through its current From 9a6505d48ad8cda2623585918e0ef5b692334ce0 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 13:48:01 -0400 Subject: [PATCH 54/64] Preserve static Windows admission and complete Mac npm custody --- cli/benchmarks/installed/benchmark.py | 6 +- cli/benchmarks/installed/test_benchmark.py | 70 ++++++++++++++++++++++ cli/ci/release_package_admission.py | 36 ++++++----- cli/ci/test_release_package_admission.py | 11 ++-- cli/protocol/OWNERSHIP.md | 7 +++ docs/packaged-sdk-design.md | 2 + 6 files changed, 112 insertions(+), 20 deletions(-) diff --git a/cli/benchmarks/installed/benchmark.py b/cli/benchmarks/installed/benchmark.py index 468b10d2..cf60d353 100644 --- a/cli/benchmarks/installed/benchmark.py +++ b/cli/benchmarks/installed/benchmark.py @@ -1243,6 +1243,8 @@ def verify_package_output( platform_value, purpose, ) + if platform_value.startswith('win32-') and 'agentsSdk' in release: + fail('PLATFORM_UNSUPPORTED', 'Windows static admission cannot contain an SDK identity') evidence_names = set(encoded) - set(artifacts) if evidence_names != REQUIRED_EVIDENCE: fail( @@ -1288,7 +1290,7 @@ def verify_package_output( artifacts, dependency_digest, ) - if release['mode'] in ('release', 'kernel-rc'): + if release['mode'] in ('release', 'kernel-rc') and not platform_value.startswith('win32-'): import sys ci_path = str(Path(__file__).resolve().parents[2] / 'ci') if ci_path not in sys.path: @@ -1481,6 +1483,8 @@ def decode_archive_members( archive_bytes: bytes, label: str, *, manifest=None ) -> dict[str, tuple[bytes, int]]: if manifest is not None and isinstance(manifest.get('agentsSdk'), dict): + if str(manifest.get('platform', '')).startswith('win32-'): + fail('PLATFORM_UNSUPPORTED', 'Windows static admission cannot contain an SDK identity') pub, custody, _ = sdk_modules() try: table = pub.decode_sdk_archive(archive_bytes, manifest, label=label) diff --git a/cli/benchmarks/installed/test_benchmark.py b/cli/benchmarks/installed/test_benchmark.py index 43ad4957..529214cf 100644 --- a/cli/benchmarks/installed/test_benchmark.py +++ b/cli/benchmarks/installed/test_benchmark.py @@ -719,12 +719,29 @@ def release_platform_manifest(members): members[index] = (name, canonical(package), mode) rewrite_tar(platform_package, release_platform_manifest) + if platform_value.startswith('win32-'): + # Windows remains a static package/sidecar authority fixture; there is + # no supported packaged SDK target or native SDK execution claim. + manifest_path.write_text(json.dumps(manifest, sort_keys=True), 'utf-8') + refresh_evidence(output) + return output import sys ci_path = str(ROOT / 'cli/ci') if ci_path not in sys.path: sys.path.insert(0, ci_path) from test_kernel_rc_evidence import sdk_fixture sdk, sdk_table = sdk_fixture(platform_value) + if platform_value.startswith('darwin-'): + # npm's pinned pacote extractor intentionally drops Link entries. + # Its install fixture must describe the complete physical tree it ships; + # separate custody tests retain the alias graph and missing-alias poisons. + sdk_table['symlinks'] = {} + receipt = json.loads(sdk_table['files']['agents-sdk-build.json'][0]) + receipt['payload']['entries'] = [row for row in receipt['payload']['entries'] + if row['type'] != 'symlink'] + encoded = json.dumps(receipt, sort_keys=True).encode() + sdk_table['files']['agents-sdk-build.json'] = (encoded, 0o644) + sdk['receiptSha256'] = digest(encoded) sdk_members = [(n, d, m) for n, (d, m) in sdk_table["files"].items()] sdk_members += [(n, b"", ("directory", m)) for n, m in sdk_table["directories"].items()] sdk_members += [(n, b"", target) for n, target in sdk_table["symlinks"].items()] @@ -2065,6 +2082,36 @@ def test_artifact_native_sbom_poison_is_rejected_after_rehash(self): +class WindowsStaticSdkBoundaryTests(unittest.TestCase): + def test_windows_release_preserves_static_sidecar_checks_without_sdk(self): + with tempfile.TemporaryDirectory() as raw: + packages = make_release_package_output(Path(raw), platform_value='win32-x64') + with mock.patch.object(BENCHMARK, 'sdk_modules', side_effect=AssertionError('No POSIX SDK decoder')): + context = BENCHMARK.verify_package_output(packages, expected_platform='win32-x64', purpose='release-invariants') + payloads = BENCHMARK.validate_package_payloads(context) + self.assertNotIn('agentsSdk', context['release']) + self.assertNotIn('agentsSdkPython', json.loads(context['encoded']['dependency-evidence.json'])['inventories']) + self.assertIn('windowsProcessHost', context['release']) + self.assertTrue(payloads) + + def test_rehashed_windows_sdk_identity_is_rejected_before_posix_decoder(self): + with tempfile.TemporaryDirectory() as raw: + packages = make_release_package_output(Path(raw), platform_value='win32-x64') + path = packages / 'release-manifest.json' + release = json.loads(path.read_bytes()); release['agentsSdk'] = {'unsupported': True} + path.write_bytes(canonical(release)) + sums = ''.join(f'{digest(p.read_bytes())} {p.name}\n' for p in sorted(packages.iterdir()) if p.name != 'SHA256SUMS') + (packages / 'SHA256SUMS').write_text(sums) + with mock.patch.object(BENCHMARK, 'sdk_modules', side_effect=AssertionError('No POSIX SDK decoder')): + with self.assertRaises(BENCHMARK.BenchmarkError) as rejected: + BENCHMARK.verify_package_output(packages, expected_platform='win32-x64', purpose='release-invariants') + self.assertEqual(rejected.exception.code, 'PLATFORM_UNSUPPORTED') + self.assertEqual(rejected.exception.message, 'Windows static admission cannot contain an SDK identity') + with self.assertRaises(BENCHMARK.BenchmarkError) as decoded: + BENCHMARK.decode_archive_members(b'not executed', 'Windows SDK poison', manifest=release) + self.assertEqual(decoded.exception.code, 'PLATFORM_UNSUPPORTED') + + class CompleteSdkInstalledTreeTests(unittest.TestCase): def fixture(self, root): BENCHMARK.sdk_modules() @@ -2100,6 +2147,29 @@ def test_generic_directory_alias_stays_forbidden(self): with self.assertRaises(BENCHMARK.BenchmarkError): BENCHMARK.capture_installed_tree(root, {link: target}) + def test_missing_physical_sdk_alias_is_not_repaired_or_ignored(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve(); sibling, context, _ = self.fixture(root) + alias = sibling / 'prose-agents-sdk-runtime/Python.framework/Python' + alias.unlink() + with self.assertRaisesRegex(ValueError, 'complete typed member inventory differs'): + BENCHMARK.capture_installed_tree(root, sdk_context=context) + + def test_npm_release_fixture_has_a_complete_alias_free_mac_payload(self): + with tempfile.TemporaryDirectory() as raw: + root = Path(raw).resolve() + packages = make_release_package_output(root, platform_value='darwin-x64') + context = BENCHMARK.verify_package_output(packages, expected_platform='darwin-x64', purpose='release-invariants') + artifact = next(row for row in context['release']['artifacts'] if row['kind'] == 'npm-platform') + members = BENCHMARK.decode_archive_members(context['encoded'][artifact['path']], artifact['path'], manifest=context['release']) + self.assertEqual(members.table['symlinks'], {}) + sibling = root / 'installed'; sibling.mkdir() + _, custody, _ = BENCHMARK.sdk_modules() + custody.materialize_sdk_members(context['release'], members.table, sibling) + installed_context = BENCHMARK.sdk_tree_context(members, 'installed') + identity = BENCHMARK.capture_installed_tree(root, sdk_context=installed_context) + self.assertNotIn('sdk-directory-symlink', [row['type'] for row in identity['entries']]) + def test_payload_poison_with_rehashed_report_identity_is_refused(self): import copy with tempfile.TemporaryDirectory() as raw: diff --git a/cli/ci/release_package_admission.py b/cli/ci/release_package_admission.py index 831f2236..f332f7d5 100644 --- a/cli/ci/release_package_admission.py +++ b/cli/ci/release_package_admission.py @@ -258,10 +258,11 @@ def validate_identity_inputs( "native-manifest.json", f"prose-rust{suffix}", f"prose-bun{suffix}", - "prose-agents-sdk", "agents-sdk-build.json", "AGENTS-SDK-NOTICES.txt", } if target_id == "win-x64": expected_files.add("openprose-windows-process-host.exe") + else: + expected_files.update({"prose-agents-sdk", "agents-sdk-build.json", "AGENTS-SDK-NOTICES.txt"}) if set(files) != expected_files: fail("NATIVE_LINEAGE_MISMATCH", "native artifact file membership differs") for name, raw in files.items(): @@ -759,18 +760,23 @@ def native_lineage( "nativeSha256": files[path]["sha256"], "packagedBinarySha256": packaged, } - sdk = payloads['release'].get('agentsSdk') - if not isinstance(sdk, dict): - fail('NATIVE_LINEAGE_MISMATCH', 'Release lacks packaged SDK identity') - npm = payloads['npmPackage'] - encoded = payloads['context']['encoded'][npm['platformName']] - import io, tarfile - with tarfile.open(fileobj=io.BytesIO(encoded), mode='r:gz') as archive: - for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): - member = archive.getmember('package/bin/' + name) - data = archive.extractfile(member).read() - if files.get(name) != {'byteLength': len(data), 'sha256': sha256(data)}: - fail('NATIVE_LINEAGE_MISMATCH', 'Packaged SDK differs from native artifact: ' + name) + if target_id == "win-x64": + if 'agentsSdk' in payloads['release']: + fail('NATIVE_LINEAGE_MISMATCH', 'Windows static authority cannot claim packaged SDK support') + sdk = 'not-applicable' + else: + sdk = payloads['release'].get('agentsSdk') + if not isinstance(sdk, dict): + fail('NATIVE_LINEAGE_MISMATCH', 'Release lacks packaged SDK identity') + npm = payloads['npmPackage'] + encoded = payloads['context']['encoded'][npm['platformName']] + import io, tarfile + with tarfile.open(fileobj=io.BytesIO(encoded), mode='r:gz') as archive: + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + member = archive.getmember('package/bin/' + name) + data = archive.extractfile(member).read() + if files.get(name) != {'byteLength': len(data), 'sha256': sha256(data)}: + fail('NATIVE_LINEAGE_MISMATCH', 'Packaged SDK differs from native artifact: ' + name) windows = native["windowsProcessHost"] if target_id == "win-x64": if windows != payloads["release"]["windowsProcessHost"]: @@ -1003,7 +1009,8 @@ def run_admission( else {} ) benchmark.verify_installed_tree( - installed["npmPrefix"], record["treeIdentity"], links + installed["npmPrefix"], record["treeIdentity"], links, + sdk_evidence=installed.get("sdkEvidence", {}), ) else: benchmark.verify_installed_tree( @@ -1011,6 +1018,7 @@ def run_admission( "destination" ], record["treeIdentity"], + sdk_evidence=installed.get("sdkEvidence", {}), ) benchmark.assert_package_output_unchanged(source_context) benchmark.assert_package_output_unchanged(snapshot_context) diff --git a/cli/ci/test_release_package_admission.py b/cli/ci/test_release_package_admission.py index a4e77398..e42e775a 100644 --- a/cli/ci/test_release_package_admission.py +++ b/cli/ci/test_release_package_admission.py @@ -241,11 +241,12 @@ def authority_inputs( "sha256": payloads["extracted"]["bun"]["binarySha256"], }, } - sdk_archive = packages / f'openprose-prose-cli-{release["platform"]}-{release["version"]}.tgz' - with tarfile.open(sdk_archive) as archive: - for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): - data = archive.extractfile('package/bin/' + name).read() - files[name] = {'byteLength': len(data), 'sha256': hashlib.sha256(data).hexdigest()} + if 'agentsSdk' in release: + sdk_archive = packages / f'openprose-prose-cli-{release["platform"]}-{release["version"]}.tgz' + with tarfile.open(sdk_archive) as archive: + for name in ('prose-agents-sdk', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + data = archive.extractfile('package/bin/' + name).read() + files[name] = {'byteLength': len(data), 'sha256': hashlib.sha256(data).hexdigest()} windows = None if target_id == "win-x64": windows = release["windowsProcessHost"] diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index afc4e653..ff4b9eda 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1128,3 +1128,10 @@ Mac payload integration lease extension: `/root/operational_design` additionally Mac SDK conformance custody clarification and caller lease: the seventeen adapter-oracle cases substitute a provider-free fixture helper; they do not execute the production SDK or require copying its unused support tree. Authenticate the original production source once and retain source references separately from explicit fixture substitution. `/root/operational_design` additionally owns `cli/ci/rehearse_release.py` and `cli/ci/test_rehearse_release.py` for the exact `--candidate-sdk-source LABEL JSON` caller integration, benchmark receipt-reference custody and matching conformance report/source validation. Preserve all existing case counts, full admission, deadlines, byte bounds and historical validation. No native/helper/heavy/Git/provider execution by agents while preservation remains active. + + +## IMP-098 full19 installed consumer repair + +Full19 failed release-package-admission after49 passing gates: npm's documented extraction drops archive aliases, and the static Windows fixture incorrectly enters the POSIX-only SDK decoder. `/root/operational_design` retains its exact `cli/benchmarks/installed/benchmark.py` and `cli/benchmarks/installed/test_benchmark.py` lease and additionally owns `cli/ci/test_release_package_admission.py` only for truthful SDK-absent Windows authority fixtures. Preserve production Mac complete-member/alias rejection, mandatory SDK custody for four supported POSIX targets, Windows ordinary artifact/sidecar/provenance integrity and blocked-before-native-execution authority. Reject unexpected Windows SDK claims; do not broaden SDK supported targets or weaken package/archive bounds. Positive npm fixtures must represent an actual complete linkless tree; independent missing-alias rejection remains required. Actual Intel directory inventory and npm compatibility remain native qualification gates. Agents run only pure/synthetic controls; root owns actual npm/native replay, full admission, source integration, push and merge. Prior full19 and prototype evidence retain their exact source and failure scope. + +Root additionally owns only `cli/ci/release_package_admission.py` for the same full19 repair: closed native artifact membership and SDK lineage must distinguish SDK-absent Windows static authority from mandatory four-platform POSIX SDK custody, preserving Windows sidecar equality and refusal before execution. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index b90bd29d..020828ac 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -17,6 +17,8 @@ The peers support package-owned platform runtimes and installation-independent p Mac packaging binds every physical support file, directory and declared relative alias in the SDK receipt. Validate the whole confined graph before copying or extraction, write regular files before aliases, and recheck complete bytes, modes and targets after relocation and helper tests. Verify every actual Mach-O target and signature after final signing. COLLECT declarations establish producer membership and alias evidence; the final inventory binds signed bytes. The twelve distribution identity fields remain unchanged because the receipt digest binds this complete inventory. +[npm's pinned pacote extractor](https://github.com/npm/pacote/blob/v21.5.1/lib/fetcher.js#L427-L430) drops archive links. Positive npm fixtures therefore use a complete linkless payload; missing required aliases remain an admission failure. Each actual Mac build must pass complete inventory checks after npm installation before that target is qualified. The linkless ARM prototype does not establish Intel compatibility. + The SDK support inventory has an explicit 8,192-entry envelope. Generic archives retain their 128-member, no-link policy. The SDK allowance applies only to an authenticated SDK context with exact declared membership, and remains subject to 256 MiB per file, 512 MiB total support and whole publication payload, 2 MiB encoded receipt and 768 MiB expanded installed payload limits. These are simultaneous bounds; they do not establish that a candidate fits. The prior Mac onefile archive already contained 2,532 support declarations, making the generic member count unsuitable for the directory bundle. Build CPython 3.10.20, existing SDK0.22.2/OpenAI3.13.0, and a hash-locked transitive dependency graph with source identities, SBOM/notices and helper hashes in package evidence. Qualified release targets remain macOS/Linux ARM64/x64, with no expansion to unadmitted Windows execution. Verify actual packaged imports, HTTPS certificates, cold discovery, cancellation and tools on native target builders. Preserve existing glibc/platform admission policy. From 046eef455ae05a77856af5e92a4e2ce3e1aecbd7 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 14:12:11 -0400 Subject: [PATCH 55/64] Distinguish explicit development SDK rehearsal fixtures --- cli/ci/rehearse_release.py | 35 +++++++++++++++---- cli/ci/test_rehearse_release.py | 59 ++++++++++++++++++++++++++++++++- 2 files changed, 86 insertions(+), 8 deletions(-) diff --git a/cli/ci/rehearse_release.py b/cli/ci/rehearse_release.py index 92db11de..05294cc7 100644 --- a/cli/ci/rehearse_release.py +++ b/cli/ci/rehearse_release.py @@ -802,13 +802,34 @@ def _npm_conformance_context(install: Path, raw_report: Mapping[str, Any]) -> di return {'prefix':str(prefix.resolve()),'platform':platform,'files':files} -def _sdk_conformance_contexts(install, raw_report, release, module): - """Bind real SDK trees before deliberately substituting the fixture oracle.""" - if not isinstance(release, dict) or 'agentsSdk' not in release: +def _sdk_release_identity(raw_report, release): + """Distinguish the explicit non-publishable development fixture from SDK custody.""" + release = require_object(release, 'explicit SDK release context') + if 'agentsSdk' not in release: + if release.get('mode') in ('release', 'kernel-rc'): + raise RehearsalError('Production release lacks an SDK identity') + if (release.get('mode') not in ('development', 'alpha') + or release.get('releaseEligible') is not False + or release.get('publicationAuthorized') is not False): + raise RehearsalError('SDK-absent context requires an explicit non-publishable historical or development package') if 'sdkEvidence' in raw_report: raise RehearsalError('SDK installation evidence lacks an explicit release identity') + return None + if release['agentsSdk'] == 'not-packaged-development-fixture': + if (release.get('mode') != 'development' + or release.get('releaseEligible') is not False + or release.get('publicationAuthorized') is not False + or 'sdkEvidence' in raw_report): + raise RehearsalError('SDK placeholder requires an explicit non-publishable development fixture') + return None + return require_object(release['agentsSdk'], 'release SDK identity') + + +def _sdk_conformance_contexts(install, raw_report, release, module): + """Bind real SDK trees before deliberately substituting the fixture oracle.""" + sdk = _sdk_release_identity(raw_report, release) + if sdk is None: return {} - sdk = require_object(release['agentsSdk'], 'release SDK identity') import qualify_installed_sdk as installed_sdk import kernel_rc_evidence as sdk_custody import sdk_native_inventory as inventory @@ -847,9 +868,9 @@ def _sdk_conformance_contexts(install, raw_report, release, module): def _sdk_report_binding(raw_report, release): - if not isinstance(release, dict) or 'agentsSdk' not in release: + sdk = _sdk_release_identity(raw_report, release) + if sdk is None: return None - sdk = require_object(release['agentsSdk'], 'release SDK identity') receipt_sha = require_sha(sdk.get('receiptSha256'), 'SDK receipt') payload_sha = 'not-applicable' if release['platform'].startswith('darwin-'): @@ -869,7 +890,7 @@ def _validate_sdk_conformance_binding(value, raw_report, release): expected_sdk = _sdk_report_binding(raw_report, release) observed_sdk = value.get('sdkSourceCustody') if expected_sdk is None: - if observed_sdk is not None or 'sdkEvidence' in raw_report: + if 'sdkSourceCustody' in value or 'sdkEvidence' in raw_report: raise RehearsalError('SDK conformance report lacks explicit release context') elif not isinstance(observed_sdk, dict) or any(observed_sdk.get(k) != v for k, v in expected_sdk.items()): raise RehearsalError('SDK conformance source differs from authenticated release') diff --git a/cli/ci/test_rehearse_release.py b/cli/ci/test_rehearse_release.py index c65f1677..701f2d6b 100644 --- a/cli/ci/test_rehearse_release.py +++ b/cli/ci/test_rehearse_release.py @@ -335,6 +335,9 @@ def verify_package_output(self, package, expected_platform=None): return { "platform": expected_platform, "release": { + "mode": "development", + "releaseEligible": False, + "publicationAuthorized": False, "version": self.raw["packageIdentity"]["version"], "source": {"revision": self.raw["packageIdentity"]["sourceRevision"]}, }, @@ -499,7 +502,8 @@ def failing_executor(argv,cwd,environment): (Path(context['prefix'])/context['files']['native']['path']).write_bytes(b'tampered') return subprocess.CompletedProcess(argv,1,b'',b'failed fixture') with self.assertRaisesRegex(rehearse_release.RehearsalError,'benchmark custody'): - rehearse_release._run_mechanical_conformance(runner,observed,install,root/'report.json',{},time.monotonic()+10,failing_executor) + rehearse_release._run_mechanical_conformance(runner,observed,install,root/'report.json',{},time.monotonic()+10,failing_executor, + release={'mode': 'development', 'releaseEligible': False, 'publicationAuthorized': False}) def test_conformance_inputs_require_a_versioned_exact_node_identity(self) -> None: _build, raw, analysis = reports() @@ -1141,6 +1145,59 @@ def test_bounds_are_closed(self) -> None: class CompleteSdkConformanceCustodyTests(unittest.TestCase): + def test_exact_development_placeholder_has_no_sdk_source_or_report_claim(self): + release = {'mode': 'development', 'releaseEligible': False, + 'publicationAuthorized': False, 'agentsSdk': 'not-packaged-development-fixture'} + self.assertEqual(rehearse_release._sdk_conformance_contexts(Path('/not-read'), {}, release, None), {}) + self.assertIsNone(rehearse_release._sdk_report_binding({}, release)) + rehearse_release._validate_sdk_conformance_binding({}, {}, release) + for claim in ({}, None, False, []): + with self.subTest(claim=claim), self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._validate_sdk_conformance_binding({'sdkSourceCustody': claim}, {}, release) + + def test_sdk_absent_historical_alpha_requires_explicit_nonpromotion_authority(self): + release = {'mode': 'alpha', 'releaseEligible': False, 'publicationAuthorized': False} + self.assertEqual(rehearse_release._sdk_conformance_contexts(Path('/not-read'), {}, release, None), {}) + self.assertIsNone(rehearse_release._sdk_report_binding({}, release)) + for key in ('releaseEligible', 'publicationAuthorized'): + changed = copy.deepcopy(release); changed.pop(key) + with self.subTest(missing=key), self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_report_binding({}, changed) + for raw_release in (None, [], 'development'): + with self.subTest(nonobject=raw_release): + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_conformance_contexts(Path('/not-read'), {}, raw_release, None) + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_report_binding({}, raw_release) + + def test_placeholder_and_sdk_absence_refuse_production_or_ambiguous_authority(self): + baseline = {'mode': 'development', 'releaseEligible': False, + 'publicationAuthorized': False, 'agentsSdk': 'not-packaged-development-fixture'} + for poison in ('release', 'kernel-rc', 'alpha', 'missing-mode', 'release-eligible', + 'missing-release-eligible', 'publication-authorized', 'missing-publication', + 'unknown-string', 'null', 'false', 'list', 'sdk-evidence'): + release = copy.deepcopy(baseline); raw = {} + if poison in ('release', 'kernel-rc', 'alpha'): release['mode'] = poison + elif poison == 'missing-mode': release.pop('mode') + elif poison == 'release-eligible': release['releaseEligible'] = True + elif poison == 'missing-release-eligible': release.pop('releaseEligible') + elif poison == 'publication-authorized': release['publicationAuthorized'] = True + elif poison == 'missing-publication': release.pop('publicationAuthorized') + elif poison == 'sdk-evidence': raw['sdkEvidence'] = {} + else: release['agentsSdk'] = {'unknown-string': 'not-packaged', 'null': None, 'false': False, 'list': []}[poison] + with self.subTest(poison=poison): + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_conformance_contexts(Path('/not-read'), raw, release, None) + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_report_binding(raw, release) + for mode in ('release', 'kernel-rc', None): + release = {'mode': mode, 'releaseEligible': False, 'publicationAuthorized': False} + with self.subTest(omitted=mode): + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_report_binding({}, release) + with self.assertRaises(rehearse_release.RehearsalError): + rehearse_release._sdk_conformance_contexts(Path('/not-read'), {}, release, None) + def fixture(self, root, platform='darwin-x64'): from test_kernel_rc_evidence import sdk_fixture import kernel_rc_evidence as custody From 5526e7a5a23e9723fe4ab603317fbe72ac6e1d20 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 14:45:30 -0400 Subject: [PATCH 56/64] Authenticate SDK npm archives and repair native Linux qualification --- cli/ci/build_agents_sdk_linux.py | 52 +++++++++++++++---- cli/ci/build_kernel_rc.py | 2 +- cli/ci/npm_alias_install.py | 40 +++++++++++++-- cli/ci/qualify_installed_sdk.py | 12 ++--- cli/ci/sdk_native_inventory.py | 2 +- cli/ci/test_build_agents_sdk_linux.py | 70 +++++++++++++++++++++++++ cli/ci/test_build_kernel_rc.py | 74 +++++++++++++++++++++++++++ cli/ci/test_sdk_native_inventory.py | 34 +++++++++++- cli/protocol/OWNERSHIP.md | 10 ++++ 9 files changed, 272 insertions(+), 24 deletions(-) diff --git a/cli/ci/build_agents_sdk_linux.py b/cli/ci/build_agents_sdk_linux.py index ff31db22..b4d1654e 100644 --- a/cli/ci/build_agents_sdk_linux.py +++ b/cli/ci/build_agents_sdk_linux.py @@ -14,6 +14,7 @@ import subprocess import selectors import signal +import stat import time import uuid import shutil @@ -79,15 +80,44 @@ def source_hashes(source): def assert_stable(source, snapshot): require(source_hashes(source)==snapshot, 'Source, runtime tests, driver or lock changed') -def check_output(output): +def check_output(output, *, live=False): + """Account without following aliases; only active entries may disappear.""" count=0; size=0 - for path in output.rglob('*'): - count+=1;require(count <= MAX_FILES, 'Owned output file limit exceeded') - # Vendor archive symlinks may exist internally; never follow these for - # accounting, and never allow symlinked owned top-level stages. - if path.is_file() and not path.is_symlink(): - size+=path.stat().st_size - require(size<=MAX_RETAINED_BYTES,'Owned output byte limit exceeded') + # Root disappearance/aliasing always refuses, including active scans. + root=os.open(output,os.O_RDONLY|os.O_DIRECTORY|os.O_NOFOLLOW) + try: + require(os.fstat(root).st_uid==os.getuid(),'Owned output directory required') + def scan(directory): + nonlocal count,size + with os.scandir(directory) as entries: + for entry in entries: + # Count each observed entry even if an active writer removes it. + count+=1;require(count<=MAX_FILES,'Owned output file limit exceeded') + try: + info=os.stat(entry.name,dir_fd=directory,follow_symlinks=False) + except FileNotFoundError: + if not live: raise + continue + if stat.S_ISREG(info.st_mode): + size+=info.st_size + require(size<=MAX_RETAINED_BYTES,'Owned output byte limit exceeded') + elif stat.S_ISDIR(info.st_mode): + try: + child=os.open(entry.name,os.O_RDONLY|os.O_DIRECTORY|os.O_NOFOLLOW,dir_fd=directory) + except FileNotFoundError: + if not live: raise + continue + try: + opened=os.fstat(child) + require((opened.st_dev,opened.st_ino)==(info.st_dev,info.st_ino), + 'Owned output directory changed during accounting') + scan(child) + finally: os.close(child) + else: + # Vendor aliases are accounted as entries, never traversed. + require(stat.S_ISLNK(info.st_mode),'Unexpected owned output file type') + scan(root) + finally: os.close(root) return size def verify_archive(path, row): @@ -467,7 +497,7 @@ def process(command, output, label, *, timeout=1200, max_bytes=MAX_LOG_BYTES, de while events.get_map(): require(time.monotonic()=next_audit: - check_output(output) + check_output(output,live=p.poll() is None) require(shutil.disk_usage(output).free>=512*1024**2,'Owned scratch reserve exhausted') next_audit=time.monotonic()+1 for key,_ in events.select(min(.1,max(0,deadline-time.monotonic()))): @@ -499,7 +529,7 @@ def stream_transport(command, output, deadline, audit): while events.get_map(): require(time.monotonic()=next_audit: - audit();check_output(output) + audit();check_output(output,live=p.poll() is None) require(shutil.disk_usage(output).free>=512*1024**2,'Owned scratch reserve exhausted') next_audit=time.monotonic()+1 for key,_ in events.select(min(.1,max(0,deadline-time.monotonic()))): @@ -508,7 +538,7 @@ def stream_transport(command, output, deadline, audit): # Flush both streams before aggregate accounting; never write a chunk # that would exceed the existing entire-job retained-byte quota. tar.flush();log.flush() - require(check_output(output)+len(data)<=MAX_RETAINED_BYTES,'Owned output byte limit exceeded') + require(check_output(output,live=p.poll() is None)+len(data)<=MAX_RETAINED_BYTES,'Owned output byte limit exceeded') if key.data=='tar':tar.write(data) else: total+=len(data) diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index 36ad5225..aa271016 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -285,7 +285,7 @@ def check(binary, runner, label, node=None): meta = next(package / a['path'] for a in manifest['artifacts'] if a['kind'] == 'npm-meta') platform_package = next(package / a['path'] for a in manifest['artifacts'] if a['kind'] == 'npm-platform') alias_install = install_alias(meta, platform_package, prefix, env=env, cwd=output, - command=command, log=logs / 'npm-install.log') + command=command, log=logs / 'npm-install.log', sdk_manifest=manifest) (logs / 'npm-alias-install.json').write_text(json.dumps(alias_install, sort_keys=True) + '\n') node = shutil.which('node', path=env.get('PATH')); require(node, 'Node is required for npm launcher') installed_helpers = [prefix / 'lib/node_modules/@openprose' / ('prose-cli-' + manifest['platform']) / 'bin/prose-agents-sdk'] diff --git a/cli/ci/npm_alias_install.py b/cli/ci/npm_alias_install.py index 377a830c..4b8971ea 100644 --- a/cli/ci/npm_alias_install.py +++ b/cli/ci/npm_alias_install.py @@ -3,17 +3,49 @@ import hashlib from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer import json +import os import threading from urllib.parse import unquote, urlsplit import publication -def install(meta, platform, prefix, *, env, cwd, command, log): - members = publication.archive_members(platform) - manifest = json.loads(members['package/package.json']) +def platform_payload(platform, *, sdk_manifest=None): + """Read SDK publications only with their explicit, exact artifact authority.""" + if sdk_manifest is None: + members = publication.archive_members(platform) + return json.loads(members['package/package.json']), platform.read_bytes() + publication.require(isinstance(sdk_manifest, dict) and isinstance(sdk_manifest.get('agentsSdk'), dict), + 'Explicit SDK publication manifest required') + artifacts = sdk_manifest.get('artifacts') + publication.require(isinstance(artifacts, list), 'SDK publication artifact list required') + matches = [row for row in artifacts if isinstance(row, dict) and row.get('kind') == 'npm-platform'] + publication.require(len(matches) == 1 and matches[0].get('path') == platform.name, + 'SDK npm artifact filename differs from publication') + artifact = matches[0] + publication.require(platform.is_file() and not platform.is_symlink() + and 0 < platform.stat().st_size <= publication.MAX_BYTES, 'Regular bounded SDK npm archive required') + before = platform.stat() + descriptor = os.open(platform, os.O_RDONLY | os.O_NOFOLLOW) + with os.fdopen(descriptor, 'rb') as source: + opened = os.fstat(source.fileno()) + publication.require((opened.st_dev, opened.st_ino) == (before.st_dev, before.st_ino), 'SDK npm archive changed before read') + payload = source.read(publication.MAX_BYTES + 1) + after = platform.stat() + identity = lambda row: (row.st_dev, row.st_ino, row.st_size, row.st_mtime_ns, row.st_ctime_ns) + publication.require(not platform.is_symlink() and len(payload) == before.st_size + and identity(before) == identity(opened) == identity(after), 'SDK npm archive changed during read') + publication.require(type(artifact.get('byteLength')) is int and artifact['byteLength'] == len(payload) + and artifact.get('sha256') == hashlib.sha256(payload).hexdigest(), 'SDK npm artifact digest or length differs') + table = publication.decode_sdk_archive(payload, sdk_manifest, label=platform.name) + # The full typed SDK tree is authenticated before projecting npm metadata. + manifest = json.loads(table['files']['package/package.json'][0], object_pairs_hook=publication.object_pairs) + return manifest, payload + + +def install(meta, platform, prefix, *, env, cwd, command, log, sdk_manifest=None): + manifest, payload = platform_payload(platform, sdk_manifest=sdk_manifest) name, version = manifest['name'], manifest['version'] publication.require(name == '@openprose/prose-cli' and manifest.get('os') and manifest.get('cpu'), 'Expected exact same-name platform package') - payload = platform.read_bytes() requests = [] class Registry(BaseHTTPRequestHandler): def do_GET(self): diff --git a/cli/ci/qualify_installed_sdk.py b/cli/ci/qualify_installed_sdk.py index 90cabac9..1fde382c 100644 --- a/cli/ci/qualify_installed_sdk.py +++ b/cli/ci/qualify_installed_sdk.py @@ -350,9 +350,9 @@ def qualify(candidate_root, previous_release, output, source, version, node, npm root = output / 'npm'; root.mkdir() env = q.environment(root, npm=True, explicit_config=True) prefix = root / 'prefix' - def install(packages, label): - return npm_alias_install.install(*packages, prefix, env=env, cwd=root, command=q.npm_command, log=q.logs / (label + '.log')) - installs = [install(candidate_npm, 'npm-fresh')] + def install(packages, label, sdk_manifest): + return npm_alias_install.install(*packages, prefix, env=env, cwd=root, command=q.npm_command, log=q.logs / (label + '.log'), sdk_manifest=sdk_manifest) + installs = [install(candidate_npm, 'npm-fresh', manifest)] launcher = prefix / 'bin/prose' def npm_payload(): native = list((prefix / 'lib/node_modules').rglob('prose-agents-sdk')) @@ -364,9 +364,9 @@ def npm_payload(): q.defaults([node, launcher], 'bun', version, source, root, env, 'npm-fresh') saved = q.save_preferences([node, launcher], root, env, 'npm') q.execute(['npm', 'uninstall', '--global', '--prefix', prefix, '--offline', '--ignore-scripts', '--no-audit', '--no-fund', '@openprose/prose-cli'], env=env, cwd=root, label='npm-remove-fresh') - installs.append(install(previous_npm, 'npm-previous')) + installs.append(install(previous_npm, 'npm-previous', None)) q.identity([node, launcher], 'bun', previous_plan['version'], previous_plan['source'], root, env, 'npm-previous') - installs.append(install(candidate_npm, 'npm-upgrade')) + installs.append(install(candidate_npm, 'npm-upgrade', manifest)) npm_payload() q.identity([node, launcher], 'bun', version, source, root, env, 'npm-upgraded') q.preferences([node, launcher], root, env, 'npm-upgraded', saved) @@ -378,7 +378,7 @@ def npm_payload(): env = q.environment(root, npm=True) # Populate only the exact platform package using the established offline # installer; npm exec itself installs the original root tarball in _npx. - install_record = npm_alias_install.install(*candidate_npm, root / 'cache-primer-prefix', env=env, cwd=root, command=q.npm_command, log=q.logs / 'npx-prime.log') + install_record = npm_alias_install.install(*candidate_npm, root / 'cache-primer-prefix', env=env, cwd=root, command=q.npm_command, log=q.logs / 'npx-prime.log', sdk_manifest=manifest) env = offline_registry_environment(env, install_record) shutil.rmtree(root / 'cache-primer-prefix') command = [node, npm, 'exec', '--offline', '--yes', '--ignore-scripts', '--no-audit', '--no-fund', '--package', candidate_npm[0], '--', 'prose'] diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index ca3195a1..730e12ea 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -367,7 +367,7 @@ def hash_value(digest): def python_origin(value): distribution = value['pythonDistribution']; root = Path(distribution['root']) data = read_file(distribution['metadataPath'], MAX_METADATA); info = load_json(data) - require(info.get('version') == 8, 'Unsupported full Python metadata version') + require(info.get('version') == '8', 'Unsupported full Python metadata version') require(info.get('python_version') == '3.10.20' and info.get('target_triple') == value['pythonArchive']['targetTriple'] and info.get('libpython_link_mode') == 'shared', 'Python metadata is not the target full shared distribution') license_paths = {info.get('license_path')} if info.get('license_path') else set() diff --git a/cli/ci/test_build_agents_sdk_linux.py b/cli/ci/test_build_agents_sdk_linux.py index 83c21db1..e612646f 100644 --- a/cli/ci/test_build_agents_sdk_linux.py +++ b/cli/ci/test_build_agents_sdk_linux.py @@ -580,5 +580,75 @@ def audit(): stream.assert_not_called() +class LiveQuotaAccountingControls(unittest.TestCase): + def disappear(self,root,live): + item=root/'temporary.pyc.owned';item.write_bytes(b'transient');original=d.os.stat + def remove(name,*args,**kwargs): + if name==item.name and kwargs.get('dir_fd') is not None:item.unlink() + return original(name,*args,**kwargs) + with patch.object(d.os,'stat',side_effect=remove):return d.check_output(root,live=live) + def test_only_live_disappearing_entry_is_tolerated(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();self.assertEqual(self.disappear(root,True),0) + with self.assertRaises(FileNotFoundError):self.disappear(root,False) + def test_live_missing_root_and_permission_failures_are_not_ignored(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();(root/'owned').write_bytes(b'x') + with self.assertRaises(FileNotFoundError):d.check_output(root/'missing',live=True) + with patch.object(d.os,'stat',side_effect=PermissionError(13,'fixture refusal')): + with self.assertRaises(PermissionError):d.check_output(root,live=True) + def test_live_observed_entry_and_regular_byte_limits_are_preserved(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve() + with patch.object(d,'MAX_FILES',0),self.assertRaisesRegex(ValueError,'file limit'):self.disappear(root,True) + (root/'temporary.pyc.owned').unlink();(root/'retained').write_bytes(b'12345') + with patch.object(d,'MAX_RETAINED_BYTES',4),self.assertRaisesRegex(ValueError,'byte limit'):d.check_output(root,live=True) + def test_external_aliases_are_counted_without_following(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();out=root/'owned';out.mkdir();external=root/'external';external.mkdir() + (external/'large').write_bytes(b'123456789');(out/'alias').symlink_to(external,target_is_directory=True);(out/'file-alias').symlink_to(external/'large') + with patch.object(d,'MAX_RETAINED_BYTES',0):self.assertEqual(d.check_output(out,live=True),0) + with patch.object(d,'MAX_FILES',1),self.assertRaisesRegex(ValueError,'file limit'):d.check_output(out,live=True) + with self.assertRaises(OSError):d.check_output(out/'alias',live=True) + def test_directory_replacement_cannot_follow_an_alias(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();out=root/'owned';out.mkdir();child=out/'stage';child.mkdir();external=root/'external';external.mkdir();original=d.os.open + def replace(name,*args,**kwargs): + if name=='stage' and kwargs.get('dir_fd') is not None:child.rmdir();child.symlink_to(external,target_is_directory=True) + return original(name,*args,**kwargs) + with patch.object(d.os,'open',side_effect=replace),self.assertRaises(OSError):d.check_output(out,live=True) + def test_directory_disappearance_is_live_only_and_identity_replacement_refuses(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();out=root/'owned';out.mkdir();original=d.os.open + for live in (True,False): + child=out/'stage';child.mkdir() + def remove(name,*args,**kwargs): + if name=='stage' and kwargs.get('dir_fd') is not None:child.rmdir() + return original(name,*args,**kwargs) + with patch.object(d.os,'open',side_effect=remove): + if live:self.assertEqual(d.check_output(out,live=True),0) + else: + with self.assertRaises(FileNotFoundError):d.check_output(out) + child=out/'stage';child.mkdir() + def replace(name,*args,**kwargs): + if name=='stage' and kwargs.get('dir_fd') is not None: + child.rename(out/'old-stage');child.mkdir() + return original(name,*args,**kwargs) + with patch.object(d.os,'open',side_effect=replace),self.assertRaisesRegex(ValueError,'directory changed'): + d.check_output(out,live=True) + def test_special_files_refuse_even_during_live_scan(self): + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();d.os.mkfifo(root/'pipe') + with self.assertRaisesRegex(ValueError,'Unexpected owned output file type'):d.check_output(root,live=True) + def test_process_scan_is_live_only_while_direct_child_runs(self): + import sys + with tempfile.TemporaryDirectory() as raw: + root=Path(raw).resolve();(root/'home').mkdir();calls=[];actual=d.check_output + def observe(output,**kwargs):calls.append(kwargs.get('live',False));return actual(output,**kwargs) + with patch.object(d,'check_output',side_effect=observe):d.process([sys.executable,'-c','import time;time.sleep(.2);print("owned")'],root,'child',timeout=2) + self.assertIn(True,calls) + self.assertEqual(d.check_output(root),sum(p.stat().st_size for p in root.iterdir() if p.is_file())) + + if __name__ == '__main__': unittest.main() diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index 86503c1f..1555c502 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -1,4 +1,6 @@ import io +import copy +import hashlib import json import os from pathlib import Path @@ -7,6 +9,7 @@ import unittest from unittest.mock import patch import build_kernel_rc as rc +import npm_alias_install class KernelRCBuildTests(unittest.TestCase): @@ -111,6 +114,77 @@ def test_tampered_artifact_is_rejected(self): +class NpmAliasSdkArchiveTests(unittest.TestCase): + def fixture(self, path): + from test_qualify_installed_sdk import sdk_archive, archive + manifest, table = sdk_archive(path, prefix='package/bin/') + encoded = table['files']['package/bin/agents-sdk-build.json'][0] + receipt = json.loads(encoded) + for index in range(140): + name = 'prose-agents-sdk-runtime/resource-' + str(index).zfill(3) + data = b'nonexecuted support fixture' + table['files']['package/bin/' + name] = (data, 0o644) + receipt['payload']['entries'].append({'path': name, 'type': 'file', 'mode': 0o644, + 'byteLength': len(data), 'sha256': hashlib.sha256(data).hexdigest()}) + receipt['payload']['totalRegularBytes'] += len(data) + receipt['payload']['entries'].sort(key=lambda row: row['path']) + encoded = json.dumps(receipt, sort_keys=True).encode() + table['files']['package/bin/agents-sdk-build.json'] = (encoded, 0o644) + manifest['agentsSdk']['receiptSha256'] = hashlib.sha256(encoded).hexdigest() + metadata = {'name': '@openprose/prose-cli', 'version': '0.15.0-rc.4', 'os': ['darwin'], 'cpu': ['x64']} + table['files']['package/package.json'] = (json.dumps(metadata).encode(), 0o644) + archive(path, table['files'], table) + manifest['artifacts'] = [{'path': path.name, 'kind': 'npm-platform', + 'byteLength': path.stat().st_size, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()}] + return manifest, table, metadata + + def test_explicit_sdk_authority_accepts_complete_tree_above_generic_limit(self): + with tempfile.TemporaryDirectory() as raw: + path = Path(raw) / 'sdk-platform.tgz'; manifest, table, metadata = self.fixture(path) + self.assertGreater(sum(len(rows) for rows in table.values()), 128) + observed, payload = npm_alias_install.platform_payload(path, sdk_manifest=manifest) + self.assertEqual(observed, metadata) + self.assertEqual(payload, path.read_bytes()) + with self.assertRaisesRegex(ValueError, 'Too many archive members'): + npm_alias_install.platform_payload(path) + + def test_rehashed_support_or_artifact_authority_poison_fails_before_server_or_commands(self): + from test_qualify_installed_sdk import archive + with tempfile.TemporaryDirectory() as raw: + root = Path(raw); path = root / 'sdk-platform.tgz' + for poison in ('filename', 'digest', 'length', 'bool-length', 'duplicate-artifact', 'missing-identity', 'support'): + manifest, table, _ = self.fixture(path) + if poison == 'filename': manifest['artifacts'][0]['path'] = 'other.tgz' + elif poison == 'digest': manifest['artifacts'][0]['sha256'] = '0' * 64 + elif poison == 'length': manifest['artifacts'][0]['byteLength'] += 1 + elif poison == 'bool-length': manifest['artifacts'][0]['byteLength'] = True + elif poison == 'duplicate-artifact': manifest['artifacts'].append(copy.deepcopy(manifest['artifacts'][0])) + elif poison == 'missing-identity': manifest.pop('agentsSdk') + else: + table['files']['package/bin/prose-agents-sdk-runtime/resource-000'] = (b'poison', 0o644) + archive(path, table['files'], table) + manifest['artifacts'][0].update(byteLength=path.stat().st_size, sha256=hashlib.sha256(path.read_bytes()).hexdigest()) + with self.subTest(poison=poison), patch.object(npm_alias_install, 'ThreadingHTTPServer') as server: + command = unittest.mock.Mock(side_effect=AssertionError('No npm call')) + with self.assertRaises(ValueError): + npm_alias_install.install(root / 'meta.tgz', path, root / 'prefix', env={}, cwd=root, + command=command, log=root / 'npm.log', sdk_manifest=manifest) + server.assert_not_called(); command.assert_not_called() + self.assertFalse((root / 'prefix').exists()) + + def test_generic_prior_archive_retains_regular_only_policy(self): + from test_qualify_installed_sdk import archive + with tempfile.TemporaryDirectory() as raw: + path = Path(raw) / 'prior.tgz' + metadata = {'name': '@openprose/prose-cli', 'version': '0.15.0-rc.3'} + files = {'package/package.json': (json.dumps(metadata).encode(), 0o644)} + archive(path, files) + self.assertEqual(npm_alias_install.platform_payload(path, sdk_manifest=None)[0], metadata) + archive(path, files, {'symlinks': {'package/alias': 'package.json'}}) + with self.assertRaisesRegex(ValueError, 'Links and special members'): + npm_alias_install.platform_payload(path, sdk_manifest=None) + + class KernelRCSdkExtractionTests(unittest.TestCase): archive = KernelRCBuildTests.archive def test_production_extraction_requires_sdk_siblings_before_writing(self): diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index d23e208a..c57183d0 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -83,7 +83,7 @@ def input_fixture(root, target='linux-x64-gnu'): python = root / 'python'; python.mkdir() (python / 'LICENSE').write_text('Python fixture license') (python / 'libpython.so').write_bytes(elf(machine=machine, symbol='Py_Initialize')) - info = {'version': 8, 'python_version': '3.10.20', 'target_triple': arch + '-unknown-linux-gnu', 'libpython_link_mode': 'shared', + info = {'version': '8', 'python_version': '3.10.20', 'target_triple': arch + '-unknown-linux-gnu', 'libpython_link_mode': 'shared', 'licenses': ['Python-2.0'], 'license_path': 'LICENSE', 'build_info': {'core': {'shared_lib': 'libpython.so'}, 'extensions': {}}} metadata = python / 'PYTHON.json'; metadata.write_text(json.dumps(info)) image = lambda floor: {'image': 'quay.io/pypa/manylinux_' + floor + '_' + arch + '@sha256:' + 'a' * 64, @@ -270,6 +270,38 @@ def test_python_owned_bytes_and_static_license_metadata(self): (root / 'zlib-license').unlink() with self.assertRaises(ValueError): native.python_origin(value) + def test_python_metadata_requires_canonical_upstream_string_version_on_both_targets(self): + for target in ('linux-x64-gnu', 'linux-arm64-gnu'): + with self.subTest(target=target), tempfile.TemporaryDirectory() as temp: + value, _, _, _ = input_fixture(Path(temp), target) + root = Path(value['pythonDistribution']['root']); metadata = root / 'PYTHON.json' + original = json.loads(metadata.read_text()) + origin, owned = native.python_origin(value) + self.assertEqual(origin['archive']['targetTriple'], original['target_triple']) + self.assertEqual(origin['metadataSha256'], native.sha(metadata.read_bytes())) + self.assertIn(native.sha((root / 'libpython.so').read_bytes()), owned) + for version in (8, 8.0, True, None, '08', '8.0', ' 8', '8 ', '9', [], {}): + changed = copy.deepcopy(original); changed['version'] = version + metadata.write_text(json.dumps(changed)) + with self.subTest(version=version), self.assertRaisesRegex(ValueError, 'metadata version'): + native.python_origin(value) + changed = copy.deepcopy(original); changed.pop('version'); metadata.write_text(json.dumps(changed)) + with self.assertRaisesRegex(ValueError, 'metadata version'): native.python_origin(value) + + def test_python_metadata_string_version_does_not_bypass_native_origin_guards(self): + with tempfile.TemporaryDirectory() as temp: + value, _, _, _ = input_fixture(Path(temp)) + metadata = Path(value['pythonDistribution']['metadataPath']); original = json.loads(metadata.read_text()) + mutations = [lambda v: v.update(python_version='3.10.19'), + lambda v: v.update(target_triple='aarch64-unknown-linux-gnu'), + lambda v: v.update(libpython_link_mode='static'), + lambda v: v.update(licenses=[]), lambda v: v.pop('license_path'), + lambda v: v['build_info']['core'].update(shared_lib='absent.so'), + lambda v: v['build_info'].update(extensions=[])] + for mutate in mutations: + changed = copy.deepcopy(original); mutate(changed); metadata.write_text(json.dumps(changed)) + with self.assertRaises(ValueError): native.python_origin(value) + def test_wheel_native_and_license_members_are_checked_against_record(self): with tempfile.TemporaryDirectory() as temp: root = Path(temp); paths = ['fixture.dist-info/RECORD', 'fixture.dist-info/LICENSE', 'extension.so'] diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index ff4b9eda..effdf4f8 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1135,3 +1135,13 @@ Mac SDK conformance custody clarification and caller lease: the seventeen adapte Full19 failed release-package-admission after49 passing gates: npm's documented extraction drops archive aliases, and the static Windows fixture incorrectly enters the POSIX-only SDK decoder. `/root/operational_design` retains its exact `cli/benchmarks/installed/benchmark.py` and `cli/benchmarks/installed/test_benchmark.py` lease and additionally owns `cli/ci/test_release_package_admission.py` only for truthful SDK-absent Windows authority fixtures. Preserve production Mac complete-member/alias rejection, mandatory SDK custody for four supported POSIX targets, Windows ordinary artifact/sidecar/provenance integrity and blocked-before-native-execution authority. Reject unexpected Windows SDK claims; do not broaden SDK supported targets or weaken package/archive bounds. Positive npm fixtures must represent an actual complete linkless tree; independent missing-alias rejection remains required. Actual Intel directory inventory and npm compatibility remain native qualification gates. Agents run only pure/synthetic controls; root owns actual npm/native replay, full admission, source integration, push and merge. Prior full19 and prototype evidence retain their exact source and failure scope. Root additionally owns only `cli/ci/release_package_admission.py` for the same full19 repair: closed native artifact membership and SDK lineage must distinguish SDK-absent Windows static authority from mandatory four-platform POSIX SDK custody, preserving Windows sidecar equality and refusal before execution. + + +## IMP-098 actual Mac npm alias installation custody repair + +Fresh native construction at046eef4 reached standalone SDK checks then failed because `npm_alias_install` decoded the complete SDK platform tarball through the ordinary128-member archive reader. `/root/operational_design` additionally owns `cli/ci/npm_alias_install.py` for an explicit optional authenticated SDK release-manifest route, plus its existing `cli/ci/test_build_kernel_rc.py`, `cli/ci/build_kernel_rc.py`, `cli/ci/qualify_installed_sdk.py`, and `cli/ci/test_qualify_installed_sdk.py` leases for current candidate fresh/upgrade/npx callers and meaningful regression controls. The prior narrow root URL-return lease is superseded for this repair. Require exact artifact filename/hash/length and complete typed SDK tree/source custody before any package.json projection, loopback server or npm command. Missing SDK context retains the original generic128/no-links policy; genuine SDK-absent RC3 installs remain on that ordinary route. Preserve all SDK limits, pins, scopes and failure evidence. No inference from an oversized archive or automatic fallback. Agents run pure/synthetic controls only; root owns native npm replay, mandatory full admission before source push, Git and merges. + + +## IMP-098 fifth native Linux qualification repairs + +Actual pinned Linux PYTHON.json records schema version as the string `"8"`; the integer8 synthetic assumption caused x64 native refusal. `/root/configuration_contract` owns its existing `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` paths to require that exact upstream string and truthful fixtures/poisons, retaining archive hashes, target/shared-library/license/source custody and all other metadata guards. Actual ARM preparation also encountered a disappearing temporary pyc file during host quota scanning. `/root/sdk_assessment` owns its existing `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` paths for bounded no-follow live accounting that tolerates only disappeared entries within owned active work, preserves file/byte caps and structural refusals, and retains closed post-stage checks. Keep supplier/native schemas, tool/image/archive pins, deadlines, original errors and cleanup evidence; no broad suppression or limit relaxation. Agents use pure/synthetic controls and read-only retained native evidence; root owns native/full execution and push/merge. From dd08df942b16a26ad2a0ecdf9aa0d2b85ecf48c6 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 14:50:09 -0400 Subject: [PATCH 57/64] Use the installed npm alias dependency path --- cli/ci/build_kernel_rc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index aa271016..0c7e089a 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -288,7 +288,7 @@ def check(binary, runner, label, node=None): command=command, log=logs / 'npm-install.log', sdk_manifest=manifest) (logs / 'npm-alias-install.json').write_text(json.dumps(alias_install, sort_keys=True) + '\n') node = shutil.which('node', path=env.get('PATH')); require(node, 'Node is required for npm launcher') - installed_helpers = [prefix / 'lib/node_modules/@openprose' / ('prose-cli-' + manifest['platform']) / 'bin/prose-agents-sdk'] + installed_helpers = [prefix / 'lib/node_modules/@openprose/prose-cli/node_modules/@openprose' / ('prose-cli-' + manifest['platform']) / 'bin/prose-agents-sdk'] require(installed_helpers[0].is_file() and not installed_helpers[0].is_symlink(), 'npm installation requires one package-owned SDK helper') sdk_before = installed_sdk_identity(installed_helpers[0].parent, manifest) From bac5ec82c32e3ae5d97a58680680e6504ebbc9c9 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 16:08:39 -0400 Subject: [PATCH 58/64] Fix packaged SDK release qualification on Mac and Linux --- cli/ci/assemble_kernel_rc.py | 38 ++- cli/ci/build_kernel_rc.py | 152 +++++++++++- cli/ci/homebrew_rehearsal.py | 7 +- cli/ci/kernel_rc_evidence.py | 342 +++++++++++++++++++++++++- cli/ci/sdk_native_inventory.py | 71 +++++- cli/ci/test_assemble_kernel_rc.py | 134 +++++++++- cli/ci/test_build_kernel_rc.py | 256 +++++++++++++++++++ cli/ci/test_homebrew_rehearsal.py | 78 ++++++ cli/ci/test_kernel_rc_evidence.py | 367 ++++++++++++++++++++++++++++ cli/ci/test_sdk_native_inventory.py | 111 ++++++++- cli/protocol/OWNERSHIP.md | 20 ++ 11 files changed, 1544 insertions(+), 32 deletions(-) diff --git a/cli/ci/assemble_kernel_rc.py b/cli/ci/assemble_kernel_rc.py index bc711834..bea07d0d 100644 --- a/cli/ci/assemble_kernel_rc.py +++ b/cli/ci/assemble_kernel_rc.py @@ -39,6 +39,7 @@ def assemble(roots, output, evidence, live_smoke=None): diagnostic, policy = manifest['embeddedDiagnosticImage'], manifest['kernelPolicy'] pub.require(manifest['version'] == version == report['version'] and manifest['source']['revision'] == source == report['sourceRevision'], 'Mixed candidate versions or source commits') pub.require(manifest['embeddedDiagnosticImage'] == diagnostic and manifest['kernelPolicy'] == policy, 'Mixed diagnostic image or kernel policy') + custody.validate_producer_command_logs(report, manifest, lambda relative: custody.read_command_log_bytes(root / relative)) for item in manifest['artifacts']: name = item['path'] pub.require(pub.safe_name(name), 'Unsafe package artifact name') @@ -75,8 +76,12 @@ def assemble(roots, output, evidence, live_smoke=None): checks = {name: pub.read_json(root / ('logs/' + name + '.json')) for name in custody.CHECKS} custody.validate_native(report, manifest, checks, native_hashes, launcher_hash) pub.require(custody.SDK_PAYLOAD_EVIDENCE in report['evidence'], 'Missing complete installed SDK payload evidence') - custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE), manifest, table) + custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE, max_bytes=16*1024*1024), manifest, table) custody.validate_sdk_producer_evidence(manifest, table, report['evidence'], lambda relative: (root / relative).read_bytes()) + custody.validate_linux_runtime_evidence(report, manifest, table, + lambda relative: custody.read_command_log_bytes(root / relative), + expected_sources=custody.read_linux_runtime_sources(Path(__file__).resolve().parents[2]) + if platform.startswith('linux-') else None) all_binary_hashes.update(native_hashes) # Retain every report-bound input, including the five structured native # observations. Publication independently repeats these bindings. @@ -92,6 +97,28 @@ def assemble(roots, output, evidence, live_smoke=None): inventory[name] = (actual, {'name': name, 'sha256': pub.digest(actual), 'size': actual.stat().st_size, 'kind': 'evidence', 'platform': platform, 'implementation': 'shared'}) reports[platform] = {'status': 'pass', 'report': name} manifests[platform] = manifest + live = pub.read_json(live_smoke) if live_smoke else {'status': 'not-run'} + live_paths = {} + live_names = set() + if live_smoke: + for runner, attempt in live.get('runners', {}).items(): + for role, record in attempt.get('evidence', {}).items(): + name = custody.live_asset_name(runner, role, record) + pub.require(name not in inventory and name not in live_names, 'Live evidence filename collision') + live_names.add(name) + live_paths[(runner, role)] = live_smoke.parent / record['path'] + custody.validate_live_smoke(live, source, version, all_binary_hashes, live_paths) + pub.require(len(live_names) == 2 * len(custody.LIVE_ROLES), 'All sixteen live evidence assets are required') + else: + # Live names have a reserved namespace; their exact file suffixes are + # bound after capture, but every required runner/role needs an asset. + pub.require(not any(name.startswith('live-bun-') or name.startswith('live-rust-') for name in inventory), + 'Offline inventory collides with reserved live evidence namespace') + pub.require('SHA256SUMS' not in inventory and 'kernel-rc-release-evidence.json' not in inventory, + 'Global release evidence filename collision') + reserved_live_count = len(live_names) if live_smoke else 2 * len(custody.LIVE_ROLES) + pub.require(len(inventory) + 2 + reserved_live_count <= 128, + 'Artifact inventory exceeds 128 including required live evidence') output.mkdir(parents=True) for name, (path, _) in inventory.items(): shutil.copyfile(path, output / name) @@ -103,20 +130,13 @@ def assemble(roots, output, evidence, live_smoke=None): pub.require(len(archives) == 13 and checksum_path.name not in inventory, 'Expected thirteen install archives') checksum_path.write_bytes(''.join(item['sha256'] + ' ' + item['name'] + '\n' for item in archives).encode('ascii')) inventory[checksum_path.name] = (checksum_path, {'name': checksum_path.name, 'sha256': pub.digest(checksum_path), 'size': checksum_path.stat().st_size, 'kind': 'evidence', 'platform': 'all', 'implementation': 'shared'}) - live = pub.read_json(live_smoke) if live_smoke else {'status': 'not-run'} preflight = {'schema': 'openprose.kernel-rc-release-evidence/1', 'version': version, 'sourceSha': source, 'status': 'pass' if live_smoke else 'incomplete', 'failures': [] if live_smoke else ['Exact-source live smoke remains required'], 'imageSource': 'published-on-run', 'embeddedDiagnosticImage': diagnostic, 'kernelPolicy': policy, 'platforms': reports, 'liveSmoke': live} if live_smoke: - paths = {} - for runner, attempt in live.get('runners', {}).items(): - for role, record in attempt.get('evidence', {}).items(): - custody.live_asset_name(runner, role, record) - paths[(runner, role)] = live_smoke.parent / record['path'] - custody.validate_live_smoke(live, source, version, all_binary_hashes, paths) for runner, attempt in live['runners'].items(): for role, record in attempt['evidence'].items(): name = custody.live_asset_name(runner, role, record) pub.require(name not in inventory and pub.safe_name(name), 'Live evidence filename collision') - actual = paths[(runner, role)] + actual = live_paths[(runner, role)] shutil.copyfile(actual, output / name) inventory[name] = (actual, {'name': name, 'sha256': record['sha256'], 'size': record['byteLength'], 'kind': 'evidence', 'platform': 'darwin-arm64', 'implementation': runner}) preflight_path = output / 'kernel-rc-release-evidence.json' diff --git a/cli/ci/build_kernel_rc.py b/cli/ci/build_kernel_rc.py index 0c7e089a..86f4a9f2 100644 --- a/cli/ci/build_kernel_rc.py +++ b/cli/ci/build_kernel_rc.py @@ -34,6 +34,139 @@ def digest(path): return h.hexdigest() +def write_installed_sdk_payloads(path, records): + """Write the fixed custody order, independent of package artifact order.""" + surfaces = ('installed-bun', 'installed-rust', 'installed-npm') + require(isinstance(records, list) and len(records) == len(surfaces), + 'Three installed SDK payload surfaces required') + by_surface = {} + for record in records: + require(isinstance(record, dict) and set(record) == {'surface', 'before', 'after'} + and record['surface'] in surfaces and record['surface'] not in by_surface + and record['before'] == record['after'], + 'Installed SDK payload surface identity differs') + by_surface[record['surface']] = record + encoded = json.dumps([by_surface[surface] for surface in surfaces], sort_keys=True).encode('utf-8') + b'\n' + require(len(encoded) <= 16 * 1024 * 1024, 'Installed SDK payload record exceeds evidence bound') + path.write_bytes(encoded) + + +def producer_command_log_evidence(output, report, manifest): + """Bundle only closed opaque logs; preserve originals and all other evidence.""" + platform_id = manifest['platform'] + require(platform_id in sdk_custody.COMMAND_LOG_PLATFORMS, 'Unsupported command-log platform') + names = (sdk_custody.COMMAND_LOG_MAC_MEMBERS if platform_id.startswith('darwin-') + else sdk_custody.COMMAND_LOG_LINUX_MEMBERS) + target = output / sdk_custody.COMMAND_LOG_EVIDENCE + require(not target.exists() and not target.is_symlink(), 'Command-log bundle must be fresh') + require(sdk_custody.COMMAND_LOG_EVIDENCE not in report['evidence'], 'Command-log bundle already recorded') + def stat_identity(path): + info = path.lstat() + return tuple(getattr(info, key) for key in + ('st_dev', 'st_ino', 'st_mode', 'st_uid', 'st_size', 'st_mtime_ns', 'st_ctime_ns')) + rows = []; originals = {}; original_metadata = {}; raw_total = encoded_total = 0 + for name in names: + original_metadata[name] = stat_identity(output / name) + data = sdk_custody.read_command_log_bytes(output / name) + require(stat_identity(output / name) == original_metadata[name], 'Original command-log metadata changed') + raw_total += len(data); encoded_total += 4 * ((len(data) + 2) // 3) + require(raw_total <= sdk_custody.COMMAND_LOG_MAX_BYTES + and encoded_total <= sdk_custody.COMMAND_LOG_MAX_BYTES, + 'Command-log bundle exceeds evidence bound') + identity = {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + recorded = report['evidence'].get(name) + require(isinstance(recorded, dict) and type(recorded.get('byteLength')) is int + and recorded == identity, 'Original command-log evidence changed') + rows.append({'path': name, **identity, 'base64': base64.b64encode(data).decode('ascii')}) + originals[name] = data + bundle = {'schema': 'openprose.kernel-rc-command-logs/1', 'platform': platform_id, + 'sourceRevision': report['sourceRevision'], 'version': report['version'], 'members': rows} + encoded = json.dumps(bundle, sort_keys=True).encode('utf-8') + b'\n' + require(0 < len(encoded) <= sdk_custody.COMMAND_LOG_MAX_BYTES, 'Command-log bundle exceeds evidence bound') + evidence = {name: record for name, record in report['evidence'].items() if name not in originals} + evidence[sdk_custody.COMMAND_LOG_EVIDENCE] = {'sha256': hashlib.sha256(encoded).hexdigest(), 'byteLength': len(encoded)} + prospective = {**report, 'evidence': evidence} + decoded = sdk_custody.validate_producer_command_logs(prospective, manifest, + lambda name: encoded if name == sdk_custody.COMMAND_LOG_EVIDENCE else None) + require(decoded == originals, 'Command-log bundle differs from original bytes') + for name, data in originals.items(): + require(sdk_custody.read_command_log_bytes(output / name) == data, + 'Original command-log bytes changed before bundling') + require(stat_identity(output / name) == original_metadata[name], 'Original command-log metadata changed') + with target.open('xb') as stream: + stream.write(encoded) + require(sdk_custody.read_command_log_bytes(target) == encoded, 'Written command-log bundle changed') + require(all(stat_identity(output / name) == original_metadata[name] for name in originals), + 'Original command-log metadata changed after bundling') + return evidence + + +def producer_linux_runtime_evidence(output, report, manifest, sdk_table, *, source_root): + """Retain the closed runtime proof without flattening its physical tree.""" + if not manifest['platform'].startswith('linux-'): + sdk_custody.validate_linux_runtime_evidence(report, manifest, sdk_table, + lambda name: sdk_custody.read_command_log_bytes(output / name), expected_sources=None) + return dict(report['evidence']) + target = output / sdk_custody.LINUX_RUNTIME_EVIDENCE + require(not target.exists() and not target.is_symlink(), 'Linux runtime packet must be fresh') + require(sdk_custody.LINUX_RUNTIME_EVIDENCE not in report['evidence'], 'Linux runtime packet already recorded') + runtime_root = output / 'agents-sdk-runtime' + raw_report = sdk_custody.read_command_log_bytes(runtime_root / 'runtime-report.json') + runtime_report = json.loads(raw_report, object_pairs_hook=pub.object_pairs) + expected_sources = sdk_custody.read_linux_runtime_sources(source_root) + sdk_custody.validate_linux_runtime_tree(runtime_root, runtime_report, sdk_table) + def stat_identity(path): + info = path.lstat() + return tuple(getattr(info, key) for key in + ('st_dev', 'st_ino', 'st_mode', 'st_uid', 'st_size', 'st_mtime_ns', 'st_ctime_ns')) + rows = []; originals = {}; metadata = {}; raw_total = encoded_total = 0 + for name in sdk_custody.LINUX_RUNTIME_MEMBER_PATHS: + if name.startswith('sources/'): + relative = name[len('sources/'):] + require(relative in expected_sources, 'Unknown Linux runtime source') + path = source_root / relative + else: + path = runtime_root / name + metadata[path] = stat_identity(path) + data = sdk_custody.read_command_log_bytes(path) + require(stat_identity(path) == metadata[path], 'Linux runtime member metadata changed') + raw_total += len(data); encoded_total += 4 * ((len(data) + 2) // 3) + require(raw_total <= sdk_custody.LINUX_RUNTIME_MAX_BYTES + and encoded_total <= sdk_custody.LINUX_RUNTIME_MAX_BYTES, + 'Linux runtime packet exceeds evidence bound') + digest_value = hashlib.sha256(data).hexdigest() + if name.startswith('sources/'): + require(digest_value == expected_sources[relative], 'Linux runtime trusted source changed') + rows.append({'path': name, 'sha256': digest_value, 'byteLength': len(data), + 'base64': base64.b64encode(data).decode('ascii')}) + originals[path] = data + packet = {'schema': 'openprose.sdk-linux-runtime-evidence/1', 'platform': manifest['platform'], + 'sourceRevision': report['sourceRevision'], 'version': report['version'], + 'runtimeReport': runtime_report, 'members': rows} + encoded = json.dumps(packet, sort_keys=True).encode('utf-8') + b'\n' + require(0 < len(encoded) <= sdk_custody.LINUX_RUNTIME_MAX_BYTES, 'Linux runtime packet exceeds evidence bound') + evidence = {**report['evidence'], sdk_custody.LINUX_RUNTIME_EVIDENCE: + {'sha256': hashlib.sha256(encoded).hexdigest(), 'byteLength': len(encoded)}} + prospective = {**report, 'evidence': evidence} + require(sdk_custody.validate_linux_runtime_evidence(prospective, manifest, sdk_table, + lambda name: encoded if name == sdk_custody.LINUX_RUNTIME_EVIDENCE else None, + expected_sources=expected_sources) == runtime_report, 'Linux runtime packet proof differs') + for path, data in originals.items(): + require(sdk_custody.read_command_log_bytes(path) == data and stat_identity(path) == metadata[path], + 'Linux runtime original changed before packet write') + require(sdk_custody.read_linux_runtime_sources(source_root) == expected_sources, 'Linux runtime source snapshot changed') + sdk_custody.validate_linux_runtime_tree(runtime_root, runtime_report, sdk_table) + with target.open('xb') as stream: + stream.write(encoded) + require(sdk_custody.validate_linux_runtime_evidence(prospective, manifest, sdk_table, + lambda name: sdk_custody.read_command_log_bytes(output / name), + expected_sources=expected_sources) == runtime_report, 'Written Linux runtime packet proof differs') + require(all(stat_identity(path) == metadata[path] for path in originals), 'Linux runtime original metadata changed') + require(sdk_custody.read_linux_runtime_sources(source_root) == expected_sources, 'Linux runtime source snapshot changed') + sdk_custody.validate_linux_runtime_tree(runtime_root, runtime_report, sdk_table) + return evidence + + def environment(output, ambient): """Keep tool caches, but exclude credentials, user npm config and image overrides.""" env = {k: ambient[k] for k in ('PATH', 'DEVELOPER_DIR', 'SDKROOT') if k in ambient} @@ -300,9 +433,7 @@ def check(binary, runner, label, node=None): sdk_after = installed_sdk_identity(installed_helpers[0].parent, manifest) require(sdk_after == sdk_before, 'Installed npm SDK tree changed during probes') sdk_records.append({'surface': 'installed-npm', 'before': sdk_before, 'after': sdk_after}) - sdk_encoded = json.dumps(sdk_records, sort_keys=True).encode('utf-8') + b'\n' - require(len(sdk_encoded) <= 16 * 1024 * 1024, 'Installed SDK payload record exceeds evidence bound') - (logs / 'installed-sdk-payloads.json').write_bytes(sdk_encoded) + write_installed_sdk_payloads(logs / 'installed-sdk-payloads.json', sdk_records) report = {'schema': 'openprose.kernel-rc-build/1', 'version': version, 'sourceRevision': revision, 'platform': manifest['platform'], 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'signing': 'unsigned', 'modelCalls': 0, 'kernelFetches': 0, @@ -311,15 +442,12 @@ def check(binary, runner, label, node=None): 'evidence': {str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} for directory in (logs, package) for p in sorted(directory.rglob('*')) if p.is_file()}} report['evidence'].update(sdk_producer_evidence(output, sdk_output)) - if manifest['platform'].startswith('darwin-'): - producer_archive = next(package / item['path'] for item in manifest['artifacts'] if item['kind'] == 'standalone-archive') - sdk_custody.validate_sdk_producer_evidence(manifest, pub.read_sdk_archive(producer_archive, manifest), - report['evidence'], lambda name: (output / name).read_bytes()) - if sys.platform.startswith('linux'): - native_evidence = output / 'agents-sdk-runtime' - report['evidence'].update({str(p.relative_to(output)): {'sha256': digest(p), 'byteLength': p.stat().st_size} - for p in sorted(native_evidence.rglob('*')) - if p.is_file() and not p.is_symlink() and 'payload' not in p.relative_to(native_evidence).parts}) + producer_archive = next(package / item['path'] for item in manifest['artifacts'] if item['kind'] == 'standalone-archive') + sdk_table = pub.read_sdk_archive(producer_archive, manifest) + sdk_custody.validate_sdk_producer_evidence(manifest, sdk_table, + report['evidence'], lambda name: (output / name).read_bytes()) + report['evidence'] = producer_command_log_evidence(output, report, manifest) + report['evidence'] = producer_linux_runtime_evidence(output, report, manifest, sdk_table, source_root=ROOT) (output / 'build-report.json').write_text(json.dumps(report, indent=2, sort_keys=True) + '\n') return report diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index cd934ce9..31eebfce 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -174,6 +174,7 @@ def verify_kernel_rc(root: Path, expected_source: str, expected_version: str) -> and path.is_file() and not path.is_symlink() and path.stat().st_size == record['byteLength'] and pub.digest(path) == record['sha256'], 'Native evidence identity mismatch: ' + relative) + custody.validate_producer_command_logs(report, manifest, lambda relative: custody.read_command_log_bytes(root / relative)) for item in artifacts: name = item['path'] custody.require(isinstance(name, str) and SAFE_NAME.fullmatch(name) and '..' not in name, @@ -214,8 +215,12 @@ def verify_kernel_rc(root: Path, expected_source: str, expected_version: str) -> custody.validate_native(report, manifest, checks, hashes, launcher_hash) if 'agentsSdk' in manifest: custody.require(custody.SDK_PAYLOAD_EVIDENCE in evidence, 'Missing complete installed SDK payload evidence') - custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE), manifest, npm_table) + custody.validate_installed_sdk_payloads(pub.read_json(root / custody.SDK_PAYLOAD_EVIDENCE, max_bytes=16*1024*1024), manifest, npm_table) custody.validate_sdk_producer_evidence(manifest, npm_table, evidence, lambda relative: (root / relative).read_bytes()) + custody.validate_linux_runtime_evidence(report, manifest, npm_table if 'agentsSdk' in manifest else None, + lambda relative: custody.read_command_log_bytes(root / relative), + expected_sources=custody.read_linux_runtime_sources(Path(__file__).resolve().parents[2]) + if manifest['platform'].startswith('linux-') else None) # Original build trees retain these files; uploaded artifact trees retain the # built probes instead. Both bind to the exact packaged executable hashes. binaries = root / 'binaries' diff --git a/cli/ci/kernel_rc_evidence.py b/cli/ci/kernel_rc_evidence.py index bc9bbc45..5821305c 100644 --- a/cli/ci/kernel_rc_evidence.py +++ b/cli/ci/kernel_rc_evidence.py @@ -14,6 +14,336 @@ def require(ok, message): raise ValueError(message) +COMMAND_LOG_EVIDENCE = 'logs/producer-command-logs.json' +COMMAND_LOG_MAX_BYTES = 16 * 1024 * 1024 +COMMAND_LOG_MAC_MEMBERS = ( + 'logs/build-bun.log', 'logs/build-rust.log', 'logs/build-sdk.log', + 'logs/npm-install-cache.log', 'logs/npm-install.log', 'logs/package.log', + 'logs/rust-ad-hoc-sign.log', 'logs/rust-ad-hoc-verify.log', +) +COMMAND_LOG_LINUX_MEMBERS = ( + 'logs/build-bun.log', 'logs/build-rust.log', 'logs/npm-install-cache.log', + 'logs/npm-install.log', 'logs/package.log', +) +COMMAND_LOG_PLATFORMS = ('darwin-arm64', 'darwin-x64', 'linux-arm64-gnu', 'linux-x64-gnu') + + +def read_command_log_bytes(path): + """Read bounded regular owned bytes with stable no-follow descriptor custody.""" + import stat + path = Path(path) + require(path.is_absolute() and path.parent.resolve(strict=True) == path.parent, + 'Command log parent must be canonical') + before = path.lstat() + require(stat.S_ISREG(before.st_mode) and before.st_uid == os.getuid() + and 0 <= before.st_size <= COMMAND_LOG_MAX_BYTES, + 'Command log must be bounded owned regular bytes') + identity = lambda value: (value.st_dev, value.st_ino, value.st_mode, value.st_uid, + value.st_size, value.st_mtime_ns, value.st_ctime_ns) + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK) + try: + require(identity(os.fstat(descriptor)) == identity(before), 'Command log changed before read') + with os.fdopen(descriptor, 'rb', closefd=False) as stream: + data = stream.read(COMMAND_LOG_MAX_BYTES + 1) + require(len(data) == before.st_size and len(data) <= COMMAND_LOG_MAX_BYTES + and identity(os.fstat(descriptor)) == identity(before) + and identity(path.lstat()) == identity(before), 'Command log changed during read') + return data + finally: + os.close(descriptor) + + +def validate_producer_command_logs(report, manifest, read_bytes): + """Validate byte-preserving opaque logs; these bytes confer no execution authority.""" + import base64 + require(isinstance(report, dict) and isinstance(manifest, dict), 'Command log producer headers absent') + platform, source, version = report.get('platform'), report.get('sourceRevision'), report.get('version') + require(report.get('schema') == 'openprose.kernel-rc-build/1' and platform in COMMAND_LOG_PLATFORMS + and isinstance(source, str) and re.fullmatch('[0-9a-f]{40}', source) + and isinstance(version, str) and re.fullmatch('[A-Za-z0-9][A-Za-z0-9.+_-]{0,127}', version), + 'Invalid command log report identity') + require(manifest.get('schema') == 'openprose.local-release-manifest/1' and manifest.get('mode') == 'kernel-rc' + and manifest.get('platform') == platform and manifest.get('version') == version + and manifest.get('source') == {'revision': source, 'verification': 'matched-product-doctor'}, + 'Command log manifest identity differs') + names = COMMAND_LOG_MAC_MEMBERS if platform.startswith('darwin-') else COMMAND_LOG_LINUX_MEMBERS + evidence = report.get('evidence') + require(isinstance(evidence, dict) and COMMAND_LOG_EVIDENCE in evidence + and not set(names).intersection(evidence), 'Mandatory command log bundle missing or originals duplicated') + record = evidence[COMMAND_LOG_EVIDENCE] + require(isinstance(record, dict) and set(record) == {'sha256', 'byteLength'} + and isinstance(record['sha256'], str) and re.fullmatch('[0-9a-f]{64}', record['sha256']) + and type(record['byteLength']) is int and 0 < record['byteLength'] <= COMMAND_LOG_MAX_BYTES, + 'Command log bundle evidence unbounded or invalid') + raw = read_bytes(COMMAND_LOG_EVIDENCE) + require(isinstance(raw, bytes) and len(raw) == record['byteLength'] and len(raw) <= COMMAND_LOG_MAX_BYTES + and hashlib.sha256(raw).hexdigest() == record['sha256'], 'Command log bundle differs from producer evidence') + def pairs(rows): + value = {} + for key, item in rows: + require(key not in value, 'Duplicate command log JSON key') + value[key] = item + return value + try: + bundle = json.loads(raw, object_pairs_hook=pairs) + except (UnicodeDecodeError, json.JSONDecodeError) as error: + raise ValueError('Invalid command log bundle JSON') from error + require(isinstance(bundle, dict) and set(bundle) == {'schema', 'platform', 'sourceRevision', 'version', 'members'} + and bundle['schema'] == 'openprose.kernel-rc-command-logs/1' and bundle['platform'] == platform + and bundle['sourceRevision'] == source and bundle['version'] == version, + 'Command log bundle header differs') + members = bundle['members'] + require(isinstance(members, list) and len(members) == len(names), 'Command log member coverage differs') + result = {} + for expected, row in zip(names, members): + require(isinstance(row, dict) and set(row) == {'path', 'sha256', 'byteLength', 'base64'} + and row['path'] == expected and isinstance(row['sha256'], str) + and re.fullmatch('[0-9a-f]{64}', row['sha256']) and type(row['byteLength']) is int + and 0 <= row['byteLength'] <= COMMAND_LOG_MAX_BYTES and isinstance(row['base64'], str), + 'Command log member identity differs') + require(len(row['base64']) == 4 * ((row['byteLength'] + 2) // 3), 'Command log base64 length differs') + try: + data = base64.b64decode(row['base64'], validate=True) + except (ValueError, UnicodeEncodeError) as error: + raise ValueError('Invalid command log base64') from error + require(base64.b64encode(data).decode('ascii') == row['base64'] + and len(data) == row['byteLength'] and hashlib.sha256(data).hexdigest() == row['sha256'], + 'Command log member bytes differ') + result[expected] = data + return result + + +LINUX_RUNTIME_EVIDENCE = 'logs/sdk-linux-runtime.json' +LINUX_RUNTIME_MAX_BYTES = 16 * 1024 * 1024 +LINUX_RUNTIME_SOURCE_PATHS = ( + 'cli/ci/verify_agents_sdk_linux.py', 'cli/ci/agents-sdk-linux-runtime.lock.json', + 'cli/ci/build_agents_sdk_linux.py', 'cli/ci/sdk_native_inventory.py', +) +LINUX_RUNTIME_JOB_PATHS = tuple(sorted(('job/pull-runtime.log', 'job/inspect-runtime.log') + + tuple('job/' + name + suffix for name in ('base', 'version', 'imports', 'tools', 'libraries') for suffix in ('.log', '.sh')))) +LINUX_RUNTIME_MEMBER_PATHS = tuple(sorted(('runtime-report.json',) + LINUX_RUNTIME_JOB_PATHS + + tuple('sources/' + name for name in LINUX_RUNTIME_SOURCE_PATHS))) + + +def read_linux_runtime_sources(source_root): + """Hash trusted explicit caller source bytes, never source identities from a packet.""" + root = Path(source_root) + require(root.is_absolute() and root.resolve(strict=True) == root and root.is_dir(), + 'Linux runtime trusted source root must be canonical') + return {name: hashlib.sha256(read_command_log_bytes(root / name)).hexdigest() for name in LINUX_RUNTIME_SOURCE_PATHS} + + +def _runtime_json(raw): + def pairs(rows): + result = {} + for key, value in rows: + require(key not in result, 'Duplicate Linux runtime JSON key'); result[key] = value + return result + try: return json.loads(raw, object_pairs_hook=pairs) + except (UnicodeDecodeError, json.JSONDecodeError) as error: raise ValueError('Invalid Linux runtime JSON') from error + + +def _runtime_equal(left, right): + return json.dumps(left, sort_keys=True, separators=(',', ':')) == json.dumps(right, sort_keys=True, separators=(',', ':')) + + +def _runtime_script_contract(source, machine): + """Extract only authenticated literal constants; never evaluate or execute Python.""" + import ast + tree = ast.parse(source) + constants = {} + def literal(node): + if isinstance(node, ast.Constant) and type(node.value) in (str, int): return node.value + if isinstance(node, ast.Tuple): return tuple(literal(item) for item in node.elts) + raise ValueError('Runtime script contract must contain literals only') + for node in tree.body: + if isinstance(node, ast.Assign): + for name in node.targets: + if isinstance(name, ast.Name) and name.id in ('BASE', 'PROBES'): + require(name.id not in constants, 'Duplicate runtime script contract') + constants[name.id] = literal(node.value) + require(set(constants) == {'BASE', 'PROBES'} and isinstance(constants['BASE'], str) + and constants['PROBES'] == (('version', '--version', 5), ('imports', '--packaged-self-test', 30), + ('tools', '--packaged-tool-self-test', 30), ('libraries', '--packaged-library-test', 30)), + 'Linux runtime probe contract differs') + scripts = {'job/base.sh': constants['BASE'].format(machine=machine).encode()} + scripts.update({'job/' + name + '.sh': ('timeout --kill-after=1s ' + str(seconds) + + 's /source/prose-agents-sdk ' + flag + '\n').encode() for name, flag, seconds in constants['PROBES']}) + return scripts + + +def _runtime_sdk_trio(table): + prefix = '' if SDK_NAMES[0] in table['files'] else sdk_archive_prefix(table) + return {name: table['files'][prefix + name][0] for name in SDK_NAMES} + + +def _runtime_file_identity(path, maximum, mode): + """Stream an owned no-follow payload leaf; never duplicate helper bytes in a packet.""" + import stat + before = path.lstat() + require(stat.S_ISREG(before.st_mode) and before.st_uid == os.getuid() + and stat.S_IMODE(before.st_mode) == mode and 0 < before.st_size <= maximum, + 'Linux runtime copied payload metadata differs') + identity = lambda st: (st.st_dev, st.st_ino, st.st_mode, st.st_uid, st.st_size, st.st_mtime_ns, st.st_ctime_ns) + fd = os.open(path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK) + try: + require(identity(os.fstat(fd)) == identity(before), 'Linux runtime payload changed before read') + h = hashlib.sha256(); total = 0 + while True: + block = os.read(fd, 65536) + if not block: break + total += len(block); require(total <= maximum, 'Linux runtime payload exceeds bound'); h.update(block) + require(total == before.st_size and identity(os.fstat(fd)) == identity(before) + and identity(path.lstat()) == identity(before), 'Linux runtime payload changed during read') + return {'sha256': h.hexdigest(), 'byteLength': total} + finally: os.close(fd) + + +def validate_linux_runtime_tree(root, runtime_report, sdk_table): + """Check every physical successful verifier member and copied payload, without execution.""" + import stat + root = Path(root) + require(root.is_absolute() and root.resolve(strict=True) == root and root.is_dir(), 'Runtime tree must be canonical') + expected_files = {'runtime-report.json', *LINUX_RUNTIME_JOB_PATHS, *('payload/' + name for name in SDK_NAMES)} + expected_dirs = {'', 'job', 'job/home', 'job/docker-config', 'payload'} + observed_files = set(); observed_dirs = set(); before = {} + def identity(st): return (st.st_dev, st.st_ino, st.st_mode, st.st_uid, st.st_size, st.st_mtime_ns, st.st_ctime_ns) + def walk(directory, relative): + st = directory.lstat() + require(stat.S_ISDIR(st.st_mode) and st.st_uid == os.getuid(), 'Runtime tree directory is aliased or unowned') + observed_dirs.add(relative); before[relative] = identity(st) + fd = os.open(directory, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW) + try: + require(identity(os.fstat(fd)) == identity(st), 'Runtime directory changed before scan') + with os.scandir(fd) as entries: + for item in entries: + name = relative + '/' + item.name if relative else item.name + info = os.stat(item.name, dir_fd=fd, follow_symlinks=False) + if stat.S_ISDIR(info.st_mode): + require(name in expected_dirs, 'Unexpected Linux runtime directory') + walk(root / name, name) + require(identity((root / name).lstat()) == identity(info), 'Runtime child directory changed') + else: + require(stat.S_ISREG(info.st_mode) and name in expected_files and info.st_uid == os.getuid(), + 'Unexpected or aliased Linux runtime file') + observed_files.add(name); before[name] = identity(info) + require(identity(os.fstat(fd)) == identity(st), 'Runtime directory changed during scan') + finally: os.close(fd) + walk(root, '') + require(observed_files == expected_files and observed_dirs == expected_dirs, 'Linux runtime physical tree coverage differs') + trio = _runtime_sdk_trio(sdk_table) + expected_payload = runtime_report['inputs']['payload'] + require(isinstance(expected_payload, dict) and set(expected_payload) == set(SDK_NAMES), 'Runtime trio coverage differs') + for name, maximum in zip(SDK_NAMES, (256 * 1024**2, 2 * 1024**2, 8 * 1024**2)): + observed = _runtime_file_identity(root / 'payload' / name, maximum, 0o555 if name == SDK_NAMES[0] else 0o444) + require(observed == {'sha256': hashlib.sha256(trio[name]).hexdigest(), 'byteLength': len(trio[name])} + and observed['sha256'] == expected_payload[name], 'Linux runtime copied trio differs') + for relative in ('runtime-report.json', *LINUX_RUNTIME_JOB_PATHS): read_command_log_bytes(root / relative) + for relative, expected in before.items(): + st = (root / relative).lstat() + require(identity(st) == expected, 'Linux runtime tree changed during validation') + + +def validate_linux_runtime_evidence(report, manifest, sdk_table, read_bytes, *, expected_sources): + """Validate retained clean-runtime proof against trusted source and actual packaged SDK bytes.""" + import base64 + require(isinstance(report, dict) and isinstance(manifest, dict), 'Linux runtime producer headers absent') + platform, source, version = report.get('platform'), report.get('sourceRevision'), report.get('version') + require(report.get('schema') == 'openprose.kernel-rc-build/1' and platform in COMMAND_LOG_PLATFORMS + and isinstance(source, str) and re.fullmatch('[0-9a-f]{40}', source) + and isinstance(version, str) and re.fullmatch('[A-Za-z0-9][A-Za-z0-9.+_-]{0,127}', version) + and manifest.get('schema') == 'openprose.local-release-manifest/1' and manifest.get('mode') == 'kernel-rc' + and manifest.get('platform') == platform and manifest.get('version') == version + and manifest.get('source') == {'revision': source, 'verification': 'matched-product-doctor'}, + 'Linux runtime producer identity differs') + evidence = report.get('evidence') + require(isinstance(evidence, dict), 'Linux runtime evidence map absent') + if platform.startswith('darwin-'): + require(LINUX_RUNTIME_EVIDENCE not in evidence, 'Mac report contains Linux runtime proof'); return None + require(isinstance(expected_sources, dict) and set(expected_sources) == set(LINUX_RUNTIME_SOURCE_PATHS) + and all(isinstance(h, str) and re.fullmatch('[0-9a-f]{64}', h) for h in expected_sources.values()), + 'Trusted Linux runtime source map absent or invalid') + record = evidence.get(LINUX_RUNTIME_EVIDENCE) + require(isinstance(record, dict) and set(record) == {'sha256', 'byteLength'} and isinstance(record['sha256'], str) + and re.fullmatch('[0-9a-f]{64}', record['sha256']) and type(record['byteLength']) is int + and 0 < record['byteLength'] <= LINUX_RUNTIME_MAX_BYTES, 'Mandatory Linux runtime packet missing or unbounded') + raw = read_bytes(LINUX_RUNTIME_EVIDENCE) + require(isinstance(raw, bytes) and len(raw) == record['byteLength'] and len(raw) <= LINUX_RUNTIME_MAX_BYTES + and hashlib.sha256(raw).hexdigest() == record['sha256'], 'Linux runtime packet differs from evidence') + packet = _runtime_json(raw) + require(isinstance(packet, dict) and set(packet) == {'schema', 'platform', 'sourceRevision', 'version', 'runtimeReport', 'members'} + and packet['schema'] == 'openprose.sdk-linux-runtime-evidence/1' and packet['platform'] == platform + and packet['sourceRevision'] == source and packet['version'] == version, 'Linux runtime packet header differs') + rows = packet['members'] + require(isinstance(rows, list) and len(rows) == len(LINUX_RUNTIME_MEMBER_PATHS), 'Linux runtime packet coverage differs') + files = {} + for name, row in zip(LINUX_RUNTIME_MEMBER_PATHS, rows): + require(isinstance(row, dict) and set(row) == {'path', 'sha256', 'byteLength', 'base64'} and row['path'] == name + and isinstance(row['sha256'], str) and re.fullmatch('[0-9a-f]{64}', row['sha256']) + and type(row['byteLength']) is int and 0 <= row['byteLength'] <= LINUX_RUNTIME_MAX_BYTES + and isinstance(row['base64'], str) and len(row['base64']) == 4 * ((row['byteLength'] + 2) // 3), + 'Linux runtime packet member differs') + try: data = base64.b64decode(row['base64'], validate=True) + except (ValueError, UnicodeEncodeError) as error: raise ValueError('Invalid Linux runtime base64') from error + require(base64.b64encode(data).decode() == row['base64'] and len(data) == row['byteLength'] + and hashlib.sha256(data).hexdigest() == row['sha256'], 'Linux runtime member bytes differ') + files[name] = data + for name, digest in expected_sources.items(): + require(hashlib.sha256(files['sources/' + name]).hexdigest() == digest, 'Linux runtime source differs from trusted checkout') + runtime = packet['runtimeReport']; keys = {'schema', 'platform', 'runtime', 'inputs', 'results', 'version', 'modelCalls', + 'networkUsed', 'networkUsageScope', 'preparationNetworkEnabled', 'cpuFloorQualified', 'qualification', 'publicationAuthorized'} + require(isinstance(runtime, dict) and set(runtime) == keys and _runtime_equal(_runtime_json(files['runtime-report.json']), runtime) + and runtime['schema'] == 'openprose.sdk-linux-clean-runtime/1' and runtime['platform'] == platform + and runtime['version'] == 'prose-agents-sdk 0.1.0' and type(runtime['modelCalls']) is int and runtime['modelCalls'] == 0 + and runtime['networkUsed'] is False and runtime['networkUsageScope'] == 'runtime-probes-only' + and runtime['preparationNetworkEnabled'] is True and runtime['cpuFloorQualified'] is False + and runtime['qualification'] == 'native-clean-glibc-2.34-only' and runtime['publicationAuthorized'] is False, + 'Linux runtime structured claims differ') + lock = _runtime_json(files['sources/cli/ci/agents-sdk-linux-runtime.lock.json']) + require(isinstance(lock, dict) and set(lock) == {'schema', 'platforms'} and lock['schema'] == 'openprose.sdk-linux-runtime-inputs/1' + and isinstance(lock['platforms'], dict) and set(lock['platforms']) == {'linux-x64-gnu', 'linux-arm64-gnu'}, + 'Linux runtime trusted lock differs') + machine, oci = ('x86_64', 'amd64') if platform == 'linux-x64-gnu' else ('aarch64', 'arm64') + target = lock['platforms'][platform] + require(isinstance(target, dict) and set(target) == {'machine', 'dockerPlatform', 'runtime'} and target['machine'] == machine + and target['dockerPlatform'] == 'linux/' + oci and isinstance(target['runtime'], dict) + and set(target['runtime']) == {'image', 'configSha256', 'metadataUrl', 'indexDigest'} + and re.fullmatch(r'quay.io/almalinuxorg/9-minimal@sha256:[0-9a-f]{64}', target['runtime']['image']) + and re.fullmatch('[0-9a-f]{64}', target['runtime']['configSha256']) + and re.fullmatch('sha256:[0-9a-f]{64}', target['runtime']['indexDigest']) + and isinstance(target['runtime']['metadataUrl'], str) and target['runtime']['metadataUrl'].startswith('https://') + and runtime['runtime'] == target['runtime'], 'Linux runtime image target differs') + inspect = _runtime_json(files['job/inspect-runtime.log']) + require(isinstance(inspect, dict) and inspect.get('Id') == 'sha256:' + target['runtime']['configSha256'] + and inspect.get('Os') == 'linux' and inspect.get('Architecture') == oci + and isinstance(inspect.get('RepoDigests'), list) and target['runtime']['image'] in inspect['RepoDigests'], + 'Linux runtime inspected image differs') + validate_sdk_archive_table(manifest, sdk_table) + trio = _runtime_sdk_trio(sdk_table); receipt = _runtime_json(trio[SDK_NAMES[1]]) + helper = trio[SDK_NAMES[0]] + require(helper[:6] == b'\x7fELF\x02\x01' and len(helper) >= 20 and int.from_bytes(helper[18:20], 'little') == (62 if machine == 'x86_64' else 183), 'Linux runtime helper ELF architecture differs') + inputs = runtime['inputs'] + slots = {'driverSha256': LINUX_RUNTIME_SOURCE_PATHS[0], 'lockSha256': LINUX_RUNTIME_SOURCE_PATHS[1], + 'lifecycleSha256': LINUX_RUNTIME_SOURCE_PATHS[2], 'inventorySha256': LINUX_RUNTIME_SOURCE_PATHS[3]} + require(isinstance(inputs, dict) and set(inputs) == {'payload', *slots} and isinstance(inputs['payload'], dict) + and inputs['payload'] == {name: hashlib.sha256(data).hexdigest() for name, data in trio.items()} + and all(inputs[slot] == expected_sources[name] for slot, name in slots.items()), 'Linux runtime input identities differ') + snapshot = receipt['linuxBuildSourceSnapshot'] + require(snapshot['driverSha256'] == inputs['lifecycleSha256'] + and snapshot['sources']['cli/ci/sdk_native_inventory.py'] == inputs['inventorySha256'], + 'Linux runtime source differs from frozen construction') + results = runtime['results'] + expected = {'imports': receipt['selfTest'], 'tools': receipt['toolSelfTest'], 'libraries': receipt['linuxLibraries']} + require(isinstance(results, dict) and _runtime_equal(results, expected) + and all(_runtime_equal(_runtime_json(files['job/' + name + '.log']), value) for name, value in expected.items()), + 'Linux runtime probe differs from frozen receipt') + require(files['job/base.log'] == ('glibc=2.34\narchitecture=' + machine + '\nnoBuildTools=true\n').encode() + and files['job/version.log'] == b'prose-agents-sdk 0.1.0\n', 'Linux runtime base or cold version differs') + scripts = _runtime_script_contract(files['sources/cli/ci/verify_agents_sdk_linux.py'], machine) + require(all(files[name] == content for name, content in scripts.items()), 'Linux runtime probe script contract differs') + return runtime + + def asset_name(platform, relative, artifacts): require(isinstance(relative, str), 'Invalid evidence path') path = PurePosixPath(relative) @@ -109,7 +439,7 @@ def validate_sdk_members(manifest, members, prefix, platform, *, historical=None require(sdk['path'] == SDK_NAMES[0] and sdk['python'] == '3.10.20' and sdk['pyinstaller'] == '6.22.3' and sdk['version'] == '0.1.0' and sdk['discovery'] == 'canonical-cli-sibling' - and sdk['selfTest'] == SDK_IMPORT_TEST and sdk['toolSelfTest'] == SDK_TOOL_TEST, + and _runtime_equal(sdk['selfTest'], SDK_IMPORT_TEST) and _runtime_equal(sdk['toolSelfTest'], SDK_TOOL_TEST), 'Packaged SDK policy or tool qualification differs') require(all(prefix + name in members for name in SDK_NAMES), 'Missing packaged SDK siblings') if table is not None: @@ -129,8 +459,8 @@ def validate_sdk_members(manifest, members, prefix, platform, *, historical=None require(receipt.get('schema') == 'openprose.agents-sdk-build/1' and receipt.get('platform') == os_name and receipt.get('architecture') == architecture and receipt.get('python') == sdk['python'] and receipt.get('pyinstaller') == sdk['pyinstaller'] - and receipt.get('helper') == {k: sdk[k] for k in ('path', 'byteLength', 'sha256')} - and receipt.get('selfTest') == SDK_IMPORT_TEST and receipt.get('toolSelfTest') == SDK_TOOL_TEST + and _runtime_equal(receipt.get('helper'), {k: sdk[k] for k in ('path', 'byteLength', 'sha256')}) + and _runtime_equal(receipt.get('selfTest'), SDK_IMPORT_TEST) and _runtime_equal(receipt.get('toolSelfTest'), SDK_TOOL_TEST) and type(receipt.get('modelCalls')) is int and receipt['modelCalls'] == 0 and receipt.get('publicationAuthorized') is False, 'Packaged SDK build receipt identity differs') @@ -140,8 +470,8 @@ def validate_sdk_members(manifest, members, prefix, platform, *, historical=None and all(isinstance(v, str) and re.fullmatch(r'[0-9a-f]{64}', v) for v in sources.values()) and sources['harnesses/agents-sdk/requirements-build.txt'] == sdk['dependencyLockSha256'], 'Packaged SDK source/lock binding differs') - require(receipt.get('notices') == {'path': SDK_NAMES[2], 'byteLength': len(notices), - 'sha256': sdk['noticesSha256']}, 'SDK notices receipt differs') + require(_runtime_equal(receipt.get('notices'), {'path': SDK_NAMES[2], 'byteLength': len(notices), + 'sha256': sdk['noticesSha256']}), 'SDK notices receipt differs') dependencies = receipt.get('dependencies') require(isinstance(dependencies, list) and dependencies, 'SDK dependency receipt is missing') seen = set() @@ -423,6 +753,7 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): require(manifest.get('embeddedDiagnosticImage') == preflight.get('embeddedDiagnosticImage') and manifest.get('kernelPolicy') == preflight.get('kernelPolicy'), 'Native manifest kernel policy differs from qualification') artifact_names = {a['path'] for a in manifest.get('artifacts', [])} evidence = report.get('evidence', {}) + validate_producer_command_logs(report, manifest, lambda relative: read_command_log_bytes(root / asset_name(platform, relative, artifact_names))) require(isinstance(evidence, dict) and set(CHECK_PATHS + SDK_PROBES).union({'package/release-manifest.json',SDK_PAYLOAD_EVIDENCE}).issubset(evidence), 'Required structured evidence is missing') seen = set() for relative, record in evidence.items(): @@ -448,6 +779,7 @@ def verify_platform_evidence(plan, root, platform, report_name, binary_hashes): def sdk_archive(name): table=pub.read_sdk_archive(root/name,manifest);tables.append(table);return table validate_sdk_archives(manifest,sdk_archive) + validate_linux_runtime_evidence(report, manifest, tables[0], lambda relative: read_command_log_bytes(root / asset_name(platform, relative, artifact_names)), expected_sources=read_linux_runtime_sources(Path(__file__).resolve().parents[2]) if platform.startswith('linux-') else None) validate_installed_sdk_payloads(pub.read_json(root/asset_name(platform,SDK_PAYLOAD_EVIDENCE,artifact_names),max_bytes=16*1024*1024),manifest,tables[0]) validate_sdk_producer_evidence(manifest,tables[0],evidence,lambda relative:(root/asset_name(platform,relative,artifact_names)).read_bytes()) for relative in SDK_PROBES: diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index 730e12ea..0d2d499e 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -364,6 +364,44 @@ def hash_value(digest): return value, sha(data), texts +# PBS 20260807 generator unions both downloads' licenses for library name "z"; +# Unix build-zlib.sh selects zlib. These inspected pins alone admit the erratum: +# https://github.com/astral-sh/python-build-standalone/tree/00c8a06113f11220667c3bcf5fab1672ff9e78ef +PBS_ZLIB_PARENTS = { + 'x86_64-unknown-linux-gnu': ('9e57324fd5e25f485fa5c8c587a2fe02f34a869092abd463e8e816c3cb1d48f2', + '181c0b9324c9f7c3bcd19464a8e49153bf086016c79330792e2b3dc59e22b359'), + 'aarch64-unknown-linux-gnu': ('9201b2d72f8ea0250d594716cfefd8cd85f7b21a8741ba473fafcde58776c619', + '277d87ea8c5603e11d5793b4cb3f59c06cc950a136fa9cbb15f3ae0a2f7264fa'), +} + + +def python_metadata_errata(archive, metadata_sha, declarations): + """Only the inspected PBS metadata overclaim has an effective-path mapping.""" + require(isinstance(declarations, list) and all(isinstance(row, dict) for row in declarations), + 'Invalid Python extension license declarations') + recognized = archive['sha256'] in {row[0] for row in PBS_ZLIB_PARENTS.values()} or metadata_sha in { + row[1] for row in PBS_ZLIB_PARENTS.values()} + if not recognized: return [] + pair = PBS_ZLIB_PARENTS.get(archive['targetTriple']) + require(pair == (archive['sha256'], metadata_sha), 'Mismatched recognized Python metadata parent') + original = ['licenses/LICENSE.zlib-ng.txt', 'licenses/LICENSE.zlib.txt'] + zlib = [row for row in declarations if row.get('name') == 'zlib'] + require(zlib == [{'name': 'zlib', 'licenses': ['Zlib'], 'licensePaths': original}], + 'Recognized Python metadata requires unchanged unique zlib declaration') + return [{'schema': 'openprose.pbs-license-metadata-erratum/1', + 'archiveSha256': pair[0], 'metadataSha256': pair[1], 'targetTriple': archive['targetTriple'], + 'extension': {'name': 'zlib', 'variant': 'default', + 'links': [{'name': 'z', 'path_static': 'build/lib/libz.a'}], 'licenses': ['Zlib'], + 'originalLicensePaths': original, 'effectiveLicensePaths': ['licenses/LICENSE.zlib.txt']}, + 'retainedLicense': {'path': 'licenses/LICENSE.zlib.txt', 'byteLength': 995, + 'sha256': '818922b2620f12801a12bf78e399644a30990e66824abd8ca8ec24d451d6f92c'}, + 'upstreamCommit': '00c8a06113f11220667c3bcf5fab1672ff9e78ef', + 'sourceSha256s': { + 'pythonbuild/utils.py': '561a16f50f2402a702153eeedab00be6590b79d1c6fd46b65c7de5f40b077157', + 'pythonbuild/downloads.py': 'eaec25f19f4e473825512528bb4130ef2b6ec66881b12b80885888bcbb2156f2', + 'cpython-unix/build-zlib.sh': '58e58a24f79cec8ecfc250adcce8ec829753e59cdfd54cf4de4d37018c45c0f9'}}] + + def python_origin(value): distribution = value['pythonDistribution']; root = Path(distribution['root']) data = read_file(distribution['metadataPath'], MAX_METADATA); info = load_json(data) @@ -392,6 +430,23 @@ def python_origin(value): require(paths, 'Declared Python native license lacks corresponding text') license_paths.update(paths) static.append({'name': name, 'licenses': declared, 'licensePaths': sorted(paths)}) + errata = python_metadata_errata(value['pythonArchive'], sha(data), static) + if errata: + extension = errata[0]['extension']; variants = extensions['zlib'] + require(len(variants) == 1 and variants[0].get('variant') == extension['variant'] + and variants[0].get('links') == extension['links'] + and variants[0].get('licenses') == extension['licenses'] + and variants[0].get('license_paths') == extension['originalLicensePaths'], + 'Recognized Python zlib metadata variant differs') + omitted = root / 'licenses/LICENSE.zlib-ng.txt' + try: omitted.lstat() + except FileNotFoundError: pass + else: raise ValueError('Recognized Python erratum requires absent zlib-ng license leaf') + license_paths.remove('licenses/LICENSE.zlib-ng.txt') + retained = errata[0]['retainedLicense']; retained_path = root / retained['path'] + require(retained_path.resolve() == root.resolve() / retained['path'], 'Python erratum license alias forbidden') + require(record(retained['path'], read_file(retained_path, 1024 * 1024)) == retained, + 'Python erratum retained zlib license differs') licenses = [] for relative in sorted(license_paths): safe_path(relative); text = read_file(root / relative, 1024 * 1024) @@ -407,7 +462,7 @@ def python_origin(value): require(core_path.is_relative_to(root.resolve()) and sha(read_file(core_path)) in owned, 'Python shared core bytes are absent') # These are full-provider declarations, not proof that each extension shipped. return {'archive': value['pythonArchive'], 'metadataSha256': sha(data), 'licenses': licenses, - 'declaredExtensionLicenses': static}, owned + 'declaredExtensionLicenses': static, 'metadataErrata': errata}, owned def wheel_origins(packages, distributions): @@ -492,7 +547,7 @@ def file_row(row, maximum=1024 * 1024): for row in supplier['licenses']: keys(row, ('path', 'sha256', 'byteLength', 'packagePath'), 'supplier license') file_row({k: row[k] for k in ('path', 'sha256', 'byteLength')}) - python = origins['python']; keys(python, ('archive', 'metadataSha256', 'licenses', 'declaredExtensionLicenses'), 'Python origin') + python = origins['python']; keys(python, ('archive', 'metadataSha256', 'licenses', 'declaredExtensionLicenses', 'metadataErrata'), 'Python origin') keys(python['archive'], ('url', 'sha256', 'byteLength', 'targetTriple', 'metadataUrl'), 'Python origin archive') digest(python['metadataSha256']); digest(python['archive']['sha256']) require(python['archive']['targetTriple'] == arch + '-unknown-linux-gnu' and @@ -501,11 +556,21 @@ def file_row(row, maximum=1024 * 1024): require(isinstance(python['licenses'], list) and python['licenses'], 'Missing Python license evidence') for row in python['licenses']: file_row(row) require(isinstance(python['declaredExtensionLicenses'], list), 'Invalid Python static dependencies') + expected_errata = python_metadata_errata(python['archive'], python['metadataSha256'], python['declaredExtensionLicenses']) + require(isinstance(python['metadataErrata'], list) and + json.dumps(python['metadataErrata'], sort_keys=True) == json.dumps(expected_errata, sort_keys=True), + 'Python metadata errata differ from recognized mapping') + if expected_errata: + retained = expected_errata[0]['retainedLicense'] + require([row for row in python['licenses'] if row['path'] == retained['path']] == [retained], + 'Python erratum retained license evidence differs') for row in python['declaredExtensionLicenses']: keys(row, ('name', 'licenses', 'licensePaths'), 'Python static dependency') require(isinstance(row['name'], str) and row['name'] and isinstance(row['licenses'], list) and row['licenses'] and isinstance(row['licensePaths'], list) and row['licensePaths'] and - all(path in {r['path'] for r in python['licenses']} for path in row['licensePaths']), + all(path in {r['path'] for r in python['licenses']} or + (bool(expected_errata) and row['name'] == 'zlib' and path == 'licenses/LICENSE.zlib-ng.txt') + for path in row['licensePaths']), 'Unbound Python static dependency licenses') wheels = origins['wheels']; require(isinstance(wheels, list) and wheels == sorted(wheels, key=lambda r: r['name']), 'Unsorted native wheel origins') names = set() diff --git a/cli/ci/test_assemble_kernel_rc.py b/cli/ci/test_assemble_kernel_rc.py index 00b0a97e..99d0c1dc 100644 --- a/cli/ci/test_assemble_kernel_rc.py +++ b/cli/ci/test_assemble_kernel_rc.py @@ -1,4 +1,5 @@ import hashlib +import base64 import io import json from pathlib import Path @@ -25,6 +26,9 @@ def linux_receipt_fixture(platform='linux-x64-gnu'): snapshot = value['sourceSnapshot'] snapshot['sources']['harnesses/agents-sdk/run.py'] = 'f' * 64 snapshot['sources']['harnesses/agents-sdk/requirements-build.txt'] = 'e' * 64 + # Source custody is real; the native bytes/results remain synthetic. + snapshot['driverSha256'] = p.digest(package.CLI.parent / 'cli/ci/build_agents_sdk_linux.py') + snapshot['sources']['cli/ci/sdk_native_inventory.py'] = p.digest(package.CLI.parent / 'cli/ci/sdk_native_inventory.py') return {'linuxBuildInputSha256': native.sha(('synthetic-native-input-' + platform).encode()), 'linuxBuildSourceSnapshot': snapshot, 'nativeDependencies': {'libraries': rows, 'symbolClosureVerified': True, 'origins': origins, 'libgccSelection': {**native.record('libgcc_s.so.1', library.read_bytes()), 'analysisTocSha256': 'a' * 64}}, @@ -35,6 +39,9 @@ def linux_receipt_fixture(platform='linux-x64-gnu'): def sdk_fixture(platform): """Hermetic bytes with production-shaped custody; never executable release evidence.""" helper = ('fixture-sdk-' + platform).encode(); notices = b'fixture-notices' + if platform.startswith('linux'): + from test_sdk_native_inventory import elf + helper = elf(machine=62 if platform == 'linux-x64-gnu' else 183) sha = lambda data: hashlib.sha256(data).hexdigest() sdk = {'path': 'prose-agents-sdk', 'byteLength': len(helper), 'sha256': sha(helper), 'noticesSha256': sha(notices), 'python': '3.10.20', 'pyinstaller': '6.22.3', 'version': '0.1.0', @@ -63,7 +70,7 @@ class AssemblyTests(unittest.TestCase): def setUp(self): self.temp = tempfile.TemporaryDirectory() self.addCleanup(self.temp.cleanup) - self.root = Path(self.temp.name) + self.root = Path(self.temp.name).resolve() self.roots = [] self.source = 'a' * 40 self.version = '0.15.0-rc.1' @@ -118,6 +125,16 @@ def setUp(self): (root / relative).write_bytes(encoded) evidence = {str(f.relative_to(root)): {'sha256': p.digest(f), 'byteLength': f.stat().st_size} for directory in (output, logs) for f in directory.iterdir()} report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, 'version': self.version, 'sourceRevision': self.source, 'imageSource': 'published-on-run', 'testSeamsEnabled': False, 'qualification': 'offline-install-only', 'publicationAuthorized': False, 'modelCalls': 0, 'kernelFetches': 0, 'checks': [{'name': n, 'status': 'passed'} for n in ('built-bun','built-rust','installed-bun','installed-rust','installed-npm')], 'evidence': evidence} + from test_kernel_rc_evidence import producer_command_log_fixture + for relative, encoded in producer_command_log_fixture(report).items(): + (root / relative).write_bytes(encoded) + report['evidence'][relative] = {'sha256': p.digest(root / relative), 'byteLength': len(encoded)} + if platform.startswith('linux'): + from test_kernel_rc_evidence import linux_runtime_fixture + relative = custody.LINUX_RUNTIME_EVIDENCE + encoded = linux_runtime_fixture(report, manifest, sdk_members, package.CLI.parent)[relative] + (root / relative).write_bytes(encoded) + report['evidence'][relative] = {'sha256': p.digest(root / relative), 'byteLength': len(encoded)} (root / 'build-report.json').write_text(json.dumps(report)) self.evidence = 'https://github.com/openprose/example-evidence/tree/' + 'b'*40 + '/test' @@ -153,6 +170,121 @@ def test_generated_packages_assemble_without_claiming_live_qualification(self): with self.assertRaisesRegex(ValueError, 'Kernel qualification'): p.load_plan(self.root/'assembly/publication-plan.json') + def test_command_bundle_is_required_and_rehashed_raw_poison_refuses_before_output(self): + root = self.roots[0]; report_path = root / 'build-report.json' + report = json.loads(report_path.read_bytes()); path = root / custody.COMMAND_LOG_EVIDENCE + original = path.read_bytes(); del report['evidence'][custody.COMMAND_LOG_EVIDENCE] + report_path.write_text(json.dumps(report)) + missing = self.root / 'missing-command-bundle' + with self.assertRaises(ValueError): a.assemble(self.roots, missing, self.evidence) + self.assertFalse(missing.exists()) + bundle = json.loads(original); bundle['members'][0]['base64'] = 'eA==' + path.write_text(json.dumps(bundle)); report['evidence'][custody.COMMAND_LOG_EVIDENCE] = { + 'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + poisoned = self.root / 'poisoned-command-bundle' + with self.assertRaises(ValueError): a.assemble(self.roots, poisoned, self.evidence) + self.assertFalse(poisoned.exists()) + + def test_actual_cohort_reserves_all_live_names_and_refuses_before_copy(self): + baseline = a.assemble(self.roots, self.root / 'budget-baseline', self.evidence) + reserved_live = len(custody.LIVE_ROLES) * 2 + self.assertEqual(reserved_live, 16) + available = 128 - reserved_live - len(baseline['artifacts']) + self.assertGreater(available, 0) + root = self.roots[0]; report_path = root / 'build-report.json'; report = json.loads(report_path.read_bytes()) + for index in range(available): + relative = 'logs/extra-opaque-' + str(index) + '.log'; path = root / relative + path.write_bytes(b'unknown log retained separately') + report['evidence'][relative] = {'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + offline = a.assemble(self.roots, self.root / 'budget-boundary', self.evidence) + self.assertEqual(len(offline['artifacts']) + reserved_live, 128) + live_path = self.root / 'budget-live.json'; live_path.write_text(json.dumps(self.live_report())) + complete = a.assemble(self.roots, self.root / 'budget-with-live', self.evidence, live_path) + self.assertEqual(len(complete['artifacts']), 128) + self.assertEqual(len([row for row in complete['artifacts'] if row['name'].startswith('live-')]), 16) + names = {row['name'] for row in complete['artifacts']} + for platform in p.PLATFORMS: + for relative in (*custody.CHECK_PATHS, *custody.SDK_PROBES, + custody.SDK_PAYLOAD_EVIDENCE, custody.COMMAND_LOG_EVIDENCE): + self.assertIn(platform + '-logs-' + Path(relative).name, names) + if platform.startswith('darwin'): + for relative in (custody.SDK_SIGNATURE_EVIDENCE, custody.SDK_COLLECT_EVIDENCE): + self.assertIn(platform + '-logs-' + Path(relative).name, names) + else: + self.assertIn(platform + '-logs-' + Path(custody.LINUX_RUNTIME_EVIDENCE).name, names) + relative = 'logs/one-too-many-opaque.log'; path = root / relative; path.write_bytes(b'kept') + report['evidence'][relative] = {'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + output = self.root / 'over-budget-before-copy' + with self.assertRaises(ValueError): a.assemble(self.roots, output, self.evidence) + self.assertFalse(output.exists()) + + def test_linux_runtime_packet_is_mandatory_before_output(self): + for platform in ('linux-x64-gnu', 'linux-arm64-gnu'): + root = next(root for root in self.roots if root.name == platform) + path = root / 'build-report.json'; original = path.read_bytes(); report = json.loads(original) + del report['evidence'][custody.LINUX_RUNTIME_EVIDENCE]; path.write_text(json.dumps(report)) + output = self.root / ('missing-runtime-' + platform) + with self.assertRaises(ValueError): a.assemble(self.roots, output, self.evidence) + self.assertFalse(output.exists()); path.write_bytes(original) + + def test_rehashed_linux_runtime_source_result_script_or_record_poison_refuses(self): + root = next(root for root in self.roots if root.name == 'linux-x64-gnu') + path = root / custody.LINUX_RUNTIME_EVIDENCE; original = path.read_bytes() + report_path = root / 'build-report.json'; original_report = report_path.read_bytes() + def replace(bundle, name, data): + row = next(row for row in bundle['members'] if row['path'] == name) + row.update(base64=base64.b64encode(data).decode(), sha256=hashlib.sha256(data).hexdigest(), byteLength=len(data)) + for poison in ('source', 'result', 'script', 'member', 'oversize'): + bundle = json.loads(original) + if poison == 'source': replace(bundle, 'sources/cli/ci/sdk_native_inventory.py', b'wrong trusted source') + elif poison == 'result': + bundle['runtimeReport']['results']['imports']['modelCalls'] = 1 + replace(bundle, 'runtime-report.json', json.dumps(bundle['runtimeReport']).encode()) + replace(bundle, 'job/imports.log', json.dumps(bundle['runtimeReport']['results']['imports']).encode()) + elif poison == 'script': replace(bundle, 'job/version.sh', b'timeout --kill-after=1s 6s /source/prose-agents-sdk --version\n') + elif poison == 'member': bundle['members'].pop() + encoded = json.dumps(bundle).encode() + if poison == 'oversize': encoded += b' ' * (16 * 1024 * 1024 + 1 - len(encoded)) + path.write_bytes(encoded); report = json.loads(original_report) + report['evidence'][custody.LINUX_RUNTIME_EVIDENCE] = {'sha256': p.digest(path), 'byteLength': len(encoded)} + report_path.write_text(json.dumps(report)); output = self.root / ('poison-runtime-' + poison) + with self.subTest(poison=poison), self.assertRaises(ValueError): a.assemble(self.roots, output, self.evidence) + self.assertFalse(output.exists()) + path.write_bytes(original); report_path.write_bytes(original_report) + + def resize_installed_sdk_evidence(self, byte_length): + root = next(root for root in self.roots if root.name == 'darwin-arm64') + path = root / custody.SDK_PAYLOAD_EVIDENCE + original = path.read_bytes() + self.assertLess(len(original), byte_length) + path.write_bytes(original + b' ' * (byte_length - len(original))) + self.assertEqual(path.stat().st_size, byte_length) + self.assertEqual(json.loads(path.read_bytes()), json.loads(original)) + report_path = root / 'build-report.json' + report = json.loads(report_path.read_bytes()) + report['evidence'][custody.SDK_PAYLOAD_EVIDENCE] = { + 'sha256': p.digest(path), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + return root, path + + def test_complete_sdk_evidence_above_generic_json_limit_assembles(self): + _, path = self.resize_installed_sdk_evidence(1024 * 1024 + 128) + with self.assertRaisesRegex(ValueError, 'Invalid JSON file'): + p.read_json(path) + plan = a.assemble(self.roots, self.root / 'large-sdk-assembly', self.evidence) + self.assertEqual(plan['qualification']['status'], 'development') + self.assertTrue((self.root / 'large-sdk-assembly/publication-plan.json').is_file()) + + def test_complete_sdk_evidence_above_specific_json_limit_refuses_assembly(self): + self.resize_installed_sdk_evidence(16 * 1024 * 1024 + 1) + output = self.root / 'oversize-sdk-assembly' + with self.assertRaisesRegex(ValueError, 'Invalid JSON file'): + a.assemble(self.roots, output, self.evidence) + self.assertFalse(output.exists()) + def test_aggregate_checksums_bind_only_all_install_archives(self): output = self.root / 'checksums' plan = a.assemble(self.roots, output, self.evidence) diff --git a/cli/ci/test_build_kernel_rc.py b/cli/ci/test_build_kernel_rc.py index 1555c502..c709b233 100644 --- a/cli/ci/test_build_kernel_rc.py +++ b/cli/ci/test_build_kernel_rc.py @@ -218,6 +218,132 @@ def test_mutated_support_is_rejected_before_installation(self): class CompleteSdkProducerEvidenceTests(unittest.TestCase): + def installed_records(self, platform): + from test_kernel_rc_evidence import sdk_fixture + sdk, table = sdk_fixture(platform) + table = {kind: {'release/' + name: value for name, value in rows.items()} + for kind, rows in table.items()} + table['files']['release/prose'] = (b'nonexecuted CLI fixture', 0o755) + prefix = rc.sdk_custody.sdk_archive_prefix(table) + identity = {name: {'sha256': hashlib.sha256(table['files'][prefix + name][0]).hexdigest(), + 'byteLength': len(table['files'][prefix + name][0])} + for name in rc.sdk_custody.SDK_NAMES} + if platform.startswith('darwin'): + payload = json.loads(table['files'][prefix + 'agents-sdk-build.json'][0])['payload'] + identity['supportTree'] = {'payload': payload, + 'sha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'byteLength': payload['totalRegularBytes'], 'entryCount': len(payload['entries'])} + return {'platform': platform, 'agentsSdk': sdk}, table, [ + {'surface': surface, 'before': copy.deepcopy(identity), 'after': copy.deepcopy(identity)} + for surface in ('installed-rust', 'installed-bun', 'installed-npm')] + + def test_actual_producer_writer_canonicalizes_shuffled_surfaces_for_custody(self): + for platform in ('darwin-arm64', 'darwin-x64'): + with self.subTest(platform=platform), tempfile.TemporaryDirectory() as directory: + manifest, table, records = self.installed_records(platform) + with self.assertRaisesRegex(ValueError, 'surface identity differs'): + rc.sdk_custody.validate_installed_sdk_payloads(records, manifest, table) + path = Path(directory) / 'installed-sdk-payloads.json' + rc.write_installed_sdk_payloads(path, records) + recorded = json.loads(path.read_bytes()) + self.assertEqual([r['surface'] for r in recorded], + ['installed-bun', 'installed-rust', 'installed-npm']) + self.assertEqual(rc.sdk_custody.validate_installed_sdk_payloads(recorded, manifest, table), recorded) + self.assertEqual({r['surface']: r for r in records}, {r['surface']: r for r in recorded}) + + def test_producer_refuses_duplicate_missing_unknown_or_changed_surfaces_before_write(self): + _, _, original = self.installed_records('darwin-arm64') + poisons = [original[:2], [original[0], original[0], original[2]]] + unknown = copy.deepcopy(original); unknown[0]['surface'] = 'installed-other'; poisons.append(unknown) + changed = copy.deepcopy(original); changed[0]['after']['prose-agents-sdk']['byteLength'] += 1; poisons.append(changed) + for records in poisons: + with self.subTest(records=records), tempfile.TemporaryDirectory() as directory: + path = Path(directory) / 'installed-sdk-payloads.json' + with self.assertRaises(ValueError): + rc.write_installed_sdk_payloads(path, records) + self.assertFalse(path.exists()) + + def command_log_fixture(self, output, platform='darwin-arm64'): + (output / 'logs').mkdir() + names = (rc.sdk_custody.COMMAND_LOG_MAC_MEMBERS if platform.startswith('darwin') + else rc.sdk_custody.COMMAND_LOG_LINUX_MEMBERS) + data = {name: (b'' if i % 2 == 0 else b'opaque\x00\xff\n' + name.encode()) + for i, name in enumerate(names)} + data['logs/unknown-future.log'] = b'keep unknown producer output' + data['logs/installed-sdk-bun.json'] = b'{"structured":"not bundled"}' + evidence = {} + for name, encoded in reversed(list(data.items())): + (output / name).write_bytes(encoded) + evidence[name] = {'sha256': hashlib.sha256(encoded).hexdigest(), 'byteLength': len(encoded)} + report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, + 'sourceRevision': 'a' * 40, 'version': '0.15.0-rc.4', 'evidence': evidence} + manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', + 'platform': platform, 'version': report['version'], + 'source': {'revision': report['sourceRevision'], 'verification': 'matched-product-doctor'}} + return report, manifest, data + + def test_actual_opaque_writer_preserves_shuffled_raw_zero_unknown_and_structured_bytes(self): + for platform in rc.sdk_custody.COMMAND_LOG_PLATFORMS: + with self.subTest(platform=platform), tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve() + report, manifest, originals = self.command_log_fixture(output, platform) + original_report = copy.deepcopy(report) + metadata = {name: ((output / name).stat().st_mode, (output / name).stat().st_mtime_ns) + for name in originals} + evidence = rc.producer_command_log_evidence(output, report, manifest) + self.assertEqual(report, original_report) + decoded = rc.sdk_custody.validate_producer_command_logs({**report, 'evidence': evidence}, manifest, + lambda name: rc.sdk_custody.read_command_log_bytes(output / name)) + self.assertEqual(decoded, {name: data for name, data in originals.items() + if name not in ('logs/unknown-future.log', 'logs/installed-sdk-bun.json')}) + self.assertEqual(set(evidence), {'logs/unknown-future.log', 'logs/installed-sdk-bun.json', + rc.sdk_custody.COMMAND_LOG_EVIDENCE}) + for name, data in originals.items(): + self.assertEqual((output / name).read_bytes(), data) + self.assertEqual(((output / name).stat().st_mode, (output / name).stat().st_mtime_ns), metadata[name]) + bundle = json.loads((output / rc.sdk_custody.COMMAND_LOG_EVIDENCE).read_bytes()) + for row in bundle['members']: + if row['byteLength'] == 0: + self.assertEqual(row['base64'], '') + self.assertEqual(row['sha256'], hashlib.sha256(b'').hexdigest()) + + def test_opaque_writer_refuses_stale_evidence_changed_metadata_and_existing_output(self): + for poison in ('stale-evidence', 'changed-mode', 'existing-output'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, _ = self.command_log_fixture(output) + name = rc.sdk_custody.COMMAND_LOG_MAC_MEMBERS[0] + target = output / rc.sdk_custody.COMMAND_LOG_EVIDENCE + if poison == 'stale-evidence': + (output / name).write_bytes(b'changed after original evidence capture') + if poison == 'existing-output': target.write_bytes(b'preserve existing bundle') + reader = rc.sdk_custody.read_command_log_bytes + def changed(path): + data = reader(path) + if path == output / name: path.chmod(0o755) + return data + with patch.object(rc.sdk_custody, 'read_command_log_bytes', + side_effect=changed if poison == 'changed-mode' else reader): + with self.assertRaises(ValueError): + rc.producer_command_log_evidence(output, report, manifest) + if poison == 'existing-output': self.assertEqual(target.read_bytes(), b'preserve existing bundle') + else: self.assertFalse(target.exists()) + + def test_opaque_writer_rejects_oversized_encoded_bundle_before_write(self): + with tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, _ = self.command_log_fixture(output) + # Exactly16MiB of base64 alone, so required JSON metadata exceeds + # the real aggregate bound even though the raw file is admissible. + name = rc.sdk_custody.COMMAND_LOG_MAC_MEMBERS[0] + data = b'x' * (12 * 1024 * 1024) + (output / name).write_bytes(data) + report['evidence'][name] = {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + for other in rc.sdk_custody.COMMAND_LOG_MAC_MEMBERS[1:]: + (output / other).write_bytes(b'') + report['evidence'][other] = {'sha256': hashlib.sha256(b'').hexdigest(), 'byteLength': 0} + with self.assertRaisesRegex(ValueError, 'exceeds evidence bound'): + rc.producer_command_log_evidence(output, report, manifest) + self.assertFalse((output / rc.sdk_custody.COMMAND_LOG_EVIDENCE).exists()) + def test_support_resource_names_cannot_select_cli_or_receipt(self): from test_qualify_installed_sdk import sdk_archive_with_resource_names with tempfile.TemporaryDirectory() as raw: @@ -287,6 +413,136 @@ def test_collect_and_silent_signature_proofs_are_retained_and_poison_refused(sel (sdk / 'codesign.log').write_bytes(original); (sdk / 'collect.toc').write_bytes(toc) +class KernelRCLinuxRuntimeProducerTests(unittest.TestCase): + def fixture(self, output, platform='linux-x64-gnu', *, large_helper=False): + from test_kernel_rc_evidence import sdk_fixture, linux_runtime_fixture + sdk, table = sdk_fixture(platform) + table = {kind: {'release/' + name: value for name, value in rows.items()} + for kind, rows in table.items()} + table['files']['release/prose'] = (b'nonexecuted CLI fixture', 0o755) + if large_helper: + helper = table['files']['release/prose-agents-sdk'][0] + helper += b'\0' * (16 * 1024 * 1024 + 1 - len(helper)) + receipt = json.loads(table['files']['release/agents-sdk-build.json'][0]) + receipt['helper'].update(sha256=hashlib.sha256(helper).hexdigest(), byteLength=len(helper)) + encoded = json.dumps(receipt, sort_keys=True).encode() + table['files']['release/prose-agents-sdk'] = (helper, 0o755) + table['files']['release/agents-sdk-build.json'] = (encoded, 0o644) + sdk.update(sha256=receipt['helper']['sha256'], byteLength=len(helper), + receiptSha256=hashlib.sha256(encoded).hexdigest()) + report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, + 'sourceRevision': 'a' * 40, 'version': '0.15.0-rc.4', 'evidence': {}} + manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', + 'platform': platform, 'version': report['version'], 'agentsSdk': sdk, + 'source': {'revision': report['sourceRevision'], 'verification': 'matched-product-doctor'}} + (output / 'logs').mkdir() + if platform.startswith('darwin'): + return report, manifest, table, {} + originals = linux_runtime_fixture(report, manifest, table, rc.ROOT) + runtime = output / 'agents-sdk-runtime' + for name in ('job/home', 'job/docker-config', 'payload'): + (runtime / name).mkdir(parents=True, exist_ok=True) + for name, data in originals.items(): + if not name.startswith('sources/') and name != rc.sdk_custody.LINUX_RUNTIME_EVIDENCE: + (runtime / name).write_bytes(data) + for name in rc.sdk_custody.SDK_NAMES: + path = runtime / 'payload' / name + path.write_bytes(table['files']['release/' + name][0]) + path.chmod(0o555 if name == 'prose-agents-sdk' else 0o444) + (output / 'logs/unknown.log').write_bytes(b'preserve other evidence') + report['evidence']['logs/unknown.log'] = { + 'sha256': hashlib.sha256(b'preserve other evidence').hexdigest(), 'byteLength': 23} + return report, manifest, table, originals + + def test_actual_runtime_packet_writer_preserves_all_originals_and_replays_both_architectures(self): + for platform in ('linux-x64-gnu', 'linux-arm64-gnu'): + with self.subTest(platform=platform), tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, originals = self.fixture(output, platform) + before_report = copy.deepcopy(report) + evidence = rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=rc.ROOT) + self.assertEqual(report, before_report) + self.assertEqual(set(evidence), {'logs/unknown.log', rc.sdk_custody.LINUX_RUNTIME_EVIDENCE}) + packet_path = output / rc.sdk_custody.LINUX_RUNTIME_EVIDENCE + runtime_report = rc.sdk_custody.validate_linux_runtime_evidence({**report, 'evidence': evidence}, + manifest, table, lambda name: rc.sdk_custody.read_command_log_bytes(output / name), + expected_sources=rc.sdk_custody.read_linux_runtime_sources(rc.ROOT)) + self.assertEqual(runtime_report, json.loads(originals['runtime-report.json'])) + packet = json.loads(packet_path.read_bytes()) + self.assertEqual(len(packet['members']), 17) + self.assertFalse(any(row['path'].startswith('payload/') for row in packet['members'])) + for name, data in originals.items(): + if name.startswith('sources/'): + self.assertEqual((rc.ROOT / name[len('sources/'):]).read_bytes(), data) + elif name != rc.sdk_custody.LINUX_RUNTIME_EVIDENCE: + self.assertEqual((output / 'agents-sdk-runtime' / name).read_bytes(), data) + for name in rc.sdk_custody.SDK_NAMES: + self.assertEqual((output / 'agents-sdk-runtime/payload' / name).read_bytes(), + table['files']['release/' + name][0]) + + def test_runtime_writer_refuses_extra_directory_payload_file_alias_or_changed_copied_trio(self): + for poison in ('extra-job-file', 'extra-directory', 'extra-payload', 'payload-alias', 'changed-trio', 'wrong-mode'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, _ = self.fixture(output) + runtime = output / 'agents-sdk-runtime'; helper = runtime / 'payload/prose-agents-sdk' + if poison == 'extra-job-file': (runtime / 'job/unknown.log').write_bytes(b'not silently discarded') + elif poison == 'extra-directory': (runtime / 'job/home/unknown').mkdir() + elif poison == 'extra-payload': (runtime / 'payload/unknown').write_bytes(b'not silently discarded') + elif poison == 'payload-alias': + helper.unlink(); helper.symlink_to(output / 'absent') + elif poison == 'changed-trio': + helper.chmod(0o755); helper.write_bytes(b'changed copied bytes'); helper.chmod(0o555) + elif poison == 'wrong-mode': helper.chmod(0o755) + with self.assertRaises(ValueError): + rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=rc.ROOT) + self.assertFalse((output / rc.sdk_custody.LINUX_RUNTIME_EVIDENCE).exists()) + + def test_runtime_writer_requires_trusted_source_bytes_and_fresh_packet(self): + for poison in ('changed-source', 'existing-packet'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, originals = self.fixture(output) + source_root = rc.ROOT + target = output / rc.sdk_custody.LINUX_RUNTIME_EVIDENCE + if poison == 'existing-packet': target.write_bytes(b'preserve prior packet') + else: + source_root = output / 'source-fixture'; source_root.mkdir() + for name in rc.sdk_custody.LINUX_RUNTIME_SOURCE_PATHS: + path = source_root / name; path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(originals['sources/' + name]) + changed = source_root / 'cli/ci/build_agents_sdk_linux.py' + changed.write_bytes(changed.read_bytes() + b'\n# changed trusted source fixture\n') + with self.assertRaises(ValueError): + rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=source_root) + if poison == 'existing-packet': self.assertEqual(target.read_bytes(), b'preserve prior packet') + else: self.assertFalse(target.exists()) + + def test_runtime_packet_cap_does_not_limit_copied_helper_to_sixteen_mebibytes(self): + with tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, _ = self.fixture(output, large_helper=True) + evidence = rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=rc.ROOT) + self.assertGreater((output / 'agents-sdk-runtime/payload/prose-agents-sdk').stat().st_size, 16 * 1024 * 1024) + self.assertLess(evidence[rc.sdk_custody.LINUX_RUNTIME_EVIDENCE]['byteLength'], 16 * 1024 * 1024) + + def test_runtime_writer_refuses_oversized_packet_before_write(self): + with tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, _ = self.fixture(output) + (output / 'agents-sdk-runtime/job/pull-runtime.log').write_bytes(b'x' * (12 * 1024 * 1024)) + with self.assertRaisesRegex(ValueError, 'exceeds evidence bound'): + rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=rc.ROOT) + self.assertFalse((output / rc.sdk_custody.LINUX_RUNTIME_EVIDENCE).exists()) + + def test_mac_absence_guard_performs_no_linux_source_lookup_or_runtime_reads(self): + with tempfile.TemporaryDirectory() as directory: + output = Path(directory).resolve(); report, manifest, table, _ = self.fixture(output, 'darwin-arm64') + with patch.object(rc.sdk_custody, 'read_linux_runtime_sources') as sources, \ + patch.object(rc.sdk_custody, 'read_command_log_bytes') as reader: + self.assertEqual(rc.producer_linux_runtime_evidence(output, report, manifest, table, + source_root=output / 'absent'), {}) + report['evidence'][rc.sdk_custody.LINUX_RUNTIME_EVIDENCE] = {'sha256': 'a' * 64, 'byteLength': 1} + with self.assertRaisesRegex(ValueError, 'Mac report contains'): + rc.producer_linux_runtime_evidence(output, report, manifest, table, source_root=output / 'absent') + sources.assert_not_called(); reader.assert_not_called() + + class NativeSdkRoutingTests(unittest.TestCase): def test_linux_requires_pin_and_never_uses_host_python(self): with tempfile.TemporaryDirectory() as d, patch.object(rc.sys, 'platform', 'linux'), patch.object(rc, 'command') as command: diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index 37c759b7..805f5f69 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -117,6 +117,70 @@ def test_non_regular_manifest_is_rejected(self): rehearsal.select_archives(self.package) +class CompleteSdkEvidenceJsonAdmissionTests(unittest.TestCase): + def setUp(self): + # Reuse the complete authenticated multi-platform package factory only; + # its native observations are synthetic and no executables are run. + from test_assemble_kernel_rc import AssemblyTests + self.fixture = AssemblyTests('test_generated_packages_assemble_without_claiming_live_qualification') + self.fixture.setUp() + self.addCleanup(self.fixture.doCleanups) + + def test_complete_sdk_evidence_above_generic_limit_verifies(self): + root, path = self.fixture.resize_installed_sdk_evidence(1024 * 1024 + 128) + with self.assertRaisesRegex(ValueError, 'Invalid JSON file'): + rehearsal.pub.read_json(path) + verified, manifest, archives = rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + self.assertEqual(verified['source'], self.fixture.source) + self.assertEqual(manifest['platform'], 'darwin-arm64') + self.assertEqual(set(archives), {'bun', 'rust'}) + + def test_complete_sdk_evidence_above_specific_limit_refuses(self): + root, _ = self.fixture.resize_installed_sdk_evidence(16 * 1024 * 1024 + 1) + with self.assertRaisesRegex(ValueError, 'Invalid JSON file'): + rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + + def test_linux_runtime_packet_missing_or_rehashed_source_refuses(self): + import base64 + root = next(root for root in self.fixture.roots if root.name == 'linux-x64-gnu') + report_path = root / 'build-report.json'; report = json.loads(report_path.read_bytes()) + path = root / rehearsal.custody.LINUX_RUNTIME_EVIDENCE; original = path.read_bytes() + del report['evidence'][rehearsal.custody.LINUX_RUNTIME_EVIDENCE]; report_path.write_text(json.dumps(report)) + with self.assertRaises(ValueError): rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + bundle = json.loads(original); row = next(row for row in bundle['members'] if row['path'] == 'sources/cli/ci/sdk_native_inventory.py') + data = b'altered source despite recomputed inner and outer digests' + row.update(base64=base64.b64encode(data).decode(), sha256=hashlib.sha256(data).hexdigest(), byteLength=len(data)) + path.write_text(json.dumps(bundle)); report['evidence'][rehearsal.custody.LINUX_RUNTIME_EVIDENCE] = { + 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + with self.assertRaises(ValueError): rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + + def test_complete_linux_runtime_packets_verify_both_targets(self): + for platform in ('linux-x64-gnu', 'linux-arm64-gnu'): + root = next(root for root in self.fixture.roots if root.name == platform) + with self.subTest(platform=platform): + verified, manifest, archives = rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + self.assertEqual(verified['source'], self.fixture.source) + self.assertEqual(manifest['platform'], platform) + self.assertEqual(set(archives), {'bun', 'rust'}) + + def test_mac_runtime_claim_refuses_before_packet_callback(self): + from unittest.mock import Mock + root = next(root for root in self.fixture.roots if root.name == 'darwin-arm64') + report_path = root / 'build-report.json'; report = json.loads(report_path.read_bytes()) + path = root / rehearsal.custody.LINUX_RUNTIME_EVIDENCE; path.write_bytes(b'unexpected Mac runtime claim') + report['evidence'][rehearsal.custody.LINUX_RUNTIME_EVIDENCE] = { + 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + callback = Mock(side_effect=AssertionError('Mac packet callback must not be used')) + actual = rehearsal.custody.validate_linux_runtime_evidence + def validate(report, manifest, table, read_bytes, *, expected_sources): + return actual(report, manifest, table, callback, expected_sources=expected_sources) + with patch.object(rehearsal.custody, 'validate_linux_runtime_evidence', side_effect=validate), self.assertRaises(ValueError): + rehearsal.verify_kernel_rc(root, self.fixture.source, self.fixture.version) + callback.assert_not_called() + + class KernelRcAdmissionTests(unittest.TestCase): SOURCE = 'a' * 40 VERSION = '0.15.0-rc.3' @@ -170,6 +234,9 @@ def write(self): (self.root / 'package/release-manifest.json').write_text(json.dumps(self.manifest)) for name, check in self.checks.items(): (self.root / 'logs' / (name + '.json')).write_text(json.dumps(check)) + from test_kernel_rc_evidence import producer_command_log_fixture + for relative, encoded in producer_command_log_fixture(self.report).items(): + (self.root / relative).write_bytes(encoded) self.report['evidence'] = {str(path.relative_to(self.root)): { 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(), 'byteLength': path.stat().st_size} for directory in ('package', 'logs') for path in (self.root / directory).iterdir()} @@ -187,6 +254,17 @@ def test_accepts_exact_offline_native_custody_without_execution(self): self.assertEqual(manifest['version'], self.VERSION) self.assertEqual(set(selected), {'bun', 'rust'}) + def test_command_bundle_missing_or_rehashed_raw_poison_is_refused(self): + report_path = self.root / 'build-report.json'; path = self.root / rehearsal.custody.COMMAND_LOG_EVIDENCE + original = path.read_bytes(); report = json.loads(report_path.read_bytes()) + del report['evidence'][rehearsal.custody.COMMAND_LOG_EVIDENCE]; report_path.write_text(json.dumps(report)) + with self.assertRaises(ValueError): self.verify() + bundle = json.loads(original); bundle['members'][0]['base64'] = 'eA==' + path.write_text(json.dumps(bundle)); report['evidence'][rehearsal.custody.COMMAND_LOG_EVIDENCE] = { + 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(), 'byteLength': path.stat().st_size} + report_path.write_text(json.dumps(report)) + with self.assertRaises(ValueError): self.verify() + def test_expected_source_and_version_are_mandatory_exact_anchors(self): for source, version in [('bad', self.VERSION), ('c' * 40, self.VERSION), (self.SOURCE, '0.15.0'), (self.SOURCE, '0.15.0-rc.4')]: diff --git a/cli/ci/test_kernel_rc_evidence.py b/cli/ci/test_kernel_rc_evidence.py index db6aa669..6bed1da5 100644 --- a/cli/ci/test_kernel_rc_evidence.py +++ b/cli/ci/test_kernel_rc_evidence.py @@ -2,6 +2,7 @@ import hashlib import json import unittest +from pathlib import Path import kernel_rc_evidence as custody import publication as pub import test_assemble_kernel_rc as assembly_tests @@ -71,6 +72,372 @@ def sdk_archive_fixture(table,prefix='root/',cli=b'fixture-cli',extra_files=None +def producer_command_log_fixture(report): + """Opaque synthetic bytes only; no build, command or native execution authority.""" + import base64 + names = custody.COMMAND_LOG_MAC_MEMBERS if report['platform'].startswith('darwin-') else custody.COMMAND_LOG_LINUX_MEMBERS + rows = [] + for i, name in enumerate(names): + data = b'' if i == 0 else b'Synthetic opaque command bytes: ' + name.encode() + b'\x00\xff' + rows.append({'path': name, 'sha256': hashlib.sha256(data).hexdigest(), + 'byteLength': len(data), 'base64': base64.b64encode(data).decode()}) + bundle = {'schema': 'openprose.kernel-rc-command-logs/1', 'platform': report['platform'], + 'sourceRevision': report['sourceRevision'], 'version': report['version'], 'members': rows} + return {custody.COMMAND_LOG_EVIDENCE: json.dumps(bundle, sort_keys=True).encode()} + + +def linux_runtime_fixture(report, manifest, sdk_table, source_root): + """Synthetic retained runtime proof from trusted source bytes; never container execution.""" + import base64 + platform = report['platform'] + if platform.startswith('darwin-'): return {} + sources = {name: custody.read_command_log_bytes(Path(source_root) / name) for name in custody.LINUX_RUNTIME_SOURCE_PATHS} + expected_sources = {name: hashlib.sha256(data).hexdigest() for name, data in sources.items()} + lock = json.loads(sources['cli/ci/agents-sdk-linux-runtime.lock.json']); target = lock['platforms'][platform] + machine = target['machine']; trio = custody._runtime_sdk_trio(sdk_table); receipt = json.loads(trio[custody.SDK_NAMES[1]]) + runtime = {'schema': 'openprose.sdk-linux-clean-runtime/1', 'platform': platform, 'runtime': target['runtime'], + 'inputs': {'payload': {name: hashlib.sha256(data).hexdigest() for name, data in trio.items()}, + 'driverSha256': expected_sources[custody.LINUX_RUNTIME_SOURCE_PATHS[0]], + 'lockSha256': expected_sources[custody.LINUX_RUNTIME_SOURCE_PATHS[1]], + 'lifecycleSha256': expected_sources[custody.LINUX_RUNTIME_SOURCE_PATHS[2]], + 'inventorySha256': expected_sources[custody.LINUX_RUNTIME_SOURCE_PATHS[3]]}, + 'results': {'imports': receipt['selfTest'], 'tools': receipt['toolSelfTest'], 'libraries': receipt['linuxLibraries']}, + 'version': 'prose-agents-sdk 0.1.0', 'modelCalls': 0, 'networkUsed': False, + 'networkUsageScope': 'runtime-probes-only', 'preparationNetworkEnabled': True, + 'cpuFloorQualified': False, 'qualification': 'native-clean-glibc-2.34-only', 'publicationAuthorized': False} + files = {'sources/' + name: data for name, data in sources.items()} + files['runtime-report.json'] = json.dumps(runtime, sort_keys=True).encode() + b'\n' + files.update(custody._runtime_script_contract(sources['cli/ci/verify_agents_sdk_linux.py'], machine)) + files['job/pull-runtime.log'] = b'' + files['job/inspect-runtime.log'] = json.dumps({'Id': 'sha256:' + target['runtime']['configSha256'], 'Os': 'linux', + 'Architecture': target['dockerPlatform'].split('/')[1], 'RepoDigests': [target['runtime']['image']]}).encode() + b'\n' + files['job/base.log'] = ('glibc=2.34\narchitecture=' + machine + '\nnoBuildTools=true\n').encode() + files['job/version.log'] = b'prose-agents-sdk 0.1.0\n' + for name, value in runtime['results'].items(): files['job/' + name + '.log'] = json.dumps(value, sort_keys=True).encode() + b'\n' + packet = {'schema': 'openprose.sdk-linux-runtime-evidence/1', 'platform': platform, + 'sourceRevision': report['sourceRevision'], 'version': report['version'], 'runtimeReport': runtime, + 'members': [{'path': name, 'sha256': hashlib.sha256(files[name]).hexdigest(), 'byteLength': len(files[name]), + 'base64': base64.b64encode(files[name]).decode()} for name in custody.LINUX_RUNTIME_MEMBER_PATHS]} + files[custody.LINUX_RUNTIME_EVIDENCE] = json.dumps(packet, sort_keys=True).encode() + return files + + +class LinuxRuntimeEvidenceTests(unittest.TestCase): + SOURCE_ROOT = Path(__file__).resolve().parents[2] + + def fixture(self, platform='linux-x64-gnu'): + report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, 'sourceRevision': 'a' * 40, + 'version': '0.15.0-rc.4', 'evidence': {}} + sdk, direct = sdk_fixture(platform) + table = {'files': {'root/prose': (b'fixture-cli', 0o755), **{'root/' + k: v for k, v in direct['files'].items()}}, + 'directories': {}, 'symlinks': {}} + manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', 'platform': platform, + 'version': report['version'], 'source': {'revision': report['sourceRevision'], 'verification': 'matched-product-doctor'}, + 'agentsSdk': sdk} + files = linux_runtime_fixture(report, manifest, table, self.SOURCE_ROOT) + self.bind(report, files) + return report, manifest, table, files + + def bind(self, report, files): + raw = files[custody.LINUX_RUNTIME_EVIDENCE] + report['evidence'][custody.LINUX_RUNTIME_EVIDENCE] = {'sha256': hashlib.sha256(raw).hexdigest(), 'byteLength': len(raw)} + + def verify(self, report, manifest, table, files, expected=None): + return custody.validate_linux_runtime_evidence(report, manifest, table, files.__getitem__, + expected_sources=expected if expected is not None else custody.read_linux_runtime_sources(self.SOURCE_ROOT)) + + def changed(self, mutate): + report, manifest, table, files = self.fixture(); packet = json.loads(files[custody.LINUX_RUNTIME_EVIDENCE]) + mutate(packet); files[custody.LINUX_RUNTIME_EVIDENCE] = json.dumps(packet).encode(); self.bind(report, files) + with self.assertRaises(ValueError): self.verify(report, manifest, table, files) + + def set_member(self, packet, name, data): + import base64 + row = next(r for r in packet['members'] if r['path'] == name) + row.update(sha256=hashlib.sha256(data).hexdigest(), byteLength=len(data), base64=base64.b64encode(data).decode()) + + def claims(self, packet, mutate): + mutate(packet['runtimeReport']); self.set_member(packet, 'runtime-report.json', json.dumps(packet['runtimeReport']).encode()) + + def test_both_targets_complete_runtime_packet_and_real_source_lookup(self): + for platform in ('linux-x64-gnu', 'linux-arm64-gnu'): + report, manifest, table, files = self.fixture(platform) + observed = self.verify(report, manifest, table, files) + self.assertEqual(observed['platform'], platform) + self.assertEqual(len(json.loads(files[custody.LINUX_RUNTIME_EVIDENCE])['members']), 17) + self.assertLess(len(files[custody.LINUX_RUNTIME_EVIDENCE]), custody.LINUX_RUNTIME_MAX_BYTES) + + def test_mac_absence_and_unexpected_claim_and_missing_linux_before_callback(self): + from unittest import mock + report, manifest, _, _ = self.fixture() + reader = mock.Mock(side_effect=AssertionError('Must not read')) + report['platform'] = manifest['platform'] = 'darwin-arm64'; report['evidence'].clear() + self.assertIsNone(custody.validate_linux_runtime_evidence(report, manifest, None, reader, expected_sources=None)) + report['evidence'][custody.LINUX_RUNTIME_EVIDENCE] = {} + with self.assertRaises(ValueError): custody.validate_linux_runtime_evidence(report, manifest, None, reader, expected_sources=None) + report['platform'] = manifest['platform'] = 'linux-x64-gnu'; report['evidence'].clear() + with self.assertRaises(ValueError): custody.validate_linux_runtime_evidence(report, manifest, None, reader, + expected_sources=custody.read_linux_runtime_sources(self.SOURCE_ROOT)) + reader.assert_not_called() + + def test_bad_headers_trusted_sources_and_packet_size_refuse_before_read(self): + from unittest import mock + for poison in ('header', 'source-map', 'source-hash', 'bool-size', 'oversize'): + report, manifest, table, files = self.fixture(); sources = custody.read_linux_runtime_sources(self.SOURCE_ROOT) + if poison == 'header': manifest['source']['verification'] = 'unverified' + elif poison == 'source-map': sources.pop(custody.LINUX_RUNTIME_SOURCE_PATHS[0]) + elif poison == 'source-hash': sources[custody.LINUX_RUNTIME_SOURCE_PATHS[0]] = False + else: report['evidence'][custody.LINUX_RUNTIME_EVIDENCE]['byteLength'] = True if poison == 'bool-size' else custody.LINUX_RUNTIME_MAX_BYTES + 1 + reader = mock.Mock(side_effect=AssertionError('Must not read')) + with self.assertRaises(ValueError): custody.validate_linux_runtime_evidence(report, manifest, table, reader, expected_sources=sources) + reader.assert_not_called() + + def test_rehashed_source_input_image_and_runtime_claims_rejected(self): + mutations = [lambda r: r.update(cpuFloorQualified=True), lambda r: r.update(networkUsed=True), + lambda r: r.update(networkUsageScope='all'), lambda r: r.update(preparationNetworkEnabled=False), + lambda r: r.update(publicationAuthorized=True), lambda r: r.update(modelCalls=False), + lambda r: r['inputs']['payload'].update({'prose-agents-sdk': '0' * 64}), + lambda r: r['inputs'].update(inventorySha256='0' * 64), + lambda r: r['runtime'].update(configSha256='0' * 64)] + for mutate in mutations: + self.changed(lambda p: self.claims(p, mutate)) + self.changed(lambda p: self.set_member(p, 'sources/cli/ci/verify_agents_sdk_linux.py', b'not trusted source')) + self.changed(lambda p: self.set_member(p, 'job/inspect-runtime.log', b'{}')) + + def test_raw_report_probe_numeric_types_cannot_coerce_into_equal_receipt(self): + def raw_report(packet): + value = dict(packet['runtimeReport']); value['modelCalls'] = False + self.set_member(packet, 'runtime-report.json', json.dumps(value).encode()) + self.changed(raw_report) + for name, value in [('imports', False), ('libraries', 0.0)]: + def raw_probe(packet, name=name, value=value): + result = dict(packet['runtimeReport']['results'][name]); result['modelCalls'] = value + self.set_member(packet, 'job/' + name + '.log', json.dumps(result).encode()) + self.changed(raw_probe) + self.changed(lambda p: self.claims(p, lambda r: r['results']['libraries'].update(elfCount=1.0))) + + def test_coherently_rehashed_sdk_numeric_identities_rejected(self): + # Mutate both authorities and regenerate runtime facts: hashes alone must not + # turn Python's bool/int/float equality into a valid fixed SDK qualification. + for field, key, value in [('selfTest', 'modelCalls', False), + ('selfTest', 'modelCalls', 0.0), + ('selfTest', 'certificates', 1), + ('toolSelfTest', 'modelCalls', False), + ('toolSelfTest', 'networkUsed', 0), + ('toolSelfTest', 'shellEffects', 1.0), + ('helper', 'byteLength', 'float'), + ('notices', 'byteLength', 'float')]: + with self.subTest(field=field, key=key, value=value): + import copy + report, manifest, table, _ = self.fixture() + manifest, table = copy.deepcopy(manifest), copy.deepcopy(table) + receipt = json.loads(table['files']['root/agents-sdk-build.json'][0]) + receipt[field][key] = float(receipt[field][key]) if value == 'float' else value + if field in ('selfTest', 'toolSelfTest'): + manifest['agentsSdk'][field][key] = value + encoded = json.dumps(receipt).encode() + table['files']['root/agents-sdk-build.json'] = (encoded, 0o644) + manifest['agentsSdk']['receiptSha256'] = hashlib.sha256(encoded).hexdigest() + files = linux_runtime_fixture(report, manifest, table, self.SOURCE_ROOT) + self.bind(report, files) + with self.assertRaisesRegex(ValueError, 'Packaged SDK policy|Packaged SDK build receipt|SDK notices receipt'): + self.verify(report, manifest, table, files) + + def test_rehashed_scripts_base_version_and_probe_values_rejected(self): + controls = [('job/version.sh', b'timeout --kill-after=1s 30s /source/prose-agents-sdk --version\n'), + ('job/base.log', b'glibc=2.34\nnoBuildTools=false\n'), + ('job/version.log', b'prose-agents-sdk 0.2.0\n'), ('job/tools.log', b'{}')] + for name, data in controls: self.changed(lambda p, name=name, data=data: self.set_member(p, name, data)) + + def test_rehashed_packet_member_and_json_envelope_poisons_rejected(self): + for mutate in [lambda p: p.update(extra=True), lambda p: p.update(sourceRevision='b' * 40), + lambda p: p['members'].pop(), lambda p: p['members'].reverse(), + lambda p: p['members'][0].update(path='unknown'), lambda p: p['members'][0].update(byteLength=False), + lambda p: p['members'][0].update(base64='!!!!', byteLength=1), + lambda p: p['members'][0].update(sha256='0' * 64)]: self.changed(mutate) + + def materialize(self, root, files, table): + (root / 'job/home').mkdir(parents=True); (root / 'job/docker-config').mkdir(); (root / 'payload').mkdir() + for name in ('runtime-report.json', *custody.LINUX_RUNTIME_JOB_PATHS): (root / name).write_bytes(files[name]) + for name, data in custody._runtime_sdk_trio(table).items(): + path = root / 'payload' / name; path.write_bytes(data); path.chmod(0o555 if name == custody.SDK_NAMES[0] else 0o444) + return json.loads(files['runtime-report.json']) + + def test_physical_tree_exact_payload_modes_and_unknowns_refuse(self): + import tempfile + for poison in ('good', 'extra-payload', 'extra-job', 'cleanup', 'mode', 'alias', 'nested', 'bytes', 'empty-home'): + report, manifest, table, files = self.fixture() + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); runtime = self.materialize(root, files, table) + if poison == 'extra-payload': (root / 'payload/unknown').write_bytes(b'x') + elif poison == 'extra-job': (root / 'job/unexpected').write_bytes(b'x') + elif poison == 'cleanup': (root / 'job/version-cleanup-error.txt').write_bytes(b'failure') + elif poison == 'mode': (root / 'payload/prose-agents-sdk').chmod(0o755) + elif poison == 'alias': + path = root / 'payload/agents-sdk-build.json'; path.unlink(); path.symlink_to('../runtime-report.json') + elif poison == 'nested': (root / 'payload/nested').mkdir() + elif poison == 'bytes': + path = root / 'payload/prose-agents-sdk'; path.chmod(0o755); path.write_bytes(b'changed'); path.chmod(0o555) + elif poison == 'empty-home': (root / 'job/home/state').write_bytes(b'x') + if poison == 'good': custody.validate_linux_runtime_tree(root, runtime, table) + else: + with self.assertRaises(ValueError): custody.validate_linux_runtime_tree(root, runtime, table) + + def test_literal_script_extraction_never_executes_python(self): + source = custody.read_command_log_bytes(self.SOURCE_ROOT / custody.LINUX_RUNTIME_SOURCE_PATHS[0]) + scripts = custody._runtime_script_contract(source + b'\nraise RuntimeError("must never execute")\n', 'x86_64') + self.assertIn(b' 5s ', scripts['job/version.sh']) + with self.assertRaises(ValueError): custody._runtime_script_contract(b'BASE = str("dynamic")\nPROBES=()\n', 'x86_64') + + +class ProducerCommandLogTests(unittest.TestCase): + def fixture(self, platform='darwin-arm64'): + report = {'schema': 'openprose.kernel-rc-build/1', 'platform': platform, + 'sourceRevision': 'a' * 40, 'version': '0.15.0-rc.4', 'evidence': {}} + manifest = {'schema': 'openprose.local-release-manifest/1', 'mode': 'kernel-rc', + 'platform': platform, 'version': report['version'], + 'source': {'revision': report['sourceRevision'], 'verification': 'matched-product-doctor'}} + files = producer_command_log_fixture(report) + self.bind(report, files) + return report, manifest, files + + def bind(self, report, files): + report['evidence'][custody.COMMAND_LOG_EVIDENCE] = { + 'sha256': hashlib.sha256(files[custody.COMMAND_LOG_EVIDENCE]).hexdigest(), + 'byteLength': len(files[custody.COMMAND_LOG_EVIDENCE])} + + def poison_bundle(self, mutate): + report, manifest, files = self.fixture() + bundle = json.loads(files[custody.COMMAND_LOG_EVIDENCE]); mutate(bundle) + files[custody.COMMAND_LOG_EVIDENCE] = json.dumps(bundle).encode(); self.bind(report, files) + with self.assertRaises(ValueError): custody.validate_producer_command_logs(report, manifest, files.__getitem__) + + def test_four_platforms_exact_opaque_bytes_and_zero_are_admitted(self): + import base64 + for platform in custody.COMMAND_LOG_PLATFORMS: + report, manifest, files = self.fixture(platform) + observed = custody.validate_producer_command_logs(report, manifest, files.__getitem__) + rows = json.loads(files[custody.COMMAND_LOG_EVIDENCE])['members'] + self.assertEqual(observed, {r['path']: base64.b64decode(r['base64']) for r in rows}) + self.assertEqual(next(iter(observed.values())), b'') + self.assertEqual(len(observed), 8 if platform.startswith('darwin-') else 5) + + def test_headers_and_mandatory_evidence_refuse_before_read(self): + from unittest import mock + for part, key, value in [('report', 'schema', 'other'), ('report', 'platform', 'win-x64'), + ('report', 'sourceRevision', 'not-source'), ('report', 'version', True), + ('manifest', 'mode', 'development'), ('manifest', 'platform', 'darwin-x64'), + ('manifest', 'version', 'different'), ('manifest', 'source', {'revision': 'a' * 40})]: + report, manifest, _ = self.fixture(); (report if part == 'report' else manifest)[key] = value + reader = mock.Mock(side_effect=AssertionError('Reader must not run')) + with self.assertRaises(ValueError): custody.validate_producer_command_logs(report, manifest, reader) + reader.assert_not_called() + for poison in ('absent', 'original', 'bool', 'negative', 'oversize', 'hash', 'extra'): + report, manifest, _ = self.fixture(); record = report['evidence'][custody.COMMAND_LOG_EVIDENCE] + if poison == 'absent': report['evidence'].clear() + elif poison == 'original': report['evidence'][custody.COMMAND_LOG_MAC_MEMBERS[0]] = record + elif poison == 'hash': record['sha256'] = 'BAD' + elif poison == 'extra': record['extra'] = 1 + else: record['byteLength'] = {'bool': True, 'negative': -1, 'oversize': custody.COMMAND_LOG_MAX_BYTES + 1}[poison] + reader = mock.Mock(side_effect=AssertionError('Reader must not run')) + with self.assertRaises(ValueError): custody.validate_producer_command_logs(report, manifest, reader) + reader.assert_not_called() + + def test_rehashed_closed_header_and_member_coverage_poisons_refuse(self): + controls = [lambda b: b.update(extra=True), lambda b: b.update(schema='other'), + lambda b: b.update(platform='linux-x64-gnu'), lambda b: b.update(sourceRevision='b' * 40), + lambda b: b.update(version='other'), lambda b: b['members'].pop(), + lambda b: b['members'].append(b['members'][0]), lambda b: b['members'].reverse(), + lambda b: b['members'][1].update(path=b['members'][0]['path']), + lambda b: b['members'][0].update(path='logs/unapproved.log'), + lambda b: b['members'][0].update(extra=True)] + for mutate in controls: + with self.subTest(mutate=mutate): self.poison_bundle(mutate) + + def test_rehashed_inner_bytes_types_and_noncanonical_base64_refuse(self): + import base64 + controls = [lambda b: b['members'][0].update(byteLength=False), + lambda b: b['members'][0].update(byteLength=-1), + lambda b: b['members'][0].update(byteLength=custody.COMMAND_LOG_MAX_BYTES + 1), + lambda b: b['members'][0].update(sha256='0' * 64), + lambda b: b['members'][1].update(base64=base64.b64encode(b'changed').decode(), byteLength=7), + lambda b: b['members'][1].update(base64='Zh==', byteLength=1, sha256=hashlib.sha256(b'f').hexdigest()), + lambda b: b['members'][1].update(base64='Zg=\n', byteLength=1), + lambda b: b['members'][0].update(base64='!!!!', byteLength=1), + lambda b: b['members'][0].update(base64='éééé', byteLength=1)] + for mutate in controls: + with self.subTest(mutate=mutate): self.poison_bundle(mutate) + + def test_outer_hash_length_and_duplicate_json_key_refuse(self): + for poison in ('hash', 'length', 'duplicate-json'): + report, manifest, files = self.fixture() + if poison == 'hash': report['evidence'][custody.COMMAND_LOG_EVIDENCE]['sha256'] = '0' * 64 + elif poison == 'length': report['evidence'][custody.COMMAND_LOG_EVIDENCE]['byteLength'] -= 1 + else: + raw = files[custody.COMMAND_LOG_EVIDENCE] + files[custody.COMMAND_LOG_EVIDENCE] = raw.replace(b'{', b'{"schema":"duplicate",', 1); self.bind(report, files) + with self.assertRaises(ValueError): custody.validate_producer_command_logs(report, manifest, files.__getitem__) + + def test_aggregate_budget_refuses_before_read_or_json_decode(self): + from unittest import mock + report, manifest, files = self.fixture() + with mock.patch.object(custody, 'COMMAND_LOG_MAX_BYTES', len(files[custody.COMMAND_LOG_EVIDENCE]) - 1): + reader = mock.Mock(side_effect=AssertionError('Reader must not run')) + with self.assertRaises(ValueError): custody.validate_producer_command_logs(report, manifest, reader) + reader.assert_not_called() + + def test_reader_nofollow_refuses_leaf_replaced_before_open(self): + import tempfile, os + from pathlib import Path + from unittest import mock + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); path = root / 'log'; path.write_bytes(b'original') + target = root / 'target'; target.write_bytes(b'outside') + real = os.open + def replace_then_open(*args, **kwargs): + path.unlink(); path.symlink_to('target'); return real(*args, **kwargs) + with mock.patch.object(custody.os, 'open', side_effect=replace_then_open): + with self.assertRaises(OSError): custody.read_command_log_bytes(path) + self.assertEqual(target.read_bytes(), b'outside') + + def test_unknown_logs_remain_bound_separately(self): + report, manifest, files = self.fixture(); data = b'unknown retained bytes' + report['evidence']['logs/unknown.log'] = {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + self.assertEqual(len(custody.validate_producer_command_logs(report, manifest, files.__getitem__)), 8) + self.assertIn('logs/unknown.log', report['evidence']) + + def test_stable_reader_accepts_zero_and_refuses_alias_special_oversize(self): + import os, tempfile + from pathlib import Path + from unittest import mock + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); path = root / 'log'; path.write_bytes(b'') + self.assertEqual(custody.read_command_log_bytes(path), b'') + alias = root / 'alias'; alias.symlink_to('log') + with self.assertRaises(ValueError): custody.read_command_log_bytes(alias) + fifo = root / 'fifo'; os.mkfifo(fifo) + with self.assertRaises(ValueError): custody.read_command_log_bytes(fifo) + path.write_bytes(b'12345') + with mock.patch.object(custody, 'COMMAND_LOG_MAX_BYTES', 4), self.assertRaises(ValueError): + custody.read_command_log_bytes(path) + + def test_stable_reader_refuses_actual_replacement_during_read(self): + import tempfile, os + from pathlib import Path + from unittest import mock + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); path = root / 'log'; path.write_bytes(b'original') + real = os.fdopen + class ReplacingReader: + def __init__(self, stream): self.stream = stream + def __enter__(self): return self + def __exit__(self, *args): self.stream.close() + def read(self, limit): + data = self.stream.read(limit); path.unlink(); path.write_bytes(data); return data + with mock.patch.object(custody.os, 'fdopen', side_effect=lambda *a, **k: ReplacingReader(real(*a, **k))): + with self.assertRaisesRegex(ValueError, 'changed during'): custody.read_command_log_bytes(path) + + class OnedirCustodyTests(unittest.TestCase): def manifest(self,platform='darwin-x64'): sdk,table=sdk_fixture(platform) diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index c57183d0..3797f3fb 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -98,7 +98,9 @@ def input_fixture(root, target='linux-x64-gnu'): # Canonical encoded URLs exactly match the approved v3 driver lock's full archive pins. version = '20260807' archive = {'url': 'https://github.com/astral-sh/python-build-standalone/releases/download/' + version + '/cpython-3.10.20%2B' + version + '-' + arch + '-unknown-linux-gnu-pgo%2Blto-full.tar.zst', - 'sha256': '9e57324fd5e25f485fa5c8c587a2fe02f34a869092abd463e8e816c3cb1d48f2' if arch == 'x86_64' else '9201b2d72f8ea0250d594716cfefd8cd85f7b21a8741ba473fafcde58776c619', + # Synthetic provider metadata must not impersonate an inspected + # archive whose exact PYTHON.json hash is part of the erratum. + 'sha256': 'e' * 64, 'byteLength': 66186516 if arch == 'x86_64' else 65985139, 'targetTriple': arch + '-unknown-linux-gnu', 'metadataUrl': 'https://api.github.com/repos/astral-sh/python-build-standalone/releases/tags/20260807'} value = {'schema': 'openprose.sdk-linux-native-input/1', 'target': target, @@ -520,5 +522,112 @@ def test_sdk_only_member_aggregate_and_receipt_bounds(self): native.validate_macos_payload(**value) + + + +# Actual pinned provider license bytes; source/archive identity is separately tested. +PBS_ZLIB_LICENSE = b"\n Copyright (C) 1995-2017 Jean-loup Gailly and Mark Adler\n\n This software is provided 'as-is', without any express or implied\n warranty. In no event will the authors be held liable for any damages\n arising from the use of this software.\n\n Permission is granted to anyone to use this software for any purpose,\n including commercial applications, and to alter it and redistribute it\n freely, subject to the following restrictions:\n\n 1. The origin of this software must not be misrepresented; you must not\n claim that you wrote the original software. If you use this software\n in a product, an acknowledgment in the product documentation would be\n appreciated but is not required.\n 2. Altered source versions must be plainly marked as such, and must not be\n misrepresented as being the original software.\n 3. This notice may not be removed or altered from any source distribution.\n\n Jean-loup Gailly Mark Adler\n jloup@gzip.org madler@alumni.caltech.edu\n" + + +class PythonMetadataErratumTests(unittest.TestCase): + def declaration(self): + return {'name': 'zlib', 'licenses': ['Zlib'], + 'licensePaths': ['licenses/LICENSE.zlib-ng.txt', 'licenses/LICENSE.zlib.txt']} + + def constructor_fixture(self, root, target): + value, _, _, _ = input_fixture(root, target) + python = Path(value['pythonDistribution']['root']); metadata = python / 'PYTHON.json' + info = json.loads(metadata.read_bytes()) + info['build_info']['extensions']['zlib'] = [{'variant': 'default', + 'links': [{'name': 'z', 'path_static': 'build/lib/libz.a'}], 'licenses': ['Zlib'], + 'license_paths': self.declaration()['licensePaths']}] + metadata.write_text(json.dumps(info)) + (python / 'licenses').mkdir(); (python / 'licenses/LICENSE.zlib.txt').write_bytes(PBS_ZLIB_LICENSE) + archive_sha, metadata_sha = native.PBS_ZLIB_PARENTS[value['pythonArchive']['targetTriple']] + value['pythonArchive']['sha256'] = archive_sha + # Controlled metadata authentication seam models a verified upstream + # record without embedding two entire94KB upstream JSON files here. + original_sha = native.sha; metadata_bytes = metadata.read_bytes() + digest = lambda data: metadata_sha if data == metadata_bytes else original_sha(data) + return value, python, mock.patch.object(native, 'sha', side_effect=digest) + + def portable_fixture(self, root, target): + value, _, _, _ = input_fixture(root, target) + machine = 62 if target == 'linux-x64-gnu' else 183 + rows, origins = native.assign_origins([native.elf_record('libgcc_s.so.1', elf(machine=machine))], value, [], lambda _: None) + python = origins['python']; archive_sha, metadata_sha = native.PBS_ZLIB_PARENTS[python['archive']['targetTriple']] + python['archive']['sha256'] = archive_sha; python['metadataSha256'] = metadata_sha + python['declaredExtensionLicenses'] = [self.declaration()] + python['licenses'].append(native.record('licenses/LICENSE.zlib.txt', PBS_ZLIB_LICENSE)) + python['metadataErrata'] = native.python_metadata_errata(python['archive'], metadata_sha, python['declaredExtensionLicenses']) + return {'libraries': rows, 'symbolClosureVerified': True, + 'libgccSelection': {**native.record('libgcc_s.so.1', elf(machine=machine)), 'analysisTocSha256': 'a' * 64}, 'origins': origins} + + def test_both_pinned_constructor_mappings_keep_original_declaration(self): + for target in ('linux-x64-gnu', 'linux-arm64-gnu'): + with self.subTest(target=target), tempfile.TemporaryDirectory() as temp: + value, _, authenticated = self.constructor_fixture(Path(temp), target) + with authenticated: origin, _ = native.python_origin(value) + self.assertEqual(origin['declaredExtensionLicenses'], [self.declaration()]) + self.assertEqual(len(origin['metadataErrata']), 1) + self.assertEqual(origin['metadataErrata'][0]['extension']['effectiveLicensePaths'], ['licenses/LICENSE.zlib.txt']) + self.assertEqual({r['path'] for r in origin['licenses']}, {'LICENSE', 'licenses/LICENSE.zlib.txt'}) + + def test_known_parent_cannot_downgrade_or_mix_identities(self): + archive_sha, metadata_sha = native.PBS_ZLIB_PARENTS['x86_64-unknown-linux-gnu'] + archive = {'sha256': archive_sha, 'targetTriple': 'x86_64-unknown-linux-gnu'} + for changed_archive, changed_meta in [({**archive, 'sha256': '0' * 64}, metadata_sha), + (archive, '0' * 64), ({**archive, 'targetTriple': 'aarch64-unknown-linux-gnu'}, metadata_sha)]: + with self.assertRaises(ValueError): native.python_metadata_errata(changed_archive, changed_meta, [self.declaration()]) + for declarations in ([], [self.declaration(), self.declaration()], [None], ['zlib']): + with self.assertRaises(ValueError): native.python_metadata_errata(archive, metadata_sha, declarations) + + def test_constructor_requires_absent_omitted_leaf_and_exact_actual_text(self): + for poison in ('present', 'dangling', 'retained-alias', 'retained-bytes', 'missing-other'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as temp: + value, root, authenticated = self.constructor_fixture(Path(temp), 'linux-x64-gnu') + retained = root / 'licenses/LICENSE.zlib.txt'; omitted = root / 'licenses/LICENSE.zlib-ng.txt' + if poison == 'present': omitted.write_bytes(PBS_ZLIB_LICENSE) + elif poison == 'dangling': omitted.symlink_to('absent') + elif poison == 'retained-alias': retained.rename(root / 'text'); retained.symlink_to('../text') + elif poison == 'retained-bytes': retained.write_bytes(b'wrong license') + else: (root / 'LICENSE').unlink() + with authenticated, self.assertRaises(ValueError): native.python_origin(value) + + def test_portable_mapping_and_every_fixed_field_are_closed_on_both_targets(self): + for target in ('linux-x64-gnu', 'linux-arm64-gnu'): + with tempfile.TemporaryDirectory() as temp: + original = self.portable_fixture(Path(temp), target) + self.assertEqual(native.validate_native_dependencies(original, target), original) + mutations = [lambda p: p.update(metadataErrata=[]), lambda p: p.pop('metadataErrata'), + lambda p: p.update(declaredExtensionLicenses=[]), + lambda p: p['declaredExtensionLicenses'].append(self.declaration()), + lambda p: p['declaredExtensionLicenses'][0].update(licenses=['MIT']), + lambda p: p['metadataErrata'].append(copy.deepcopy(p['metadataErrata'][0])), + lambda p: p['metadataErrata'][0].update(extra=True), + lambda p: p['metadataErrata'][0].update(upstreamCommit='0' * 40), + lambda p: p['metadataErrata'][0]['sourceSha256s'].update({'pythonbuild/utils.py': '0' * 64}), + lambda p: p['metadataErrata'][0]['extension'].update(variant='other'), + lambda p: p['metadataErrata'][0]['extension'].update(links=[]), + lambda p: p['metadataErrata'][0]['extension'].update(effectiveLicensePaths=[]), + lambda p: p['metadataErrata'][0]['retainedLicense'].update(byteLength=995.0), + lambda p: p['licenses'][-1].update(sha256='0' * 64), + lambda p: p['licenses'][-1].update(byteLength=994), + lambda p: p.update(declaredExtensionLicenses=[None])] + for mutate in mutations: + changed = copy.deepcopy(original); mutate(changed['origins']['python']) + with self.assertRaises(ValueError): native.validate_native_dependencies(changed, target) + + def test_unknown_provider_never_skips_missing_declared_text(self): + with tempfile.TemporaryDirectory() as temp: + value, _, _, _ = input_fixture(Path(temp)) + root = Path(value['pythonDistribution']['root']); metadata = root / 'PYTHON.json' + origin, _ = native.python_origin(value); self.assertEqual(origin['metadataErrata'], []) + info = json.loads(metadata.read_bytes()); info['build_info']['extensions']['zlib'] = [ + {'licenses': ['Zlib'], 'license_paths': self.declaration()['licensePaths']}] + metadata.write_text(json.dumps(info)) + with self.assertRaises(ValueError): native.python_origin(value) + + if __name__ == '__main__': unittest.main() diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index effdf4f8..2c1322aa 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1145,3 +1145,23 @@ Fresh native construction at046eef4 reached standalone SDK checks then failed be ## IMP-098 fifth native Linux qualification repairs Actual pinned Linux PYTHON.json records schema version as the string `"8"`; the integer8 synthetic assumption caused x64 native refusal. `/root/configuration_contract` owns its existing `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` paths to require that exact upstream string and truthful fixtures/poisons, retaining archive hashes, target/shared-library/license/source custody and all other metadata guards. Actual ARM preparation also encountered a disappearing temporary pyc file during host quota scanning. `/root/sdk_assessment` owns its existing `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` paths for bounded no-follow live accounting that tolerates only disappeared entries within owned active work, preserves file/byte caps and structural refusals, and retains closed post-stage checks. Keep supplier/native schemas, tool/image/archive pins, deadlines, original errors and cleanup evidence; no broad suppression or limit relaxation. Agents use pure/synthetic controls and read-only retained native evidence; root owns native/full execution and push/merge. + +## IMP-098 complete installed SDK evidence readers + +Fresh dd08 construction passed, but its 3,386,183-byte complete installed payload record was rejected by the shared verifier's generic1MiB JSON reader. Root owns `cli/ci/homebrew_rehearsal.py`, `cli/ci/assemble_kernel_rc.py` and their existing tests for explicit use of the already established16MiB installed SDK evidence bound at those two consumers. `/root/configuration_contract` may edit only their matching test files to add meaningful large-record acceptance, oversize refusal and unchanged generic-reader controls. Keep checksum/source/tree/schema/physical bounds and all generic JSON defaults unchanged. Historical successful construction and failed qualification/audit retain dd08 provenance; root owns source integration, full admission, actual reruns, Git and merge. + +The retained actual record also reveals producer rows in Rust/Bun/npm order, contradicting the validator's canonical Bun/Rust/npm order. `/root/operational_design` retains its `cli/ci/build_kernel_rc.py` and `cli/ci/test_build_kernel_rc.py` lease to canonicalize producer evidence before writing and add a meaningful shuffled-order producer control. Preserve strict unique surfaces and the existing validator contract; do not relabel or rewrite historical records. Pure controls only; root retains all actual execution, integration and push/merge. + +## IMP-098 pinned Python metadata license erratum + +Root and both peers reviewed the exact original x64/ARM provider archives, metadata and upstream20260807 source evidence. The pinned generator unions licenses for two suppliers sharing library name `z`, although its Unix recipe selects ordinary zlib; both archives omit only the overclaimed zlib-ng text. `/root/configuration_contract` retains its `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` lease to implement the reviewed closed Python-origin `metadataErrata` contract, with original declarations preserved and exact original-to-effective path mapping for only the two authenticated provider/metadata/target pairs. Recognized archive or metadata identity requires the exact complete pair and exactly one literal erratum plus one unchanged zlib declaration; mixed/partial/altered identities never fall back. Unknown ordinary records require an empty errata list and all original text guards. The omitted leaf must be absent including dangling links; actual regular995-byte zlib text is hash-bound, and every other declared text remains mandatory. Pin upstream commit/source proof in the record; no generic existence filtering, renamed/fabricated text or shipped-zlib-ng claim. Keep all metadata/native/supplier/wheel/ELF/symbol/runtime limits and guards, Mac twelve-field manifest and genuine RC3 routes unchanged. `/root/sdk_assessment` independently reviews actual metadata shapes and meaningful poison controls. Agents perform pure/read-only work only; root owns full/native qualification, source integration and push/merge. + +## IMP-098 full-cohort evidence budget + +Root and both peers accepted contract `full-cohort-artifact-budget-and-opaque-log-proposal-1.json`, SHA256 `ff039b8695de9d820ad60fc412d934eba71dc530e257e999806ea2326f7e36cc`, following actual Mac raw-byte closure and a modeled full cohort exceeding128 with mandatory live assets. `/root/operational_design` owns its existing build/test pair for fixed `logs/producer-command-logs.json`, exact eight Mac/five Linux raw member names, bounded stable no-follow reads, canonical base64/hash/length closure including empty logs, and16MiB serialized bound before fresh write. Keep original raw files locally unchanged; report exclusion is only the exact successfully validated enclosed names, with unknown logs retained as evidence. `/root/sdk_assessment` owns its existing `cli/ci/kernel_rc_evidence.py` and `cli/ci/test_kernel_rc_evidence.py` pair for shared closed bundle validation, bounded stable no-follow file reading and mandatory final platform-consumer integration. Root owns `homebrew_rehearsal.py` and `assemble_kernel_rc.py` caller integration and prewrite inventory/live reservation; `/root/configuration_contract` owns their two matching test files for truthful complete bundle fixtures and before-write/count refusal controls. Existing pure SDK fixtures may be updated only in those leased test files and the central/build matching test files; no production bundle-presence fallback. Bind bundle to report/manifest source, version and platform; require the exact closed ordered member list, integer lengths, canonical base64, hashes and aggregate cap. All structured native/SDK JSON, payload, npm-alias, signature and COLLECT paths remain separate. Keep global128/generic policies and all byte/native guards unchanged. Actual four-platform unique artifact inventory plus all sixteen required live roles must fit before paid qualification; modeled115-117 headroom is not actual fit evidence. Agents use pure/read-only work only; root retains full/native/provider execution, source integration and push/merge. + +## IMP-098 retained Linux minimal runtime packet + +Root and both peers accepted the runtime packet design with exact copied-payload closure, frozen as `linux-minimal-runtime-fixed-packet-proposal-3.json`, SHA256 `4527e4194207ed9e20eea6928f4c5dd84c1afa0acb083ae0e7b1b979bff96590`. `/root/sdk_assessment` owns its existing `cli/ci/kernel_rc_evidence.py` and `cli/ci/test_kernel_rc_evidence.py` pair for the closed mandatory Linux packet validator, trusted explicit four-source map lookup, stable bounded readers and final platform-consumer integration. `/root/operational_design` owns its existing `cli/ci/build_kernel_rc.py` and `cli/ci/test_build_kernel_rc.py` pair for exact physical runtime closure, fixed seventeen-member packet formation and replacement of blind runtime-path enumeration. Root owns `cli/ci/homebrew_rehearsal.py` and `cli/ci/assemble_kernel_rc.py` integration; `/root/configuration_contract` owns their two matching tests for truthful fixtures and refusal controls. Other runtime driver/lock/lifecycle/inventory source paths are read-only for this extension. + +Retain every original file unchanged. Require exactly thirteen nonpayload originals, canonical expected directories including empty home/docker-config, and exactly three regular copied payload files with helper0555/receipt0444/notices0444, bounded stable streaming hashes matching both runtime inputs and the authenticated packaged SDK table. No aliases, unknown files or directories may be ignored. Keep helper256MiB/receipt2MiB/notices8MiB and packet16MiB bounds; do not embed copied helper bytes. Packet source bytes must hash to an exact four-source map supplied from each caller's current reviewed checkout before parsing lock data; packet declarations cannot establish source authority. Bind original/raw/structured runtime results, scripts, native target, pinned runtime image and SDK receipt, retaining genuine five-second cold-version checks and honest network/CPU limitations. Linux packet is mandatory; Mac refuses unexpected packet claims before callbacks. Preserve global128 and all existing structured proof paths; actual full cohort plus sixteen live assets must fit before paid calls. Agents use pure/synthetic/read-only work only; root owns full/native/provider execution, Git, source push and merges. From 4e80f69cde7e83ba3d68bb124d356cb92cfdc0de Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 16:43:07 -0400 Subject: [PATCH 59/64] Verify final freezer selection and preserve Homebrew SDK payloads --- cli/ci/homebrew_rehearsal.py | 2 + cli/ci/sdk_native_inventory.py | 30 +++++++++---- cli/ci/test_build_agents_sdk.py | 9 +++- cli/ci/test_homebrew_rehearsal.py | 66 +++++++++++++++++++++++++++++ cli/ci/test_sdk_native_inventory.py | 54 ++++++++++++++++++++--- cli/protocol/OWNERSHIP.md | 10 +++++ docs/packaged-sdk-design.md | 7 +++ 7 files changed, 164 insertions(+), 14 deletions(-) diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index 31eebfce..b4315242 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -132,6 +132,8 @@ def formula(version: str, implementation: str, archive: Path, sha256: str) -> st ' desc "Development-only Prose packaging rehearsal"', ' homepage "https://prose.md"', f' version "{version}"', ' license "MIT"', f' url "{archive.as_uri()}"', f' sha256 "{sha256}"', '', + ' preserve_rpath', + ' skip_clean "bin/prose", "bin/prose-agents-sdk", "bin/prose-agents-sdk-runtime"', '', ' def install', ' bin.install "prose"', ' bin.install "prose-agents-sdk" if File.exist?("prose-agents-sdk")', ' bin.install "prose-agents-sdk-runtime" if File.directory?("prose-agents-sdk-runtime")', diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index 0d2d499e..2b925d3a 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -246,19 +246,33 @@ def inspect_tree(root): 'libraries': rows, 'modelCalls': 0} +def _analysis_final_binaries(value): + # PyInstaller 6.22.3 Analysis._GUTS / Target._save_guts serialize twenty + # fields: input binaries at 10 and normalized final binaries at 15. + # https://github.com/pyinstaller/pyinstaller/blob/v6.22.3/PyInstaller/building/build_main.py + require(type(value) is tuple and len(value) == 20, 'Unsupported PyInstaller analysis TOC shape') + rows = value[15] + require(type(rows) is list and len(rows) <= MAX_FILES, 'Unsupported PyInstaller final binaries shape') + for row in rows: + require(type(row) is tuple and len(row) == 3 + and all(isinstance(text, str) and 0 < len(text.encode('utf-8')) <= 4096 + and all(ord(character) >= 32 and ord(character) != 127 for character in text) + for text in row), 'Invalid PyInstaller final binary row') + safe_path(row[0]) + require(row[0] != '.' and row[2] in ('BINARY', 'EXTENSION', 'SYMLINK'), + 'Invalid PyInstaller final binary destination or type') + return rows + + def verify_analysis_toc(path, supplier): data = read_file(path, MAX_METADATA) try: value = ast.literal_eval(data.decode('utf-8')) except (ValueError, SyntaxError, RecursionError) as error: raise ValueError('Invalid PyInstaller analysis TOC') from error - found = [] - def visit(node, depth=0): - require(depth <= 64, 'PyInstaller TOC nesting exceeds bound') - if isinstance(node, (list, tuple)): - if len(node) == 3 and node[0] == 'libgcc_s.so.1' and node[2] == 'BINARY': found.append(node) - else: - for child in node: visit(child, depth + 1) - visit(value) + rows = _analysis_final_binaries(value) + found = [row for row in rows if PurePosixPath(row[0]).name == 'libgcc_s.so.1'] require(len(found) == 1, 'PyInstaller must select exactly one libgcc binary') + require(found[0][0] == 'libgcc_s.so.1' and found[0][2] == 'BINARY', + 'PyInstaller libgcc destination or type differs') require(Path(found[0][1]).resolve() == Path(supplier).resolve() and sha(read_file(found[0][1])) == sha(read_file(supplier)), 'PyInstaller selected an ambient libgcc binary') return sha(data) diff --git a/cli/ci/test_build_agents_sdk.py b/cli/ci/test_build_agents_sdk.py index b6d00f61..e9ade7a7 100644 --- a/cli/ci/test_build_agents_sdk.py +++ b/cli/ci/test_build_agents_sdk.py @@ -186,7 +186,14 @@ def simulated_run(command, *, env, cwd, log, timeout=900): self.assertEqual((cwd / 'sdk_native_inventory.py').read_bytes(), Path(sdk.native.__file__).read_bytes()) (cwd / 'dist').mkdir(); (cwd / 'dist' / sdk.NAME).write_bytes(b'nonexecuted fixture helper') work = cwd / 'work' / sdk.NAME; work.mkdir(parents=True) - (work / 'Analysis-00.toc').write_text(repr(([], [('libgcc_s.so.1', str(supplier), 'BINARY')]))) + # Pinned Analysis._GUTS distinguishes explicit input from + # final binaries; neither occurrence is a duplicate output. + analysis = [[] for _ in range(20)] + analysis[4] = {}; analysis[7] = False; analysis[8] = {}; analysis[9] = 0 + analysis[12] = 'synthetic Python version' + analysis[10] = [('libgcc_s.so.1', str(supplier), 'BINARY')] + analysis[15] = [('libgcc_s.so.1', str(supplier), 'BINARY')] + (work / 'Analysis-00.toc').write_text(repr(tuple(analysis))) log.write_text('simulated freeze') elif '--packaged-self-test' in command: log.write_text(json.dumps({'schema':'openprose.sdk-packaged-self-test/1','openaiAgents':'0.22.2','openai':'3.13.0','certificates':True,'modelCalls':0})) diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index 805f5f69..151a8be2 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -54,6 +54,7 @@ def test_sdk_archive_identity_is_required_before_homebrew_mutation(self): self.manifest['agentsSdk'] = sdk with self.assertRaises(ValueError): self.select() + for item in self.manifest['artifacts']: path = self.package / item['path'] package.tar_gz(path, [('package/prose', item['implementation'].encode(), 0o755), @@ -65,6 +66,22 @@ def test_sdk_archive_identity_is_required_before_homebrew_mutation(self): with self.assertRaises(ValueError): self.select() + def test_prebuilt_formula_preserves_only_the_immutable_product_paths(self): + for implementation in ('bun', 'rust'): + rendered = rehearsal.formula('0.15.0-rc.4', implementation, + self.package / f'{implementation}.tar.gz', 'a' * 64) + declarations = [line.strip() for line in rendered.splitlines()] + self.assertEqual(declarations.count('preserve_rpath'), 1) + self.assertEqual([line for line in declarations if line.startswith('skip_clean ')], [ + 'skip_clean "bin/prose", "bin/prose-agents-sdk", "bin/prose-agents-sdk-runtime"']) + self.assertNotIn(':all', rendered) + self.assertNotIn('HOMEBREW_', rendered) + self.assertNotIn('fix_dynamic_linkage', rendered) + self.assertNotIn('codesign', rendered) + self.assertIn('bin.install "prose"', rendered) + self.assertIn('bin.install "prose-agents-sdk" if File.exist?("prose-agents-sdk")', rendered) + self.assertIn('bin.install "prose-agents-sdk-runtime" if File.directory?("prose-agents-sdk-runtime")', rendered) + def test_changed_bytes_and_size_are_rejected(self): with (self.package / 'bun.tar.gz').open('ab') as stream: stream.write(b'changed') @@ -698,6 +715,55 @@ def mutate_support(argv, **kwargs): self.exercise(previous=False) self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + def test_cleaner_style_support_mode_change_fails_before_helper_probe(self): + original = self.execute + changed = [] + def change_mode(argv, **kwargs): + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'install']: + name = next(name for name, (_, mode) in self.sdk_table['files'].items() + if name.startswith('prose-agents-sdk-runtime/') and mode == 0o644) + path = self.active.resolve().parent / name + before = path.read_bytes() + self.assertEqual(path.stat().st_mode & 0o777, 0o644) + path.chmod(0o444) + self.assertEqual(path.read_bytes(), before) + changed.append(name) + return result + with patch.object(self, 'execute', side_effect=change_mode), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertEqual(len(changed), 1) + self.assertFalse(any(Path(argv[0]).name == 'prose-agents-sdk' for argv, _ in self.calls)) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + + def test_pruned_declared_empty_support_directory_fails_before_helper_probe(self): + name = 'prose-agents-sdk-runtime/declared-empty-data' + self.sdk_table['directories'][name] = 0o755 + receipt = json.loads(self.sdk_table['files']['agents-sdk-build.json'][0]) + receipt['payload']['entries'].append({'path': name, 'type': 'directory', 'mode': 0o755}) + receipt['payload']['entries'].sort(key=lambda row: row['path']) + encoded = json.dumps(receipt).encode() + self.sdk_table['files']['agents-sdk-build.json'] = (encoded, 0o644) + self.manifest['agentsSdk']['receiptSha256'] = hashlib.sha256(encoded).hexdigest() + self.verified['sdkBuild'] = receipt + original = self.execute + pruned = [] + def prune_empty(argv, **kwargs): + result = original(argv, **kwargs) + if argv[:2] == ['brew', 'install']: + import sdk_native_inventory + view = sdk_native_inventory.read_macos_payload( + self.active.resolve().parent, receipt['payload'], 'arm64') + self.assertIn(name, view['directories']) + (self.active.resolve().parent / name).rmdir() + pruned.append(name) + return result + with patch.object(self, 'execute', side_effect=prune_empty), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertEqual(pruned, [name]) + self.assertFalse(any(Path(argv[0]).name == 'prose-agents-sdk' for argv, _ in self.calls)) + self.assertFalse((self.output / 'homebrew-rehearsal.json').exists()) + def test_lingering_support_after_all_version_uninstall_is_refused(self): original = self.execute def leave_support(argv, **kwargs): diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index 3797f3fb..62d78a3a 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -177,12 +177,56 @@ def test_analysis_toc_must_select_exact_explicit_supplier(self): root = Path(temp); supplier = root / 'libgcc_s.so.1'; supplier.write_bytes(elf()) ambient = root / 'ambient'; ambient.write_bytes(elf(symbol='other')) toc = root / 'Analysis-00.toc' - toc.write_text(repr(([], [('libgcc_s.so.1', str(supplier), 'BINARY')]))) + row = ('libgcc_s.so.1', str(supplier), 'BINARY') + def fixture(rows): + # Pinned twenty-field envelope, with distinct explicit input + # and final output fields. Native bytes remain synthetic. + value = [[] for _ in range(20)] + value[4] = {}; value[7] = False; value[8] = {}; value[9] = 0 + value[10] = [row]; value[12] = 'synthetic Python version' + value[15] = rows + return tuple(value) + toc.write_text(repr(fixture([row]))) self.assertEqual(native.verify_analysis_toc(toc, supplier), native.sha(toc.read_bytes())) - for rows in ([], [('libgcc_s.so.1', str(ambient), 'BINARY')], - [('libgcc_s.so.1', str(supplier), 'BINARY')] * 2): - toc.write_text(repr(([], rows))) - with self.assertRaises(ValueError): native.verify_analysis_toc(toc, supplier) + original_digest = native.sha(toc.read_bytes()) + value = list(fixture([row])); value[0] = [('irrelevant', 'input detail', 'not final')] + toc.write_text(repr(tuple(value))) + self.assertEqual(native.verify_analysis_toc(toc, supplier), native.sha(toc.read_bytes())) + self.assertNotEqual(original_digest, native.sha(toc.read_bytes())) + value[10] = [] + toc.write_text(repr(tuple(value))) + self.assertEqual(native.verify_analysis_toc(toc, supplier), native.sha(toc.read_bytes())) + for rows in ([], [('libgcc_s.so.1', str(ambient), 'BINARY')], [row] * 2, + [('nested/libgcc_s.so.1', str(supplier), 'BINARY')], + [row, ('nested/libgcc_s.so.1', str(supplier), 'BINARY')], + [('libgcc_s.so.1', str(supplier), 'EXTENSION')]): + with self.subTest(rows=rows): + toc.write_text(repr(fixture(rows))) + with self.assertRaises(ValueError): native.verify_analysis_toc(toc, supplier) + # The correct bytes under a different canonical name are still + # ambient; recursive input discovery cannot rescue final output. + ambient.write_bytes(supplier.read_bytes()) + toc.write_text(repr(fixture([('libgcc_s.so.1', str(ambient), 'BINARY')]))) + with self.assertRaises(ValueError): native.verify_analysis_toc(toc, supplier) + + def test_analysis_final_binary_shape_is_closed_without_recursive_fallback(self): + row = ('libgcc_s.so.1', '/synthetic/supplier.so.1', 'BINARY') + def fixture(rows): + value = [[] for _ in range(20)]; value[10] = [row]; value[15] = rows + return tuple(value) + good = fixture([row, ('extension.so', '/synthetic/extension.so', 'EXTENSION'), + ('alias.so', 'extension.so', 'SYMLINK')]) + self.assertEqual(native._analysis_final_binaries(good), good[15]) + for value in (list(good), good[:19], good + ([],), ([], [row]), + fixture((row,)), fixture(None), fixture([list(row)]), fixture([[row]]), + fixture([('libgcc_s.so.1', 1, 'BINARY')]), fixture([('x.so', '', 'BINARY')]), + fixture([('/absolute.so', 'source', 'BINARY')]), + fixture([('../escape.so', 'source', 'BINARY')]), + fixture([('.', 'source', 'BINARY')]), + fixture([('x.so', 'source\n', 'BINARY')]), + fixture([('x.so', 'x' * 4097, 'BINARY')]), + fixture([('x.so', 'source', 'DATA')]), fixture([row] * (native.MAX_FILES + 1))): + with self.subTest(value=value), self.assertRaises(ValueError): native._analysis_final_binaries(value) def test_valid_v3_encoded_full_archive_input_both_architectures(self): for target in ('linux-x64-gnu', 'linux-arm64-gnu'): diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 2c1322aa..91eecc4e 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1165,3 +1165,13 @@ Root and both peers accepted contract `full-cohort-artifact-budget-and-opaque-lo Root and both peers accepted the runtime packet design with exact copied-payload closure, frozen as `linux-minimal-runtime-fixed-packet-proposal-3.json`, SHA256 `4527e4194207ed9e20eea6928f4c5dd84c1afa0acb083ae0e7b1b979bff96590`. `/root/sdk_assessment` owns its existing `cli/ci/kernel_rc_evidence.py` and `cli/ci/test_kernel_rc_evidence.py` pair for the closed mandatory Linux packet validator, trusted explicit four-source map lookup, stable bounded readers and final platform-consumer integration. `/root/operational_design` owns its existing `cli/ci/build_kernel_rc.py` and `cli/ci/test_build_kernel_rc.py` pair for exact physical runtime closure, fixed seventeen-member packet formation and replacement of blind runtime-path enumeration. Root owns `cli/ci/homebrew_rehearsal.py` and `cli/ci/assemble_kernel_rc.py` integration; `/root/configuration_contract` owns their two matching tests for truthful fixtures and refusal controls. Other runtime driver/lock/lifecycle/inventory source paths are read-only for this extension. Retain every original file unchanged. Require exactly thirteen nonpayload originals, canonical expected directories including empty home/docker-config, and exactly three regular copied payload files with helper0555/receipt0444/notices0444, bounded stable streaming hashes matching both runtime inputs and the authenticated packaged SDK table. No aliases, unknown files or directories may be ignored. Keep helper256MiB/receipt2MiB/notices8MiB and packet16MiB bounds; do not embed copied helper bytes. Packet source bytes must hash to an exact four-source map supplied from each caller's current reviewed checkout before parsing lock data; packet declarations cannot establish source authority. Bind original/raw/structured runtime results, scripts, native target, pinned runtime image and SDK receipt, retaining genuine five-second cold-version checks and honest network/CPU limitations. Linux packet is mandatory; Mac refuses unexpected packet claims before callbacks. Preserve global128 and all existing structured proof paths; actual full cohort plus sixteen live assets must fit before paid calls. Agents use pure/synthetic/read-only work only; root owns full/native/provider execution, Git, source push and merges. + + +## IMP-098 actual seventh-cohort freezer and Homebrew repair + +Root selected the primary-source PyInstaller6.22.3 final-binaries contract proposal SHA256 `b51da6cc94810bcd4d0b3c3cd9af62b54db32d67a30b3cc84ebb59ce4dca72b5` and public Homebrew preservation proposal2 SHA256 `40e0c9c02f224955cd7fc1541ced895edd67fbc1c164810fe60defa409e0c394`, with comparison recorded in IMP-098 before implementation. `/root/configuration_contract` retains only `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` for the narrow final-binaries selector after independent design acceptance. Require pinned tuple20/final field15 shape, exactly one final libgcc destination/type, supplier path/hash and complete original TOC SHA; input/other-field decoys cannot establish final selection. Preserve all downstream CArchive ELF/native-origin/symbol/runtime guards, every pin/limit and prior failed evidence. No recursive deduplication or index fallback. + +Root owns `cli/ci/homebrew_rehearsal.py` and `docs/packaged-sdk-design.md` for public preserve_rpath and exact skip_clean paths bin/prose, bin/prose-agents-sdk and bin/prose-agents-sdk-runtime. `/root/operational_design` owns only `cli/ci/test_homebrew_rehearsal.py` for the matching conventional formula regression and existing immutable-tree refusal controls, superseding configuration agent's earlier test lease for this repair. Canonical sibling installation/discovery, all other Homebrew linkage checks, complete tree/modes/bytes/signatures, collision refusal, genuine upgrades and actual five-second probes remain. No broad :all/env/installer override, libexec-only assumption, re-sign or restoration of changed bytes. Agents use only pure/synthetic/read-only work and independent cross-review. Root owns Git, full24 admission before push and all actual native/helper/installed/Homebrew replay. + + +Seventh-cohort selector fixture lease extension: `/root/configuration_contract` additionally owns only `cli/ci/test_build_agents_sdk.py` to replace the legacy tuple2 positive mocked Linux Analysis fixture with the pinned tuple20 shape containing separate input field10 and final field15 supplier rows. Preserve all original builder supplier/archive/receipt/ELF controls; no production builder/source change or actual execution is authorized by this test-only extension. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 020828ac..d37a95c8 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -111,3 +111,10 @@ The runner owns fixed recovery actions from the shared SDK oracle, for both huma and JSON output. Unknown/missing reasons stay execution failures. Never forward arbitrary native messages or infer request completion, usage or billing from a setup classification; preserve valid observations already received. + + +### Homebrew preservation of packaged payloads + +The native Homebrew rehearsal uses the public Formula `preserve_rpath` convention to retain the SDK libraries' relative dylib identities and `skip_clean` scoped to `bin/prose`, `bin/prose-agents-sdk` and `bin/prose-agents-sdk-runtime`. Homebrew continues its other linkage checks. The complete signed SDK tree must retain its declared bytes, modes and directories through installation; neither a cleanup exclusion nor relative-ID preservation alone establishes that custody. Real install, genuine upgrade, collision refusal and cold helper probes remain qualification gates. The formula keeps the executable and its SDK helper/support tree as canonical keg siblings. + +The Linux freezer verifier reads the final `binaries` field in the pinned PyInstaller Analysis TOC. Its input binary list is a distinct record and cannot establish final selection. Exactly one final libgcc destination must bind to the pinned supplier path and bytes; complete TOC provenance and downstream archive, ELF, origin and symbol-closure checks remain required. From 27690a409061f73203257137d4b6dcf56ea4594e Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 17:15:13 -0400 Subject: [PATCH 60/64] Qualify owning wheel RECORD and explicit development Homebrew context --- cli/ci/homebrew_rehearsal.py | 39 ++++++-- cli/ci/sdk_native_inventory.py | 36 ++++++- cli/ci/test_homebrew_rehearsal.py | 103 ++++++++++++++++++-- cli/ci/test_sdk_native_inventory.py | 146 +++++++++++++++++++++++++++- cli/protocol/OWNERSHIP.md | 8 ++ docs/packaged-sdk-design.md | 5 + 6 files changed, 319 insertions(+), 18 deletions(-) diff --git a/cli/ci/homebrew_rehearsal.py b/cli/ci/homebrew_rehearsal.py index b4315242..c47f28fa 100644 --- a/cli/ci/homebrew_rehearsal.py +++ b/cli/ci/homebrew_rehearsal.py @@ -72,6 +72,24 @@ def verify_previous_release(root: Path, platform: str, candidate_version: str) - 'root': root.resolve(), 'archives': archives, 'binaryHashes': binary_hashes} +def sdk_context(manifest: dict[str, Any], verified: dict[str, Any] | None = None) -> dict[str, Any] | None: + """Keep the explicit development placeholder distinct from a real SDK.""" + if 'agentsSdk' in manifest: + try: + sdk = rehearse_release._sdk_release_identity(verified or {}, manifest) + except rehearse_release.RehearsalError as error: + raise ValueError(str(error)) from error + else: + if manifest.get('mode') in ('release', 'kernel-rc'): + raise ValueError('Production release lacks an SDK identity') + sdk = None + if sdk is None: + for context in (manifest, verified or {}): + if any(name in context for name in ('sdkBuild', 'sdkEvidence', 'sdkSourceCustody')): + raise ValueError('SDK-absent Homebrew context has SDK evidence') + return sdk + + def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, Any]]]: if package.is_symlink() or not package.is_dir(): raise ValueError('A regular verified package directory is required') @@ -85,6 +103,7 @@ def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, raise ValueError('Unsafe formula version') if manifest.get('platform') not in PLATFORMS: raise ValueError('Unsupported native platform') + sdk = sdk_context(manifest) selected = {} for item in manifest.get('artifacts', []): if item.get('kind') != 'standalone-archive': @@ -104,7 +123,7 @@ def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, raise ValueError('Invalid archive byte limit') if archive.stat().st_size != item['byteLength'] or hashlib.sha256(archive.read_bytes()).hexdigest() != item.get('sha256'): raise ValueError('Native archive identity mismatch') - if 'agentsSdk' in manifest: + if sdk is not None: table = pub.read_sdk_archive(archive, manifest) custody.validate_sdk_archive_table(manifest, table) cli_path = custody.sdk_archive_prefix(table) + 'prose' @@ -117,6 +136,9 @@ def select_archives(package: Path) -> tuple[dict[str, Any], dict[str, dict[str, parts = Path(member.name).parts if member.name.startswith('/') or '..' in parts or not (member.isfile() or member.isdir()): raise ValueError('Unsafe archive member') + if any(part in {'prose-agents-sdk', 'prose-agents-sdk-runtime', + 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'} for part in parts): + raise ValueError('SDK-absent archive contains SDK members') executables = [m for m in members if Path(m.name).name == 'prose' and m.isfile()] if len(executables) != 1 or len(Path(executables[0].name).parts) != 2: raise ValueError('Archive requires one prose executable under one package root') @@ -315,6 +337,7 @@ def report(suffix: str, argv: list[str], code: int = 0, additions: dict[str, str def exercise(package: Path, manifest: dict[str, Any], archives: dict[str, Any], verified: dict[str, Any], output: Path, brew: str, *, previous: dict[str, Any] | None = None) -> dict[str, Any]: + sdk = sdk_context(manifest, verified) output.mkdir(exist_ok=False) output = output.resolve() env = {key: os.environ[key] for key in ('PATH', 'HOME', 'USER', 'LOGNAME', 'TMPDIR', 'DEVELOPER_DIR', 'SDKROOT') if key in os.environ} @@ -348,10 +371,10 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False, if active.exists() or active.is_symlink(): raise ValueError('Rehearsal refuses to overwrite an existing prose command') active_helper = prefix / 'bin/prose-agents-sdk' - if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): + if sdk is not None and (active_helper.exists() or active_helper.is_symlink()): raise ValueError('Rehearsal refuses to overwrite an existing SDK helper command') active_support = prefix / 'bin/prose-agents-sdk-runtime' - if manifest.get('agentsSdk') and (active_support.exists() or active_support.is_symlink()): + if sdk is not None and (active_support.exists() or active_support.is_symlink()): raise ValueError('Rehearsal refuses to overwrite an existing SDK support directory') tap = output / 'tap' (tap / 'Formula').mkdir(parents=True) @@ -383,13 +406,13 @@ def command(label: str, argv: list[str], *, expected_failure: bool = False, digest = hashlib.sha256(active.read_bytes()).hexdigest() if digest != verified['packageIdentity'][f'{implementation}BinarySha256']: raise ValueError('Homebrew executable differs from the verified candidate') - if isinstance(manifest.get('agentsSdk'), dict): + if sdk is not None: sdk_payload_checks.append(qualify_installed_sdk(active, manifest, verified, command, 'fresh-' + implementation)) command(f'unlink-{implementation}', [brew, 'unlink', name]) command('link-bun', [brew, 'link', f'{TAP}/prose-bun']) original = hashlib.sha256(active.read_bytes()).hexdigest() def sdk_link_identity(): - if not manifest.get('agentsSdk'): + if sdk is None: return None helper = active.resolve(strict=True).parent / 'prose-agents-sdk' if active_helper.resolve(strict=True) != helper or pub.digest(helper) != manifest['agentsSdk']['sha256']: @@ -495,9 +518,9 @@ def preserve(label: str) -> None: raise ValueError('The rehearsal packages did not uninstall cleanly') if active.exists() or active.is_symlink(): raise ValueError('The rehearsal command did not uninstall cleanly') - if manifest.get('agentsSdk') and (active_helper.exists() or active_helper.is_symlink()): + if sdk is not None and (active_helper.exists() or active_helper.is_symlink()): raise ValueError('The rehearsal SDK helper command did not uninstall cleanly') - if manifest.get('agentsSdk') and (active_support.exists() or active_support.is_symlink()): + if sdk is not None and (active_support.exists() or active_support.is_symlink()): raise ValueError('The rehearsal SDK support directory did not uninstall cleanly') if previous: preserve('settings-after-final-cleanup') @@ -510,7 +533,7 @@ def preserve(label: str) -> None: receipt['sdkPayloadChecks'] = sdk_payload_checks receipt['sdkLinkConflictPreserved'] = True receipt['upgradeQualification'] = 'passed-genuine-upgrade-both-selections' if previous else 'not-requested-fresh-install-only' - receipt['kernelRetrieval'] = 'published-kernel-may-be-acquired-by-provider-free-dry-run' if manifest.get('agentsSdk') else 'not-measured' + receipt['kernelRetrieval'] = 'published-kernel-may-be-acquired-by-provider-free-dry-run' if sdk is not None else 'not-measured' if previous: receipt['previousRelease'] = {'version': previous['version'], 'manifestSha256': previous['manifestSha256'], 'archives': list(previous['archives'].values())} diff --git a/cli/ci/sdk_native_inventory.py b/cli/ci/sdk_native_inventory.py index 2b925d3a..53c5b417 100644 --- a/cli/ci/sdk_native_inventory.py +++ b/cli/ci/sdk_native_inventory.py @@ -484,10 +484,42 @@ def wheel_origins(packages, distributions): for package in packages: dist = distributions(package['name']) require(dist.version == package['version'], 'Installed wheel differs from lock') + # Distribution.files is the complete owning RECORD inventory, including + # vendored distributions' metadata. Only the direct owner is authority: + # https://packaging.python.org/en/latest/specifications/recording-installed-packages/ + def normalized_name(name): + require(isinstance(name, str) and len(name) <= 1024 and + re.fullmatch(r'[A-Za-z0-9](?:[A-Za-z0-9._-]*[A-Za-z0-9])?', name), + 'Invalid wheel distribution name') + return re.sub(r'[-_.]+', '-', name).lower() + owner = normalized_name(package['name']) + require(normalized_name(dist.metadata.get('Name')) == owner, + 'Installed wheel metadata name differs from lock') files = dist.files or [] - record_files = [entry for entry in files if str(entry).endswith('.dist-info/RECORD')] + record_files = [] + for entry in files: + name = str(entry) + if not name.endswith('.dist-info/RECORD'): continue + safe_path(name) + require(len(name.encode('utf-8')) <= 4096 and ':' not in name and + not any(ord(char) < 32 or ord(char) == 127 for char in name), + 'Unsafe wheel RECORD path') + parts = PurePosixPath(name).parts + if len(parts) != 2: continue + stem = parts[0][:-len('.dist-info')] + distribution_name, separator, version = stem.rpartition('-') + if separator and normalized_name(distribution_name) == owner and version == package['version']: + record_files.append(entry) require(len(record_files) == 1, 'Wheel RECORD is absent or ambiguous') - record_bytes = read_file(dist.locate_file(record_files[0]), MAX_METADATA) + record_path = Path(dist.locate_file(record_files[0])) + try: + base = Path(dist.locate_file('')).resolve(strict=True) + canonical_owner = (not record_path.parent.is_symlink() and + record_path.resolve(strict=True) == base / str(record_files[0])) + except (OSError, RuntimeError): + canonical_owner = False + require(canonical_owner, 'Wheel RECORD must belong to its distribution base') + record_bytes = read_file(record_path, MAX_METADATA) licenses = [] for entry in files: path = Path(dist.locate_file(entry)) diff --git a/cli/ci/test_homebrew_rehearsal.py b/cli/ci/test_homebrew_rehearsal.py index 151a8be2..0e39e307 100644 --- a/cli/ci/test_homebrew_rehearsal.py +++ b/cli/ci/test_homebrew_rehearsal.py @@ -82,6 +82,48 @@ def test_prebuilt_formula_preserves_only_the_immutable_product_paths(self): self.assertIn('bin.install "prose-agents-sdk" if File.exist?("prose-agents-sdk")', rendered) self.assertIn('bin.install "prose-agents-sdk-runtime" if File.directory?("prose-agents-sdk-runtime")', rendered) + def development_sentinel(self): + return dict(self.manifest, mode='development', releaseEligible=False, + publicationAuthorized=False, agentsSdk='not-packaged-development-fixture') + + def test_development_placeholder_uses_ordinary_archives_without_sdk_decoder(self): + manifest = self.development_sentinel() + with patch.object(rehearsal.pub, 'read_sdk_archive', side_effect=AssertionError('No SDK decoder')): + observed, selected = self.select(manifest) + self.assertEqual(observed, manifest) + self.assertEqual(set(selected), {'bun', 'rust'}) + + def test_placeholder_authority_and_malformed_sdk_refuse_before_mutation(self): + good = self.development_sentinel() + poisons = [dict(good, mode=mode) for mode in ('release', 'kernel-rc', 'alpha')] + poisons += [dict(good, **{field: value}) for field in ('releaseEligible', 'publicationAuthorized') + for value in (True, 0, None)] + poisons += [{k: v for k, v in good.items() if k != field} + for field in ('releaseEligible', 'publicationAuthorized')] + poisons += [dict(good, agentsSdk=value) for value in ('unknown', None, [], False, {})] + poisons += [dict(self.manifest, mode=mode) for mode in ('release', 'kernel-rc')] + for manifest in poisons: + with self.subTest(manifest=manifest): + with self.assertRaises(ValueError): self.select(manifest) + + def test_development_placeholder_manifest_sdk_claim_presence_refuses(self): + for field in ('sdkBuild', 'sdkEvidence', 'sdkSourceCustody'): + for value in (None, {}, False): + with self.subTest(field=field, value=value), self.assertRaises(ValueError): + self.select(dict(self.development_sentinel(), **{field: value})) + + def test_sdk_absent_archive_members_cannot_hide_behind_placeholder(self): + for name in ('prose-agents-sdk', 'prose-agents-sdk-runtime', 'agents-sdk-build.json', 'AGENTS-SDK-NOTICES.txt'): + path = self.package / 'bun.tar.gz' + with tarfile.open(path, 'w:gz') as archive: + for member_name in ('package/prose', 'package/nested/' + name): + member = tarfile.TarInfo(member_name); member.size = 1 + archive.addfile(member, io.BytesIO(b'x')) + self.manifest['artifacts'][0].update(byteLength=path.stat().st_size, + sha256=hashlib.sha256(path.read_bytes()).hexdigest()) + with self.subTest(name=name), self.assertRaisesRegex(ValueError, 'SDK-absent archive'): + self.select(self.development_sentinel()) + def test_changed_bytes_and_size_are_rejected(self): with (self.package / 'bun.tar.gz').open('ab') as stream: stream.write(b'changed') @@ -214,6 +256,9 @@ def setUp(self): 'imageSource': 'published-on-run', 'releaseEligible': False, 'publicationAuthorized': False, 'buildProfiles': {i: {'profile': 'release', 'testSeamsEnabled': False} for i in ('bun', 'rust')}, 'artifacts': []} + from test_kernel_rc_evidence import sdk_fixture + sdk, self.sdk_table = sdk_fixture('darwin-arm64') + self.manifest['agentsSdk'] = sdk for implementation in ('bun', 'rust'): self.archive(implementation + '.tar.gz', 'standalone-archive', implementation, 'darwin-arm64', 'package/prose', implementation.encode()) @@ -238,11 +283,13 @@ def setUp(self): def archive(self, filename, kind, implementation, platform, member_name, payload): path = self.root / 'package' / filename - with tarfile.open(path, 'w:gz') as archive: - member = tarfile.TarInfo(member_name) - member.mode = 0o755 - member.size = len(payload) - archive.addfile(member, io.BytesIO(payload)) + import package_local as package + prefix = str(Path(member_name).parent) + '/' + rows = [(member_name, payload, 0o755)] + if kind != 'npm-meta': + rows += [(prefix + name, data, mode) for name, (data, mode) in self.sdk_table['files'].items()] + package.tar_gz(path, rows, 0, sdk_table=self.sdk_table if kind != 'npm-meta' else None, + sdk_prefix=prefix if kind != 'npm-meta' else '') item = {'kind': kind, 'implementation': implementation, 'platform': platform, 'path': filename, 'byteLength': path.stat().st_size, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest()} self.manifest['artifacts'] = [a for a in self.manifest['artifacts'] if a['path'] != filename] + [item] @@ -252,6 +299,22 @@ def write(self): for name, check in self.checks.items(): (self.root / 'logs' / (name + '.json')).write_text(json.dumps(check)) from test_kernel_rc_evidence import producer_command_log_fixture + from test_kernel_rc_evidence import sdk_producer_fixture + for relative in rehearsal.custody.SDK_PROBES: + (self.root / relative).write_text(json.dumps(rehearsal.custody.SDK_TOOL_TEST + if 'sdk-tools-' in relative else rehearsal.custody.SDK_IMPORT_TEST)) + identities = {name: {'sha256': hashlib.sha256(data).hexdigest(), 'byteLength': len(data)} + for name, (data, _) in self.sdk_table['files'].items() + if name in rehearsal.custody.SDK_NAMES} + payload = json.loads(self.sdk_table['files']['agents-sdk-build.json'][0])['payload'] + identities['supportTree'] = {'payload': payload, + 'sha256': hashlib.sha256(json.dumps(payload, sort_keys=True, separators=(',', ':')).encode()).hexdigest(), + 'byteLength': payload['totalRegularBytes'], 'entryCount': len(payload['entries'])} + (self.root / rehearsal.custody.SDK_PAYLOAD_EVIDENCE).write_text(json.dumps([ + {'surface': 'installed-' + runner, 'before': identities, 'after': identities} + for runner in ('bun', 'rust', 'npm')])) + for relative, encoded in sdk_producer_fixture(self.sdk_table).items(): + (self.root / relative).write_bytes(encoded) for relative, encoded in producer_command_log_fixture(self.report).items(): (self.root / relative).write_bytes(encoded) self.report['evidence'] = {str(path.relative_to(self.root)): { @@ -473,7 +536,7 @@ def execute(self, argv, **kwargs): keg = self.prefix / 'Cellar' / implementation / version / 'bin'; keg.mkdir(parents=True, exist_ok=True) binary = keg / 'prose' binary.write_bytes((('new-' if version.endswith('.4') else 'old-') + implementation).encode()) - if version.endswith('.4'): + if version.endswith('.4') and 'sdkBuild' in self.verified: import sdk_native_inventory as native view = self.sdk_table files = {name: row for name, row in view['files'].items() @@ -551,6 +614,34 @@ def exercise(self, previous=True): return rehearsal.exercise(self.package, self.manifest, self.archives, self.verified, self.output, 'brew', previous=self.previous if previous else None) + def test_development_placeholder_completes_cli_only_failed_link_and_cleanup(self): + self.manifest.update(mode='development', releaseEligible=False, publicationAuthorized=False, + agentsSdk='not-packaged-development-fixture') + before = copy.deepcopy(self.manifest) + self.verified.pop('sdkBuild'); self.verified['custodyKind'] = 'development-rehearsal' + with patch.object(rehearsal, 'qualify_installed_sdk', side_effect=AssertionError('No SDK probe')): + receipt = self.exercise(previous=False) + self.assertEqual(self.manifest, before) + self.assertEqual(receipt['kernelRetrieval'], 'not-measured') + self.assertNotIn('sdkPayloadChecks', receipt) + self.assertFalse(any(Path(argv[0]).name == 'prose-agents-sdk' for argv, _ in self.calls)) + self.assertTrue(any(argv[:2] == ['brew', 'link'] for argv, _ in self.calls)) + self.assertTrue(receipt['uninstallPassed']) + self.assertEqual(self.kegs, {}) + + def test_development_placeholder_rejects_sdk_claim_presence_before_output(self): + self.manifest.update(mode='development', releaseEligible=False, publicationAuthorized=False, + agentsSdk='not-packaged-development-fixture') + self.verified.pop('sdkBuild') + for field in ('sdkBuild', 'sdkEvidence', 'sdkSourceCustody'): + for value in (None, {}, False): + self.verified[field] = value + with self.subTest(field=field, value=value), self.assertRaises(ValueError): + self.exercise(previous=False) + self.assertFalse(self.output.exists()) + self.assertEqual(self.calls, []) + del self.verified[field] + def test_both_genuine_upgrades_preserve_selection_settings_and_scrub_credentials(self): with patch.dict(rehearsal.os.environ, {'OPENAI_API_KEY': 'ambient-secret', 'HTTPS_PROXY': 'ambient-proxy'}): receipt = self.exercise() diff --git a/cli/ci/test_sdk_native_inventory.py b/cli/ci/test_sdk_native_inventory.py index 62d78a3a..1f49ac10 100644 --- a/cli/ci/test_sdk_native_inventory.py +++ b/cli/ci/test_sdk_native_inventory.py @@ -3,6 +3,7 @@ import copy import hashlib import json +import os from pathlib import Path import struct import sys @@ -350,7 +351,7 @@ def test_python_metadata_string_version_does_not_bypass_native_origin_guards(sel def test_wheel_native_and_license_members_are_checked_against_record(self): with tempfile.TemporaryDirectory() as temp: - root = Path(temp); paths = ['fixture.dist-info/RECORD', 'fixture.dist-info/LICENSE', 'extension.so'] + root = Path(temp).resolve(); paths = ['fixture-1.0.dist-info/RECORD', 'fixture-1.0.dist-info/LICENSE', 'extension.so'] files = [] for name in paths: path = root / name; path.parent.mkdir(exist_ok=True); path.write_bytes(elf(imported=True) if name.endswith('.so') else b'fixture license or RECORD') @@ -358,7 +359,7 @@ def test_wheel_native_and_license_members_are_checked_against_record(self): class Entry(str): pass entry = Entry(name); entry.hash = SimpleNamespace(mode='sha256', value=digest); entry.size = path.stat().st_size files.append(entry) - dist = SimpleNamespace(version='1.0', files=files, locate_file=lambda e: root / str(e)) + dist = SimpleNamespace(version='1.0', metadata={'Name': 'fixture'}, files=files, locate_file=lambda e: root / str(e)) package = {'name': 'fixture', 'version': '1.0', 'wheelSha256': ['a' * 64]} origins, owned = native.wheel_origins([package], lambda _: dist) self.assertEqual(origins[0]['wheelCandidateSha256s'], ['a' * 64]) @@ -673,5 +674,146 @@ def test_unknown_provider_never_skips_missing_declared_text(self): with self.assertRaises(ValueError): native.python_origin(value) +class OwningWheelRecordTests(unittest.TestCase): + # Mirrors the authenticated setuptools84 owning inventory: twelve vendored + # RECORD resources plus one direct owner, without importing wheel code. + vendors = ('autocommand-2.2.2', 'backports.tarfile-1.2.0', 'importlib_metadata-8.7.1', + 'jaraco.text-4.0.0', 'jaraco_context-6.1.0', 'jaraco_functools-4.4.0', + 'more_itertools-10.8.0', 'packaging-26.0', 'platformdirs-4.4.0', + 'tomli-2.4.0', 'wheel-0.46.3', 'zipp-3.23.0') + + def fixture(self, root, name='setuptools', directory='setuptools-84.0.0.dist-info'): + root = root.resolve() + names = [directory + '/RECORD', directory + '/LICENSE', 'extension.so'] + [ + 'setuptools/_vendor/' + vendor + '.dist-info/RECORD' for vendor in self.vendors] + class Entry(str): pass + files = [] + for path in names: + data = elf(imported=True) if path.endswith('.so') else (b'license text' if path.endswith('/LICENSE') else b'vendor resource') + target = root / path; target.parent.mkdir(parents=True, exist_ok=True); target.write_bytes(data) + entry = Entry(path); entry.hash = SimpleNamespace(mode='sha256', value=base64.urlsafe_b64encode(hashlib.sha256(data).digest()).rstrip(b'=').decode()); entry.size = len(data); files.append(entry) + record_path = root / names[0] + record_path.write_bytes(('\n'.join(str(e) + ',' + ('sha256=' + e.hash.value if e != files[0] else '') + ',' + (str(e.size) if e != files[0] else '') for e in files) + '\r\n').encode()) + dist = SimpleNamespace(metadata={'Name': name}, version='84.0.0', files=files, locate_file=lambda entry: root / str(entry)) + package = {'name': name, 'version': '84.0.0', 'wheelSha256': ['a' * 64]} + return package, dist, record_path + + def test_nested_vendor_inventory_keeps_exact_owner_bytes_and_native_license_guards(self): + with tempfile.TemporaryDirectory() as temp: + package, dist, record_path = self.fixture(Path(temp)) + self.assertEqual(sum(str(e).endswith('.dist-info/RECORD') for e in dist.files), 13) + origins, owned = native.wheel_origins([package], lambda _: dist) + self.assertEqual(origins[0]['recordSha256'], native.sha(record_path.read_bytes())) + self.assertEqual(len(origins[0]['licenses']), 1) + self.assertEqual(owned[native.sha(elf(imported=True))], 'setuptools') + original = record_path.read_bytes(); record_path.write_bytes(original + b'\n') + self.assertNotEqual(native.wheel_origins([package], lambda _: dist)[0][0]['recordSha256'], origins[0]['recordSha256']) + vendor = Path(dist.locate_file(dist.files[3])); vendor.write_bytes(b'different nested RECORD') + self.assertEqual(native.wheel_origins([package], lambda _: dist)[0][0]['recordSha256'], native.sha(record_path.read_bytes())) + + def test_missing_duplicate_and_foreign_owner_refuse_without_vendor_fallback(self): + for mutation in ('missing', 'duplicate', 'foreign-name', 'foreign-version'): + with self.subTest(mutation=mutation), tempfile.TemporaryDirectory() as temp: + package, dist, _ = self.fixture(Path(temp)) + if mutation == 'missing': dist.files = dist.files[1:] + elif mutation == 'duplicate': dist.files.append(dist.files[0]) + else: dist.files[0] = ('foreign-84.0.0' if mutation == 'foreign-name' else 'setuptools-83.0.0') + '.dist-info/RECORD' + with self.assertRaisesRegex(ValueError, 'absent or ambiguous'): native.wheel_origins([package], lambda _: dist) + + def test_metadata_name_and_exact_version_must_match_lock(self): + for field, value in [('name', 'foreign'), ('name', None), ('name', 'bad/name'), ('version', '83.0.0')]: + with self.subTest(field=field, value=value), tempfile.TemporaryDirectory() as temp: + package, dist, _ = self.fixture(Path(temp)) + if field == 'name': dist.metadata['Name'] = value + else: dist.version = value + with self.assertRaises(ValueError): native.wheel_origins([package], lambda _: dist) + + def test_documented_name_normalization_applies_to_lock_metadata_and_directory(self): + for directory in ('FOO_bar', 'foo.bar', 'foo-bar'): + with self.subTest(directory=directory), tempfile.TemporaryDirectory() as temp: + package, dist, _ = self.fixture(Path(temp), name='Foo-Bar', directory=directory + '-84.0.0.dist-info') + dist.metadata['Name'] = 'FOO_bar' + self.assertEqual(native.wheel_origins([package], lambda _: dist)[0][0]['name'], 'Foo-Bar') + + def test_unsafe_record_paths_refuse(self): + for path in ('/setuptools-84.0.0.dist-info/RECORD', '../setuptools-84.0.0.dist-info/RECORD', + './setuptools-84.0.0.dist-info/RECORD', 'C:setuptools-84.0.0.dist-info/RECORD', + 'setuptools-84.0.0.dist-info/\x00.dist-info/RECORD', 'setuptools\n-84.0.0.dist-info/RECORD', + 'setuptools\\-84.0.0.dist-info/RECORD'): + with self.subTest(path=path), tempfile.TemporaryDirectory() as temp: + package, dist, _ = self.fixture(Path(temp)); dist.files[0] = path + with self.assertRaises(ValueError): native.wheel_origins([package], lambda _: dist) + + def test_owner_leaf_and_parent_aliases_refuse_even_with_identical_bytes(self): + for alias in ('leaf', 'parent'): + with self.subTest(alias=alias), tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); package, dist, record_path = self.fixture(root) + if alias == 'leaf': + original = root / 'original-record'; record_path.rename(original); record_path.symlink_to(original) + else: + original = root / 'original-metadata'; record_path.parent.rename(original); record_path.parent.symlink_to(original, target_is_directory=True) + with self.assertRaises(ValueError): native.wheel_origins([package], lambda _: dist) + + def test_native_and_license_bytes_hash_size_algorithm_and_origin_ambiguity_stay_strict(self): + for poison in ('native-bytes', 'license-bytes', 'size', 'algorithm', 'missing-hash', 'duplicate-native'): + with self.subTest(poison=poison), tempfile.TemporaryDirectory() as temp: + package, dist, _ = self.fixture(Path(temp)); entry = dist.files[2] + if poison == 'native-bytes': Path(dist.locate_file(entry)).write_bytes(elf(imported=True, symbol='changed')) + elif poison == 'license-bytes': Path(dist.locate_file(dist.files[1])).write_bytes(b'changed license') + elif poison == 'size': entry.size += 1 + elif poison == 'algorithm': entry.hash.mode = 'md5' + elif poison == 'missing-hash': entry.hash = None + else: dist.files.append(entry) + with self.assertRaises(ValueError): native.wheel_origins([package], lambda _: dist) + + def test_consistent_public_base_alias_and_relative_base_are_supported(self): + for layout in ('alias', 'relative'): + with self.subTest(layout=layout), tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); package, dist, record_path = self.fixture(root) + if layout == 'alias': + proxy = root / 'alias-root'; proxy.symlink_to(root, target_is_directory=True) + dist.locate_file = lambda entry: proxy / str(entry) + result = native.wheel_origins([package], lambda _: dist) + else: + relative_base = Path(os.path.relpath(root)) + dist.locate_file = lambda entry: relative_base / str(entry) + result = native.wheel_origins([package], lambda _: dist) + self.assertEqual(result[0][0]['recordSha256'], native.sha(record_path.read_bytes())) + + def test_inconsistent_public_base_cannot_select_identical_foreign_record(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); package, dist, _ = self.fixture(root / 'owner') + _, _, other = self.fixture(root / 'other') + dist.locate_file = lambda entry: root / 'owner' if str(entry) == '' else root / 'other' / str(entry) + with self.assertRaisesRegex(ValueError, 'distribution base'): + native.wheel_origins([package], lambda _: dist) + + def test_native_digest_cannot_claim_two_different_distributions(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve() + first, first_dist, _ = self.fixture(root / 'first') + second, second_dist, _ = self.fixture(root / 'second', name='other', directory='other-84.0.0.dist-info') + with self.assertRaisesRegex(ValueError, 'Ambiguous installed native wheel origin'): + native.wheel_origins([first, second], lambda name: first_dist if name == first['name'] else second_dist) + + def test_missing_parent_and_relative_locator_fail_closed(self): + for location in ('absent', 'relative'): + with self.subTest(location=location), tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); package, dist, _ = self.fixture(root) + dist.locate_file = lambda entry: (root / 'absent' / str(entry)) if location == 'absent' else Path(str(entry)) + with self.assertRaisesRegex(ValueError, 'distribution base'): + native.wheel_origins([package], lambda _: dist) + + def test_nested_license_and_native_resources_remain_in_complete_inventory(self): + with tempfile.TemporaryDirectory() as temp: + root = Path(temp).resolve(); package, dist, _ = self.fixture(root) + extra = copy.copy(dist.files[1]); extra = type(extra)('setuptools/_vendor/vendor-1.0.dist-info/LICENSE') + extra.hash = SimpleNamespace(mode='sha256', value=base64.urlsafe_b64encode(hashlib.sha256(b'vendor license').digest()).rstrip(b'=').decode()); extra.size = len(b'vendor license') + p = root / str(extra); p.parent.mkdir(parents=True); p.write_bytes(b'vendor license'); dist.files.append(extra) + self.assertEqual(len(native.wheel_origins([package], lambda _: dist)[0][0]['licenses']), 2) + p.write_bytes(b'bad vendor license') + with self.assertRaisesRegex(ValueError, 'wheel RECORD'): native.wheel_origins([package], lambda _: dist) + + if __name__ == '__main__': unittest.main() diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 91eecc4e..76a3e067 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1175,3 +1175,11 @@ Root owns `cli/ci/homebrew_rehearsal.py` and `docs/packaged-sdk-design.md` for p Seventh-cohort selector fixture lease extension: `/root/configuration_contract` additionally owns only `cli/ci/test_build_agents_sdk.py` to replace the legacy tuple2 positive mocked Linux Analysis fixture with the pinned tuple20 shape containing separate input field10 and final field15 supplier rows. Preserve all original builder supplier/archive/receipt/ELF controls; no production builder/source change or actual execution is authorized by this test-only extension. + + +## IMP-098 eighth-cohort owning wheel RECORD repair + +After completed independent current Mac audits, root and both peers accepted proposal `c2deacc436e3ee17784ab6e4b13e71019e138acbe75917e3dbfb73093a20b7da` and original pinned-wheel/CPython evidence `81441015f407d5719a3ba296a78dd141b7c69a2271409edf1c9ebec1eb06f129`, recorded before implementation in workspace main64652c6. `/root/configuration_contract` owns only `cli/ci/sdk_native_inventory.py` and `cli/ci/test_sdk_native_inventory.py` for direct owning RECORD selection. Require normalized resolved metadata Name agreement with locked package Name, exact locked Version, exactly one safe two-component owning name/version .dist-info/RECORD and canonical original regular path including parent-alias refusal. Do not use private metadata paths, nested suffix fallback, text round-trip or skipped original inventory rows. Retain complete original RECORD bytes/digest, all native/license hash/size/algorithm/ambiguity and ELF/supplier/closure/runtime guards. Meaningful controls must cover real pinned nested-vendor shape, missing/duplicate/foreign owner, wrong metadata identity, unsafe path/parent aliases and unchanged modified native/license refusals. Fix only existing synthetic owning fixtures as needed inside the leased test file. Agents use pure/synthetic/read-only checks only; root owns full25, actual native/helper qualification, Git, push and merge. Prior4e80 artifacts and failed originals remain unchanged and source-scoped. + + +Eighth-cohort development Homebrew routing lease: root owns `cli/ci/homebrew_rehearsal.py`; `/root/operational_design` owns only `cli/ci/test_homebrew_rehearsal.py` for peer-accepted proposal `1c5373c19fc83141a69ec0e7b90d2fd610199497e2ea7e5dae66c696fa90105b`, recorded before implementation in IMP-098. Reuse strict existing rehearsal SDK identity classification for present SDK claims and use its result consistently in archive and exercise paths without changing original manifest/provenance. Literal sentinel requires development and false eligibility/publication flags; production missing SDK, malformed present claims and no-SDK helper/support/receipt/notices archive or SDK evidence/source claims refuse. Preserve genuine production complete custody, all existing collision/upgrade/preservation controls and original failed evidence. Agents run focused pure/synthetic controls only; root owns full25, actual native/Git/push/merge. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index d37a95c8..11c20dc1 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -118,3 +118,8 @@ setup classification; preserve valid observations already received. The native Homebrew rehearsal uses the public Formula `preserve_rpath` convention to retain the SDK libraries' relative dylib identities and `skip_clean` scoped to `bin/prose`, `bin/prose-agents-sdk` and `bin/prose-agents-sdk-runtime`. Homebrew continues its other linkage checks. The complete signed SDK tree must retain its declared bytes, modes and directories through installation; neither a cleanup exclusion nor relative-ID preservation alone establishes that custody. Real install, genuine upgrade, collision refusal and cold helper probes remain qualification gates. The formula keeps the executable and its SDK helper/support tree as canonical keg siblings. The Linux freezer verifier reads the final `binaries` field in the pinned PyInstaller Analysis TOC. Its input binary list is a distinct record and cannot establish final selection. Exactly one final libgcc destination must bind to the pinned supplier path and bytes; complete TOC provenance and downstream archive, ELF, origin and symbol-closure checks remain required. + + +Wheel origin inspection selects the direct owning distribution's `.dist-info/RECORD`, binding its normalized metadata name and exact version to the lock. `Distribution.files` also lists vendored metadata resources; those remain part of the complete inventory but cannot substitute for the owning RECORD. The original RECORD bytes and native/license file hashes and sizes remain required. Selection anchors to the public distribution base, permitting consistent base aliases while refusing aliases of the owning metadata directory and foreign locators. + +The development Homebrew rehearsal recognizes the existing explicit SDK-free fixture marker only in non-publishable development context. It retains the original manifest and uses the classified SDK identity consistently for archive and installation checks. An SDK-free context cannot contain packaged helper/support/receipt/notices members or SDK evidence claims. Production qualification continues to require the complete packaged SDK identity and custody. From ba212da856255b421b55c4bfde26b9b60866936d Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 17:44:24 -0400 Subject: [PATCH 61/64] fix: permit bounded Linux runtime extraction mappings --- cli/ci/build_agents_sdk_linux.py | 6 ++++- cli/ci/test_build_agents_sdk_linux.py | 34 +++++++++++++++++++++++++++ cli/protocol/OWNERSHIP.md | 5 ++++ docs/packaged-sdk-design.md | 3 +++ 4 files changed, 47 insertions(+), 1 deletion(-) diff --git a/cli/ci/build_agents_sdk_linux.py b/cli/ci/build_agents_sdk_linux.py index b4d1654e..aab2cc2c 100644 --- a/cli/ci/build_agents_sdk_linux.py +++ b/cli/ci/build_agents_sdk_linux.py @@ -161,7 +161,11 @@ def validate_image(info,row,key): def container_command(row, image_key, source, output, script, *, network='none', owner=None): + require(isinstance(image_key,str) and image_key in ('supplier','freezer','runtime'), + 'Closed container image role required') require(network in ('none','bridge'),'Closed network mode') + require(image_key != 'runtime' or network == 'none', 'Runtime network must be none') + tmpfs = '/tmp:rw,exec,nosuid,nodev,size=536870912' if image_key == 'runtime' else '/tmp:rw,nosuid,nodev,size=536870912' owner=owner or uuid.uuid4().hex require(re.fullmatch('[0-9a-f]{32}',owner), 'Invalid container owner') return ['docker','--config',str(output/'docker-config'),'run','--rm','--pull=never', @@ -169,7 +173,7 @@ def container_command(row, image_key, source, output, script, *, network='none', '--label',OWNER_LABEL+'='+owner, '--platform',row['dockerPlatform'],'--network='+network,'--read-only', '--cap-drop=ALL','--security-opt=no-new-privileges','--user',str(os.getuid())+':'+str(os.getgid()), - '--tmpfs','/tmp:rw,nosuid,nodev,size=536870912', + '--tmpfs',tmpfs, '--mount','type=bind,src='+str(source)+',dst=/source,readonly', '--mount','type=bind,src='+str(output)+',dst=/job', '--env','HOME=/job/home','--env','TMPDIR=/tmp','--env','PYTHONNOUSERSITE=1', diff --git a/cli/ci/test_build_agents_sdk_linux.py b/cli/ci/test_build_agents_sdk_linux.py index e612646f..78fecc83 100644 --- a/cli/ci/test_build_agents_sdk_linux.py +++ b/cli/ci/test_build_agents_sdk_linux.py @@ -60,6 +60,40 @@ def test_preparation_network_is_explicit_and_freeze_is_not(self): self.assertIn('--network=bridge',d.container_command(r,'freezer',Path('/source'),Path('/job'),'prepare.sh',network='bridge')) with self.assertRaises(ValueError):d.container_command(r,'freezer',Path('/source'),Path('/job'),'freeze.sh',network='host') + def test_both_runtime_targets_use_bounded_executable_tmpfs_without_widening_isolation(self): + import verify_agents_sdk_linux as runtime + for target,row in runtime.load_lock()['platforms'].items(): + with self.subTest(target=target): + c=d.container_command(row,'runtime',Path('/source-owned'),Path('/job-owned'),'version.sh',owner='a'*32) + self.assertEqual(c[c.index('--tmpfs')+1],'/tmp:rw,exec,nosuid,nodev,size=536870912') + self.assertEqual(c.count('--tmpfs'),1);self.assertEqual(c.count('--mount'),2) + for option in ('--network=none','--read-only','--pull=never','--rm','--cap-drop=ALL','--security-opt=no-new-privileges'): + self.assertIn(option,c) + self.assertEqual(c[c.index('--user')+1],str(d.os.getuid())+':'+str(d.os.getgid())) + self.assertEqual(c[c.index('--platform')+1],row['dockerPlatform']) + self.assertIn('type=bind,src=/source-owned,dst=/source,readonly',c) + self.assertIn('type=bind,src=/job-owned,dst=/job',c) + self.assertIn('TMPDIR=/tmp',c);self.assertIn('--name',c) + self.assertIn(d.OWNER_LABEL+'='+'a'*32,c) + self.assertEqual(c[-4:],[row['runtime']['image'],'-euo','pipefail','/job/version.sh']) + self.assertNotIn('--privileged',c);self.assertNotIn('--cap-add',c);self.assertNotIn('--env-file',c) + self.assertEqual(runtime.PROBES,(('version','--version',5),('imports','--packaged-self-test',30),('tools','--packaged-tool-self-test',30),('libraries','--packaged-library-test',30))) + def test_unknown_roles_and_runtime_network_refuse_before_owner_or_command_formation(self): + row=d.load_lock()['platforms']['linux-x64-gnu'] + for role,network in [('unknown','none'),('', 'none'),(None,'none'),([], 'none'),(True,'none'),('runtime','bridge'),('runtime','host')]: + with self.subTest(role=role,network=network),patch.object(d.uuid,'uuid4',side_effect=AssertionError('Must refuse before ownership')): + with self.assertRaises(ValueError):d.container_command(row,role,Path('/source'),Path('/job'),'version.sh',network=network) + def test_both_preparation_roles_retain_original_tmpfs_arguments(self): + for target,row in d.load_lock()['platforms'].items(): + for role,script,network in [('supplier','supplier.sh','none'),('freezer','prepare.sh','bridge'),('freezer','freeze.sh','none')]: + with self.subTest(target=target,role=role,script=script): + c=d.container_command(row,role,Path('/source-owned'),Path('/job-owned'),script,network=network,owner='a'*32) + self.assertEqual(c[c.index('--tmpfs')+1],'/tmp:rw,nosuid,nodev,size=536870912') + self.assertIn('--network='+network,c);self.assertIn('TMPDIR=/tmp',c) + self.assertEqual(c.count('--tmpfs'),1);self.assertEqual(c.count('--mount'),2) + self.assertNotIn('/tmp:rw,exec,nosuid,nodev,size=536870912',c) + + class RealOrchestratorControls(unittest.TestCase): diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 76a3e067..0a28ca75 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1183,3 +1183,8 @@ After completed independent current Mac audits, root and both peers accepted pro Eighth-cohort development Homebrew routing lease: root owns `cli/ci/homebrew_rehearsal.py`; `/root/operational_design` owns only `cli/ci/test_homebrew_rehearsal.py` for peer-accepted proposal `1c5373c19fc83141a69ec0e7b90d2fd610199497e2ea7e5dae66c696fa90105b`, recorded before implementation in IMP-098. Reuse strict existing rehearsal SDK identity classification for present SDK claims and use its result consistently in archive and exercise paths without changing original manifest/provenance. Literal sentinel requires development and false eligibility/publication flags; production missing SDK, malformed present claims and no-SDK helper/support/receipt/notices archive or SDK evidence/source claims refuse. Preserve genuine production complete custody, all existing collision/upgrade/preservation controls and original failed evidence. Agents run focused pure/synthetic controls only; root owns full25, actual native/Git/push/merge. + + +## IMP-098 ninth-cohort minimal runtime tmpfs compatibility + +After complete27690 Mac custody audits, root and both peers selected proposal `0ac213f2ad3f54b99c66bd9aa31bd292604fb308b5fa24d5e61d8f4cf892270a` and primary/actual-x64 evidence `e360df420fd1672feeef0d1ca4308080e33eafbfad916730964d93f0b2e7a19f`, with Docker/PyInstaller comparison recorded in IMP-098 before implementation. `/root/sdk_assessment` owns only `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` for exact runtime-only `/tmp:rw,exec,nosuid,nodev,size=536870912` selection in the shared lifecycle command. Close image roles to supplier/freezer/runtime and refuse runtime network bridge or unknown/non-string roles before command formation. Supplier/freezer tmpfs arguments remain byte-identical. No arbitrary mount/execution parameter, extra host bind, baked extraction path, privileged/user/capability widening or limit change. Preserve pinned image/platform/source/trio/receipt/native closure,512MiB size, read-only root/source, cap-drop/no-new-privileges/unprivileged uid/gid, owner/cleanup and exact five-second/30-second probes. Meaningful pure controls cover both targets, runtime isolation/role/network guards and unchanged preparation roles. Configuration agent independently reviews final code/controls. Agents use pure/synthetic/read-only work; root owns full26, actual native replay, source integration/Git/push/merge. Original failures and successful27690 artifacts remain immutable and scoped. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 11c20dc1..796437f9 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -123,3 +123,6 @@ The Linux freezer verifier reads the final `binaries` field in the pinned PyInst Wheel origin inspection selects the direct owning distribution's `.dist-info/RECORD`, binding its normalized metadata name and exact version to the lock. `Distribution.files` also lists vendored metadata resources; those remain part of the complete inventory but cannot substitute for the owning RECORD. The original RECORD bytes and native/license file hashes and sizes remain required. Selection anchors to the public distribution base, permitting consistent base aliases while refusing aliases of the owning metadata directory and foreign locators. The development Homebrew rehearsal recognizes the existing explicit SDK-free fixture marker only in non-publishable development context. It retains the original manifest and uses the classified SDK identity consistently for archive and installation checks. An SDK-free context cannot contain packaged helper/support/receipt/notices members or SDK evidence claims. Production qualification continues to require the complete packaged SDK identity and custody. + + +Linux minimal-runtime verification uses explicit `exec` on its existing 512 MiB temporary filesystem so PyInstaller onefile can map the libraries it extracts. Docker's default noexec tmpfs conflicts with this documented extraction requirement. The runtime role remains network-free, unprivileged, read-only outside its existing mounts and subject to the same capability, ownership, cleanup and five-second/30-second probe guards. Supplier and freezer temporary mount options remain unchanged. This is a qualification-container setting; artifact extraction paths and consumer configuration are not overridden. From ec2ab54bf00b0a0bafd99043fe6d3ee137388c83 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 18:26:10 -0400 Subject: [PATCH 62/64] Retain shell ownership through subprocess cancellation --- cli/protocol/OWNERSHIP.md | 5 + docs/packaged-sdk-design.md | 3 + harnesses/agents-sdk/run.py | 126 +++++++++-- harnesses/agents-sdk/test_run.py | 369 +++++++++++++++++++++++++++++-- 4 files changed, 462 insertions(+), 41 deletions(-) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index 0a28ca75..e5a5bb22 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1188,3 +1188,8 @@ Eighth-cohort development Homebrew routing lease: root owns `cli/ci/homebrew_reh ## IMP-098 ninth-cohort minimal runtime tmpfs compatibility After complete27690 Mac custody audits, root and both peers selected proposal `0ac213f2ad3f54b99c66bd9aa31bd292604fb308b5fa24d5e61d8f4cf892270a` and primary/actual-x64 evidence `e360df420fd1672feeef0d1ca4308080e33eafbfad916730964d93f0b2e7a19f`, with Docker/PyInstaller comparison recorded in IMP-098 before implementation. `/root/sdk_assessment` owns only `cli/ci/build_agents_sdk_linux.py` and `cli/ci/test_build_agents_sdk_linux.py` for exact runtime-only `/tmp:rw,exec,nosuid,nodev,size=536870912` selection in the shared lifecycle command. Close image roles to supplier/freezer/runtime and refuse runtime network bridge or unknown/non-string roles before command formation. Supplier/freezer tmpfs arguments remain byte-identical. No arbitrary mount/execution parameter, extra host bind, baked extraction path, privileged/user/capability widening or limit change. Preserve pinned image/platform/source/trio/receipt/native closure,512MiB size, read-only root/source, cap-drop/no-new-privileges/unprivileged uid/gid, owner/cleanup and exact five-second/30-second probes. Meaningful pure controls cover both targets, runtime isolation/role/network guards and unchanged preparation roles. Configuration agent independently reviews final code/controls. Agents use pure/synthetic/read-only work; root owns full26, actual native replay, source integration/Git/push/merge. Original failures and successful27690 artifacts remain immutable and scoped. + + +## IMP-098 tenth-cohort shell acquisition and cancellation + +Root recorded conventional asyncio/CPython comparison before implementation: proposal `aa48ff2dc1b0cf5321c952bce1392f93bf67d25fe556972e3aa69772dab26477`, independent design review `86005eaa69ff8b2d590fd2b4e00db24086f3b3b48879885e12f73e129ab995d9`. `/root/sdk_assessment` owns only `harnesses/agents-sdk/run.py` and `harnesses/agents-sdk/test_run.py`: retain shielded spawn acquisition and cleanup through repeated cancellation; attempt owned group kill, reader settlement and reap while preserving primary exceptions. Use explicit descendant readiness and post-cancellation effect release in the fixture. Closed safe secondary cleanup phase/type/integer-errno details survive existing tool/native errors through conditional shellCleanupFailures; no raw exception/command/environment/output or new event types. Remove unconditional group-termination guarantees. Preserve shell argv/cwd/env/new-session/PIPE/draining/output limits, model/usage/defaults and actual completion timeout. Existing acquisition/exceptional reap stay outside that timeout; no total cleanup deadline or detached containment is asserted. Configuration peer reviews final implementation and meaningful focused controls. Root owns docs/ownership, integration, mandatory full27, native replay, Git/push/merge; original failed and qualified evidence remains immutable. diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 796437f9..5013f7e2 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -126,3 +126,6 @@ The development Homebrew rehearsal recognizes the existing explicit SDK-free fix Linux minimal-runtime verification uses explicit `exec` on its existing 512 MiB temporary filesystem so PyInstaller onefile can map the libraries it extracts. Docker's default noexec tmpfs conflicts with this documented extraction requirement. The runtime role remains network-free, unprivileged, read-only outside its existing mounts and subject to the same capability, ownership, cleanup and five-second/30-second probe guards. Supplier and freezer temporary mount options remain unchanged. This is a qualification-container setting; artifact extraction paths and consumer configuration are not overridden. + + +Shell cancellation retains and shields subprocess acquisition until it obtains the owned process handle, then retains cleanup through repeated cancellation. Cleanup attempts process-group termination, capture settlement and reaping without replacing the initiating error. Safe secondary cleanup failures are carried in existing tool or native error records; cancellation and timeout messages do not promise group termination. Acquisition and exceptional reaping retain their existing placement outside the command completion timeout. This does not establish a total cleanup deadline or detached-process containment. The descendant cancellation fixture explicitly orders readiness, cancellation and effect release rather than racing fixed sleeps. diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index c7037987..ee56576c 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -97,11 +97,15 @@ class BudgetExceeded(Exception): async def shell(command, cwd, timeout, env, output_limit=30000): - process = await asyncio.create_subprocess_exec( + # Retain ownership while subprocess creation connects its pipes. Cancellation + # of that constructor otherwise lets asyncio kill only the direct child. + acquisition = asyncio.create_task(asyncio.create_subprocess_exec( '/bin/bash', '-c', command, cwd=cwd, env=env, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, start_new_session=True, - ) + )) + process = None + readers = [] async def capture(stream): kept = bytearray() total = 0 @@ -111,22 +115,53 @@ async def capture(stream): return bytes(kept), total > output_limit total += len(chunk) kept.extend(chunk[:max(0, output_limit - len(kept))]) - readers = [asyncio.create_task(capture(stream)) for stream in (process.stdout, process.stderr)] async def complete(): captured = await asyncio.gather(*readers) await process.wait() return captured try: + process = await asyncio.shield(acquisition) + readers = [asyncio.create_task(capture(stream)) for stream in (process.stdout, process.stderr)] stdout, stderr = await asyncio.wait_for(complete(), timeout) - except BaseException: - try: - os.killpg(process.pid, signal.SIGKILL) - except ProcessLookupError: - pass - for reader in readers: - reader.cancel() - await asyncio.gather(*readers, return_exceptions=True) - await process.wait() + except BaseException as primary: + async def cleanup(): + failures = [] + owned = process + if owned is None: + try: + owned = await acquisition + except BaseException as error: + failures.append(_shell_cleanup_failure('acquire', error)) + if owned is not None: + try: + os.killpg(owned.pid, signal.SIGKILL) + except ProcessLookupError: + pass + except BaseException as error: + failures.append(_shell_cleanup_failure('kill', error)) + for reader in readers: + reader.cancel() + settled = await asyncio.gather(*readers, return_exceptions=True) + for error in settled: + if isinstance(error, BaseException) and not isinstance(error, asyncio.CancelledError): + failures.append(_shell_cleanup_failure('readers', error)) + if owned is not None: + try: + await owned.wait() + except BaseException as error: + failures.append(_shell_cleanup_failure('reap', error)) + return failures + # Acquisition and exceptional reaping retain their existing placement + # outside the command-completion timeout. Shield is not a wall-time bound. + retained = asyncio.create_task(cleanup()) + while not retained.done(): + try: + await asyncio.shield(retained) + except asyncio.CancelledError: + continue + failures = retained.result() + if failures: + primary._shell_cleanup_failures = failures raise return {'exit_code': process.returncode, 'stdout': stdout[0].decode(errors='replace'), @@ -134,6 +169,51 @@ async def complete(): 'stdout_truncated': stdout[1], 'stderr_truncated': stderr[1]} +def _shell_cleanup_failure(phase, error): + allowed = (OSError, PermissionError, ProcessLookupError, FileNotFoundError, + BrokenPipeError, RuntimeError, ValueError, asyncio.TimeoutError, + asyncio.CancelledError) + result = {'phase': phase, 'errorType': type(error).__name__ if type(error) in allowed else 'ExecutionError'} + if type(error) in allowed and type(getattr(error, 'errno', None)) is int: + result['errno'] = error.errno + return result + + +def _shell_cleanup_details(error): + # Python 3.10 wait_for may wrap CancelledError and leave its original on + # __context__. Only this closed private metadata is copied, never messages. + seen = set() + for _ in range(8): + if not isinstance(error, BaseException) or id(error) in seen: + break + seen.add(id(error)) + # Use the built-in descriptors, rather than custom exception properties. + metadata = BaseException.__dict__['__dict__'].__get__(error) + failures = metadata.get('_shell_cleanup_failures') + if type(failures) is list and 0 < len(failures) <= 5: + counts = {'acquire': 0, 'kill': 0, 'readers': 0, 'reap': 0} + safe = [] + for row in failures: + if (type(row) is not dict or any(type(key) is not str for key in row) + or set(row) not in ({'phase', 'errorType'}, {'phase', 'errorType', 'errno'}) + or type(row['phase']) is not str or row['phase'] not in counts + or type(row['errorType']) is not str or row['errorType'] not in ( + 'OSError', 'PermissionError', 'ProcessLookupError', 'FileNotFoundError', + 'BrokenPipeError', 'RuntimeError', 'ValueError', 'TimeoutError', + 'CancelledError', 'ExecutionError') + or ('errno' in row and type(row['errno']) is not int)): + break + counts[row['phase']] += 1 + if counts[row['phase']] > (2 if row['phase'] == 'readers' else 1): + break + safe.append(dict(row)) + else: + return {'shellCleanupFailures': safe} + cause = BaseException.__cause__.__get__(error) + error = cause if cause is not None else BaseException.__context__.__get__(error) + return {} + + async def retrieve_public(url, timeout, output_limit=60000): """HTTPS GET with DNS-pinned public addresses, bounded body, no redirects.""" parsed = urlsplit(url) @@ -295,10 +375,10 @@ def emit(kind, **data): 'maxAggregateRequests': args.max_turns, 'maxAggregateHostedWebCalls': args.max_turns, 'maxAggregateFunctionTools': max_tools, 'maxObservedTotalTokens': max_total_tokens, 'maxRequestInputBytes': max_input_bytes, 'maxChildren': 8, 'maxChildDepth': 1} observation = UsageObservation(args.max_turns, max_tools, max_total_tokens) - def fail(kind, message, setup_reason=None): + def fail(kind, message, setup_reason=None, cleanup_error=None): extra = {"setup_reason": setup_reason} if kind == "SetupError" and setup_reason in ("credential-or-permission", "model-unavailable", "local-input") else {} emit('error', error_type=kind, message=message, limits=limits, - usageObservation=observation.summary(), **extra) + usageObservation=observation.summary(), **extra, **_shell_cleanup_details(cleanup_error)) return 1 emit('start', model=args.model, cwd=args.cwd, limits=limits, permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, @@ -334,10 +414,10 @@ async def execute_shell(command: str) -> str: emit('tool_call', name='execute_shell', command=command) try: result = await shell(command, cwd, min(args.tool_timeout, remaining()), tool_env) - except asyncio.TimeoutError: - result = {'error': 'shell timeout; process group terminated'} - except OSError: - result = {'error': 'shell could not start; check working directory and bash availability'} + except asyncio.TimeoutError as error: + result = {'error': 'shell command timed out', **_shell_cleanup_details(error)} + except OSError as error: + result = {'error': 'shell could not start; check working directory and bash availability', **_shell_cleanup_details(error)} emit('tool_result', name='execute_shell', result=result) return json.dumps(result) @function_tool(failure_error_function=None) @@ -394,16 +474,16 @@ async def delegate(task: str) -> str: 'output_tokens': usage['output_tokens'], 'total_tokens': usage['total_tokens']}, usageObservation=observation.summary()) return 0 - except asyncio.CancelledError: - return fail('CancelledError', 'Execution cancelled; active tool process groups terminated.') + except asyncio.CancelledError as error: + return fail('CancelledError', 'Execution cancelled.', cleanup_error=error) except Exception as error: name = type(error).__name__ if name in ('AuthenticationError', 'PermissionDeniedError'): - return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.', setup_reason='credential-or-permission') + return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.', setup_reason='credential-or-permission', cleanup_error=error) if name == 'NotFoundError': - return fail('SetupError', 'The configured OpenAI model is unavailable to this account. Check the model selection and account access; no fallback was selected.', setup_reason='model-unavailable') + return fail('SetupError', 'The configured OpenAI model is unavailable to this account. Check the model selection and account access; no fallback was selected.', setup_reason='model-unavailable', cleanup_error=error) safe = name if name in ('MaxTurnsExceeded', 'TimeoutError', 'BudgetExceeded') else 'ExecutionError' - return fail(safe, 'Execution stopped. Check configured limits, tool availability and OpenAI account access.') + return fail(safe, 'Execution stopped. Check configured limits, tool availability and OpenAI account access.', cleanup_error=error) finally: await client.close() diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index f6afb979..cf36b242 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -16,7 +16,39 @@ harness = importlib.util.module_from_spec(spec) spec.loader.exec_module(harness) +GATED_DESCENDANT = '(touch ready; read release < effect-gate; touch late) & wait' + +@contextlib.contextmanager +def descendant_gate(directory): + os.mkfifo(Path(directory, 'effect-gate')) + descriptor = os.open(Path(directory, 'effect-gate'), os.O_RDWR | os.O_NONBLOCK) + try: + yield descriptor + finally: + os.close(descriptor) + +async def descendant_ready(directory): + async def ready(): + while not Path(directory, 'ready').exists(): + await asyncio.sleep(.001) + await asyncio.wait_for(ready(), 1) + class ShellTest(unittest.IsolatedAsyncioTestCase): + async def test_descendant_effect_gate_releases_without_cancellation(self): + with tempfile.TemporaryDirectory() as directory, descendant_gate(directory) as release: + task = asyncio.create_task(harness.shell(GATED_DESCENDANT, directory, 2, {'PATH': os.environ['PATH']})) + try: + await descendant_ready(directory) + self.assertFalse(Path(directory, 'late').exists()) + os.write(release, b'go\n') + result = await asyncio.wait_for(task, 1) + self.assertEqual(result['exit_code'], 0) + self.assertTrue(Path(directory, 'late').exists()) + finally: + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + async def test_read_write_and_exit_status(self): with tempfile.TemporaryDirectory() as directory: Path(directory, 'in.txt').write_text('a\nb\n') @@ -40,14 +72,281 @@ async def test_large_output_is_bounded_while_pipe_drains(self): self.assertEqual(result['exit_code'], 0) async def test_cancellation_stops_delayed_descendant_effect(self): - with tempfile.TemporaryDirectory() as directory: - task = asyncio.create_task(harness.shell('(sleep 0.4; touch late) & wait', directory, 2, {'PATH': os.environ['PATH']})) - await asyncio.sleep(.05) + with tempfile.TemporaryDirectory() as directory, descendant_gate(directory) as release: + task = asyncio.create_task(harness.shell(GATED_DESCENDANT, directory, 2, {'PATH': os.environ['PATH']})) + try: + await descendant_ready(directory) + self.assertFalse(Path(directory, 'late').exists()) + task.cancel() + with self.assertRaises(asyncio.CancelledError): + await task + finally: + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + # This owner is outside the killed group. A surviving ready + # descendant receives the release rather than relying on a timer. + os.write(release, b'go\n') + await asyncio.sleep(.5) + self.assertFalse(Path(directory, 'late').exists()) + +class ShellOwnershipTest(unittest.IsolatedAsyncioTestCase): + class Process: + pid = 900001 # Every signal call in this class is mocked. + returncode = None + def __init__(self): + self.reading = asyncio.Event() + self.reaping = asyncio.Event() + self.reap_release = None + self.waits = 0 + self.stdout = SimpleNamespace(read=self.read) + self.stderr = SimpleNamespace(read=self.read) + async def read(self, size): + self.reading.set() + await asyncio.Future() + async def wait(self): + self.waits += 1 + self.reaping.set() + if self.reap_release is not None: + await self.reap_release.wait() + self.returncode = -9 + return -9 + + async def test_cancel_pending_acquisition_retains_handle_and_reaps(self): + for cancel_before_start in (False, True): + with self.subTest(cancel_before_start=cancel_before_start): + created, release = asyncio.Event(), asyncio.Event() + process = self.Process() + async def spawn(*args, **kwargs): + created.set() + await release.wait() + return process + with patch.object(harness.asyncio, 'create_subprocess_exec', spawn), patch.object(harness.os, 'killpg') as kill: + task = asyncio.create_task(harness.shell('private-command', '.', 1, {})) + if cancel_before_start: + # Start shell so its retained acquisition exists, but + # cancel before the acquisition coroutine is scheduled. + await asyncio.sleep(0) + else: + await asyncio.wait_for(created.wait(), 1) + task.cancel() + await asyncio.wait_for(created.wait(), 1) + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + release.set() + with self.assertRaises(asyncio.CancelledError): + await asyncio.wait_for(task, 1) + kill.assert_called_once_with(process.pid, harness.signal.SIGKILL) + self.assertEqual(process.waits, 1) + + async def test_repeated_cancel_during_reap_does_not_abandon_cleanup(self): + process = self.Process() + process.reap_release = asyncio.Event() + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg') as kill: + task = asyncio.create_task(harness.shell('ignored', '.', 1, {})) + await asyncio.wait_for(process.reading.wait(), 1) + task.cancel() + await asyncio.wait_for(process.reaping.wait(), 1) + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + process.reap_release.set() + with self.assertRaises(asyncio.CancelledError): + await asyncio.wait_for(task, 1) + kill.assert_called_once() + self.assertEqual(process.waits, 1) + + async def test_repeated_cancel_during_reader_settlement_does_not_skip_reap(self): + process = self.Process() + settling, release = asyncio.Event(), asyncio.Event() + async def read(size): + process.reading.set() + try: + await asyncio.Future() + except asyncio.CancelledError: + settling.set() + await release.wait() + raise + process.stdout = SimpleNamespace(read=read) + process.stderr = SimpleNamespace(read=read) + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg') as kill: + task = asyncio.create_task(harness.shell('ignored', '.', 1, {})) + await asyncio.wait_for(process.reading.wait(), 1) + task.cancel() + await asyncio.wait_for(settling.wait(), 1) + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + release.set() + with self.assertRaises(asyncio.CancelledError): + await asyncio.wait_for(task, 1) + kill.assert_called_once() + self.assertEqual(process.waits, 1) + + async def test_cancel_before_shell_start_creates_no_process(self): + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock()) as spawn, patch.object(harness.os, 'killpg') as kill: + task = asyncio.create_task(harness.shell('ignored', '.', 1, {})) task.cancel() with self.assertRaises(asyncio.CancelledError): await task - await asyncio.sleep(.5) - self.assertFalse(Path(directory, 'late').exists()) + spawn.assert_not_called() + kill.assert_not_called() + + async def test_acquisition_failure_and_late_failure_preserve_primary(self): + error = FileNotFoundError(2, 'private-path') + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(side_effect=error)), patch.object(harness.os, 'killpg') as kill: + with self.assertRaises(FileNotFoundError) as caught: + await harness.shell('ignored', '.', 1, {}) + self.assertIs(caught.exception, error) + kill.assert_not_called() + self.assertEqual(error._shell_cleanup_failures, [{'phase': 'acquire', 'errorType': 'FileNotFoundError', 'errno': 2}]) + created, release = asyncio.Event(), asyncio.Event() + async def late(*args, **kwargs): + created.set() + await release.wait() + raise error + with patch.object(harness.asyncio, 'create_subprocess_exec', late), patch.object(harness.os, 'killpg') as kill: + task = asyncio.create_task(harness.shell('ignored', '.', 1, {})) + await asyncio.wait_for(created.wait(), 1) + task.cancel() + await asyncio.sleep(0) + release.set() + with self.assertRaises(asyncio.CancelledError) as caught: + await asyncio.wait_for(task, 1) + self.assertEqual(harness._shell_cleanup_details(caught.exception)['shellCleanupFailures'][0]['phase'], 'acquire') + kill.assert_not_called() + + async def test_kill_failure_preserves_cancel_and_still_settles_and_reaps(self): + for error in (ProcessLookupError(3, 'private'), PermissionError(1, 'private'), OSError(5, 'private')): + process = self.Process() + with self.subTest(error=type(error).__name__), patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg', side_effect=error): + task = asyncio.create_task(harness.shell('ignored', '.', 1, {})) + await asyncio.wait_for(process.reading.wait(), 1) + task.cancel() + with self.assertRaises(asyncio.CancelledError) as caught: + await asyncio.wait_for(task, 1) + self.assertEqual(process.waits, 1) + details = harness._shell_cleanup_details(caught.exception) + if isinstance(error, ProcessLookupError): + self.assertEqual(details, {}) + else: + self.assertEqual(details['shellCleanupFailures'], [{'phase': 'kill', 'errorType': type(error).__name__, 'errno': error.errno}]) + self.assertNotIn('private', json.dumps(details)) + + async def test_reader_and_reap_failures_are_bounded_safe_and_primary_survives(self): + process = self.Process() + secret = type('SecretCommandAndToken', (Exception,), {})('private-output') + process.stdout = SimpleNamespace(read=AsyncMock(side_effect=secret)) + process.stderr = SimpleNamespace(read=AsyncMock(side_effect=secret)) + process.wait = AsyncMock(side_effect=OSError(5, 'private-path')) + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg') as kill: + with self.assertRaises(type(secret)) as caught: + await asyncio.wait_for(harness.shell('private-command', '.', 1, {}), 1) + self.assertIs(caught.exception, secret) + kill.assert_called_once() + process.wait.assert_awaited_once() + self.assertEqual(secret._shell_cleanup_failures, [ + {'phase': 'readers', 'errorType': 'ExecutionError'}, + {'phase': 'readers', 'errorType': 'ExecutionError'}, + {'phase': 'reap', 'errorType': 'OSError', 'errno': 5}]) + self.assertNotIn('private', json.dumps(harness._shell_cleanup_details(secret))) + self.assertNotIn('SecretCommandAndToken', json.dumps(harness._shell_cleanup_details(secret))) + + async def test_timeout_preserves_primary_and_cleans_owned_process(self): + process = self.Process() + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg', side_effect=PermissionError(1, 'private')): + with self.assertRaises(asyncio.TimeoutError) as caught: + await harness.shell('ignored', '.', .001, {}) + self.assertEqual(process.waits, 1) + self.assertEqual(caught.exception._shell_cleanup_failures, [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}]) + + async def test_empty_and_invalid_output_preserves_result_and_draining(self): + process = self.Process() + process.stdout = SimpleNamespace(read=AsyncMock(side_effect=[b'\xffabcd', b'ef', b''])) + process.stderr = SimpleNamespace(read=AsyncMock(return_value=b'')) + with patch.object(harness.asyncio, 'create_subprocess_exec', AsyncMock(return_value=process)), patch.object(harness.os, 'killpg') as kill: + result = await harness.shell('ignored', '.', 1, {}, output_limit=3) + self.assertEqual(result['stdout'], '\ufffdab') + self.assertTrue(result['stdout_truncated']) + self.assertEqual(result['stderr'], '') + self.assertFalse(result['stderr_truncated']) + self.assertEqual(process.stdout.read.await_count, 3) + kill.assert_not_called() + + def test_cleanup_metadata_is_closed_bounded_and_context_safe(self): + class Hostile(Exception): + @property + def _shell_cleanup_failures(self): + raise AssertionError('private property must not execute') + @property + def __context__(self): + raise AssertionError('private property must not execute') + @property + def __cause__(self): + raise AssertionError('private property must not execute') + self.assertEqual(harness._shell_cleanup_details(Hostile()), {}) + for row in ({'phase': 'private-command', 'errorType': 'OSError'}, + {'phase': 'kill', 'errorType': 'SecretToken'}, + {'phase': 'kill', 'errorType': 'OSError', 'errno': True}, + {'phase': 'kill', 'errorType': 'OSError', 'message': 'private'}, + {'phase': 'kill', 'errorType': 'OSError', 'errno': 1.0}): + error = RuntimeError('private-message') + error._shell_cleanup_failures = [row] + self.assertEqual(harness._shell_cleanup_details(error), {}) + error._shell_cleanup_failures = [{'phase': 'readers', 'errorType': 'ExecutionError'}] * 3 + self.assertEqual(harness._shell_cleanup_details(error), {}) + error.__context__ = error + self.assertEqual(harness._shell_cleanup_details(error), {}) + error._shell_cleanup_failures = [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}] + outer = asyncio.CancelledError() + outer.__context__ = error + self.assertEqual(harness._shell_cleanup_details(outer), {'shellCleanupFailures': error._shell_cleanup_failures}) + for _ in range(8): + wrapper = asyncio.CancelledError() + wrapper.__context__ = outer + outer = wrapper + self.assertEqual(harness._shell_cleanup_details(outer), {}) + + def test_hostile_chain_truth_and_metadata_key_methods_are_not_executed(self): + class HostileCause(Exception): + def __bool__(self): + raise AssertionError('exception truth must not execute') + cause = HostileCause() + cause._shell_cleanup_failures = [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}] + outer = asyncio.CancelledError() + outer.__cause__ = cause + self.assertEqual(harness._shell_cleanup_details(outer), {'shellCleanupFailures': cause._shell_cleanup_failures}) + class HostileKey: + armed = False + calls = 0 + def __hash__(self): + if self.armed: + self.calls += 1 + raise AssertionError('metadata key hash must not execute') + return hash('phase') + def __eq__(self, other): + if self.armed: + self.calls += 1 + raise AssertionError('metadata key equality must not execute') + return False + key = HostileKey() + row = {key: 'private', 'errorType': 'OSError'} + key.armed = True + error = RuntimeError() + error._shell_cleanup_failures = [row] + self.assertEqual(harness._shell_cleanup_details(error), {}) + self.assertEqual(key.calls, 0) + + async def test_wait_for_cancel_wrapper_preserves_safe_cleanup_context(self): + primary = asyncio.CancelledError('private-command') + primary._shell_cleanup_failures = [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}] + async def cancelled(): + raise primary + with self.assertRaises(asyncio.CancelledError) as caught: + await asyncio.wait_for(cancelled(), 1) + self.assertEqual(harness._shell_cleanup_details(caught.exception), {'shellCleanupFailures': primary._shell_cleanup_failures}) + self.assertNotIn('private', json.dumps(harness._shell_cleanup_details(caught.exception))) class BudgetTest(unittest.IsolatedAsyncioTestCase): def setUp(self): @@ -245,23 +544,22 @@ def response(number,name,arguments): return Response.model_validate(dict(id=str(number),object='response',created_at=1, model='fixture',output=[dict(type='function_call',id='call'+str(number),call_id='call'+str(number),name=name,arguments=json.dumps(arguments),status='completed')],parallel_tool_calls=False,tool_choice='auto',tools=[],status='completed', usage=dict(input_tokens=3,output_tokens=2,total_tokens=5,input_tokens_details=dict(cached_tokens=0,cache_write_tokens=0),output_tokens_details=dict(reasoning_tokens=0)))) - create=AsyncMock(side_effect=[response(1,'delegate',{'task':'explicit child only'}),response(2,'execute_shell',{'command':'(sleep 0.4; touch late) & wait'})]) + create=AsyncMock(side_effect=[response(1,'delegate',{'task':'explicit child only'}),response(2,'execute_shell',{'command':GATED_DESCENDANT})]) client=SimpleNamespace(responses=SimpleNamespace(create=create),base_url='https://api.openai.com/v1/',close=AsyncMock()) - started=asyncio.Event() - actual_shell=harness.shell - async def tracked_shell(*args,**kwargs): - # Set after process creation has had a chance to settle; cancellation remains in actual shell. - task=asyncio.create_task(actual_shell(*args,**kwargs)) - await asyncio.sleep(.03) - started.set() - return await task - with tempfile.TemporaryDirectory() as directory, patch.object(harness,'AsyncOpenAI',return_value=client), patch.object(harness,'shell',tracked_shell): + with tempfile.TemporaryDirectory() as directory, descendant_gate(directory) as release, patch.object(harness,'AsyncOpenAI',return_value=client): output=io.StringIO() with contextlib.redirect_stdout(output): task=asyncio.create_task(harness.run(self.args(directory))) - await asyncio.wait_for(started.wait(),1) - task.cancel() - self.assertEqual(await asyncio.wait_for(task,1),1) + try: + await descendant_ready(directory) + self.assertFalse(Path(directory,'late').exists()) + task.cancel() + self.assertEqual(await asyncio.wait_for(task,1),1) + finally: + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + os.write(release, b'go\n') await asyncio.sleep(.5) self.assertFalse(Path(directory,'late').exists()) records=[json.loads(line) for line in output.getvalue().splitlines()] @@ -403,6 +701,41 @@ async def test_cancelled_parent_has_safe_terminal_error(self): self.assertEqual(await harness.run(self.args(directory)), 1) self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'CancelledError') + async def test_shell_cleanup_failures_visible_in_native_errors_without_secrets(self): + for error in (asyncio.CancelledError('private-token'), RuntimeError('private-command')): + error._shell_cleanup_failures = [harness._shell_cleanup_failure('kill', PermissionError(1, 'private-path'))] + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', AsyncMock(side_effect=error)): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 1) + record = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(record['shellCleanupFailures'], [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}]) + self.assertEqual(record['error_type'], 'CancelledError' if isinstance(error, asyncio.CancelledError) else 'ExecutionError') + if isinstance(error, asyncio.CancelledError): + self.assertEqual(record['message'], 'Execution cancelled.') + self.assertNotIn('private', output.getvalue()) + self.assertNotIn('groups terminated', output.getvalue()) + + async def test_shell_cleanup_failures_visible_in_existing_tool_results(self): + for error in (asyncio.TimeoutError('private-command'), OSError(5, 'private-path')): + error._shell_cleanup_failures = [harness._shell_cleanup_failure('reap', RuntimeError('private-token'))] + async def invoke(agent, *args, **kwargs): + tool = next(tool for tool in agent.tools if tool.name == 'execute_shell') + raw = await tool.on_invoke_tool(ToolContext(context=None, tool_name='execute_shell', tool_call_id='fixture', tool_arguments='{}'), '{"command":"fixture"}') + self.assertEqual(json.loads(raw)['shellCleanupFailures'], [{'phase': 'reap', 'errorType': 'RuntimeError'}]) + return SimpleNamespace(final_output='done', context_wrapper=SimpleNamespace(usage=SimpleNamespace(requests=0,input_tokens=0,output_tokens=0,total_tokens=0))) + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', invoke), patch.object(harness, 'shell', AsyncMock(side_effect=error)): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(self.args(directory)), 0) + records = [json.loads(line) for line in output.getvalue().splitlines()] + result = next(row['result'] for row in records if row['type'] == 'tool_result') + self.assertEqual(result['shellCleanupFailures'], [{'phase': 'reap', 'errorType': 'RuntimeError'}]) + if isinstance(error, asyncio.TimeoutError): + self.assertEqual(result['error'], 'shell command timed out') + self.assertNotIn('private', output.getvalue()) + self.assertNotIn('process group terminated', output.getvalue()) + async def test_auth_and_model_failures_do_not_leak_bodies(self): for name in ('AuthenticationError', 'PermissionDeniedError', 'NotFoundError'): with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', AsyncMock(side_effect=type(name, (Exception,), {})('credential-value'))): From 5b10b3d502b478ed19002811da0af69021980173 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 19:19:53 -0400 Subject: [PATCH 63/64] Settle owned SDK tools and report kernel retrieval failures accurately --- cli/bun/src/core/errors.ts | 16 + cli/bun/src/core/kernel-startup.ts | 39 +- cli/bun/src/core/types.ts | 1 + cli/bun/test/kernel-startup.test.ts | 113 +- cli/protocol/OWNERSHIP.md | 14 + .../crates/prose-runner-core/src/error.rs | 13 +- .../prose-runner-core/src/kernel_startup.rs | 171 +- cli/shared/errors/taxonomy.v1.json | 8 + .../fixtures/adapters/kernel-startup.json | 47 +- cli/shared/fixtures/human/runner-errors.json | 17 + .../fixtures/kernel-startup/release.json | 1639 ++++++++++++++++- cli/shared/schemas/runner-error.schema.json | 206 ++- cli/shared/tests/test_contracts.py | 63 +- cli/shared/tests/test_service_contract.py | 14 +- docs/packaged-sdk-design.md | 6 + harnesses/agents-sdk/run.py | 116 +- harnesses/agents-sdk/test_run.py | 333 +++- 17 files changed, 2767 insertions(+), 49 deletions(-) diff --git a/cli/bun/src/core/errors.ts b/cli/bun/src/core/errors.ts index c540bf94..df7acb0d 100644 --- a/cli/bun/src/core/errors.ts +++ b/cli/bun/src/core/errors.ts @@ -46,3 +46,19 @@ export function hostedRunFailed(details: Record & { reason: str export function invocationFailure(reason: string): RunnerFailure { return failure("INVOCATION_INVALID", { reason }); } + +export type KernelRetrievalStage = "entry" | "descriptor" | "inventory" | "kernel"; +/** Fixed runner-owned diagnostics only; retryability does not initiate a retry. */ +export function kernelRetrievalFailure(stage: KernelRetrievalStage, httpStatus?: number): RunnerFailure { + if (!["entry", "descriptor", "inventory", "kernel"].includes(stage) || + (httpStatus !== undefined && (!Number.isInteger(httpStatus) || httpStatus < (stage === "entry" ? 400 : 100) || httpStatus > 599 || httpStatus === 200))) { + throw new RangeError("Invalid kernel retrieval observation."); + } + const error = failure("KERNEL_RETRIEVAL_FAILED", { + reason: httpStatus === undefined ? "Published kernel transport or response read failed." : "Published kernel HTTP request failed.", + stage, failureKind: httpStatus === undefined ? "transport" : "http", origin: "https://pkg.prose.md", + ...(httpStatus === undefined ? {} : { httpStatus }), + }); + const { schema: _schema, ...shape } = error.toJSON(); + return new RunnerFailure({ ...shape, retryable: httpStatus === undefined || [408, 429, 500, 502, 503, 504].includes(httpStatus) }); +} diff --git a/cli/bun/src/core/kernel-startup.ts b/cli/bun/src/core/kernel-startup.ts index 261dac60..6f19d66a 100644 --- a/cli/bun/src/core/kernel-startup.ts +++ b/cli/bun/src/core/kernel-startup.ts @@ -3,9 +3,9 @@ import template from "../../../shared/image/kernel-startup/manifest.template.jso import task from "../../../shared/image/kernel-startup/contracts/task-envelope.schema.json" with { type: "text" }; import terminal from "../../../shared/image/kernel-startup/contracts/terminal.schema.json" with { type: "text" }; import framing from "../../../shared/image/kernel-startup/contracts/framing.txt" with { type: "text" }; -import { failure } from "./errors"; +import { failure, kernelRetrievalFailure, type KernelRetrievalStage } from "./errors"; import { sha256, verifyRuntimeImage } from "./image"; -import type { RuntimeImageManifest, VerifiedRuntimeImage } from "./types"; +import { RunnerFailure, type RuntimeImageManifest, type VerifiedRuntimeImage } from "./types"; export interface KernelResponse { status: number; location?: string | undefined; bytes: Uint8Array } export type KernelGet = (url: string, limit: number, signal: AbortSignal) => Promise; @@ -16,13 +16,15 @@ async function httpGet(url: string, limit: number, signal: AbortSignal): Promise const response = await fetch(url, { redirect: "manual", signal, headers: { "User-Agent": policy.userAgent } }); const location = response.headers.get("location") ?? undefined; if (response.status !== 200) { - await response.body?.cancel(); + // Discarded response cleanup must not mask the observed HTTP status. + try { await response.body?.cancel(); } catch { /* No response bytes are consumed. */ } return { status: response.status, location, bytes: new Uint8Array() }; } const chunks: Uint8Array[] = []; let length = 0; const reader = response.body?.getReader(); if (!reader) throw invalid("Published kernel response has no body."); + let failed = false; try { while (true) { const { value, done } = await reader.read(); @@ -31,7 +33,12 @@ async function httpGet(url: string, limit: number, signal: AbortSignal): Promise if (length > limit) throw failure("IMAGE_TOO_LARGE", { maximumBytes: limit }); chunks.push(value); } - } finally { await reader.cancel(); } + } catch (error) { + failed = true; + throw error; + } finally { + try { await reader.cancel(); } catch (error) { if (!failed) throw error; } + } const bytes = new Uint8Array(length); let offset = 0; for (const chunk of chunks) { bytes.set(chunk, offset); offset += chunk.length; } @@ -43,7 +50,15 @@ export async function publishedKernel(get: KernelGet = httpGet, cancellation?: A const timeout = AbortSignal.timeout(policy.timeoutMs); const signal = cancellation ? AbortSignal.any([timeout, cancellation]) : timeout; try { - const entry = await get(policy.entry, policy.maxMetadataBytes, signal); + const request = async (url: string, limit: number, stage: KernelRetrievalStage) => { + try { return await get(url, limit, signal); } + catch (error) { + if (error instanceof RunnerFailure && ["IMAGE_INVALID", "IMAGE_TOO_LARGE", "CANCELLED", "STARTUP_TIMEOUT"].includes(error.code)) throw error; + throw kernelRetrievalFailure(stage); + } + }; + const entry = await request(policy.entry, policy.maxMetadataBytes, "entry"); + if (entry.status >= 400 && entry.status <= 599) throw kernelRetrievalFailure("entry", entry.status); if (![301, 302, 303, 307, 308].includes(entry.status) || !entry.location) throw invalid("Kernel entry must redirect to an immutable release."); const location = entry.location.startsWith("/") && !entry.location.startsWith("//") ? policy.origin + entry.location : entry.location; if (!location.startsWith(policy.origin + "/releases/") || /[%?#]/.test(location)) throw invalid("Kernel entry selected an unsupported release URL."); @@ -52,20 +67,20 @@ export async function publishedKernel(get: KernelGet = httpGet, cancellation?: A if (url.origin !== policy.origin || url.username || url.password || url.search || url.hash || !match) throw invalid("Kernel entry selected an unsupported release URL."); const release = match[1]!; const root = url.href.slice(0, -"README.md".length); - const read = async (url: string, limit: number) => { - const r = await get(url, limit, signal); - if (r.status !== 200) throw invalid("Published kernel artifact retrieval failed."); + const read = async (url: string, limit: number, stage: KernelRetrievalStage) => { + const r = await request(url, limit, stage); + if (r.status !== 200) throw kernelRetrievalFailure(stage, r.status); if (r.bytes.length > limit) throw failure("IMAGE_TOO_LARGE", { maximumBytes: limit }); return r.bytes; }; const json = (bytes: Uint8Array) => JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)); - const descriptor = json(await read(root + "descriptor.json", policy.maxMetadataBytes)); + const descriptor = json(await read(root + "descriptor.json", policy.maxMetadataBytes, "descriptor")); if (descriptor.identity !== "openprose/core" || descriptor.release !== release || descriptor.exports?.entry !== "README.md" || descriptor.inventory !== `releases/${release}/core/inventory.json` || !/^[a-f0-9]{40}$/.test(descriptor.source?.commit ?? "") || !/^[a-f0-9]{64}$/.test(descriptor.inventory_sha256 ?? "")) throw invalid("Published kernel descriptor identity is invalid."); - const inventoryBytes = await read(root + "inventory.json", policy.maxMetadataBytes); + const inventoryBytes = await read(root + "inventory.json", policy.maxMetadataBytes, "inventory"); if (await sha256(inventoryBytes) !== descriptor.inventory_sha256) throw invalid("Published kernel inventory digest mismatch."); const expected = json(inventoryBytes)["README.md"]; if (!expected || expected.mode !== "100644" || !/^[a-f0-9]{64}$/.test(expected.sha256 ?? "")) throw invalid("Published kernel inventory entry is invalid."); - const kernel = await read(url.href, policy.maxKernelBytes); + const kernel = await read(url.href, policy.maxKernelBytes, "kernel"); if (!kernel.length || await sha256(kernel) !== expected.sha256) throw invalid("Published kernel content digest mismatch."); const manifest = structuredClone(template) as RuntimeImageManifest; manifest.imageVersion = `kernel-${release}`; @@ -83,7 +98,7 @@ export async function publishedKernel(get: KernelGet = httpGet, cancellation?: A return await verifyRuntimeImage({ manifest, files }); } catch (error) { if (signal.aborted) throw failure(cancellation?.aborted ? "CANCELLED" : "STARTUP_TIMEOUT", { reason: "Kernel retrieval was cancelled or exceeded its startup deadline." }); - if (error && typeof error === "object" && "code" in error) throw error; + if (error instanceof RunnerFailure) throw error; throw invalid("Cannot retrieve or verify the published kernel; no fallback was used."); } } diff --git a/cli/bun/src/core/types.ts b/cli/bun/src/core/types.ts index 072cb62e..a4d76dac 100644 --- a/cli/bun/src/core/types.ts +++ b/cli/bun/src/core/types.ts @@ -128,6 +128,7 @@ export type RunnerErrorCode = | "HARNESS_NEEDS_AUTH" | "TRANSPORT_UNSUPPORTED" | "PROMPT_CHANNEL_UNSUPPORTED" + | "KERNEL_RETRIEVAL_FAILED" | "IMAGE_INVALID" | "IMAGE_TOO_LARGE" | "RECURSIVE_INVOCATION" diff --git a/cli/bun/test/kernel-startup.test.ts b/cli/bun/test/kernel-startup.test.ts index f816b56f..59c98b9a 100644 --- a/cli/bun/test/kernel-startup.test.ts +++ b/cli/bun/test/kernel-startup.test.ts @@ -30,7 +30,9 @@ const bad: Record)=>void> = { "HTTP failure": r=>{r[root+"descriptor.json"]!.status=503;}, }; for (const [name, change] of Object.entries(bad)) test(`published startup rejects ${name} without fallback`, async()=>{ - const {get}=setup(change); await expect(publishedKernel(get)).rejects.toBeDefined(); + const {get}=setup(change); + const error = await publishedKernel(get).catch(error => error); + expect(error.code).toBe(name === "oversize kernel" ? "IMAGE_TOO_LARGE" : name === "HTTP failure" ? "KERNEL_RETRIEVAL_FAILED" : "IMAGE_INVALID"); }); test("published startup propagates cancellation without a model call",async()=>{ const control=new AbortController();control.abort(); @@ -46,3 +48,112 @@ test("published startup rejects invalid UTF-8 even with matching release hashes" const get:KernelGet=async(url,limit,signal)=>url.endsWith('/descriptor.json')?{status:200,bytes:new TextEncoder().encode(JSON.stringify(descriptor))}:url.endsWith('/inventory.json')?{status:200,bytes:inventory}:url.endsWith('/README.md')?{status:200,bytes}:original(url,limit,signal); await expect(publishedKernel(get)).rejects.toBeDefined(); }); + +// Exercise the real HTTP/stream implementation against the independently frozen oracle. +for (const c of fixture.retrievalFailures) test(`retrieval oracle: ${c.id}`, async () => { + const calls: string[] = []; + const original = globalThis.fetch; + globalThis.fetch = (async (input: string | URL | Request) => { + const url = String(input); calls.push(url); + const stage = [entry, root + "descriptor.json", root + "inventory.json", root + "README.md"].indexOf(url); + const selected = ["entry", "descriptor", "inventory", "kernel"][stage] === c.stage; + if (selected && c.trigger === "transport") throw new Error("private-network-cause"); + if (selected && c.trigger === "body-read") return { + status: 200, headers: new Headers(), body: { getReader: () => ({ + read: async () => { throw new Error("private-read-cause"); }, cancel: async () => {}, + }) }, + } as unknown as globalThis.Response; + if (selected && c.trigger === "http") return { status: "httpStatus" in c ? c.httpStatus : 404, headers: new Headers(), body: null } as unknown as globalThis.Response; + const r = (fixture.responses as Record)[url]!; + return new globalThis.Response(r.status === 200 ? r.text : null, { status: r.status, headers: r.location ? { location: r.location } : {} }); + }) as typeof fetch; + try { + const error = await publishedKernel().then(() => { throw Error("unexpected success"); }, error => error); + expect(error.toJSON()).toEqual(c.error); + expect(calls).toEqual(c.expectedCalls); + expect(JSON.stringify(error.toJSON())).not.toContain("private-"); + } finally { globalThis.fetch = original; } +}); + +for (const c of fixture.unrecognizedTransportObjects) test(`untrusted thrown code: ${c.thrownCode}`, async () => { + const { get: original, calls } = setup(); + const error = await publishedKernel(async (url, limit, signal) => { + if (url === root + "README.md") { calls.push(url); throw { code: c.thrownCode, message: "private-cause" }; } + return original(url, limit, signal); + }).catch(error => error); + expect(error.toJSON()).toEqual(c.error); + expect(calls).toHaveLength(4); +}); + +test("retrieval factory rejects invalid phase/status observations", async () => { + const { kernelRetrievalFailure } = await import("../src/core/errors"); + for (const status of [200, 99, 600, 404.5, NaN, true as unknown as number, "503" as unknown as number, null as unknown as number]) expect(() => kernelRetrievalFailure("kernel", status)).toThrow(); + for (const status of [100, 204, 302, 399]) expect(() => kernelRetrievalFailure("entry", status)).toThrow(); + expect(() => kernelRetrievalFailure("other" as "entry")).toThrow(); +}); + +for (const code of ["IMAGE_TOO_LARGE", "CANCELLED", "STARTUP_TIMEOUT"] as const) test(`reader cancellation rejection preserves ${code}`, async () => { + const { failure } = await import("../src/core/errors"); + const original = globalThis.fetch; + let cancels = 0; + globalThis.fetch = (async () => ({ status: 200, headers: new Headers(), body: { getReader: () => ({ + read: async () => { + if (code === "IMAGE_TOO_LARGE") return { done: false, value: new Uint8Array(262145) }; + throw failure(code, { reason: "owned interrupt" }); + }, + cancel: async () => { cancels++; throw new Error("private-cleanup-cause"); }, + }) } })) as unknown as typeof fetch; + try { + const error = await publishedKernel().catch(error => error); + expect(error.code).toBe(code); expect(cancels).toBe(1); + expect(JSON.stringify(error.toJSON())).not.toContain("private-cleanup"); + } finally { globalThis.fetch = original; } +}); + +test("discarded HTTP body cancellation failure preserves observed status", async () => { + const original = globalThis.fetch; + globalThis.fetch = (async () => ({ status: 503, headers: new Headers(), body: { + cancel: async () => { throw Error("private-cleanup-cause"); }, + } })) as unknown as typeof fetch; + try { + const error = await publishedKernel().catch(error => error); + expect(error.toJSON()).toEqual(fixture.retrievalFailures.find(c => c.id === "entry-http-503")!.error); + } finally { globalThis.fetch = original; } +}); + +for (const status of [100, 200, 204, 302, 399]) test(`entry policy status ${status} remains IMAGE_INVALID`, async () => { + const error = await publishedKernel(async () => ({ status, bytes: new Uint8Array() })).catch(error => error); + expect(error.code).toBe("IMAGE_INVALID"); +}); + +test("body read and secondary cleanup failure retain the transport phase without raw causes", async () => { + const original = globalThis.fetch; + globalThis.fetch = (async () => ({ status: 200, headers: new Headers(), body: { getReader: () => ({ + read: async () => { throw Error("private-primary-read"); }, + cancel: async () => { throw Error("private-secondary-cleanup"); }, + }) } })) as unknown as typeof fetch; + try { + const error = await publishedKernel().catch(error => error); + expect(error.toJSON()).toEqual(fixture.retrievalFailures.find(c => c.id === "entry-body-read")!.error); + } finally { globalThis.fetch = original; } +}); + +test("actual aborted reader and secondary cleanup failure retain cancellation", async () => { + const control = new AbortController(); + const original = globalThis.fetch; + globalThis.fetch = (async () => ({ status: 200, headers: new Headers(), body: { getReader: () => ({ + read: async () => { control.abort(); throw Error("private-primary-abort"); }, + cancel: async () => { throw Error("private-secondary-cleanup"); }, + }) } })) as unknown as typeof fetch; + try { + const error = await publishedKernel(undefined, control.signal).catch(error => error); + expect(error.code).toBe("CANCELLED"); + expect(error.details).toEqual({ reason: "Kernel retrieval was cancelled or exceeded its startup deadline." }); + } finally { globalThis.fetch = original; } +}); + +test("unrelated own runner failures do not bypass retrieval classification", async () => { + const { failure } = await import("../src/core/errors"); + const error = await publishedKernel(async () => { throw failure("HARNESS_FAILED", { reason: "private-unrelated-cause" }); }).catch(error => error); + expect(error.toJSON()).toEqual(fixture.retrievalFailures.find(c => c.id === "entry-transport")!.error); +}); diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index e5a5bb22..d7e15ff8 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1193,3 +1193,17 @@ After complete27690 Mac custody audits, root and both peers selected proposal `0 ## IMP-098 tenth-cohort shell acquisition and cancellation Root recorded conventional asyncio/CPython comparison before implementation: proposal `aa48ff2dc1b0cf5321c952bce1392f93bf67d25fe556972e3aa69772dab26477`, independent design review `86005eaa69ff8b2d590fd2b4e00db24086f3b3b48879885e12f73e129ab995d9`. `/root/sdk_assessment` owns only `harnesses/agents-sdk/run.py` and `harnesses/agents-sdk/test_run.py`: retain shielded spawn acquisition and cleanup through repeated cancellation; attempt owned group kill, reader settlement and reap while preserving primary exceptions. Use explicit descendant readiness and post-cancellation effect release in the fixture. Closed safe secondary cleanup phase/type/integer-errno details survive existing tool/native errors through conditional shellCleanupFailures; no raw exception/command/environment/output or new event types. Remove unconditional group-termination guarantees. Preserve shell argv/cwd/env/new-session/PIPE/draining/output limits, model/usage/defaults and actual completion timeout. Existing acquisition/exceptional reap stay outside that timeout; no total cleanup deadline or detached containment is asserted. Configuration peer reviews final implementation and meaningful focused controls. Root owns docs/ownership, integration, mandatory full27, native replay, Git/push/merge; original failed and qualified evidence remains immutable. + + +## IMP-098 eleventh-cohort application-owned tool settlement + +Root published proposal `3e6ccfe19f1f735e2701e2321f320975400b148785b9588fdec33caceec29132`, causal locked-SDK/mock-provider/fake-shell evidence `5d8291c4435fedc13c64ab2e720912f11d83e13544c323199b3366e7adb43b92` and independent design review `7680cd8255214c2a1b975bf2671e116c2211090878884e79f55109d783845e75` before implementation. `/root/sdk_assessment` owns only `harnesses/agents-sdk/run.py` and `test_run.py`: explicit per-invocation strong shell/retrieval/child operation retention, synchronous closing guards at tool/child-lock/provider boundaries, protected settlement before terminal/client close despite repeated cancellation. Preserve initiating error, first cancellation on normal shutdown, and refuse unsupported success with outstanding work. Capture safe shell diagnostics once per owned shell; per-shell parser stays five with existing closed quotas/type/integer errno. Run aggregate uses existing shellCleanupFailures bounded five times effective existing function-tool budget, no value-based dedup or reader-bound widening. Preserve shell acquisition/reap placement, arguments/drains/caps/retrieval protection/child serialization/budgets/model/defaults/usage/event vocabulary. No private SDK patch/global task scan/new timeout/containment guarantee. Meaningful real locked-SDK/mock-provider/fake-gate controls and owned stdlib FIFO fixtures cover cancellation, repeats, late guards, normal/error/timeout and multi-shell safe diagnostics. Configuration peer reviews final code; root owns docs/integration/full28/native/Git. Agents may run focused SDK tests/owned stdlib fixtures only, no native/compiled helper/full/provider/Git. Original failures remain retained. + + +## IMP-098 eleventh-cohort actionable kernel retrieval errors + +Root published proposal `c6d0c8b2825e8afda35973c3a89291a81fe9e6c08e508552e8cb9d12961a533e` and SDK independent review `a47533208cea806f77e3b892323820266d84c150646ef5dcfb0fce09ec416eea` before code. Configuration agent owns exactly the proposal's six shared taxonomy/schema/test/fixture paths, four Bun core/test paths and two Rust error/kernel-startup paths. First freeze shared oracle and meaningful controls; root then authorizes port implementation. Add closed KERNEL_RETRIEVAL_FAILED image/exit20 with fixed phase/kind/public-origin/observedHTTPstatus and diagnostic-only retryability. Preserve integrity/redirect failures, cancellation/deadline/oversize priorities and original network bounds/request counts/no fallback. Do not pass arbitrary thrown code objects as recognized failures or mask a primary refusal with reader cancellation failure. SDK peer reviews final source; root owns docs/integration/Rust/full28/native/Git. Only provider-free shared Python/focused Bun tests permitted to this lane; no compiled CLI/helper/native/full/provider/Git. + +Root extends only the shared error-oracle lease to `cli/shared/tests/test_service_contract.py`: reconcile the old fixed43 taxonomy count with the new additive44th retrieval code using explicit identity/parity checks while retaining all service cases. First81/82 replay is retained; freeze a passing shared oracle before ports. No other lease changes. + +New-code oracle clarification from independent review: KERNEL_RETRIEVAL_FAILED/http excludes status200. Entry/http permits only400..599; other entry statuses violate the immutable redirect contract and stay IMAGE_INVALID. Descriptor/inventory/kernel permit observed100..599 except200, including refused redirects. Body read failure after200 is transport with no status. These consistency restrictions apply only to the new code; existing policies are unchanged. Freeze strict poisons before ports. diff --git a/cli/rust/crates/prose-runner-core/src/error.rs b/cli/rust/crates/prose-runner-core/src/error.rs index 0a3d4a94..b6b656fa 100644 --- a/cli/rust/crates/prose-runner-core/src/error.rs +++ b/cli/rust/crates/prose-runner-core/src/error.rs @@ -174,6 +174,7 @@ pub enum ErrorCode { HarnessNeedsAuth, TransportUnsupported, PromptChannelUnsupported, + KernelRetrievalFailed, ImageInvalid, ImageTooLarge, RecursiveInvocation, @@ -224,6 +225,7 @@ impl ErrorCode { Self::HarnessNeedsAuth => "HARNESS_NEEDS_AUTH", Self::TransportUnsupported => "TRANSPORT_UNSUPPORTED", Self::PromptChannelUnsupported => "PROMPT_CHANNEL_UNSUPPORTED", + Self::KernelRetrievalFailed => "KERNEL_RETRIEVAL_FAILED", Self::ImageInvalid => "IMAGE_INVALID", Self::ImageTooLarge => "IMAGE_TOO_LARGE", Self::RecursiveInvocation => "RECURSIVE_INVOCATION", @@ -293,6 +295,7 @@ impl ErrorCode { | Self::ServiceResponseTooLarge => 10, Self::TransportUnsupported | Self::PromptChannelUnsupported + | Self::KernelRetrievalFailed | Self::ImageInvalid | Self::ImageTooLarge | Self::RecursiveInvocation => 20, @@ -312,7 +315,7 @@ impl ErrorCode { impl ErrorCode { /// Every taxonomy code, in declaration order. - pub const ALL: [Self; 43] = [ + pub const ALL: [Self; 44] = [ Self::ConfigInvalid, Self::InvocationInvalid, Self::HarnessUnavailable, @@ -320,6 +323,7 @@ impl ErrorCode { Self::HarnessNeedsAuth, Self::TransportUnsupported, Self::PromptChannelUnsupported, + Self::KernelRetrievalFailed, Self::ImageInvalid, Self::ImageTooLarge, Self::RecursiveInvocation, @@ -498,6 +502,12 @@ impl RunnerError { "Choose a strict adapter reported by the `cli harness list` runner operation.", false, ), + ErrorCode::KernelRetrievalFailed => ( + "image", + "The published OpenProse kernel could not be retrieved.", + "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + false, + ), ErrorCode::ImageInvalid => ( "image", "The Skill Runtime Image failed structural or digest verification.", @@ -1228,6 +1238,7 @@ mod tests { ErrorCode::HarnessNeedsAuth, ErrorCode::TransportUnsupported, ErrorCode::PromptChannelUnsupported, + ErrorCode::KernelRetrievalFailed, ErrorCode::ImageInvalid, ErrorCode::ImageTooLarge, ErrorCode::RecursiveInvocation, diff --git a/cli/rust/crates/prose-runner-core/src/kernel_startup.rs b/cli/rust/crates/prose-runner-core/src/kernel_startup.rs index 5ae3232d..869572a7 100644 --- a/cli/rust/crates/prose-runner-core/src/kernel_startup.rs +++ b/cli/rust/crates/prose-runner-core/src/kernel_startup.rs @@ -26,6 +26,52 @@ pub struct KernelResponse { fn invalid(reason: &str) -> RunnerError { RunnerError::catalog(ErrorCode::ImageInvalid).with_detail("reason", reason) } +fn retrieval(stage: &str, status: Option) -> RunnerError { + if !["entry", "descriptor", "inventory", "kernel"].contains(&stage) + || status.is_some_and(|value| { + !(100..=599).contains(&value) || value == 200 || (stage == "entry" && value < 400) + }) + { + return invalid(UNVERIFIED); + } + let mut error = RunnerError::catalog(ErrorCode::KernelRetrievalFailed) + .with_detail("stage", stage) + .with_detail("origin", "https://pkg.prose.md") + .with_detail( + "failureKind", + if status.is_some() { + "http" + } else { + "transport" + }, + ) + .with_detail( + "reason", + if status.is_some() { + "Published kernel HTTP request failed." + } else { + "Published kernel transport or response read failed." + }, + ); + error.retryable = status.is_none_or(|value| [408, 429, 500, 502, 503, 504].contains(&value)); + if let Some(value) = status { + error = error.with_detail("httpStatus", value); + } + error +} +// Transport is staged at the request call site, never inferred from an external URL. +fn transport() -> RunnerError { + RunnerError::catalog(ErrorCode::KernelRetrievalFailed) +} +fn staged(error: RunnerError, stage: &str) -> RunnerError { + match error.code { + ErrorCode::ImageInvalid + | ErrorCode::ImageTooLarge + | ErrorCode::Cancelled + | ErrorCode::StartupTimeout => error, + _ => retrieval(stage, None), + } +} fn hash_valid(value: &str, length: usize) -> bool { value.len() == length && value @@ -61,7 +107,7 @@ pub fn published_kernel( let agent = |url: &str| { crate::service::http::agent_builder(url, &crate::service::http::process_environment) .map(ureq::AgentBuilder::build) - .map_err(|_| invalid(UNVERIFIED)) + .map_err(|_| transport()) }; let user_agent = policy["userAgent"].as_str().expect("user agent").to_owned(); let interrupted = || { @@ -95,7 +141,7 @@ pub fn published_kernel( Ok(outcome) => return outcome, Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {} Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => { - return Err(invalid(UNVERIFIED)); + return Err(transport()); } } } @@ -114,7 +160,7 @@ pub fn published_kernel( fn fetch(request: ureq::Request, limit: usize) -> Result { let response = match request.call() { Ok(response) | Err(ureq::Error::Status(_, response)) => response, - Err(ureq::Error::Transport(_)) => return Err(invalid(UNVERIFIED)), + Err(ureq::Error::Transport(_)) => return Err(transport()), }; let status = response.status(); let location = response.header("Location").map(str::to_owned); @@ -124,7 +170,7 @@ fn fetch(request: ureq::Request, limit: usize) -> Result limit { return Err(too_large(limit)); } @@ -157,7 +203,10 @@ pub fn published_kernel_with( let origin = policy["origin"].as_str().expect("origin"); let metadata_limit = usize::try_from(policy["maxMetadataBytes"].as_u64().expect("limit")) .map_err(|_| invalid("Kernel metadata limit exceeds this platform"))?; - let response = get(entry, metadata_limit)?; + let response = get(entry, metadata_limit).map_err(|error| staged(error, "entry"))?; + if (400..=599).contains(&response.status) { + return Err(retrieval("entry", Some(response.status))); + } let location = response .location .filter(|location| { @@ -185,19 +234,22 @@ pub fn published_kernel_with( return Err(invalid("Kernel entry selected an unsupported release URL.")); } let root = url.strip_suffix("README.md").expect("verified suffix"); - let mut read = |url: &str, limit: usize| -> Result, RunnerError> { - let response = get(url, limit)?; + let mut read = |url: &str, limit: usize, stage: &str| -> Result, RunnerError> { + let response = get(url, limit).map_err(|error| staged(error, stage))?; if response.status != 200 { - return Err(invalid("Published kernel artifact retrieval failed.")); + return Err(retrieval(stage, Some(response.status))); } if response.bytes.len() > limit { return Err(too_large(limit)); } Ok(response.bytes) }; - let descriptor: Value = - serde_json::from_slice(&read(&format!("{root}descriptor.json"), metadata_limit)?) - .map_err(|_| invalid(UNVERIFIED))?; + let descriptor: Value = serde_json::from_slice(&read( + &format!("{root}descriptor.json"), + metadata_limit, + "descriptor", + )?) + .map_err(|_| invalid(UNVERIFIED))?; let commit = descriptor["source"]["commit"].as_str().unwrap_or(""); let inventory_hash = descriptor["inventory_sha256"].as_str().unwrap_or(""); if descriptor["identity"] != "openprose/core" @@ -209,7 +261,11 @@ pub fn published_kernel_with( { return Err(invalid("Published kernel descriptor identity is invalid.")); } - let inventory = read(&format!("{root}inventory.json"), metadata_limit)?; + let inventory = read( + &format!("{root}inventory.json"), + metadata_limit, + "inventory", + )?; if sha256_hex(&inventory) != inventory_hash { return Err(invalid("Published kernel inventory digest mismatch.")); } @@ -222,6 +278,7 @@ pub fn published_kernel_with( &url, usize::try_from(policy["maxKernelBytes"].as_u64().expect("kernel limit")) .map_err(|_| invalid("Kernel byte limit exceeds this platform"))?, + "kernel", )?; if kernel.is_empty() || sha256_hex(&kernel) != kernel_hash { return Err(invalid("Published kernel content digest mismatch.")); @@ -279,6 +336,84 @@ mod tests { ); } #[test] + fn frozen_retrieval_errors_are_staged_without_extra_requests() { + let fixture: Value = serde_json::from_str(FIXTURE).unwrap(); + for case in fixture["retrievalFailures"].as_array().unwrap() { + let expected_calls = case["expectedCalls"].as_array().unwrap(); + let failed_url = expected_calls.last().unwrap().as_str().unwrap(); + let mut calls = Vec::new(); + let error = published_kernel_with(|url, _| { + calls.push(url.to_owned()); + if url == failed_url { + if case["trigger"] != "http" { + return Err(transport()); + } + return Ok(KernelResponse { + status: u16::try_from(case["httpStatus"].as_u64().unwrap()).unwrap(), + location: None, + bytes: Vec::new(), + }); + } + let response = &fixture["responses"][url]; + Ok(KernelResponse { + status: u16::try_from(response["status"].as_u64().unwrap()).unwrap(), + location: response["location"].as_str().map(str::to_owned), + bytes: response["text"].as_str().unwrap().as_bytes().to_vec(), + }) + }) + .unwrap_err(); + assert_eq!( + serde_json::to_value(error).unwrap(), + case["error"], + "{}", + case["id"] + ); + assert_eq!(json!(calls), case["expectedCalls"], "{}", case["id"]); + } + } + #[test] + fn retrieval_observation_factory_is_closed() { + for (stage, status) in [ + ("other", None), + ("kernel", Some(200)), + ("kernel", Some(99)), + ("kernel", Some(600)), + ("entry", Some(302)), + ] { + assert_eq!(retrieval(stage, status).code, ErrorCode::ImageInvalid); + } + } + #[test] + fn interruption_size_and_entry_policy_failures_keep_their_classification() { + for code in [ + ErrorCode::ImageTooLarge, + ErrorCode::Cancelled, + ErrorCode::StartupTimeout, + ] { + let original = RunnerError::catalog(code).with_detail("reason", "owned interruption"); + let error = published_kernel_with(|_, _| Err(original.clone())).unwrap_err(); + assert_eq!(error, original); + } + let unrelated = RunnerError::catalog(ErrorCode::HarnessFailed) + .with_detail("reason", "private-unrelated-cause"); + let error = published_kernel_with(|_, _| Err(unrelated.clone())).unwrap_err(); + assert_eq!( + serde_json::to_value(error).unwrap(), + serde_json::from_str::(FIXTURE).unwrap()["retrievalFailures"][0]["error"] + ); + for status in [100, 200, 204, 302, 399] { + let error = published_kernel_with(|_, _| { + Ok(KernelResponse { + status, + location: None, + bytes: Vec::new(), + }) + }) + .unwrap_err(); + assert_eq!(error.code, ErrorCode::ImageInvalid); + } + } + #[test] fn rejects_changed_origin_metadata_content_and_size() { let fixture: Value = serde_json::from_str(FIXTURE).unwrap(); let entry = "https://pkg.prose.md/kernel.md"; @@ -302,8 +437,16 @@ mod tests { (format!("{root}descriptor.json"), "status", json!(503)), ] { let mut mutated = fixture.clone(); - mutated["responses"][url][field] = value; - assert!(run(&mutated).is_err()); + mutated["responses"][url][field] = value.clone(); + let error = run(&mutated).unwrap_err(); + let expected = if field == "status" && value == json!(503) { + ErrorCode::KernelRetrievalFailed + } else if field == "text" && value.as_str().is_some_and(|text| text.len() > 32768) { + ErrorCode::ImageTooLarge + } else { + ErrorCode::ImageInvalid + }; + assert_eq!(error.code, expected); } } } diff --git a/cli/shared/errors/taxonomy.v1.json b/cli/shared/errors/taxonomy.v1.json index fb2281f7..d3d8b6c3 100644 --- a/cli/shared/errors/taxonomy.v1.json +++ b/cli/shared/errors/taxonomy.v1.json @@ -57,6 +57,14 @@ "message": "The selected adapter cannot use a manifest-permitted instruction placement.", "action": "Choose a strict adapter reported by the `cli harness list` runner operation." }, + { + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used." + }, { "code": "IMAGE_INVALID", "boundary": "image", diff --git a/cli/shared/fixtures/adapters/kernel-startup.json b/cli/shared/fixtures/adapters/kernel-startup.json index 211f33fe..22688070 100644 --- a/cli/shared/fixtures/adapters/kernel-startup.json +++ b/cli/shared/fixtures/adapters/kernel-startup.json @@ -48,5 +48,50 @@ "oversize", "timeout", "http-failure" - ] + ], + "retrievalError": { + "code": "KERNEL_RETRIEVAL_FAILED", + "stages": [ + "entry", + "descriptor", + "inventory", + "kernel" + ], + "failureKinds": [ + "transport", + "http" + ], + "origin": "https://pkg.prose.md", + "retryableHttpStatuses": [ + 408, + 429, + 500, + 502, + 503, + 504 + ], + "automaticRetries": 0, + "rawCausesRetained": false, + "preservesIntegrityCode": "IMAGE_INVALID", + "requestPolicy": { + "timeoutMs": 15000, + "maxMetadataBytes": 262144, + "maxKernelBytes": 32768, + "redirects": "manual", + "fallback": false + }, + "httpStatusPolicy": { + "entry": { + "minimum": 400, + "maximum": 599 + }, + "artifact": { + "minimum": 100, + "maximum": 599, + "excluded": [ + 200 + ] + } + } + } } diff --git a/cli/shared/fixtures/human/runner-errors.json b/cli/shared/fixtures/human/runner-errors.json index ca3dcff2..02c350d2 100644 --- a/cli/shared/fixtures/human/runner-errors.json +++ b/cli/shared/fixtures/human/runner-errors.json @@ -51,6 +51,23 @@ } }, "rendered": "HOSTED_UNAVAILABLE at hosted-service: line\\nbreak\nDetail: tab\\there\nAction: Use the exact runner invocation {{RUNNER}} for runner operations. act\\u{001B}[31m\n" + }, + { + "id": "kernel-retrieval-http", + "error": { + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 404 + } + }, + "rendered": "KERNEL_RETRIEVAL_FAILED at image: The published OpenProse kernel could not be retrieved.\nDetail: Published kernel HTTP request failed.\nAction: Use the exact runner invocation {{RUNNER}} for runner operations. Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.\n" } ] } diff --git a/cli/shared/fixtures/kernel-startup/release.json b/cli/shared/fixtures/kernel-startup/release.json index 18abec6a..e280b3d2 100644 --- a/cli/shared/fixtures/kernel-startup/release.json +++ b/cli/shared/fixtures/kernel-startup/release.json @@ -20,5 +20,1642 @@ }, "kernel": "# Provider-free kernel fixture\n", "sha256": "3d9beb296067894962d3d2fc37e15ff734e472fe6ab583118a285edbff65c85e", - "release": "fixture-1" + "release": "fixture-1", + "retrievalFailures": [ + { + "id": "entry-transport", + "stage": "entry", + "trigger": "transport", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "entry", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "entry-body-read", + "stage": "entry", + "trigger": "body-read", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "entry", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "entry-http-403", + "stage": "entry", + "trigger": "http", + "httpStatus": 403, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 403 + } + } + }, + { + "id": "entry-http-404", + "stage": "entry", + "trigger": "http", + "httpStatus": 404, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 404 + } + } + }, + { + "id": "entry-http-408", + "stage": "entry", + "trigger": "http", + "httpStatus": 408, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 408 + } + } + }, + { + "id": "entry-http-429", + "stage": "entry", + "trigger": "http", + "httpStatus": 429, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 429 + } + } + }, + { + "id": "entry-http-500", + "stage": "entry", + "trigger": "http", + "httpStatus": 500, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 500 + } + } + }, + { + "id": "entry-http-501", + "stage": "entry", + "trigger": "http", + "httpStatus": 501, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 501 + } + } + }, + { + "id": "entry-http-502", + "stage": "entry", + "trigger": "http", + "httpStatus": 502, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 502 + } + } + }, + { + "id": "entry-http-503", + "stage": "entry", + "trigger": "http", + "httpStatus": 503, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 503 + } + } + }, + { + "id": "entry-http-504", + "stage": "entry", + "trigger": "http", + "httpStatus": 504, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 504 + } + } + }, + { + "id": "entry-http-505", + "stage": "entry", + "trigger": "http", + "httpStatus": 505, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "entry", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 505 + } + } + }, + { + "id": "descriptor-transport", + "stage": "descriptor", + "trigger": "transport", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "descriptor", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "descriptor-body-read", + "stage": "descriptor", + "trigger": "body-read", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "descriptor", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "descriptor-http-403", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 403, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 403 + } + } + }, + { + "id": "descriptor-http-404", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 404, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 404 + } + } + }, + { + "id": "descriptor-http-408", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 408, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 408 + } + } + }, + { + "id": "descriptor-http-429", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 429, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 429 + } + } + }, + { + "id": "descriptor-http-500", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 500, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 500 + } + } + }, + { + "id": "descriptor-http-501", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 501, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 501 + } + } + }, + { + "id": "descriptor-http-502", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 502, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 502 + } + } + }, + { + "id": "descriptor-http-503", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 503, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 503 + } + } + }, + { + "id": "descriptor-http-504", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 504, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 504 + } + } + }, + { + "id": "descriptor-http-505", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 505, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 505 + } + } + }, + { + "id": "inventory-transport", + "stage": "inventory", + "trigger": "transport", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "inventory", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "inventory-body-read", + "stage": "inventory", + "trigger": "body-read", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "inventory", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "inventory-http-403", + "stage": "inventory", + "trigger": "http", + "httpStatus": 403, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 403 + } + } + }, + { + "id": "inventory-http-404", + "stage": "inventory", + "trigger": "http", + "httpStatus": 404, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 404 + } + } + }, + { + "id": "inventory-http-408", + "stage": "inventory", + "trigger": "http", + "httpStatus": 408, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 408 + } + } + }, + { + "id": "inventory-http-429", + "stage": "inventory", + "trigger": "http", + "httpStatus": 429, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 429 + } + } + }, + { + "id": "inventory-http-500", + "stage": "inventory", + "trigger": "http", + "httpStatus": 500, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 500 + } + } + }, + { + "id": "inventory-http-501", + "stage": "inventory", + "trigger": "http", + "httpStatus": 501, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 501 + } + } + }, + { + "id": "inventory-http-502", + "stage": "inventory", + "trigger": "http", + "httpStatus": 502, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 502 + } + } + }, + { + "id": "inventory-http-503", + "stage": "inventory", + "trigger": "http", + "httpStatus": 503, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 503 + } + } + }, + { + "id": "inventory-http-504", + "stage": "inventory", + "trigger": "http", + "httpStatus": 504, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 504 + } + } + }, + { + "id": "inventory-http-505", + "stage": "inventory", + "trigger": "http", + "httpStatus": 505, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 505 + } + } + }, + { + "id": "kernel-transport", + "stage": "kernel", + "trigger": "transport", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "kernel", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "kernel-body-read", + "stage": "kernel", + "trigger": "body-read", + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "kernel", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "kernel-http-403", + "stage": "kernel", + "trigger": "http", + "httpStatus": 403, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 403 + } + } + }, + { + "id": "kernel-http-404", + "stage": "kernel", + "trigger": "http", + "httpStatus": 404, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 404 + } + } + }, + { + "id": "kernel-http-408", + "stage": "kernel", + "trigger": "http", + "httpStatus": 408, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 408 + } + } + }, + { + "id": "kernel-http-429", + "stage": "kernel", + "trigger": "http", + "httpStatus": 429, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 429 + } + } + }, + { + "id": "kernel-http-500", + "stage": "kernel", + "trigger": "http", + "httpStatus": 500, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 500 + } + } + }, + { + "id": "kernel-http-501", + "stage": "kernel", + "trigger": "http", + "httpStatus": 501, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 501 + } + } + }, + { + "id": "kernel-http-502", + "stage": "kernel", + "trigger": "http", + "httpStatus": 502, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 502 + } + } + }, + { + "id": "kernel-http-503", + "stage": "kernel", + "trigger": "http", + "httpStatus": 503, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 503 + } + } + }, + { + "id": "kernel-http-504", + "stage": "kernel", + "trigger": "http", + "httpStatus": 504, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 504 + } + } + }, + { + "id": "kernel-http-505", + "stage": "kernel", + "trigger": "http", + "httpStatus": 505, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 505 + } + } + }, + { + "id": "descriptor-http-100", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 100, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 100 + } + } + }, + { + "id": "descriptor-http-204", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 204, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 204 + } + } + }, + { + "id": "descriptor-http-302", + "stage": "descriptor", + "trigger": "http", + "httpStatus": 302, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "descriptor", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 302 + } + } + }, + { + "id": "inventory-http-100", + "stage": "inventory", + "trigger": "http", + "httpStatus": 100, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 100 + } + } + }, + { + "id": "inventory-http-204", + "stage": "inventory", + "trigger": "http", + "httpStatus": 204, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 204 + } + } + }, + { + "id": "inventory-http-302", + "stage": "inventory", + "trigger": "http", + "httpStatus": 302, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "inventory", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 302 + } + } + }, + { + "id": "kernel-http-100", + "stage": "kernel", + "trigger": "http", + "httpStatus": 100, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 100 + } + } + }, + { + "id": "kernel-http-204", + "stage": "kernel", + "trigger": "http", + "httpStatus": 204, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 204 + } + } + }, + { + "id": "kernel-http-302", + "stage": "kernel", + "trigger": "http", + "httpStatus": 302, + "expectedCalls": [ + "https://pkg.prose.md/kernel.md", + "https://pkg.prose.md/releases/fixture-1/core/descriptor.json", + "https://pkg.prose.md/releases/fixture-1/core/inventory.json", + "https://pkg.prose.md/releases/fixture-1/core/README.md" + ], + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": false, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel HTTP request failed.", + "stage": "kernel", + "failureKind": "http", + "origin": "https://pkg.prose.md", + "httpStatus": 302 + } + } + } + ], + "primaryPriorityCases": [ + { + "id": "body-cleanup-after-IMAGE_TOO_LARGE", + "initiatingCode": "IMAGE_TOO_LARGE", + "secondaryFailure": "reader-cancel-rejection", + "expectedCode": "IMAGE_TOO_LARGE" + }, + { + "id": "body-cleanup-after-CANCELLED", + "initiatingCode": "CANCELLED", + "secondaryFailure": "reader-cancel-rejection", + "expectedCode": "CANCELLED" + }, + { + "id": "body-cleanup-after-STARTUP_TIMEOUT", + "initiatingCode": "STARTUP_TIMEOUT", + "secondaryFailure": "reader-cancel-rejection", + "expectedCode": "STARTUP_TIMEOUT" + } + ], + "integrityCases": [ + { + "id": "entry-http-200", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "entry-missing-location", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "entry-unsupported-redirect", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "redirect-outside-origin", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "unsafe-release-path", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "descriptor-json", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "descriptor-identity", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "inventory-json", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "inventory-digest", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "kernel-digest", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "entry-http-100", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "entry-http-204", + "expectedCode": "IMAGE_INVALID" + }, + { + "id": "entry-http-302-without-location", + "expectedCode": "IMAGE_INVALID" + } + ], + "unrecognizedTransportObjects": [ + { + "id": "plain-object-cancelled", + "thrownCode": "CANCELLED", + "stage": "kernel", + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "kernel", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "plain-object-image_too_large", + "thrownCode": "IMAGE_TOO_LARGE", + "stage": "kernel", + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "kernel", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + }, + { + "id": "plain-object-kernel_retrieval_failed", + "thrownCode": "KERNEL_RETRIEVAL_FAILED", + "stage": "kernel", + "error": { + "schema": "openprose.runner-error/1", + "code": "KERNEL_RETRIEVAL_FAILED", + "boundary": "image", + "exitCode": 20, + "retryable": true, + "message": "The published OpenProse kernel could not be retrieved.", + "action": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used.", + "details": { + "reason": "Published kernel transport or response read failed.", + "stage": "kernel", + "failureKind": "transport", + "origin": "https://pkg.prose.md" + } + } + } + ] } diff --git a/cli/shared/schemas/runner-error.schema.json b/cli/shared/schemas/runner-error.schema.json index ff9cce16..8fa6a9dc 100644 --- a/cli/shared/schemas/runner-error.schema.json +++ b/cli/shared/schemas/runner-error.schema.json @@ -27,6 +27,7 @@ "HARNESS_NEEDS_AUTH", "TRANSPORT_UNSUPPORTED", "PROMPT_CHANNEL_UNSUPPORTED", + "KERNEL_RETRIEVAL_FAILED", "IMAGE_INVALID", "IMAGE_TOO_LARGE", "RECURSIVE_INVOCATION", @@ -111,8 +112,12 @@ "adapterDiagnostic": { "$ref": "adapter-diagnostic.schema.json" }, - "usageObservation": { "$ref": "sdk-observation.schema.json#/$defs/usageObservation" }, - "modelIdentity": { "$ref": "sdk-observation.schema.json#/$defs/modelIdentity" }, + "usageObservation": { + "$ref": "sdk-observation.schema.json#/$defs/usageObservation" + }, + "modelIdentity": { + "$ref": "sdk-observation.schema.json#/$defs/modelIdentity" + }, "nativeFailure": { "$ref": "native-failure.schema.json" }, @@ -302,7 +307,10 @@ }, "processResource": { "description": "The owned process service whose settlement failed; contains no machine path.", - "enum": ["owned-prime-harness-service", "owned-private-transport-files"] + "enum": [ + "owned-prime-harness-service", + "owned-private-transport-files" + ] }, "resource": { "description": "The resource a SERVICE_RESOURCE_NOT_FOUND names: its kind (from the operation's manifest notFound entry, or the handler) and the identifier as the caller gave it.", @@ -335,6 +343,198 @@ } }, "allOf": [ + { + "if": { + "properties": { + "code": { + "const": "KERNEL_RETRIEVAL_FAILED" + } + }, + "required": [ + "code" + ] + }, + "then": { + "required": [ + "details" + ], + "properties": { + "boundary": { + "const": "image" + }, + "exitCode": { + "const": 20 + }, + "message": { + "const": "The published OpenProse kernel could not be retrieved." + }, + "action": { + "const": "Check your network connection and access to https://pkg.prose.md. Retry when the published kernel is available; no fallback was used." + }, + "details": { + "type": "object", + "additionalProperties": false, + "required": [ + "reason", + "stage", + "failureKind", + "origin" + ], + "properties": { + "reason": { + "type": "string" + }, + "stage": { + "enum": [ + "entry", + "descriptor", + "inventory", + "kernel" + ] + }, + "failureKind": { + "enum": [ + "transport", + "http" + ] + }, + "origin": { + "const": "https://pkg.prose.md" + }, + "httpStatus": { + "type": "integer", + "minimum": 100, + "maximum": 599, + "not": { + "const": 200 + } + } + } + } + }, + "allOf": [ + { + "if": { + "properties": { + "details": { + "properties": { + "failureKind": { + "const": "transport" + } + }, + "required": [ + "failureKind" + ] + } + } + }, + "then": { + "properties": { + "retryable": { + "const": true + }, + "details": { + "properties": { + "reason": { + "const": "Published kernel transport or response read failed." + } + }, + "not": { + "required": [ + "httpStatus" + ] + } + } + } + }, + "else": { + "properties": { + "details": { + "required": [ + "httpStatus" + ], + "properties": { + "reason": { + "const": "Published kernel HTTP request failed." + } + } + } + }, + "allOf": [ + { + "if": { + "properties": { + "details": { + "properties": { + "httpStatus": { + "enum": [ + 408, + 429, + 500, + 502, + 503, + 504 + ] + } + }, + "required": [ + "httpStatus" + ] + } + } + }, + "then": { + "properties": { + "retryable": { + "const": true + } + } + }, + "else": { + "properties": { + "retryable": { + "const": false + } + } + } + } + ] + } + }, + { + "if": { + "properties": { + "details": { + "properties": { + "stage": { + "const": "entry" + }, + "failureKind": { + "const": "http" + } + }, + "required": [ + "stage", + "failureKind" + ] + } + } + }, + "then": { + "properties": { + "details": { + "properties": { + "httpStatus": { + "minimum": 400 + } + } + } + } + } + } + ] + } + }, { "if": { "properties": { diff --git a/cli/shared/tests/test_contracts.py b/cli/shared/tests/test_contracts.py index 25970bdc..7555b17b 100755 --- a/cli/shared/tests/test_contracts.py +++ b/cli/shared/tests/test_contracts.py @@ -95,6 +95,62 @@ def test_kernel_startup_instruction_recipes_and_provider_routes(self): self.assertIn(route, admissions[harness]["credentialGroups"]) self.assertEqual(fixture["failurePolicy"], "no-echo-or-provider-fallback") + def test_kernel_retrieval_failure_oracle_and_closed_diagnostics(self): + fixture = load_json(FIXTURES / "kernel-startup/release.json") + policy = load_json(SHARED / "image/kernel-startup/policy.json") + contract = load_json(FIXTURES / "adapters/kernel-startup.json")["retrievalError"] + self.assertEqual(contract["automaticRetries"], 0) + self.assertFalse(contract["requestPolicy"]["fallback"]) + for key in ("timeoutMs", "maxMetadataBytes", "maxKernelBytes"): + self.assertEqual(contract["requestPolicy"][key], policy[key]) + cases = fixture["retrievalFailures"] + self.assertEqual(len(cases), 57) + self.assertEqual(len({case["id"] for case in cases}), 57) + self.assertEqual({case["stage"] for case in cases}, {"entry", "descriptor", "inventory", "kernel"}) + urls = list(fixture["responses"]) + validator = self.validator("runner-error.schema.json") + for case in cases: + error = case["error"] + self.assert_valid("runner-error.schema.json", error) + self.assertEqual(case["expectedCalls"], urls[:contract["stages"].index(case["stage"]) + 1]) + self.assertEqual(error["details"]["stage"], case["stage"]) + self.assertEqual(error["retryable"], case["trigger"] != "http" or case["httpStatus"] in contract["retryableHttpStatuses"]) + for key, bad in (("stage", "provider"), ("failureKind", "integrity"), ("origin", "https://evil.invalid"), ("reason", "secret-provider-body")): + poisoned = deepcopy(error) + poisoned["details"][key] = bad + self.assertTrue(list(validator.iter_errors(poisoned)), (case["id"], key)) + poisoned = deepcopy(error) + poisoned["details"]["rawCause"] = "secret" + self.assertTrue(list(validator.iter_errors(poisoned))) + poisoned = deepcopy(error) + poisoned["retryable"] = not error["retryable"] + self.assertTrue(list(validator.iter_errors(poisoned))) + http = deepcopy(next(case["error"] for case in cases if case["trigger"] == "http")) + for bad in (None, True, "404", 404.5, 99, 600): + http["details"]["httpStatus"] = bad + self.assertTrue(list(validator.iter_errors(http))) + for stage in contract["stages"]: + for status in (200, 100, 204, 302, 399, 400, 599): + sample = deepcopy(next(case["error"] for case in cases if case.get("httpStatus") == 404)) + sample["details"].update(stage=stage, httpStatus=status) + valid = status != 200 and (stage != "entry" or status >= 400) + self.assertEqual(not list(validator.iter_errors(sample)), valid, (stage, status)) + del http["details"]["httpStatus"] + self.assertTrue(list(validator.iter_errors(http))) + absent = deepcopy(cases[0]["error"]) + del absent["details"] + self.assertTrue(list(validator.iter_errors(absent))) + for case in fixture["unrecognizedTransportObjects"]: + self.assertEqual(case["error"]["code"], "KERNEL_RETRIEVAL_FAILED") + self.assertEqual(case["error"]["details"]["failureKind"], "transport") + self.assert_valid("runner-error.schema.json", case["error"]) + transport = deepcopy(cases[0]["error"]) + transport["details"]["httpStatus"] = 503 + self.assertTrue(list(validator.iter_errors(transport))) + for case in fixture["primaryPriorityCases"]: + self.assertEqual(case["initiatingCode"], case["expectedCode"]) + self.assertEqual({case["expectedCode"] for case in fixture["integrityCases"]}, {"IMAGE_INVALID"}) + def test_safe_transport_diagnostics(self): validator = self.validator("transport-diagnostic.schema.json") for fixture in load_json(FIXTURES / "transport-diagnostics.json"): @@ -1023,7 +1079,7 @@ def test_error_taxonomy_is_complete_unique_and_schema_valid(self) -> None: "CONFIG_INVALID", "INVOCATION_INVALID", "HARNESS_UNAVAILABLE", "HARNESS_INCOMPATIBLE", "HARNESS_NEEDS_AUTH", "TRANSPORT_UNSUPPORTED", - "PROMPT_CHANNEL_UNSUPPORTED", "IMAGE_INVALID", "IMAGE_TOO_LARGE", + "PROMPT_CHANNEL_UNSUPPORTED", "KERNEL_RETRIEVAL_FAILED", "IMAGE_INVALID", "IMAGE_TOO_LARGE", "RECURSIVE_INVOCATION", "STARTUP_TIMEOUT", "PROTOCOL_MALFORMED", "PROTOCOL_TRUNCATED", "HARNESS_FAILED", "SEMANTIC_STATUS_UNKNOWN", "CANCELLED", "PROCESS_CLEANUP_FAILED", "HOSTED_UNAVAILABLE", @@ -1041,7 +1097,10 @@ def test_error_taxonomy_is_complete_unique_and_schema_valid(self) -> None: self.assertEqual({record["code"] for record in records}, expected_codes) self.assertEqual(len(records), len(expected_codes)) for record in records: - self.assert_valid("runner-error.schema.json", {"schema": "openprose.runner-error/1", **record}) + sample = {"schema": "openprose.runner-error/1", **record} + if record["code"] == "KERNEL_RETRIEVAL_FAILED": + sample["details"] = load_json(FIXTURES / "kernel-startup/release.json")["retrievalFailures"][3]["error"]["details"] + self.assert_valid("runner-error.schema.json", sample) self.assertNotIn("prose cli", record["action"]) self.assertNotIn("$PROSE", record["action"]) cleanup = next(record for record in records if record["code"] == "PROCESS_CLEANUP_FAILED") diff --git a/cli/shared/tests/test_service_contract.py b/cli/shared/tests/test_service_contract.py index b07dd62b..368e6c1f 100644 --- a/cli/shared/tests/test_service_contract.py +++ b/cli/shared/tests/test_service_contract.py @@ -175,8 +175,13 @@ def test_service_status_projects_only_health_and_models(self): self.assert_invalid(self.result_ref("model.list"), {"models": [], "default_model": "model-sol", "hidden": []}) # ----------------------------------------------------------------- taxonomy - def test_taxonomy_grows_additively_to_43_codes_with_parity(self): - self.assertEqual(43, len(self.taxonomy)) + def test_taxonomy_grows_additively_to_44_codes_with_parity(self): + self.assertEqual(44, len(self.taxonomy)) + retrieval = self.taxonomy["KERNEL_RETRIEVAL_FAILED"] + self.assertEqual((retrieval["boundary"], retrieval["exitCode"], retrieval["retryable"]), + ("image", 20, False)) + self.assertIn("https://pkg.prose.md", retrieval["action"]) + self.assertNotEqual(retrieval["action"], self.taxonomy["IMAGE_INVALID"]["action"]) for code, exit_code in NEW_CODES.items(): self.assertEqual(exit_code, self.taxonomy[code]["exitCode"], code) schema = load_json(SCHEMAS / "runner-error.schema.json") @@ -190,7 +195,10 @@ def test_taxonomy_grows_additively_to_43_codes_with_parity(self): self.assertEqual(value, self.taxonomy[code][field], (code, field)) for record in self.taxonomy.values(): self.assertNotIn("prose cli", record["action"]) - self.assert_valid("runner-error.schema.json", {"schema": "openprose.runner-error/1", **record}) + sample = {"schema": "openprose.runner-error/1", **record} + if record["code"] == "KERNEL_RETRIEVAL_FAILED": + sample["details"] = load_json(SHARED / "fixtures" / "kernel-startup" / "release.json")["retrievalFailures"][3]["error"]["details"] + self.assert_valid("runner-error.schema.json", sample) def test_hosted_unavailable_text_is_unchanged(self): self.assertEqual(HOSTED_UNAVAILABLE, self.taxonomy["HOSTED_UNAVAILABLE"]) diff --git a/docs/packaged-sdk-design.md b/docs/packaged-sdk-design.md index 5013f7e2..e592d442 100644 --- a/docs/packaged-sdk-design.md +++ b/docs/packaged-sdk-design.md @@ -129,3 +129,9 @@ Linux minimal-runtime verification uses explicit `exec` on its existing 512 MiB Shell cancellation retains and shields subprocess acquisition until it obtains the owned process handle, then retains cleanup through repeated cancellation. Cleanup attempts process-group termination, capture settlement and reaping without replacing the initiating error. Safe secondary cleanup failures are carried in existing tool or native error records; cancellation and timeout messages do not promise group termination. Acquisition and exceptional reaping retain their existing placement outside the command completion timeout. This does not establish a total cleanup deadline or detached-process containment. The descendant cancellation fixture explicitly orders readiness, cancellation and effect release rather than racing fixed sleeps. + + +The locked SDK's ordinary parent cancellation can leave tool invocation cleanup pending. The harness therefore owns shell, retrieval and child operations explicitly, closes admission during shutdown and settles owned operations before terminal output and client closure. Late tool/child/provider callbacks cannot start work after closing. Initiating failures and cancellation remain visible; safe shell cleanup diagnostics retain their per-shell bound and a run aggregate bounded by the existing function-tool budget. This adds no total cleanup deadline, detached-process containment or billing guarantee. + + +Published-kernel network and HTTP failures report a distinct retrieval error with a fixed resource stage, an observed HTTP status when available, and guidance to check access to `https://pkg.prose.md`. Invalid redirects, structure and digests retain integrity errors. Diagnostic retryability does not trigger automatic retries; the existing deadline and byte limits remain in force, and no fallback kernel is selected. diff --git a/harnesses/agents-sdk/run.py b/harnesses/agents-sdk/run.py index ee56576c..33f00daa 100644 --- a/harnesses/agents-sdk/run.py +++ b/harnesses/agents-sdk/run.py @@ -96,6 +96,83 @@ class BudgetExceeded(Exception): """An aggregate invocation budget was exhausted.""" +async def _settle_protected(task): + """Join retained cleanup despite cancellation; report the first interruption.""" + interrupted = None + while not task.done(): + try: + await asyncio.shield(task) + except asyncio.CancelledError as error: + if interrupted is None: + interrupted = error + task.result() + return interrupted + + +class OwnedOperations: + """Own application effects independently of SDK background cancellation.""" + def __init__(self, max_tools): + self.max_tools = max_tools + self.tasks = [] + self.shell_failures = {} + self.closing = False + self.shutdown_task = None + self.pending_at_shutdown = False + + def require_open(self): + if self.closing: + raise asyncio.CancelledError() + + async def perform(self, factory, *, is_shell=False): + self.require_open() + if len(self.tasks) >= self.max_tools: + raise BudgetExceeded() + async def tracked(): + try: + self.require_open() + return await factory() + except BaseException as error: + if is_shell: + rows = _shell_cleanup_details(error).get('shellCleanupFailures', []) + if rows: + self.shell_failures[task] = rows + raise + task = asyncio.create_task(tracked()) + self.tasks.append(task) + try: + return await asyncio.shield(task) + except BaseException as primary: + if not task.done(): + task.cancel() + async def settle(): + await asyncio.gather(task, return_exceptions=True) + retained = asyncio.create_task(settle()) + await _settle_protected(retained) + if is_shell and task in self.shell_failures: + primary._shell_cleanup_failures = self.shell_failures[task] + raise + + async def shutdown(self, primary=None): + self.closing = True + if self.shutdown_task is None: + self.pending_at_shutdown = any(not task.done() for task in self.tasks) + async def settle(): + for task in self.tasks: + if not task.done(): + task.cancel() + await asyncio.gather(*self.tasks, return_exceptions=True) + self.shutdown_task = asyncio.create_task(settle()) + interrupted = await _settle_protected(self.shutdown_task) + if primary is None and interrupted is not None: + raise interrupted + + def cleanup_details(self): + # Keep identical failures from distinct operations. Each entry was + # validated once by the single-shell <=5-row parser at its boundary. + rows = [row for task in self.tasks for row in self.shell_failures.get(task, [])] + return {'shellCleanupFailures': rows} if rows else {} + + async def shell(command, cwd, timeout, env, output_limit=30000): # Retain ownership while subprocess creation connects its pipes. Cancellation # of that constructor otherwise lets asyncio kill only the direct child. @@ -310,13 +387,16 @@ async def exchange(): class GuardedResponses: """Guard the actual SDK request after instruction/history/tool conversion.""" - def __init__(self, resource, max_input_bytes, observed_models, observed_tiers=None): + def __init__(self, resource, max_input_bytes, observed_models, observed_tiers=None, admission=None): self.resource = resource self.max_input_bytes = max_input_bytes self.observed_models = observed_models self.observed_tiers = observed_tiers if observed_tiers is not None else set() + self.admission = admission async def create(self, **kwargs): + if self.admission is not None: + self.admission() payload = {field: kwargs[field] for field in ('instructions', 'input', 'tools', 'text') if field in kwargs and isinstance(kwargs[field], (str, list, dict))} if len(json.dumps(payload, ensure_ascii=False).encode('utf-8')) > self.max_input_bytes: @@ -333,12 +413,13 @@ async def create(self, **kwargs): class GuardedClient: """Retain request guards when the SDK uses public client.with_options().""" - def __init__(self, client, max_input_bytes, observed_models, observed_tiers): + def __init__(self, client, max_input_bytes, observed_models, observed_tiers, admission=None): self.client = client self.max_input_bytes = max_input_bytes self.observed_models = observed_models self.observed_tiers = observed_tiers - self.responses = GuardedResponses(client.responses, max_input_bytes, observed_models, observed_tiers) + self.admission = admission + self.responses = GuardedResponses(client.responses, max_input_bytes, observed_models, observed_tiers, admission) def __getattr__(self, name): return getattr(self.client, name) @@ -346,7 +427,7 @@ def __getattr__(self, name): def with_options(self, **kwargs): clone = getattr(self.client, 'with_options', None) client = clone(**kwargs) if callable(clone) else self.client - return GuardedClient(client, self.max_input_bytes, self.observed_models, self.observed_tiers) + return GuardedClient(client, self.max_input_bytes, self.observed_models, self.observed_tiers, self.admission) async def run(args): @@ -369,6 +450,7 @@ def emit(kind, **data): print(json.dumps({'type': kind, 'event': kind, 'elapsed_seconds': round(time.monotonic()-start, 3), **data}), flush=True) max_input_bytes = getattr(args, 'max_input_bytes', 256000) max_tools = getattr(args, 'max_tools', 80) + operations = OwnedOperations(max_tools) max_total_tokens = getattr(args, 'max_total_tokens', 500000) limits = {'maxTurns': args.max_turns, 'timeoutSeconds': args.timeout, 'toolTimeoutSeconds': args.tool_timeout, 'maxOutputTokens': args.max_output_tokens, @@ -378,7 +460,8 @@ def emit(kind, **data): def fail(kind, message, setup_reason=None, cleanup_error=None): extra = {"setup_reason": setup_reason} if kind == "SetupError" and setup_reason in ("credential-or-permission", "model-unavailable", "local-input") else {} emit('error', error_type=kind, message=message, limits=limits, - usageObservation=observation.summary(), **extra, **_shell_cleanup_details(cleanup_error)) + usageObservation=observation.summary(), **extra, + **(operations.cleanup_details() or _shell_cleanup_details(cleanup_error))) return 1 emit('start', model=args.model, cwd=args.cwd, limits=limits, permissions={'shell': 'host_os_permissions', 'filesystemSandbox': False, @@ -409,49 +492,56 @@ def remaining(): @function_tool(failure_error_function=None) async def execute_shell(command: str) -> str: """Execute bash in the working directory with host OS permissions; read/edit files. This is not a sandbox.""" + operations.require_open() if len(command.encode()) > 30000: return json.dumps({'error': 'shell command exceeds 30000 bytes'}) emit('tool_call', name='execute_shell', command=command) try: - result = await shell(command, cwd, min(args.tool_timeout, remaining()), tool_env) + result = await operations.perform(lambda: shell(command, cwd, min(args.tool_timeout, remaining()), tool_env), is_shell=True) except asyncio.TimeoutError as error: result = {'error': 'shell command timed out', **_shell_cleanup_details(error)} except OSError as error: result = {'error': 'shell could not start; check working directory and bash availability', **_shell_cleanup_details(error)} + operations.require_open() emit('tool_result', name='execute_shell', result=result) return json.dumps(result) @function_tool(failure_error_function=None) async def retrieve_url(url: str) -> str: """Retrieve text from a public HTTPS URL. Private addresses, credentials, non-443 ports and redirects are rejected.""" + operations.require_open() emit('tool_call', name='retrieve_url', url=url[:4096]) try: - result = await retrieve_public(url, min(args.tool_timeout, remaining())) + result = await operations.perform(lambda: retrieve_public(url, min(args.tool_timeout, remaining()))) except (OSError, ValueError, asyncio.TimeoutError, asyncio.IncompleteReadError, asyncio.LimitOverrunError): result = {'error': 'public retrieval failed; check the HTTPS URL, server availability and response format'} + operations.require_open() emit('tool_result', name='retrieve_url', result=result) return json.dumps(result) client = AsyncOpenAI(api_key=key.strip(), max_retries=0, timeout=args.timeout) - client = GuardedClient(client, max_input_bytes, observed_models, observed_tiers) + client = GuardedClient(client, max_input_bytes, observed_models, observed_tiers, operations.require_open) config = RunConfig(tracing_disabled=True, model_provider=OpenAIProvider(openai_client=client)) def make_agent(depth): @function_tool(failure_error_function=None) async def delegate(task: str) -> str: """Run an independent fresh agent context. Supply all needed task inputs explicitly; no parent conversation is inherited. Files remain shared.""" nonlocal children + operations.require_open() if depth >= 1 or children >= 8: raise BudgetExceeded() if len(task.encode()) > 60000: return json.dumps({'error': 'child input exceeds 60000 bytes'}) # A single lock covers the complete child: sibling provider calls never overlap. async with child_lock: + operations.require_open() if children >= 8: raise BudgetExceeded() children += 1 child_id = children emit('tool_call', name='delegate', child_id=child_id, depth=depth + 1) child = make_agent(depth + 1) - result = await Runner.run(child, task, max_turns=args.max_turns, - run_config=config, hooks=observation) + result = await operations.perform(lambda: Runner.run(child, task, max_turns=args.max_turns, + run_config=config, hooks=observation)) + operations.require_open() child_usage.append(result.context_wrapper.usage) output = str(result.final_output) if len(output.encode()) > 60000: @@ -467,6 +557,9 @@ async def delegate(task: str) -> str: try: result = await asyncio.wait_for(Runner.run(make_agent(0), args.prompt, max_turns=args.max_turns, run_config=config, hooks=observation), remaining()) + await operations.shutdown() + if operations.pending_at_shutdown or operations.cleanup_details(): + raise RuntimeError('Application tool work remained active at completion') normalized = [result.context_wrapper.usage] + child_usage usage = {field: sum(getattr(value, field) for value in normalized) for field in ('requests', 'input_tokens', 'output_tokens', 'total_tokens')} emit('final', output=result.final_output, usage={ @@ -475,8 +568,10 @@ async def delegate(task: str) -> str: usageObservation=observation.summary()) return 0 except asyncio.CancelledError as error: + await operations.shutdown(error) return fail('CancelledError', 'Execution cancelled.', cleanup_error=error) except Exception as error: + await operations.shutdown(error) name = type(error).__name__ if name in ('AuthenticationError', 'PermissionDeniedError'): return fail('SetupError', 'OpenAI rejected the credential or account permissions. Check the API key and model access; no fallback was selected.', setup_reason='credential-or-permission', cleanup_error=error) @@ -485,6 +580,7 @@ async def delegate(task: str) -> str: safe = name if name in ('MaxTurnsExceeded', 'TimeoutError', 'BudgetExceeded') else 'ExecutionError' return fail(safe, 'Execution stopped. Check configured limits, tool availability and OpenAI account access.', cleanup_error=error) finally: + await operations.shutdown() await client.close() diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index cf36b242..19babc9b 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -348,6 +348,124 @@ async def cancelled(): self.assertEqual(harness._shell_cleanup_details(caught.exception), {'shellCleanupFailures': primary._shell_cleanup_failures}) self.assertNotIn('private', json.dumps(harness._shell_cleanup_details(caught.exception))) +class OwnedOperationsTest(unittest.IsolatedAsyncioTestCase): + async def test_shutdown_joins_effects_despite_repeated_cancellation(self): + operations = harness.OwnedOperations(80) + entered, cleaning, release = (asyncio.Event() for _ in range(3)) + async def effect(): + entered.set() + try: + await asyncio.Future() + except asyncio.CancelledError: + cleaning.set() + await release.wait() + raise + invocation = asyncio.create_task(operations.perform(effect)) + await asyncio.wait_for(entered.wait(), 1) + primary = RuntimeError('private-primary') + closing = asyncio.create_task(operations.shutdown(primary)) + await asyncio.wait_for(cleaning.wait(), 1) + closing.cancel() + await asyncio.sleep(0) + self.assertFalse(closing.done()) + release.set() + await asyncio.wait_for(closing, 1) + await asyncio.gather(invocation, return_exceptions=True) + self.assertTrue(all(task.done() for task in operations.tasks)) + await operations.shutdown(primary) + + async def test_first_cancel_during_normal_shutdown_is_preserved(self): + operations = harness.OwnedOperations(1) + entered, cleaning, release = (asyncio.Event() for _ in range(3)) + async def effect(): + entered.set() + try: + await asyncio.Future() + except asyncio.CancelledError: + cleaning.set() + await release.wait() + raise + invocation = asyncio.create_task(operations.perform(effect)) + await asyncio.wait_for(entered.wait(), 1) + closing = asyncio.create_task(operations.shutdown()) + await asyncio.wait_for(cleaning.wait(), 1) + closing.cancel() + closing.cancel() + await asyncio.sleep(0) + self.assertFalse(closing.done()) + release.set() + with self.assertRaises(asyncio.CancelledError): + await asyncio.wait_for(closing, 1) + await asyncio.gather(invocation, return_exceptions=True) + self.assertTrue(all(task.done() for task in operations.tasks)) + + async def test_registered_factory_cannot_start_after_shutdown_closes(self): + operations = harness.OwnedOperations(1) + started = asyncio.Event() + async def effect(): + started.set() + factory = AsyncMock(side_effect=effect) + invocation = asyncio.create_task(operations.perform(factory)) + await asyncio.sleep(0) + self.assertEqual(len(operations.tasks), 1) + factory.assert_not_called() + await operations.shutdown(RuntimeError('private-primary')) + outcomes = await asyncio.gather(invocation, return_exceptions=True) + self.assertIsInstance(outcomes[0], asyncio.CancelledError) + factory.assert_not_called() + self.assertFalse(started.is_set()) + self.assertTrue(all(task.done() for task in operations.tasks)) + + async def test_closed_factory_and_provider_clones_never_start(self): + operations = harness.OwnedOperations(1) + await operations.shutdown() + factory = AsyncMock() + with self.assertRaises(asyncio.CancelledError): + await operations.perform(factory) + factory.assert_not_called() + resource = SimpleNamespace(create=AsyncMock()) + client = SimpleNamespace(responses=resource) + guarded = harness.GuardedClient(client, 1000, set(), set(), operations.require_open) + with self.assertRaises(asyncio.CancelledError): + await guarded.with_options().responses.create(input='fixture') + resource.create.assert_not_called() + + async def test_single_shell_validation_and_distinct_operation_failures_retained(self): + operations = harness.OwnedOperations(80) + rows = [{'phase': 'acquire', 'errorType': 'ExecutionError'}, + {'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}, + {'phase': 'readers', 'errorType': 'ExecutionError'}, + {'phase': 'readers', 'errorType': 'ExecutionError'}, + {'phase': 'reap', 'errorType': 'OSError', 'errno': 5}] + async def failure(): + error = OSError(5, 'private-message') + error._shell_cleanup_failures = rows + raise error + for _ in range(80): + with self.assertRaises(OSError): + await operations.perform(failure, is_shell=True) + actual = operations.cleanup_details()['shellCleanupFailures'] + self.assertEqual(actual, rows * 80) + self.assertEqual(len(actual), 400) + self.assertLess(len(json.dumps(operations.cleanup_details()).encode()), 1024 * 1024) + self.assertNotIn('private', json.dumps(operations.cleanup_details())) + factory = AsyncMock() + with self.assertRaises(harness.BudgetExceeded): + await operations.perform(factory, is_shell=True) + factory.assert_not_called() + await operations.shutdown() + + async def test_hostile_shell_metadata_not_adopted(self): + operations = harness.OwnedOperations(1) + async def failure(): + error = RuntimeError('private-message') + error._shell_cleanup_failures = [{'phase': 'kill', 'errorType': 'PrivateToken', 'message': 'private'}] + raise error + with self.assertRaises(RuntimeError): + await operations.perform(failure, is_shell=True) + self.assertEqual(operations.cleanup_details(), {}) + await operations.shutdown() + class BudgetTest(unittest.IsolatedAsyncioTestCase): def setUp(self): self.env = patch.dict(os.environ, {'OPENAI_API_KEY': 'fixture-no-provider-access'}) @@ -576,6 +694,217 @@ def response(number,name,arguments): self.assertEqual(create.await_count,2) client.close.assert_awaited_once() + async def test_real_sdk_parent_waits_for_nested_cleanup_before_terminal_and_close(self): + from openai.types.responses import Response + def response(number, name, arguments): + return Response.model_validate(dict(id=str(number), object='response', created_at=1, + model='fixture', output=[dict(type='function_call', id='call'+str(number), + call_id='call'+str(number), name=name, arguments=json.dumps(arguments), status='completed')], + parallel_tool_calls=False, tool_choice='auto', tools=[], status='completed', + usage=dict(input_tokens=3, output_tokens=2, total_tokens=5, + input_tokens_details=dict(cached_tokens=0, cache_write_tokens=0), + output_tokens_details=dict(reasoning_tokens=0)))) + entered, cleaning, release, settled = (asyncio.Event() for _ in range(4)) + async def fake_shell(*args, **kwargs): + entered.set() + try: + await asyncio.Future() + except asyncio.CancelledError as error: + cleaning.set() + await release.wait() + error._shell_cleanup_failures = [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}] + settled.set() + raise + create = AsyncMock(side_effect=[response(1, 'delegate', {'task': 'explicit child only'}), + response(2, 'execute_shell', {'command': 'fake-only'})]) + client = SimpleNamespace(responses=SimpleNamespace(create=create), base_url='https://api.openai.com/v1/', close=AsyncMock()) + with tempfile.TemporaryDirectory() as directory, patch.object(harness, 'AsyncOpenAI', return_value=client), patch.object(harness, 'shell', fake_shell): + output = io.StringIO() + with contextlib.redirect_stdout(output): + task = asyncio.create_task(harness.run(self.args(directory))) + try: + await asyncio.wait_for(entered.wait(), 1) + task.cancel() + await asyncio.wait_for(cleaning.wait(), 1) + await asyncio.sleep(0) + self.assertFalse(task.done()) + self.assertFalse(settled.is_set()) + self.assertFalse(any(json.loads(line)['type'] in ('error', 'final') for line in output.getvalue().splitlines())) + client.close.assert_not_awaited() + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + release.set() + self.assertEqual(await asyncio.wait_for(task, 1), 1) + finally: + release.set() + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['error_type'], 'CancelledError') + self.assertEqual(final['shellCleanupFailures'], [{'phase': 'kill', 'errorType': 'PermissionError', 'errno': 1}]) + self.assertTrue(settled.is_set()) + self.assertEqual(create.await_count, 2) + client.close.assert_awaited_once() + + async def test_native_normal_error_timeout_and_first_shutdown_cancel_join_work(self): + for mode in ('normal', 'error', 'timeout', 'first-normal-cancel'): + with self.subTest(mode=mode): + entered, cleaning, release, settled = (asyncio.Event() for _ in range(4)) + tool_invocations = [] + captured = {} + async def fake_shell(*args, **kwargs): + entered.set() + try: + await asyncio.Future() + except asyncio.CancelledError: + cleaning.set() + await release.wait() + settled.set() + raise + async def runner(agent, *args, **kwargs): + captured['agent'] = agent + tool = next(tool for tool in agent.tools if tool.name == 'execute_shell') + invocation = asyncio.create_task(tool.on_invoke_tool(ToolContext(context=None, tool_name='execute_shell', tool_call_id='fixture', tool_arguments='{}'), '{"command":"fake"}')) + tool_invocations.append(invocation) + await entered.wait() + if mode == 'error': + raise RuntimeError('private-primary') + if mode == 'timeout': + raise asyncio.TimeoutError('private-primary') + return SimpleNamespace(final_output='done', context_wrapper=SimpleNamespace(usage=SimpleNamespace(requests=0,input_tokens=0,output_tokens=0,total_tokens=0))) + resource = SimpleNamespace(create=AsyncMock()) + client = SimpleNamespace(responses=resource, close=AsyncMock()) + providers = [] + def provider(**kwargs): + value = SimpleNamespace(client=kwargs['openai_client']) + providers.append(value) + return value + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner), patch.object(harness, 'shell', fake_shell) as shell_mock, patch.object(harness, 'AsyncOpenAI', return_value=client), patch.object(harness, 'OpenAIProvider', provider), patch.object(harness, 'retrieve_public', AsyncMock()) as retrieve: + output = io.StringIO() + with contextlib.redirect_stdout(output): + task = asyncio.create_task(harness.run(self.args(directory))) + try: + await asyncio.wait_for(cleaning.wait(), 1) + self.assertFalse(task.done()) + client.close.assert_not_awaited() + self.assertFalse(any(json.loads(line)['type'] in ('error', 'final') for line in output.getvalue().splitlines())) + for tool_name, arguments in [('execute_shell', {'command': 'late'}), ('retrieve_url', {'url': 'https://example.invalid'}), ('delegate', {'task': 'late'})]: + tool = next(tool for tool in captured['agent'].tools if tool.name == tool_name) + with self.assertRaises(asyncio.CancelledError): + await tool.on_invoke_tool(ToolContext(context=None, tool_name=tool_name, tool_call_id='late', tool_arguments='{}'), json.dumps(arguments)) + with self.assertRaises(asyncio.CancelledError): + await providers[0].client.with_options().responses.create(input='late') + resource.create.assert_not_called() + retrieve.assert_not_called() + if mode in ('error', 'first-normal-cancel'): + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + release.set() + self.assertEqual(await asyncio.wait_for(task, 1), 1) + finally: + release.set() + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + await asyncio.gather(*tool_invocations, return_exceptions=True) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['error_type'], {'timeout': 'TimeoutError', 'first-normal-cancel': 'CancelledError'}.get(mode, 'ExecutionError')) + self.assertTrue(settled.is_set()) + self.assertFalse(any(json.loads(line)['type'] == 'final' for line in output.getvalue().splitlines())) + self.assertNotIn('private', output.getvalue()) + client.close.assert_awaited_once() + + async def test_delegate_waiting_for_child_lock_cannot_start_after_closing(self): + entered = asyncio.Event() + invocations = [] + child_starts = [] + async def runner(agent, *args, **kwargs): + if any(tool.name == 'delegate' for tool in agent.tools): + delegate = next(tool for tool in agent.tools if tool.name == 'delegate') + for number in range(2): + invocations.append(asyncio.create_task(delegate.on_invoke_tool(ToolContext(context=None, tool_name='delegate', tool_call_id=str(number), tool_arguments='{}'), '{"task":"fixture"}'))) + await entered.wait() + raise RuntimeError('private-primary') + child_starts.append(agent) + entered.set() + await asyncio.Future() + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(await asyncio.wait_for(harness.run(self.args(directory)), 1), 1) + outcomes = await asyncio.wait_for(asyncio.gather(*invocations, return_exceptions=True), 1) + self.assertEqual(len(child_starts), 1) + self.assertTrue(all(isinstance(value, asyncio.CancelledError) for value in outcomes)) + self.assertEqual(json.loads(output.getvalue().splitlines()[-1])['error_type'], 'ExecutionError') + + async def test_active_retrieval_settles_before_native_cancellation_returns(self): + entered, cleaning, release, settled = (asyncio.Event() for _ in range(4)) + invocations = [] + async def retrieval(*args): + entered.set() + try: + await asyncio.Future() + except asyncio.CancelledError: + cleaning.set() + await release.wait() + settled.set() + raise + async def runner(agent, *args, **kwargs): + tool = next(tool for tool in agent.tools if tool.name == 'retrieve_url') + invocations.append(asyncio.create_task(tool.on_invoke_tool(ToolContext(context=None, tool_name='retrieve_url', tool_call_id='fixture', tool_arguments='{}'), '{"url":"https://example.invalid"}'))) + await asyncio.Future() + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner), patch.object(harness, 'retrieve_public', retrieval): + output = io.StringIO() + with contextlib.redirect_stdout(output): + task = asyncio.create_task(harness.run(self.args(directory))) + try: + await asyncio.wait_for(entered.wait(), 1) + task.cancel() + await asyncio.wait_for(cleaning.wait(), 1) + self.assertFalse(task.done()) + task.cancel() + await asyncio.sleep(0) + self.assertFalse(task.done()) + release.set() + self.assertEqual(await asyncio.wait_for(task, 1), 1) + finally: + release.set() + if not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + await asyncio.gather(*invocations, return_exceptions=True) + self.assertTrue(settled.is_set()) + final = json.loads(output.getvalue().splitlines()[-1]) + self.assertEqual(final['error_type'], 'CancelledError') + self.assertNotIn('shellCleanupFailures', final) + + async def test_native_aggregate_preserves_identical_distinct_shell_failures_once(self): + rows = [{'phase': 'reap', 'errorType': 'OSError', 'errno': 5}] + async def failure(*args, **kwargs): + error = OSError(5, 'private-path') + error._shell_cleanup_failures = rows + raise error + async def runner(agent, *args, **kwargs): + tool = next(tool for tool in agent.tools if tool.name == 'execute_shell') + await asyncio.gather(*(tool.on_invoke_tool(ToolContext(context=None, tool_name='execute_shell', tool_call_id=str(number), tool_arguments='{}'), '{"command":"fixture"}') for number in range(2))) + return SimpleNamespace(final_output='done', context_wrapper=SimpleNamespace(usage=SimpleNamespace(requests=0,input_tokens=0,output_tokens=0,total_tokens=0))) + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner), patch.object(harness, 'shell', failure): + output = io.StringIO() + args = self.args(directory) + args.max_tools = 2 + with contextlib.redirect_stdout(output): + self.assertEqual(await harness.run(args), 1) + records = [json.loads(line) for line in output.getvalue().splitlines()] + self.assertEqual(records[-1]['error_type'], 'ExecutionError') + self.assertEqual(records[-1]['shellCleanupFailures'], rows * 2) + results = [row for row in records if row['type'] == 'tool_result'] + self.assertEqual(len(results), 2) + self.assertTrue(all(row['result']['shellCleanupFailures'] == rows for row in results)) + self.assertNotIn('private', output.getvalue()) + async def test_service_tier_observations_are_allowlisted_and_never_inferred(self): observed_models, observed_tiers = set(), set() resource = SimpleNamespace(create=AsyncMock()) @@ -727,12 +1056,14 @@ async def invoke(agent, *args, **kwargs): with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', invoke), patch.object(harness, 'shell', AsyncMock(side_effect=error)): output = io.StringIO() with contextlib.redirect_stdout(output): - self.assertEqual(await harness.run(self.args(directory)), 0) + self.assertEqual(await harness.run(self.args(directory)), 1) records = [json.loads(line) for line in output.getvalue().splitlines()] result = next(row['result'] for row in records if row['type'] == 'tool_result') self.assertEqual(result['shellCleanupFailures'], [{'phase': 'reap', 'errorType': 'RuntimeError'}]) if isinstance(error, asyncio.TimeoutError): self.assertEqual(result['error'], 'shell command timed out') + self.assertEqual(records[-1]['error_type'], 'ExecutionError') + self.assertEqual(records[-1]['shellCleanupFailures'], result['shellCleanupFailures']) self.assertNotIn('private', output.getvalue()) self.assertNotIn('process group terminated', output.getvalue()) From 2b9ee58200e1f7a01069be74151aafe56d3fd072 Mon Sep 17 00:00:00 2001 From: mm Date: Wed, 7 Oct 2026 23:36:20 -0400 Subject: [PATCH 64/64] Isolate client construction in retrieval cancellation fixture --- cli/protocol/OWNERSHIP.md | 7 +++++++ harnesses/agents-sdk/test_run.py | 15 +++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/cli/protocol/OWNERSHIP.md b/cli/protocol/OWNERSHIP.md index d7e15ff8..a34afe2c 100644 --- a/cli/protocol/OWNERSHIP.md +++ b/cli/protocol/OWNERSHIP.md @@ -1207,3 +1207,10 @@ Root published proposal `c6d0c8b2825e8afda35973c3a89291a81fe9e6c08e508552e8cb9d1 Root extends only the shared error-oracle lease to `cli/shared/tests/test_service_contract.py`: reconcile the old fixed43 taxonomy count with the new additive44th retrieval code using explicit identity/parity checks while retaining all service cases. First81/82 replay is retained; freeze a passing shared oracle before ports. No other lease changes. New-code oracle clarification from independent review: KERNEL_RETRIEVAL_FAILED/http excludes status200. Entry/http permits only400..599; other entry statuses violate the immutable redirect contract and stay IMAGE_INVALID. Descriptor/inventory/kernel permit observed100..599 except200, including refused redirects. Body read failure after200 is transport with no status. These consistency restrictions apply only to the new code; existing policies are unchanged. Freeze strict poisons before ports. + + +## IMP-098 twelfth-cohort retrieval cancellation fixture isolation + +Root and independent peer accepted actual CI12 Intel logs and fixture assessment `a5cf9470fd30d8e43edb522a58e3672ebda3a7cfe0edcaef257e8bbbcba79a9c`, review `685d90fc2ec15e77e1abf6965b202ca56b3eb5d10f1a8020b0b5e6f2d861e4b7`. The active retrieval cancellation fixture leaves irrelevant synchronous real-client initialization inside its one-second readiness wait; both Intel failures occurred before cancellation assertions. Isolate that existing dependency through the same explicit fake-client seam used by neighboring SDK tests. Actual constructor-specific latency remains unmeasured; no product ownership failure is inferred. + +`/root/sdk_assessment` owns only `harnesses/agents-sdk/test_run.py` for this narrow fixture repair. Inject an explicit client with forbidden responses.create and checked close. Preserve real tool invocation, GuardedClient/provider/Agent/OwnedOperations, exact one-second guards, Events, both cancellations, pending-before-release and settled-before-terminal checks. Assert factory arguments/no request and client close once after settlement. No production code, deadlines, retries, skips, dependency pins or defaults change. Configuration peer reviews focused controls. Agents run only focused provider-free SDK tests; root owns source integration, mandatory full29, native qualification, Git/push/merge. Original failed CI and successful full28 records remain source-scoped and unchanged. diff --git a/harnesses/agents-sdk/test_run.py b/harnesses/agents-sdk/test_run.py index 19babc9b..708b07b1 100644 --- a/harnesses/agents-sdk/test_run.py +++ b/harnesses/agents-sdk/test_run.py @@ -843,6 +843,11 @@ async def runner(agent, *args, **kwargs): async def test_active_retrieval_settles_before_native_cancellation_returns(self): entered, cleaning, release, settled = (asyncio.Event() for _ in range(4)) invocations = [] + async def close(): + self.assertTrue(settled.is_set()) + create = AsyncMock(side_effect=AssertionError('provider request forbidden')) + client = SimpleNamespace(responses=SimpleNamespace(create=create), + base_url='https://api.openai.com/v1/', close=AsyncMock(side_effect=close)) async def retrieval(*args): entered.set() try: @@ -856,10 +861,11 @@ async def runner(agent, *args, **kwargs): tool = next(tool for tool in agent.tools if tool.name == 'retrieve_url') invocations.append(asyncio.create_task(tool.on_invoke_tool(ToolContext(context=None, tool_name='retrieve_url', tool_call_id='fixture', tool_arguments='{}'), '{"url":"https://example.invalid"}'))) await asyncio.Future() - with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner), patch.object(harness, 'retrieve_public', retrieval): + with tempfile.TemporaryDirectory() as directory, patch.object(harness.Runner, 'run', runner), patch.object(harness, 'retrieve_public', retrieval), patch.object(harness, 'AsyncOpenAI', return_value=client) as factory: + args = self.args(directory) output = io.StringIO() with contextlib.redirect_stdout(output): - task = asyncio.create_task(harness.run(self.args(directory))) + task = asyncio.create_task(harness.run(args)) try: await asyncio.wait_for(entered.wait(), 1) task.cancel() @@ -880,6 +886,11 @@ async def runner(agent, *args, **kwargs): final = json.loads(output.getvalue().splitlines()[-1]) self.assertEqual(final['error_type'], 'CancelledError') self.assertNotIn('shellCleanupFailures', final) + factory.assert_called_once_with(api_key='fixture-no-provider-access', + max_retries=0, timeout=args.timeout) + create.assert_not_called() + create.assert_not_awaited() + client.close.assert_awaited_once() async def test_native_aggregate_preserves_identical_distinct_shell_failures_once(self): rows = [{'phase': 'reap', 'errorType': 'OSError', 'errno': 5}]