diff --git a/CHANGELOG.md b/CHANGELOG.md index e4af90fd6..76c000007 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ All notable changes to OriginWeave are documented in this file. The format follo - Cumulative interactive-first RAM, VRAM, batch, local-model, admission, pause, and compositor-pressure mitigation plans, including active-consumer reduction at exact hard limits. - Real pinned-Chrome WebDriver evidence for the controlled Agent Task fixture: the CI lane uses an isolated profile, disables extensions, types and submits synthetic text, observes the same-document post-condition, and proves profile cleanup; this does not claim a shipped OriginWeave browser adapter. - Active pinned-Chrome Agent Task evidence verifies browser-computed role/name for controlled input and submit targets before action; this remains test-harness semantic evidence and does not claim a product semantic observer or authority. +- Active pinned-Chrome Agent Task evidence records browser-process RSS, semantic-observation bytes, action latency, and task duration from bounded trusted adapter inputs; this remains test evidence and does not claim process-set attribution or product resource telemetry. - Universally value-redacted network evidence with explicit path, metadata, and provenance bounds; ambiguous path rejection; validated source URLs; lowercase SHA-256 identifiers; and verification state. - Rust 1.97.1 build contract, strict Clippy and rustdoc gates, and exact production function, line, region, and branch coverage enforcement. - Hourly bounded OpenCode product-development workflow using `NVIDIA_NIM_API_KEY`, an unprivileged disposable workspace, loopback-only model broker, independently verified patches, and publication through a dedicated `OPENCODE_PR_TOKEN` that cannot review or merge. diff --git a/docs/DOCUMENTATION_FITNESS.md b/docs/DOCUMENTATION_FITNESS.md index f82bd4fe8..640ed7111 100644 --- a/docs/DOCUMENTATION_FITNESS.md +++ b/docs/DOCUMENTATION_FITNESS.md @@ -164,9 +164,9 @@ Active #65 supplies a deterministic synthetic local web fixture with a labelled ### 3.20 Bounded browser process-set resource evidence -Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled. +Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled. Active PR #72 records browser-process RSS, semantic-observation bytes, action latency, and task duration for the controlled pinned-Chrome fixture from bounded trusted adapter inputs. -**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. The implementation does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store. +**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. PR #72 is bounded resource evidence for test repeatability; it does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store, and does not turn the fixture into a product resource adapter. ## 4. Durable product decisions captured by the canonical graph diff --git a/docs/TEST_STRATEGY.md b/docs/TEST_STRATEGY.md index faf7f54b3..a2e898ff1 100644 --- a/docs/TEST_STRATEGY.md +++ b/docs/TEST_STRATEGY.md @@ -90,6 +90,12 @@ controlled input and submit target before the real WebDriver action. CSS remains a fixture-harness locator; this does not establish OriginWeave node authority, semantic provenance or policy dispatch. +Active PR #72 additionally records bounded browser-process RSS, +semantic-observation bytes, action latency and task duration for the same +controlled fixture. These are test-harness resource evidence from trusted +adapter inputs; they do not establish Chromium process-set attribution, +GPU/VRAM telemetry or a product resource adapter. + ### 3.5 Buyer acceptance Versioned task packs measure repeatable product outcomes rather than one lucky agent run. The benchmark artifact records browser build, OriginWeave version, model/provider/reasoning configuration, seed where supported, policy profile, hardware profile and source fixtures. diff --git a/docs/traceability/action-postcondition-evidence.md b/docs/traceability/action-postcondition-evidence.md index 482538394..866e0404f 100644 --- a/docs/traceability/action-postcondition-evidence.md +++ b/docs/traceability/action-postcondition-evidence.md @@ -71,6 +71,14 @@ PR #71 extends the pinned-Chrome fixture lane by reading WebDriver's browser-com This is bounded browser-computed evidence for a synthetic test target, not the OriginWeave semantic observation adapter. CSS locators remain test-harness selectors, and the lane does not create OriginWeave node handles, source-channel provenance, policy authority, or permission to execute page-advertised actions. +### PR #72 — bounded Agent Task resource evidence + +**Capability maturity:** `IMPLEMENTED_ON_ACTIVE_PR` + +PR #72 records browser-process RSS, semantic-observation bytes, action latency, and total task duration while the pinned-Chrome fixture runs. The measurements are bounded, positive observations from the trusted ChromeDriver process identifier and the controlled semantic payload; they make the real fixture's resource and timing evidence inspectable without introducing a new telemetry subsystem. + +This is resource evidence for the active test harness, not process-set attribution or a product resource adapter. It does not discover Chromium children, prove task ownership or ancestry, walk cgroups, sample GPU/VRAM, or export durable product telemetry. + ## 4. Non-transitive success semantics The intended first-slice chain is: @@ -96,7 +104,7 @@ VerifiedActionOutcomeEvidence type existence -/> proof of real Chromium executio controlled fixture success -/> proof of an OriginWeave product browser runtime ``` -PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture and PR #71 adds browser-computed role/name evidence, but their test-harness CSS locators and direct WebDriver calls are not the OriginWeave adapter/runtime composition required under issue #28. +PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture, PR #71 adds browser-computed role/name evidence, and PR #72 adds bounded resource evidence, but their test-harness CSS locators, direct WebDriver calls, and fixture-scoped measurements are not the OriginWeave adapter/runtime composition required under issue #28. ## 5. Active prerequisite graph for issue #28 @@ -111,9 +119,10 @@ The first real Chromium vertical slice remains distributed across bounded active - PR #64 — verified and caller-timestamp-ordered post-condition action-outcome evidence; and - PR #65 — controlled hostile local Agent Task workflow fixture; and - PR #70 — real WebDriver execution of that fixture on pinned Chrome, without claiming a product browser adapter; and -- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer. +- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer; and +- PR #72 — bounded browser-process RSS, semantic-observation byte, latency, and task-duration resource evidence, without claiming process-set attribution or a product resource adapter. -These active PRs are non-shipped evidence. PR #70/#71 prove bounded browser-level and semantic evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime. +These active PRs are non-shipped evidence. PR #70/#71/#72 prove bounded browser-level, semantic, and resource evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime. ## 6. Remaining issue #28 boundary diff --git a/scripts/ci/run_mv3_compatibility.py b/scripts/ci/run_mv3_compatibility.py index 747fc9b01..6f5c3f3e6 100755 --- a/scripts/ci/run_mv3_compatibility.py +++ b/scripts/ci/run_mv3_compatibility.py @@ -9,8 +9,9 @@ restart-persistence behavior. It also executes the controlled Agent Task fixture with extensions disabled in a fresh profile, verifies browser-computed role/name for the controlled action targets, performs real WebDriver input and click -operations, verifies the observable post-condition, and proves profile cleanup -without treating page content as instruction or authority. +operations, verifies the observable post-condition, proves the controlled action +preserves its loaded URL, and records bounded runtime resource evidence without +treating page content as instruction or authority. """ from __future__ import annotations @@ -41,6 +42,8 @@ STARTUP_TIMEOUT_SECONDS = 20.0 FIXTURE_TIMEOUT_SECONDS = 20.0 MAX_WEBDRIVER_RESPONSE_BYTES = 1_048_576 +MAX_PROC_STATUS_CHARACTERS = 65_536 +MAX_U64 = (1 << 64) - 1 W3C_ELEMENT_KEY = "element-6066-11e4-a52e-4f735466cecf" PATH_TOKEN_CHARACTERS = frozenset(string.ascii_letters + string.digits + "-_.") @@ -202,6 +205,43 @@ def _get_element_semantics( return role, label +def _parse_linux_proc_status_rss_bytes(status_text: str) -> int: + """Parse exactly one positive Linux ``VmRSS`` kB field into bounded bytes.""" + + rss_values: list[int] = [] + for line in status_text.splitlines(): + if not line.startswith("VmRSS:"): + continue + fields = line.split() + if len(fields) != 3 or fields[0] != "VmRSS:" or fields[2] != "kB": + raise ValueError("malformed Linux VmRSS field") + raw_kibibytes = fields[1] + if not raw_kibibytes.isascii() or not raw_kibibytes.isdigit(): + raise ValueError("malformed Linux VmRSS value") + kibibytes = int(raw_kibibytes, 10) + if kibibytes <= 0: + raise ValueError("Linux VmRSS must be positive") + if kibibytes > MAX_U64 // 1024: + raise OverflowError("Linux VmRSS exceeds u64 byte range") + rss_values.append(kibibytes * 1024) + if len(rss_values) != 1: + raise ValueError("Linux proc status must contain exactly one VmRSS field") + return rss_values[0] + + +def _sample_linux_process_rss_bytes(process_id: int) -> int: + """Read one attributed Linux process RSS through a bounded ``/proc`` status file.""" + + if isinstance(process_id, bool) or not isinstance(process_id, int) or process_id <= 0: + raise ValueError("invalid Linux process identifier") + status_path = pathlib.Path("/proc") / str(process_id) / "status" + with status_path.open("r", encoding="utf-8", errors="strict") as status_file: + status_text = status_file.read(MAX_PROC_STATUS_CHARACTERS + 1) + if len(status_text) > MAX_PROC_STATUS_CHARACTERS: + raise RuntimeError("Linux proc status exceeded the bounded text limit") + return _parse_linux_proc_status_rss_bytes(status_text) + + def _wait_for_extension_evidence( driver_port: int, session_id: str, @@ -490,7 +530,7 @@ def _run_agent_task_browser_pass( fixture_url: str, profile_dir: str, ) -> dict[str, Any]: - """Execute one synthetic Agent Task through real WebDriver input in pinned Chrome.""" + """Execute one synthetic Agent Task and measure bounded real-browser evidence.""" started = time.monotonic() driver_port = _free_loopback_port() @@ -535,15 +575,22 @@ def _run_agent_task_browser_pass( capabilities = session.get("capabilities", {}) if not isinstance(raw_session_id, str): raise RuntimeError("ChromeDriver did not return an Agent Task session id") + if not isinstance(capabilities, dict): + raise RuntimeError("ChromeDriver Agent Task capabilities are malformed") session_id = _path_token(raw_session_id, "session identifier") - browser_version = ( - capabilities.get("browserVersion") if isinstance(capabilities, dict) else None - ) + browser_version = capabilities.get("browserVersion") + browser_process_id = capabilities.get("goog:processID") if browser_version != PINNED_CHROME_VERSION: raise RuntimeError( f"unexpected Agent Task Chrome version: expected {PINNED_CHROME_VERSION}, " f"got {browser_version!r}" ) + if ( + isinstance(browser_process_id, bool) + or not isinstance(browser_process_id, int) + or browser_process_id <= 0 + ): + raise RuntimeError("ChromeDriver did not return a valid browser process id") _json_request( driver_port, @@ -557,9 +604,8 @@ def _run_agent_task_browser_pass( _webdriver_path(session_id, "/url"), ).get("value") if initial_url != fixture_url: - raise RuntimeError( - f"Agent Task initial URL mismatch: expected {fixture_url!r}, got {initial_url!r}" - ) + raise RuntimeError("Agent Task did not load the requested fixture URL") + input_element = _find_element(driver_port, session_id, "#task-text") input_role, input_name = _get_element_semantics( driver_port, @@ -568,18 +614,6 @@ def _run_agent_task_browser_pass( ) if input_role != "textbox" or input_name != "Task text": raise RuntimeError("Agent Task input semantic evidence mismatch") - _json_request( - driver_port, - "POST", - _element_command_path(session_id, input_element, "/clear"), - {}, - ) - _json_request( - driver_port, - "POST", - _element_command_path(session_id, input_element, "/value"), - {"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)}, - ) submit_element = _find_element( driver_port, session_id, @@ -592,12 +626,44 @@ def _run_agent_task_browser_pass( ) if submit_role != "button" or submit_name != "Submit task": raise RuntimeError("Agent Task submit semantic evidence mismatch") + semantic_observation = { + "input": {"role": input_role, "name": input_name}, + "submit": {"role": submit_role, "name": submit_name}, + } + semantic_observation_bytes = len( + json.dumps( + semantic_observation, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ) + if semantic_observation_bytes <= 0: + raise RuntimeError("Agent Task semantic observation was empty") + + action_started = time.monotonic() + _json_request( + driver_port, + "POST", + _element_command_path(session_id, input_element, "/clear"), + {}, + ) + _json_request( + driver_port, + "POST", + _element_command_path(session_id, input_element, "/value"), + {"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)}, + ) _json_request( driver_port, "POST", _element_command_path(session_id, submit_element, "/click"), {}, ) + action_latency_ms = round((time.monotonic() - action_started) * 1000, 3) + if action_latency_ms <= 0: + raise RuntimeError("Agent Task measured a non-positive action latency") + post_submit_url = _json_request( driver_port, "GET", @@ -606,6 +672,7 @@ def _run_agent_task_browser_pass( url_unchanged = post_submit_url == initial_url if not url_unchanged: raise RuntimeError("Agent Task URL changed during submission") + result_element = _find_element(driver_port, session_id, "#task-result") state = _json_request( driver_port, @@ -620,6 +687,10 @@ def _run_agent_task_browser_pass( _validate_agent_task_submitted_state(state) if text != AGENT_TASK_INPUT_VALUE: raise RuntimeError("Agent Task result did not match the synthetic typed value") + browser_process_rss_bytes = _sample_linux_process_rss_bytes(browser_process_id) + task_duration_ms = round((time.monotonic() - started) * 1000, 3) + if task_duration_ms <= 0: + raise RuntimeError("Agent Task measured a non-positive task duration") return { "browser_version": browser_version, "post_condition": True, @@ -628,7 +699,11 @@ def _run_agent_task_browser_pass( "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled": True, - "duration_ms": round((time.monotonic() - started) * 1000), + "browser_process_rss_bytes": browser_process_rss_bytes, + "semantic_observation_bytes": semantic_observation_bytes, + "action_latency_ms": action_latency_ms, + "task_duration_ms": task_duration_ms, + "duration_ms": round(task_duration_ms), } finally: try: @@ -677,6 +752,10 @@ def _run_agent_task_trial( "input_semantics_verified": result["input_semantics_verified"], "submit_semantics_verified": result["submit_semantics_verified"], "extensions_disabled": result["extensions_disabled"], + "browser_process_rss_bytes": result["browser_process_rss_bytes"], + "semantic_observation_bytes": result["semantic_observation_bytes"], + "action_latency_ms": result["action_latency_ms"], + "task_duration_ms": result["task_duration_ms"], "profile_cleaned": profile_cleaned, "duration_ms": round((time.monotonic() - trial_started) * 1000), } @@ -806,6 +885,14 @@ def main() -> int: and trial.get("submit_semantics_verified") is True and trial.get("extensions_disabled") is True and trial.get("profile_cleaned") is True + and isinstance(trial.get("browser_process_rss_bytes"), int) + and trial["browser_process_rss_bytes"] > 0 + and isinstance(trial.get("semantic_observation_bytes"), int) + and trial["semantic_observation_bytes"] > 0 + and isinstance(trial.get("action_latency_ms"), (int, float)) + and trial["action_latency_ms"] > 0 + and isinstance(trial.get("task_duration_ms"), (int, float)) + and trial["task_duration_ms"] >= trial["action_latency_ms"] for trial in agent_task_trials if trial.get("passed") is True ) diff --git a/tests/test_agent_task_pinned_chrome_contract.py b/tests/test_agent_task_pinned_chrome_contract.py index 2571cabd4..196cbdb62 100644 --- a/tests/test_agent_task_pinned_chrome_contract.py +++ b/tests/test_agent_task_pinned_chrome_contract.py @@ -116,6 +116,45 @@ def test_agent_task_observes_computed_role_and_name_before_action(self) -> None: with self.subTest(expected=expected): self.assertIn(expected, runner) + def test_agent_task_records_real_bounded_resource_evidence(self) -> None: + """The real task must report measured browser/runtime resource evidence.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_resource_contract") + runner = RUNNER.read_text(encoding="utf-8") + for expected in ( + "_parse_linux_proc_status_rss_bytes", + "_sample_linux_process_rss_bytes", + ): + with self.subTest(expected=expected): + self.assertIn(expected, namespace) + for expected in ( + '"goog:processID"', + '"browser_process_rss_bytes"', + '"semantic_observation_bytes"', + '"action_latency_ms"', + '"task_duration_ms"', + ): + with self.subTest(expected=expected): + self.assertIn(expected, runner) + + def test_linux_rss_parser_is_strict_and_overflow_safe(self) -> None: + """Runner-side RSS evidence must not accept ambiguous proc status input.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_rss_contract") + parser = namespace["_parse_linux_proc_status_rss_bytes"] + self.assertEqual(parser("Name:\tchrome\nVmRSS:\t123 kB\n"), 123 * 1024) + for malformed in ( + "Name:\tchrome\n", + "VmRSS:\t0 kB\n", + "VmRSS:\t123 MB\n", + "VmRSS:\t123 kB extra\n", + "VmRSS:\t123 kB\nVmRSS:\t124 kB\n", + "VmRSS:\t18446744073709551616 kB\n", + ): + with self.subTest(malformed=malformed): + with self.assertRaises((ValueError, OverflowError)): + parser(malformed) + def test_agent_task_fixture_runs_under_the_existing_pinned_chrome_job(self) -> None: """No floating browser or second workflow may be introduced for this slice.""" @@ -146,6 +185,10 @@ def test_documentation_separates_active_browser_evidence_from_product_runtime(se self.assertIn("PR #71", traceability) self.assertIn("computed role/name", traceability) self.assertIn("computed role/name", fitness) + self.assertIn("browser-process RSS", changelog) + self.assertIn("PR #72", traceability) + self.assertIn("resource evidence", traceability) + self.assertIn("resource evidence", fitness) if __name__ == "__main__":