diff --git a/docs/cost-context-governance-design.md b/docs/cost-context-governance-design.md new file mode 100644 index 0000000..89e80ac --- /dev/null +++ b/docs/cost-context-governance-design.md @@ -0,0 +1,118 @@ +# Cost & Context Governance — v0.7.0 Phase 1 Design + +## Version +This document captures the MiniCISO-side design and documentation updates aligned to the Hermes cost/context governance v0.7.0 rollout. + +## Goal +Implement a mandatory, provider-independent governance layer for MiniCISO that preserves evidence quality and independent QA while preventing runaway token/context/tool consumption. + +## Repositories inspected +- MiniCISO overlay checkout: `` +- Hermes runtime checkout: `` + +## Existing integration points + +### MiniCISO overlay +- `profiles/chief-of-staff/SOUL.md`: core Chief-of-Staff operating instructions. Best place to make the governance skill mandatory at the procedural layer. +- `config/chief-of-staff.public.yaml`: public example config that can expose governance defaults. +- No existing MiniCISO-owned runtime package for deterministic governance. + +### Hermes runtime +- `run_agent.py`: `AIAgent` wrappers, turn execution entrypoints, tool execution dispatch, `delegate_task` dispatch, context compression hook. +- `agent/conversation_loop.py`: main model-call loop, iteration handling, context growth, turn completion path. +- `agent/context_compressor.py`: existing compaction/externalization mechanism. +- `agent/tool_executor.py`: deterministic tool execution path; best insertion point for tool-call accounting and pre-tool circuit breakers. +- `agent/tool_guardrails.py`: existing repeated-tool/no-progress primitive that can be complemented by broader engagement governance. +- `tools/delegate_tool.py`: child-agent creation, task context packaging, timeout/error/summary handling; best insertion point for child budget allocation, bounded child context manifests, and structured partial handoffs. +- `model_tools.py`: tool-definition assembly; best insertion point for exact allowlist filtering and tool-schema overhead measurement. +- `hermes_cli/config.py`: default config surface for observe/enforce/disabled modes and budget profiles. + +## Selected implementation strategy +Use a dual-layer design: + +1. **Procedural layer (MiniCISO-owned)** + - Add mandatory skill `cost-context-governance`. + - Patch Chief-of-Staff SOUL to require loading/classification on every request. + - Keep conversational requests lightweight via explicit pass-through classification. + +2. **Runtime layer (Hermes-side generic enforcement, MiniCISO-enabled by config)** + - Add a new controller module that: + - classifies requests; + - creates an engagement workspace + JSON artifacts; + - allocates hierarchical envelopes; + - records telemetry/context manifests/tool-schema overhead; + - enforces model/tool/delegation/time/context thresholds; + - preserves partial handoffs on timeout/hard stop. + - Enable by default in the active Chief-of-Staff profile config. + +## Affected files + +### New files +- `/agent/cost_context_governance.py` +- `/tests/agent/test_cost_context_governance.py` +- `/tests/tools/test_delegate.py` +- `skills/cost-context-governance/SKILL.md` +- `docs/cost-context-governance-design.md` + +### Patched Hermes files +- `run_agent.py` +- `agent/conversation_loop.py` +- `agent/tool_executor.py` +- `tools/delegate_tool.py` +- `model_tools.py` +- `hermes_cli/config.py` + +### Patched MiniCISO/profile files +- `profiles/chief-of-staff/SOUL.md` +- `/SOUL.md` +- `/config.yaml` +- `/skills/cost-context-governance/SKILL.md` + +## Runtime data layout +Under the active profile: +- `engagements//brief.json` +- `engagements//budget.json` +- `engagements//telemetry.jsonl` +- `engagements//evidence_ledger.jsonl` +- `engagements//claim_ledger.jsonl` +- `engagements//checkpoints/*.json` +- `engagements//partial_handoffs/*.json` + +## Scope of first implementation +Implemented now for immediate VPS effect: +- observe/enforce/disabled modes; +- root + child envelope accounting with file-lock persistence; +- configurable budget profiles and QA reserve; +- model/tool/delegation/time/context thresholds; +- context manifests and tool-schema overhead metrics; +- bounded child context package metadata; +- structured partial handoffs on timeout / limit / hard stop; +- role/toolset filtering via runtime allowlist intersection; +- local JSONL telemetry and engagement summaries. + +## Compatibility risks +- Hermes currently lacks a native pre-model-call governance abstraction, so the first implementation must patch runtime call sites directly. +- Tool filtering must not break existing sessions that intentionally grant broader toolsets; fallback is intersection-only when governance is active. +- Child timeout handling varies by provider/runtime path; partial handoff synthesis must work even when the provider returns no summary. +- Existing MiniCISO repo has no skill-sync convention yet; immediate VPS install will patch the active profile directly and also stage the skill inside the overlay repo. + +## Test plan +Automated tests will cover at least: +- threshold/circuit-breaker evaluation before 50 unbounded child calls; +- concurrency-safe shared root budget; +- timeout → persisted partial handoff (not null); +- QA reserve protection; +- per-task tool filtering before schema assembly; +- measurable unused tool-schema overhead; +- compaction/externalization before threshold breach; +- lightweight conversational pass-through; +- usage estimation fallback when provider metadata is missing. + +## Why MiniCISO-level enforcement alone is insufficient +MiniCISO prompt/skill instructions can require planning and checkpointing, but they cannot deterministically: +- prevent an agent from sending oversized tool schemas; +- stop a child before the next model call once budget is exhausted; +- atomically coordinate concurrent child consumption from a shared root budget; +- force structured partial handoffs when runtime timeouts occur. + +Therefore the smallest required upstream-compatible change is a generic Hermes runtime governance controller that MiniCISO enables by config. The fallback for unsupported runtimes is visible observation-only mode with explicit warning in child/task results. diff --git a/docs/dependencies-and-configuration.md b/docs/dependencies-and-configuration.md index 2c708c6..b9ac231 100644 --- a/docs/dependencies-and-configuration.md +++ b/docs/dependencies-and-configuration.md @@ -69,6 +69,7 @@ env -u VIRTUAL_ENV uv run bigua-analyzer --help - KAG query builder - deterministic retrieval selector - manual wrapper with shadow-mode logging +- RTK execution output optimizer experiment for narrow operational command classes **Repo-side location:** - `tools/headroom_phase1/` @@ -79,6 +80,13 @@ env -u VIRTUAL_ENV uv run bigua-analyzer --help - no raw evidence artifacts in the repo - absence in retrieval pack must remain `not_verified_in_raw` - keep selection-first logs and code separable from confidential engagement data +- RTK reduced output is never authoritative +- RTK default mode is `shadow` +- `MINICISO_EXECUTION_OUTPUT_OPTIMIZER=0` must preserve rollback to passthrough + +**RTK MVP scope:** +- included: `git_status`, `git_diff_stat`, `ls`, `find`, `tree`, `git_fetch` +- excluded: `read_file`, `search_files`, `grep`, reports/findings, SARIF, SBOM, PoCs, HTTP traces, SME/Security QA responses ### ProjectDiscovery Cloud / passive discovery layer **Purpose:** passive asset discovery and cloud-assisted recon support when the assessment model includes authorized external inventory work. diff --git a/docs/github-pr-access.md b/docs/github-pr-access.md index 0c2cb94..3e27953 100644 --- a/docs/github-pr-access.md +++ b/docs/github-pr-access.md @@ -40,7 +40,7 @@ If you use a classic PAT instead of a fine-grained one: Store it outside the repo, in the active profile: ```bash -/home/vpsadmin/.hermes/profiles/chief-of-staff/.env +/.env ``` Add: @@ -74,8 +74,8 @@ gh auth status Always valid: ```bash -git -C /home/vpsadmin/miniCISO remote -v -git -C /home/vpsadmin/miniCISO ls-remote origin +git -C remote -v +git -C ls-remote origin ``` To validate the API directly: @@ -97,7 +97,7 @@ PY ### With `gh` ```bash -cd /home/vpsadmin/miniCISO +cd git checkout -b chore/my-change # edit / export / validate git add -A diff --git a/docs/headroom-kag-selective-retrieval.md b/docs/headroom-kag-selective-retrieval.md index 8c1deef..78521a7 100644 --- a/docs/headroom-kag-selective-retrieval.md +++ b/docs/headroom-kag-selective-retrieval.md @@ -45,6 +45,49 @@ The initial recommendation is to run in **shadow mode**: 4. continue running the raw/full flow in parallel; 5. compare savings, recovered evidence, and `decision_delta`. +## RTK execution output optimizer (experimental) + +A second, narrower experiment now lives beside the wrapper in `tools/headroom_phase1/`: the **RTK execution output optimizer**. + +Its contract is intentionally strict: + +- `raw` remains authoritative at all times; +- the optimizer runs only in **shadow mode** by default; +- the reduced view is derivative/log-only and never replaces the effective payload; +- a single env var kill switch restores baseline behavior immediately. + +### Included MVP operation classes + +- `git_status` +- `git_diff_stat` +- `ls` +- `find` +- `tree` +- `git_fetch` + +### Explicitly excluded from this MVP + +- `read_file` +- `search_files` +- `grep` +- evidence artifacts +- reports / findings +- SARIF / SBOM / PoC material +- HTTP traces +- SME or Security QA responses + +### Runtime knobs + +```text +MINICISO_EXECUTION_OUTPUT_OPTIMIZER=1 +MINICISO_EXECUTION_OUTPUT_OPTIMIZER_MODE=shadow +MINICISO_EXECUTION_OUTPUT_OPTIMIZER_ALLOWLIST=git_status,git_diff_stat,ls,find,tree,git_fetch +``` + +### Rollback rule + +Set `MINICISO_EXECUTION_OUTPUT_OPTIMIZER=0` to force passthrough and preserve the pre-experiment baseline without changing the wrapper's real output path. + ## Minimum pack provenance Each selected slice must preserve, at minimum: diff --git a/docs/self-update-capability.md b/docs/self-update-capability.md index fdf8aef..74295c4 100644 --- a/docs/self-update-capability.md +++ b/docs/self-update-capability.md @@ -55,8 +55,8 @@ You can also point to explicit sources: ```bash python3 scripts/export_safe_self_state.py \ - --source-workspace /home/vpsadmin/miniciso-security \ - --source-profile /home/vpsadmin/.hermes/profiles/chief-of-staff \ + --source-workspace \ + --source-profile \ --apply ``` diff --git a/scripts/bootstrap.ps1 b/scripts/bootstrap.ps1 index 5316f67..c346b71 100644 --- a/scripts/bootstrap.ps1 +++ b/scripts/bootstrap.ps1 @@ -91,7 +91,7 @@ if (-not $SkipProviderSetup) { Invoke-Hermes -Arguments @('setup') } -$profileRoot = Join-Path $HOME '.hermes\profiles' +$profileRoot = Join-Path $HermesHome 'profiles' $profiles = Get-ChildItem -LiteralPath (Join-Path $repoRoot 'profiles') -Directory | Sort-Object Name if ($profiles.Count -ne 9) { throw "Expected 9 profiles; found $($profiles.Count)." } diff --git a/scripts/bootstrap.sh b/scripts/bootstrap.sh index 334c760..773b580 100755 --- a/scripts/bootstrap.sh +++ b/scripts/bootstrap.sh @@ -72,7 +72,7 @@ if [[ "$SKIP_PROVIDER_SETUP" == false ]]; then hermes setup fi -profile_root="$HOME/.hermes/profiles" +profile_root="$HERMES_HOME/profiles" profiles=() for profile_dir in "$REPO_ROOT"/profiles/*; do [[ -d "$profile_dir" ]] && profiles+=("${profile_dir##*/}") diff --git a/scripts/export_safe_self_state.py b/scripts/export_safe_self_state.py index c381018..9526f35 100644 --- a/scripts/export_safe_self_state.py +++ b/scripts/export_safe_self_state.py @@ -37,8 +37,16 @@ def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser(description="Export public/safe MiniCISO state from the VPS into this repo.") parser.add_argument("--repo-root", default=str(Path(__file__).resolve().parents[1])) - parser.add_argument("--source-workspace", default="/home/vpsadmin/miniciso-security") - parser.add_argument("--source-profile", default="/home/vpsadmin/.hermes/profiles/chief-of-staff") + parser.add_argument( + "--source-workspace", + default=str(Path.home() / "miniciso-security"), + help="Path to the local MiniCISO workspace to export from.", + ) + parser.add_argument( + "--source-profile", + default=str(Path.home() / ".hermes/profiles/chief-of-staff"), + help="Path to the local Hermes profile root to export from.", + ) parser.add_argument("--apply", action="store_true", help="Actually write files. Default is dry-run.") return parser.parse_args() diff --git a/scripts/smoke-test.ps1 b/scripts/smoke-test.ps1 index 1779206..8b616fd 100644 --- a/scripts/smoke-test.ps1 +++ b/scripts/smoke-test.ps1 @@ -3,6 +3,7 @@ param([switch]$Online) $ErrorActionPreference = 'Stop' $repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path +$hermesHome = if ($env:HERMES_HOME) { $env:HERMES_HOME } else { Join-Path $HOME '.hermes' } $profiles = Get-ChildItem -LiteralPath (Join-Path $repoRoot 'profiles') -Directory | Sort-Object Name $hermes = Get-Command hermes -ErrorAction Stop $profileList = (& $hermes.Source profile list 2>&1 | Out-String) @@ -12,7 +13,7 @@ foreach ($profile in $profiles) { if ($profileList -notmatch [regex]::Escape($profile.Name)) { throw "Profile not registered in Hermes: $($profile.Name)" } - $installedSoul = Join-Path $HOME ".hermes\profiles\$($profile.Name)\SOUL.md" + $installedSoul = Join-Path $hermesHome "profiles\$($profile.Name)\SOUL.md" if (-not (Test-Path -LiteralPath $installedSoul)) { throw "SOUL.md not installed: $($profile.Name)" } diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh index 5d0b619..cb9f2f1 100755 --- a/scripts/smoke-test.sh +++ b/scripts/smoke-test.sh @@ -15,7 +15,7 @@ done for profile in "${profiles[@]}"; do grep -Fq "$profile" <<<"$profile_list" || { echo "Profile not registered: $profile" >&2; exit 1; } - [[ -f "$HOME/.hermes/profiles/$profile/SOUL.md" ]] || { echo "SOUL.md not installed: $profile" >&2; exit 1; } + [[ -f "${HERMES_HOME:-$HOME/.hermes}/profiles/$profile/SOUL.md" ]] || { echo "SOUL.md not installed: $profile" >&2; exit 1; } echo "OK: $profile" if [[ "$ONLINE" == true ]]; then hermes -p "$profile" chat -Q -q 'Answer in one line starting with OK and state your role in MiniCISO.' diff --git a/tools/headroom_phase1/hr_manual_wrapper.py b/tools/headroom_phase1/hr_manual_wrapper.py index 50b5956..0f5986a 100755 --- a/tools/headroom_phase1/hr_manual_wrapper.py +++ b/tools/headroom_phase1/hr_manual_wrapper.py @@ -98,7 +98,11 @@ def parse_args() -> argparse.Namespace: help="Human/QA verdict for this artifact run", ) p.add_argument("--note", default="", help="Free-form operator note") - p.add_argument("--log-dir", default="/home/vpsadmin/miniciso-security/headroom_phase1/logs", help="Log directory") + p.add_argument( + "--log-dir", + default=str(Path.home() / "miniciso-security/headroom_phase1/logs"), + help="Log directory", + ) p.add_argument("--selection-index", default="", help="Path to structural index JSON for selection-first shadow mode") p.add_argument("--selection-query", default="", help="Path to KAG query JSON for selection-first shadow mode") p.add_argument("--selection-pack", default="", help="Path to retrieval pack JSON for selection-first shadow mode")