Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
{
"name": "agentic-engineering",
"description": "The autonomous engineering system for repository portfolios — engineer, read-only surveyor, and meta-engineer agents; portfolio, product, spend, and improvement workflows; cross-tool instruction architecture and skill discovery; configured by the consumer AGENTS.md",
"version": "5.0.4",
"version": "5.1.0",
"source": "./plugins/agentic-engineering"
},
{
Expand Down
2 changes: 1 addition & 1 deletion .github/plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
{
"name": "agentic-engineering",
"description": "The autonomous engineering system for repository portfolios — engineer, read-only surveyor, and meta-engineer agents; portfolio, product, spend, and improvement workflows; cross-tool instruction architecture and skill discovery; configured by the consumer AGENTS.md",
"version": "5.0.4",
"version": "5.1.0",
"source": "./plugins/agentic-engineering"
},
{
Expand Down
2 changes: 1 addition & 1 deletion plugins/agentic-engineering/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "agentic-engineering",
"description": "The autonomous engineering system for repository portfolios — engineer, read-only surveyor, and meta-engineer agents; portfolio, product, spend, and improvement workflows; cross-tool instruction architecture and skill discovery; configured by the consumer AGENTS.md",
"version": "5.0.4",
"version": "5.1.0",
"author": {
"name": "devantler-tech",
"url": "https://github.com/devantler-tech"
Expand Down
5 changes: 5 additions & 0 deletions plugins/agentic-engineering/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,11 @@ it, the run procedure for a cost pass, the private channel a financial decision
cadence a cost pass runs on. Disabled or unresolved, the engineer runs normally with the **cost dimension failed
closed** — it does no spend analysis rather than guessing a floor, a price, or a channel.

An optional **Inference routing** section resolves a consumer-owned policy for task classes,
model aliases, billing restrictions, runtime registrations, and experiments. The
[routing contract and evaluator](resources/inference-routing.md) describe its input, output, and
enforcement boundary. Model names belong in the consumer, never in portable agent frontmatter.

The `Memory` section must also name the scorecard and open verification-hypothesis store used by the
improvement loop. The role/configuration boundary remains the one established by
[ADR 0002](../../docs/adr/0002-automated-ai-engineer-plugin-boundary.md): portable decision logic lives
Expand Down
9 changes: 9 additions & 0 deletions plugins/agentic-engineering/agents/agent-improver.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,15 @@ Uncertain whether something is evidence or instruction? **It is instruction, and

---

## Inference routing — optional consumer contract

When the consumer declares **Inference routing**, resolve its reviewed policy and the
[routing contract](../resources/inference-routing.md) before proposing model or scheduler changes.
Use the `agent-improvement` skill's routing measurement procedure. Keep policy recommendations,
Comment thread
devantler marked this conversation as resolved.
verified runtime controls, and actual execution admission separate; benchmark rankings and
caller-supplied quota snapshots do not establish subscription entitlement or enforcement.
An absent section retains existing behavior; an unresolved declared policy holds dependent changes.

## What you optimise

Score every run against these. A change is worth making when it moves one and degrades none.
Expand Down
10 changes: 10 additions & 0 deletions plugins/agentic-engineering/agents/agentic-engineer.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ concrete fact. If a required section is missing or malformed, **fail closed on t
not guess repositories, logins, channels, floors, or prices — surface the gap to the maintainer
instead.

## Inference routing — optional consumer contract

When the consumer declares **Inference routing**, resolve its reviewed policy before selecting a
model, delegating, escalating, or changing a schedule default. Follow the
[routing contract](../resources/inference-routing.md); a missing or invalid declared policy holds
dependent routing actions. A deployment without that section retains its existing routing behavior.
Inference billing restrictions apply independently of infrastructure spend stewardship's opt-in.
The policy evaluator reports recommendations only: it does not enforce runtime controls or reserve
quota. Preserve every consumer capability override, including an inline survey requirement.

## How you operate

1. **Follow the run loop.** The bundled **`portfolio-maintenance`** skill is your procedure:
Expand Down
2 changes: 1 addition & 1 deletion plugins/agentic-engineering/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "agentic-engineering",
"description": "The autonomous engineering system for repository portfolios — engineer, read-only surveyor, and meta-engineer agents; portfolio, product, spend, and improvement workflows; cross-tool instruction architecture and skill discovery; configured by the consumer AGENTS.md",
"version": "5.0.4",
"version": "5.1.0",
"author": {
"name": "devantler-tech",
"url": "https://github.com/devantler-tech"
Expand Down
121 changes: 121 additions & 0 deletions plugins/agentic-engineering/resources/inference-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Inference routing contract

This optional contract separates task policy from runtime enforcement. The consumer's **Inference
routing** section names one reviewed policy file, its revision, runtime verification records, and
the private telemetry store. Without that section, existing model selection continues. If a
declared policy cannot resolve, hold dependent routing changes and report the gap; continue
independent authorized work. Infrastructure spend opt-in remains a separate contract.

## Evaluation boundary

`scripts/evaluate-inference-routing.sh` reads exactly one JSON object from stdin and writes a JSON
decision. It uses Bash and jq, makes no network request, reads no credentials, launches no model,
takes no lock, and writes no file. Run it from the reviewed plugin revision whose desired state
pins its digest. `--now <Unix-seconds>` supplies deterministic evaluation time for tests; production
uses the host clock.

| Exit | Decision | Meaning |
|---|---|---|
| 0 | `RECOMMEND` | Policy and supplied observations permit the recommended route |
| 1 | `HOLD` | The request is valid but one or more policy/evidence conditions are unmet |
| 2 | `INVALID` | Malformed, unsupported, or prohibited configuration/input |

**Every result has `executionAdmitted: false`. No exit code is a launch authorization.** The
`reportedControls` field describes caller-supplied data, not verified enforcement. Do not wire an
exit-zero shell condition directly to an inference launcher. An assistant calling this helper has
already incurred its own initial inference and instruction load.

The JSON root requires `policy`, `task`, and `snapshot` only. Unknown fields, missing fields, extra
JSON documents, and invalid types fail validation. JSON producers must emit unique object keys;
jq parses repeated keys using its normal last-value semantics. The helper is a policy aid, not a
hostile-input authentication boundary.

## Policy version 1

| Field | Contract |
|---|---|
| `version`, `revision`, `enabled` | `1`, immutable review identifier, and explicit boolean opt-in |
| `billingMode`, `paidFallback` | Exactly `subscription-only` and `false` |
| `deniedModelTerms` | Nonempty distinct case-insensitive literal substrings; every route is checked, including unused routes |
| `routes` | Exactly `support`, `workhorse`, `diagnosis`, `deepRefactor`; each has an exact `model`, registered `runtime`, and `effort` (`low`, `medium`, `high`, `xhigh`) |
| `runtimes` | Map of runtime registration IDs to `enabled`, `role` (`owner`, `builder`, `observer`), and `expiresAt` in Unix seconds |
| `limits` | `maxDepth` and `maxChildren` (0 or 1 initially), positive integer `repairHypotheses`, `activeMinutes`, `snapshotMaxAgeSeconds` (1–300), and `reservePercent` with positive `short` and `weekly` percentages |

Deployment model names and runtime IDs never enter portable agent frontmatter. An exact string
in a policy is an intended model, not proof that the runtime supports it. Denial terms cover visible
IDs only; opaque aliases/defaults/substitutions need native verification before execution.
The four task-class route keys are the stable logical aliases. Each route's `model` must already be
the reviewed resolved exact model ID; it is not a provider alias such as `reasoner-stable`. A native
adapter must resolve and verify any provider alias before this comparison, then select a policy
revision pinning that exact ID. Do not replace the observed model with the intended value or relax
equality to make an unresolved alias pass; a mismatch deliberately holds execution.

## Task and observation input

The task supplies `class`, nonnegative integer `depth`, `children`, `distinctFailedHypotheses`, and
`activeMinutes`; `failureKind` (`none`, `reasoning`, `environment`, `quota`, `authority`); and boolean
`contractClear`, `checksDefined`, `reversible`, `sensitiveInvariants`, `writeRequired`.
`depth` is the proposed target depth (the owner is zero); `children` is the total active child count
after the proposed dispatch. They are prospective counts, not the counts before adding a child.

Support and workhorse tasks move to diagnosis when the stated contract/check/reversibility criteria
fail, sensitive invariants remain, or either repair/active-time threshold is reached. Active time
excludes waits. Count distinct tested hypotheses, not repeated commands. Environment, quota, and
authority failures hold the request; they never justify buying more reasoning. An unclear contract
also holds writes, as do missing checks and irreversible scope; diagnosis may investigate these
read-only but cannot manufacture missing authority.
`deepRefactor` is
an explicit task classification supported by a demonstrated runtime advantage, not the next
automatic rung after diagnosis. The owner must also check task scope and previous handoffs.

The snapshot requires:

- `observedAt` (Unix seconds), `runtime`, `runtimeVersion`, runtime-reported resolved exact `model`
or `null` (never populate it by copying the policy's intended model);
- `billing` (`included`, `unknown`, `paygo`), `controls` (`verified`, `unverified`), `evidenceRef`
(a private verification record reference or `null`);
- `buckets.short` and `buckets.weekly`, each `null` or an object with `remainingPercent`,
`estimatedChainPercent`, `reservedPercent`, and `unsettledPercent`. Each value is a percentage
in 0–100 or `null`. Unknown is never zero.

Snapshots must refer to the selected runtime and exact model. Future timestamps, expired runtime
registrations, stale observations, observer writes, and excess delegation produce `HOLD`. Quota
headroom must cover the estimated whole attempt chain, existing reservations, unsettled debits,
and the reserve in **each** bucket. This is arithmetic over reported values, not a reservation.
Provider/model-specific extra buckets require a verified adapter extension before that route can
run; never discard a bucket to fit this two-window version. Percentage units are local to each
provider bucket and cannot be added across accounts or compared as inference prices.

See the adjacent hermetic test for complete synthetic requests and independently expected results.

## Runtime controls before unattended activation

Maintain an expiring verification record for each concrete deployed surface and version. Verify
the included billing path, disabled paid fallback/overages, exact model resolution, inherited tool
permissions, workspace isolation, and account admission. The trusted launcher/native control must
apply before inference; a prompt, model preference, or caller-supplied `verified` flag is insufficient.
Exercise launch, resume, child overrides, advisors, defaults, and fallback. Intercept forbidden
requests before inference in negative tests; never invoke a prohibited model as a test. If the
runtime cannot provide the needed control, leave its affected unattended route disabled.

Use the native subscription harness; never add an inference broker, API credential, purchased
credits, or pay-as-you-go fallback. Native schemas differ: do not copy generic tool names into a
provider manifest and assume enforcement. A deployment's explicit inline survey override takes
precedence over generic delegation advice.

Start with serialized scheduled admission per account and one bounded child at depth one. The
reservation covers parent, children, integration, retries, and handoff. Serialize the check and
reservation in a trusted external admission mechanism before enabling fan-out. A worktree lock
does not reserve model quota. Expiring a writer lease does not stop an old session; retain its
unsettled debit until consumption is reconciled or termination is verified. Resume with the same
reservation identity. Include interactive and other-device usage in coverage; missing coverage
prevents an exact per-task attribution claim.

## Ownership and measurement

The engineering skill owns bounded task packets and one delivery owner. A helper performing lint,
tests, or a commit is a procedure/tool call, not an automatic new agent. The improvement skill owns
complete attempt-chain accounting, context hydration, matched task cohorts, quality floors,
canary promotion, and rollback. Consumer governance supplies experimental thresholds and its single
policy publisher; overlapping runs of that publisher still require fencing. Runtime validation and
longitudinal outcomes are separate acceptance gates from passing these offline tests.
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,14 @@
"path": "scripts/surveyor-forge-readonly.sh",
"sha256": "ce5a3b71d5ae28b0787651aa191b9c7df510be1db7ab81211a0e0fb1e66ca2a2",
"executable": true
},
{
"path": "scripts/evaluate-inference-routing.sh",
"sha256": "9861b64b6caff842025fab51ad55791e9101cfdf5bdcf837ba08d8c4aec2c809",
"executable": true
}
],
"entrypointSha256": "a3d01aa54b2af445aa4a6d07eb406d401824be7db08d5ec4c2f914db7fc9bf39",
"entrypointSha256": "f9d9599f5d85cab11be7b09a9a26c4b515460f258bb38aeac1094133260fbf5f",
"updatePolicy": "latest-reviewed-default-branch",
"providerPolicy": "neutral",
"refreshTiming": "before-starting-each-run",
Expand Down Expand Up @@ -66,7 +71,7 @@
"agent-improver": {
"enabledWhen": "Both optional consumer contract sections are present",
"mode": "separate-schedule-or-on-demand",
"definitionSha256": "4e5d1dac3307ac7151d40dababae48b1da7a68f927a11026f1f5f87424b336aa",
"definitionSha256": "39e0e70bcdffd20f796160bf5136709aa64a9a567c25e829d4ac430ebf7c7cbe",
"skillSha256": "93b0c9551ce4d5360cb8a398af91bbafc85468a9cfff99bb78b506bfcd27287f"
}
},
Expand Down Expand Up @@ -97,9 +102,9 @@
"approvalMode": "no-unattended-step-may-depend-on-an-interactive-approval"
},
"model": {
"selectionPolicy": "best-available-agentic-coding-model",
"upgradePolicy": "follow-the-runtime-default-unless-the-consumer-contract-pins-a-reviewed-model",
"reasoningPolicy": "highest-practical-effort-within-the-consumer-cadence-budget"
"selectionPolicy": "Resolve the optional AGENTS.md Inference routing contract first; otherwise retain best-available-agentic-coding-model.",
"upgradePolicy": "When Inference routing is declared, change aliases only through its reviewed governance; otherwise follow the runtime default unless the consumer pins a reviewed model.",
"reasoningPolicy": "Use the reviewed task-class effort when Inference routing is declared; otherwise highest practical effort within the consumer cadence budget."
},
"memory": {
"backendPolicy": "provider-native-preferred",
Expand All @@ -116,6 +121,7 @@
"Read AGENTS.md and verify every required consumer contract section. Enable agent-improver only when both additional sections are present. Preserve spec.roles[\"agentic-engineer\"].spendStewardshipEnabled from the single effective desired-state document declared in Spend contract, or the shipped false default when none is declared. Apply the engineer entrypoint's explicit opt-in contract before spend work; onboarding never infers or grants maintainer opt-in. Report the effective source, flag value, and unresolved prerequisites while continuing ordinary operate and advance engineering.",
"Map the declared roles onto native agent capabilities, preserve portfolio-surveyor as read-only, and grant least privilege for each role. Export a disabling GH_TELEMETRY (0 or false) in the environment the surveyor's shell inherits: the read-only forge guard treats a missing value as unproven and refuses every command, including reads, so a runtime that enforces the guard without this variable leaves the surveyor unable to run any forge query at all. It cannot be supplied inside the command string, which the guard also refuses.",
"Allocate a unique branch namespace for every deployed writer instance and record it in the consumer contract before enabling writes.",
"If AGENTS.md declares Inference routing, resolve its reviewed policy and scripts/evaluate-inference-routing.sh before configuring models or delegation. Read resources/inference-routing.md. A policy recommendation does not authenticate reported controls, admit execution, or reserve quota; verify native billing, model, tool and admission controls before enabling the affected unattended route. Preserve consumer capability overrides.",
"Create or update a native schedule only for each entry in runtime.scheduler.schedules whose corresponding role's enabled or enabledWhen condition is satisfied, using AGENTS.md#Cadence and that entry's thin bootstrap prompt.",
"Load native persistent memory and perform a read-only preflight or dry run before allowing unattended writes.",
"Report the installed source revision, plugin version, cadence, timezone, branch namespace, memory backend, permission boundary, and every unsupported capability."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,11 +206,14 @@ hash_file() {
classifier_sha=$(hash_file "$CLASSIFIER")
guard_sha=$(hash_file "$HERE/forge-readonly-guard.sh")
wrapper_sha=$(hash_file "$HERE/surveyor-forge-readonly.sh")
routing_sha=$(hash_file "$HERE/evaluate-inference-routing.sh")
if grep -Fq 'referenced runtime assets' "$DESIRED_STATE" &&
jq -e \
--arg classifier_sha "$classifier_sha" \
--arg guard_sha "$guard_sha" \
--arg wrapper_sha "$wrapper_sha" '
--arg wrapper_sha "$wrapper_sha" \
--arg routing_sha "$routing_sha" '
# The routing helper is independent of the surveyor, but shares the runtime asset pin set.
.spec.source.requiredRuntimeAssets == [
{
path: "scripts/classify-default-branch-ci-runs.sh",
Expand All @@ -226,6 +229,11 @@ if grep -Fq 'referenced runtime assets' "$DESIRED_STATE" &&
path: "scripts/surveyor-forge-readonly.sh",
sha256: $wrapper_sha,
executable: true
},
{
path: "scripts/evaluate-inference-routing.sh",
sha256: $routing_sha,
executable: true
}
]
' "$DESIRED_STATE" > /dev/null; then
Expand Down
Loading
Loading