Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ When either changes at MINOR or above, a new entry below records the diff and th

### Changed

- *(none — all additions are net-new docs that cite, but do not modify, the canonical artifacts.)*
- `spec.md` 0.1.0 → 0.2.0 — make budget enforcement concurrency-safe and accounting deterministic (FR-011, FR-112, FR-112a, FR-113, FR-115, FR-116, FR-116a, FR-128; [#45](https://github.com/CiscoDevNet/foundry-security-spec/issues/45)).
- Budget operations, observability, substrate-contract, glossary, and reference documentation updated for atomic cost reservation, idempotent settlement, active-runtime semantics, deterministic trailing windows, and hot-reload/stop behavior.

### Removed

Expand Down
4 changes: 2 additions & 2 deletions GLOSSARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ A condensed quick-card version (one row per term, fewer terms) is at [`docs/refe
| **Detection rule** | A reusable, versioned check for one vulnerability class, applied by the Detector to each function in scope. The rule corpus is an artifact independent of the agent code. |
| **Rule-gap** | A record that an exploratory finding was confirmed `true-positive` and no detection rule would have produced it; the input to growing the rule corpus. See FR-042. |
| **Coverage** | The degree to which the evaluation goals have been credibly attempted. |
| **Yield** | Severity-weighted confirmed findings per unit of spend, measured over a trailing window. |
| **Yield** | Severity-weighted confirmed findings per unit of settled spend, measured over a deterministic trailing spend window. |
| **Work queue** | The shared, ordered list of tasks agents claim from. See [`spec.md` §8](spec.md#8-coordination-substrate). |
| **Finding store** | The durable, fingerprint-indexed record of every finding at every lifecycle stage; internal, queryable by every role. Distinct from the issue tracker. |
| **Coverage log** | The append-only record of which (area × technique) pairs the fleet has attempted; an audit trail, not a stop-list. See FR-046. |
| **Budget governor** | The substrate component that tracks spend, runtime, and trailing yield against operator caps and signals the Orchestrator to halt. See [`spec.md` §9.3–§9.4](spec.md#93-budget). |
| **Budget governor** | The substrate component that atomically reserves and settles model-call cost, tracks active runtime and trailing yield against operator caps, and signals the Orchestrator to halt. See [`spec.md` §9.3–§9.4](spec.md#93-budget). |
| **Help request** | An operator-filed issue asking the fleet to perform a specific action; resolved by the Orchestrator's conversational facet. See FR-015. |
| **Operator message** | An agent-authored, asynchronous, one-way note to the operator (blocker, request, feedback, or informational), deduplicated across the fleet. The agent→operator counterpart of a help request. See FR-102a. |
| **Proof-of-concept (PoC)** | A self-contained, runnable artifact that demonstrates a finding's headline impact against the testbed. See FR-063. |
Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**An open specification for agentic AI security evaluation, from Cisco.**

[![Status](https://img.shields.io/badge/Status-Seed_v0.1.0-orange.svg)](CHANGELOG.md)
[![Status](https://img.shields.io/badge/Status-Seed_v0.2.0-orange.svg)](CHANGELOG.md)

Cisco's Advanced Security Initiatives Group has built and operated an agentic security evaluation internally across several iterations and deployment models, and along the way accumulated a long list of design decisions that turned out to matter and a longer list that turned out not to.

Expand Down Expand Up @@ -206,4 +206,3 @@ Original authors of the spec and constitution:
- Theo Morales (@kh0rvus)
- John Allbritten (@jallbrit)


12 changes: 6 additions & 6 deletions constitution.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
<!--
SYNC IMPACT REPORT — maintained by /speckit.constitution
═══════════════════════════════════════════════════════
Version change : 0.1.0 → 0.2.0 [MINOR: III scope narrowed]
Principles : III narrowed (work-reclamation only; session rotation per FR-118 carved out)
Sections changed : III
Version change : constitution 0.2.0 unchanged; spec 0.1.0 → 0.2.0 [MINOR: budget semantics strengthened]
Principles : V, VI, XI unchanged and re-validated against FR-112a, FR-113, FR-115, FR-116a
Sections changed : spec §1.6–§1.7, §3.2–§3.3, §4.4–§4.5, §5.1, §9.3–§9.4, §11.2, §12
Templates needing update : n/a
Downstream re-check : spec.md FR-005, FR-118 README.md ✓ plan.md n/a tasks.md n/a
Downstream re-check : spec.md ✓ README.md principle-fr-matrix.md ✓ plan.md n/a tasks.md n/a
Follow-up TODOs : none
Last sync : 2026-05-04
Last sync : 2026-09-10
═══════════════════════════════════════════════════════
This block is regenerated on every constitution change; do not hand-edit below the rule.
This block is regenerated on every constitution change and on spec changes that require downstream re-validation; do not hand-edit below the rule.
-->

# Foundry Constitution
Expand Down
3 changes: 2 additions & 1 deletion docs/adoption/clarification-playbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,9 @@ The seed describes the *contract* each integration must satisfy. Your answer nam

### §11.2 LLM provider

- Good: "Provider X via internal gateway Y; rate limits are exposed via response headers."
- Good: "Provider X via internal gateway Y; rate limits are exposed via response headers; requests enforce a maximum output allowance; responses report input/output/cached/reasoning tokens; versioned rate cards are stored with accounting events."
- Anti: "Multi-provider, we'll abstract over them." (Principle V — the provider is the rate arbiter — gets harder, not easier, behind an abstraction.)
- Anti: "Use the provider's current pricing page for all estimates." (Historical settled cost must not change when rates change.)

### §11.3 Datastore

Expand Down
6 changes: 5 additions & 1 deletion docs/adoption/integration-decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ Required capabilities the spec assumes:

- The provider's responses include rate-limit headers or equivalent backoff signals.
- Calls support timeouts at the call layer (separately from agent liveness).
- Token usage / cost is observable for budget enforcement (FR-112, FR-113).
- Every request supports a maximum output-token allowance so its conservative maximum cost can be reserved before dispatch (FR-112a).
- Token usage is observable by billable category; direct cost is preferable when the provider reports it (FR-113).
- Provider/model pricing can be captured as an immutable rate-card snapshot rather than a mutable “current price”.

## §11.3 Datastore

Expand All @@ -54,6 +56,8 @@ Constraint: Principle XI — persist atomically. The finding store must never be

**Anti-pattern:** "delete old, write new" persistence flows. Always write new, then atomically rename or commit (see [`spec.md` §8.6](../../spec.md#86-atomic-persistence)).

The same transaction boundary must protect budget admission: checking remaining spend and writing the reservation are one operation. A datastore that can atomically claim work but cannot atomically reserve budget does not satisfy FR-112a for a concurrent fleet.

## §11.4 Vector search

Constraint: optional per FR-023. If absent, the Indexer drops similarity search and the Variant-Hunter extension cannot be enabled (see [`extension-roles-when.md`](extension-roles-when.md)).
Expand Down
29 changes: 29 additions & 0 deletions docs/architecture/substrate-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,35 @@ provider_call(args) -> result

---

## Budget governance contract (§9.3–§9.4)

```text
budget.admit(call_id, maximum_cost, rate_card_id) -> reservation | DENIED
budget.settle(call_id, billable_cost, reported_or_estimated) -> accounting_event
budget.snapshot() -> {settled, reserved, active_runtime, admission_state, stop_reason}
budget.evaluate_yield(window_size) -> {full, boundary, spend, points, yield}
budget.stop(reason) -> persisted_stop # idempotent
```

### Required guarantees

| Guarantee | FR(s) | Principle |
|---|---|---|
| Concurrent admission cannot reserve more than the remaining operator-authored spend cap. | FR-112a | XI |
| Settlement and stop transitions are idempotent. | FR-113, FR-116a | XI |
| Rate-card identity is immutable for each reservation and settlement. | FR-112a, FR-113 | — |
| Budget admission does not impose a provider-rate or concurrency ceiling while authorized spend remains. | FR-112a, FR-105 | V |
| Every consumer derives the trailing window from the same ordered settled-event stream. | FR-115, FR-124 | VI, XI |

### Anti-patterns

- Read remaining budget, release the lock, then write a reservation — concurrent agents can all spend the same remainder.
- Dispatch before reservation commits — a denied call has already incurred cost.
- Recompute old settlements from the current provider price sheet — historical totals change under the operator.
- Use the budget path to cap calls per second — that is rate governance and contradicts Principle V.

---

## Putting it together

A reasonable substrate implementation is:
Expand Down
18 changes: 16 additions & 2 deletions docs/governance/sync-impact-reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,21 @@ If any check fails, the PR is not ready.

These are preserved here as a navigable history. Git is authoritative; this archive is for convenience.

### 0.1.0 → 0.2.0 (current as of writing — see `constitution.md`)
### Spec 0.1.0 → 0.2.0 (current as of 2026-09-10 — constitution unchanged)

```
Version change : constitution 0.2.0 unchanged; spec 0.1.0 → 0.2.0 [MINOR: budget semantics strengthened]
Principles : V, VI, XI unchanged and re-validated against FR-112a, FR-113, FR-115, FR-116a
Sections changed : spec §1.6–§1.7, §3.2–§3.3, §4.4–§4.5, §5.1, §9.3–§9.4, §11.2, §12
Templates needing update : n/a
Downstream re-check : spec.md ✓ README.md ✓ principle-fr-matrix.md ✓ plan.md n/a tasks.md n/a
Follow-up TODOs : none
Last sync : 2026-09-10
```

**Why:** FR-011, FR-112–FR-116, and FR-128 previously allowed concurrent calls to overshoot a hard spend cap and left independent implementations free to compute different accounting totals, restart behavior, and trailing windows from the same events. The amendment adds atomic reservation, idempotent settlement and stopping, active-runtime semantics, immutable rate-card attribution, deterministic spend-window boundaries, and explicit hot-reload/reset behavior while preserving Principles V, VI, and XI.

### Constitution 0.1.0 → 0.2.0 (archived 2026-05-04)

```
Version change : 0.1.0 → 0.2.0 [MINOR: III scope narrowed]
Expand All @@ -118,7 +132,7 @@ Last sync : 2026-05-04

### Future entries

When the constitution next changes, a new entry appears here, *above* the current entry, dated and tier-tagged. The constitution itself shows only the most recent block; this archive carries the trail.
When the report is next regenerated for a constitution change or a spec change that forces downstream re-validation, a new entry appears here, *above* the current entry, dated and tier-tagged. The constitution itself shows only the most recent block; this archive carries the trail.

## See also

Expand Down
Loading