Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ jobs:
diff -q /tmp/requirements.json docs/conformance/requirements.json \
|| (echo "Inventory is stale: re-run scripts/extract_requirements.py" && exit 1)
- name: Coverage map is valid and has not regressed
run: python3 scripts/conformance_coverage.py --baseline 51 --markdown docs/conformance/coverage-report.md
run: python3 scripts/conformance_coverage.py --baseline 57 --markdown docs/conformance/coverage-report.md

doc-examples:
name: Doc Examples Compile
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/SemVer

## [Unreleased]

### Added
- **`facet-fct agent`: the host execution loop.** `run` compiles one request and
stops; `agent` drives the turn cycle — model answers, guard decides, tools
execute, results enter the next turn. The model sits behind a `ModelClient`
trait, so this is a runtime rather than a vendor wrapper; `ScriptedClient`
replays recorded turns. This makes §16 guard behaviour and Appendix F
provenance observable for the first time outside mocks: a real `tool_call`
decision, `F454` from a real denial, and one hash chain with contiguous `seq`
across every turn. A denied run still writes its artifact (§18.1.3).
- `docs/19-agent-loop.md`, and `tests/agent_loop_tests.rs` covering the allowed
path, the denied path, multi-turn chaining, replay reproducibility, and a call
to a tool the contract never declared.
- Specification coverage rises to 57 of 206 as the loop unblocks §16.6 and
Appendix F requirements that could not previously be exercised.

### Changed
- `run` and `agent` share one compile path (`compile_contract`), so a turn built
by the loop is byte-identical to the same contract compiled by `run`.

## [0.1.4] - 2026-08-22

### Added
Expand Down
26 changes: 17 additions & 9 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,8 @@ that is a smoke test, not a conformance suite.
language chapters are now largely attributed: §12 9/17, §13 6/9,
§14 9/13, §15 3/3
- [ ] Fill gaps by risk order: §5 syntax → §8 FTS → §11 layout → §16 policy.
§16 sits at 12/65 and most of the remainder is guard *behaviour*, which
cannot be observed until item 2 closes the loop; §17 (5), §2 profiles (10)
and Appendix F (10) are likewise blocked on execution
§16 sits at 15/65; the loop from item 2 unblocked guard behaviour, so the
rest is now writable rather than blocked
- [x] Publish the coverage report next to the compliance report
(`docs/conformance/coverage-report.md`, regenerated by CI)

Expand All @@ -43,14 +42,23 @@ recorded as untestable-here with a reason.
effect classes, fail-closed denial — and Appendix F's hash chain are exercised
only in simulation. Deterministic agent execution is specified, not executed.

- [ ] Host runtime: canonical JSON → provider → parse tool calls → guard →
`ToolExecutor` → next turn
- [ ] Multi-turn Execution Artifact: one hash chain across the whole run
- [ ] Replay: re-run a recorded artifact and assert an identical chain head
- [ ] `F454` / `F455` raised from a real denied call, not a mock
- [x] Host runtime: canonical JSON → model → parse tool calls → guard →
`ToolExecutor` → next turn (`facet-fct agent`, `src/commands/agent.rs`).
The model sits behind a `ModelClient` trait so this stays a runtime, not a
vendor wrapper; `ScriptedClient` replays recorded turns
- [x] Multi-turn Execution Artifact: one hash chain across the whole run, with
contiguous `seq` over every turn
- [x] Replay: the same script reproduces the chain head byte for byte
- [x] `F454` raised from a real denied call, and the denied decision is still
written to the artifact — a refused run is exactly when provenance matters
- [ ] `F455` from a genuinely undecidable guard state (needs a condition that
fails at evaluation time, not a policy that says no)
- [ ] Feed tool results back through the contract rather than as appended
messages, so a turn is a compiled artifact rather than a patched payload

**Done when** an agent run that calls a tool produces a verifiable provenance
chain, and replaying it reproduces the chain head byte for byte.
chain, and replaying it reproduces the chain head byte for byte. *(Reached for
the scripted client; a real provider arrives with item 3.)*

## 3. Ship one real adapter

Expand Down
95 changes: 95 additions & 0 deletions docs/19-agent-loop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
permalink: /19-agent-loop.html
title: Agent Loop
---

# 19. The agent loop

`run` compiles one request and stops. An agent is the loop around it: the model
answers, asks for tools, the guard decides, tools execute, and their results
enter the next turn.

FACET does not standardize that loop — the host owns it. What FACET standardizes
is what must hold *while* it runs: the Runtime Guard (§16.6), effect classes
(§16.5), and the provenance record (Appendix F). None of it is observable in a
single compile-and-stop invocation, which is why the loop lives here.

## The model is behind a trait

```rust
pub trait ModelClient {
fn complete(&mut self, turn: usize, request: &CanonicalPayload) -> Result<ModelTurn>;
}
```

This keeps the compiler a runtime rather than a vendor wrapper. `ScriptedClient`
replays recorded turns, so guard behaviour is testable without a network; a
provider adapter is the same trait with a socket behind it.

## Running a scripted conversation

```bash
facet-fct agent --input contract.facet --script turns.json --artifact execution.json --exec
```

`turns.json` records what the model returns and what the tools produce:

```json
{
"turns": [
{ "tool_calls": [ { "id": "c1", "name": "WeatherAPI.get_current", "arguments": { "city": "Minsk" } } ] },
{ "text": "It is raining in Minsk." }
],
"tool_results": { "WeatherAPI.get_current": "Rain, 11C" }
}
```

Each turn:

1. the client returns an answer for the current canonical payload
2. no tool calls → the run completes with that text
3. otherwise, for each call: the declared `effect` is looked up from
`@interface`, the guard decides, and only then does the tool execute
4. the result enters the next turn's context

Tools are registered from the contract's own `@interface` blocks, so a call to
something the contract never declared has nowhere to land.

## What lands in the artifact

One hash chain covers the whole run, with `seq` contiguous across turns
(Appendix F.3–F.4):

```
seq=1 message_emit system#1 allowed
seq=2 message_emit user#1 allowed
seq=3 tool_expose WeatherAPI.get_current allowed rule=expose-weather
seq=4 tool_call WeatherAPI.get_current allowed rule=call-weather
```

Remove the `tool_call` rule from `@policy` and the same script produces:

```
seq=4 tool_call WeatherAPI.get_current denied
```

The run stops with `F454` and the tool never executes — **and the artifact is
still written**. A refused run is exactly the case where the provenance record
matters, so the denial is recorded before the error is reported (§18.1.3).

## Replay

The same contract and the same script reproduce the same chain head byte for
byte. That is the property that makes an artifact worth signing: an attestation
over `hash_chain.head` is only meaningful if the head is reproducible.

## Current limits

- `F455` (undecidable guard state) is not yet reachable from the loop; it needs
a condition that fails at evaluation time rather than a policy that says no.
- Tool results are appended to the rendered payload as messages. A turn is
therefore a patched payload rather than a recompiled contract — acceptable for
provenance, but the next step is to feed results back through the document so
every turn is a compiled artifact in its own right.
- The only client is `ScriptedClient`. A real provider adapter is the next
roadmap item.
6 changes: 3 additions & 3 deletions docs/conformance/coverage-report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Specification coverage

51 of 206 normative statements are exercised by a named test (24%); 0 are recorded as not testable here.
57 of 206 normative statements are exercised by a named test (27%); 0 are recorded as not testable here.

| Chapter | Requirements | Covered | Waived |
|---------|-------------:|--------:|-------:|
Expand All @@ -18,12 +18,12 @@
| §13 | 9 | 6 | 0 |
| §14 | 13 | 9 | 0 |
| §15 | 3 | 3 | 0 |
| §16 | 65 | 12 | 0 |
| §16 | 65 | 15 | 0 |
| §17 | 5 | 0 | 0 |
| §18 | 11 | 1 | 0 |
| §19 | 1 | 0 | 0 |
| §20 | 2 | 0 | 0 |
| §Appendix A | 1 | 0 | 0 |
| §Appendix C | 2 | 0 | 0 |
| §Appendix D | 1 | 0 | 0 |
| §Appendix F | 10 | 0 | 0 |
| §Appendix F | 10 | 3 | 0 |
39 changes: 39 additions & 0 deletions docs/conformance/coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -346,5 +346,44 @@
"policy_validation_tests::policy_accepts_valid_rule_and_condition"
],
"note": "§16.3 PolicyCond forms"
},
"9494d30f50b323dd": {
"tests": [
"agent_loop_tests::allowed_tool_call_is_executed_and_recorded",
"agent_loop_tests::denied_tool_call_stops_the_run_and_is_still_recorded"
],
"note": "§16.6.1 the guard runs on every tool_call, allowed and denied alike"
},
"d937a7d07bc662de": {
"tests": [
"agent_loop_tests::denied_tool_call_stops_the_run_and_is_still_recorded",
"agent_loop_tests::an_undeclared_tool_cannot_be_called"
],
"note": "§16.6.1a the decision precedes execution: a denied call never runs"
},
"997c6920fd1fc224": {
"tests": [
"agent_loop_tests::denied_tool_call_stops_the_run_and_is_still_recorded"
],
"note": "§16.6.6 a deterministic deny is F454"
},
"2bea6c78116dc1ac": {
"tests": [
"agent_loop_tests::allowed_tool_call_is_executed_and_recorded"
],
"note": "Appendix F.3 GuardDecision shape: op, name, effect_class, decision, input_hash"
},
"632a37ff15ef05d7": {
"tests": [
"agent_loop_tests::every_turn_shares_one_hash_chain",
"agent_loop_tests::replaying_the_same_script_reproduces_the_chain_head"
],
"note": "Appendix F.2 artifact shape; contiguous seq and a chain over the whole run"
},
"3a81c34d065c7024": {
"tests": [
"agent_loop_tests::replaying_the_same_script_reproduces_the_chain_head"
],
"note": "Appendix F.1 identical inputs reproduce the artifact"
}
}
Loading
Loading