feat(server): agent runtime runner and Anthropic Messages adapter with typed failures and configurable timeout (part 1 of #20)

[XxHugheadxX]

Refs #20

This is the first part of #20: the provider-neutral runner and the Anthropic adapter. Wiring the runner to hires, the demo manifests and the testnet end-to-end run come in follow-up PRs. Their prerequisites are in this comment on #20.

Summary

• AgentRunner (lib/src/runtime/agent_runner.dart) runs one prompt-only task (ADR-0004) on any ModelRuntime. It applies the rules that hold for every provider:
  • the manifest's input.max_chars: input over the limit fails before the model is called;
  • the manifest's output.max_chars;
  • non-blank output;
  • the runtime timeout, which must end before the job's expired_at (ADR-0005 D3/D4).
• RuntimeFailure (lib/src/runtime/runtime_task.dart) is a sealed set of typed failures. Each has a code that is safe to store as the hire's failureReason and to show to the client: timeout, provider_error:<type>, refused, output_truncated, input_too_long, output_too_long, empty_output, unsupported_provider. Codes never include provider messages, prompts, inputs or keys. RuntimeProviderFailed.retryable is true only when no response arrived, or the status is 429 or 5xx.
• AnthropicRuntime (lib/src/runtime/adapters/anthropic_runtime.dart) makes one non-streaming POST /v1/messages over HTTP with the existing http package. There is no official Dart SDK, so it adds no new dependency.
  • Request: model, system and the user input come from the task, so the manifest drives behavior. It sets max_tokens: 16000 and no sampling or thinking parameters.
  • Fallbacks: on claude-opus-5-5, claude-sonnet-5-5, claude-opus-5 and claude-fable-5-1, it asks for server-side fallbacks on a refusal (fallbacks: "default" plus the server-side-fallback-2026-07-01 beta header).
  • Response: text blocks are joined and thinking blocks are ignored. A refusal stop reason becomes RuntimeRefused with its category, and max_tokens becomes RuntimeOutputTruncated.
  • Key: the API key goes only in x-api-key. It never appears in a failure or in toString().
• RuntimeConfig (lib/src/runtime/runtime_config.dart) reads two variables:
  • PULS3_ANTHROPIC_API_KEY: an unset key disables the runtime.
  • PULS3_RUNTIME_TIMEOUT_SECONDS: defaults to 120 s. A value that is not a positive whole number fails fast.

Acceptance criteria (#20)

• End to end on testnet, a paid hire of each demo agent reaches Delivered (now submitted, ADR-0005 D6) with a real result. Follow-up PR.
• Changing a demo agent's manifest (for example its prompt) changes its behavior without code changes. The test a different manifest prompt changes the request, with no code change covers it at the adapter level; the manifest-to-task mapping lands with feat: AgentManifest domain model and validation #34.
• A provider error and a timeout are each tested as typed failures. Moving the hire to failed (now runtimeStatus = failed via Hire.failRun, feat: align Hire lifecycle with ERC-8183 #73) comes with the wiring PR.
• The timeout value is configurable (PULS3_RUNTIME_TIMEOUT_SECONDS).
• The provider's API key comes from an env var and is never logged (both done and tested). Still to do: .env.example, which waits for the env inventory in chore(infra): testnet configuration and secrets management #106 to avoid a conflict.
• The backend depends only on the runtime interface, and only the adapter knows the provider API (see the grep below).

Verification evidence

$ cd puls3_server
$ dart format --output=none --set-exit-if-changed lib test
Formatted 74 files (0 changed)
$ dart analyze --fatal-infos
No issues found!
$ dart test test/unit/runtime
00:00 +32: All tests passed!
$ dart test test/unit
00:01 +288: All tests passed!

$ grep -rnE "api\.anthropic\.com|x-api-key|anthropic-version" puls3_server/lib puls3_server/bin | grep -v "lib/src/runtime/adapters/"
(empty)

I broke each guard one at a time to check that the tests catch it. All 6 changes made the suite fail:

Removed or changed guard	Failing tests
input limit check	1
timeout	1
output limit check	1
refusal detection	1
hard-coded system prompt	2
provider check	1

Notes for reviewers

• Nothing calls the runner yet. It is not wired into the server, so this PR changes no runtime behavior.
• The timeout lives in AgentRunner, not in the adapter, so every provider gets the same rule. The 120 s default is a placeholder until ADR-0005 D3 sets the real values.
• Output over output.max_chars fails the run instead of being truncated, so a client never receives a cut-off result as if it were complete.
• No retries yet. retryable is exposed so the hire wiring can decide whether to retry within the timeout.

[cloudflare-workers-and-pages]

Deploying puls3 with    Cloudflare Pages

Latest commit:	c39033c
Status:	✅  Deploy successful!
Preview URL:	https://398c3bb4.puls3-4lw.pages.dev
Branch Preview URL:	https://feat-20-agent-runtime.puls3-4lw.pages.dev

View logs

[TOMOKI977]

Great foundation, thanks. The Claude API usage checks out against the docs: model IDs, anthropic-version: 2023-06-01, fallbacks: "default" with server-side-fallback-2026-07-01 only on the four models that support it, thinking blocks ignored, refusal and max_tokens handled. The key never leaks, consumer input stays in the user turn, and the tests pin the exact wire request. Requesting changes for a small set before this lands:

1. The timeout doesn't cancel the call (agent_runner.dart:29). Future.timeout only stops waiting: the http.post keeps running and is billed (up to 16000 output tokens), and a retry starts a second paid call while the first is still in flight. package:http 1.6 (already our version) supports aborting requests: send an AbortableRequest with an abortTrigger (see io_client.dart), and have the runner pass the deadline down to the runtime so it can abort.
2. The API key name breaks the PULS3_* convention. In this repo PULS3_* holds public values only (StellarConfig doc, puls3_server/README.md), and #106's docs/infra/secrets.md plans the LLM key as a Serverpod password (SERVERPOD_PASSWORD_<name>, read through passwords.yaml / Serverpod Cloud secrets). Reading it that way also keeps it out of the shared root .env that the Flutter app loads with --dart-define-from-file.
3. max_tokens is fixed at 16000 (anthropic_runtime.dart:29) regardless of the manifest's output.max_chars. A small output cap still pays for up to 16000 tokens and then fails as output_too_long. Deriving it from maxOutputChars (with a ceiling) caps cost per hire.

Fine as follow-ups for the wiring PR:

• No model allow-list yet: any manifest modelId reaches the paid API. I know it's planned as manifest validation in #34.
• The 120 s default is short for non-streaming long turns, and nothing ties the timeout to the job's expired_at (the runner takes a fixed duration).
• String.length counts UTF-16 code units, so emoji count twice against max_chars. Untested for non-ASCII output.
• pause_turn falls into unexpected_stop_reason. 408/409 aren't retryable, and retry-after on 429 is dropped.
• No request-id or usage captured, so failures can't be traced and spend per hire isn't measurable.
• ModelRuntime sits next to the domain's AgentRuntimePort (ADR-0001, "implemented by the agent runtime (#20)"). A line on how AgentRunner relates to it would help.
• RuntimeConfig throws FormatException while TrackerLoopConfig throws ArgumentError for the same kind of bad input.

[TOMOKI977]

@XxHugheadxX Heads-up: CI changes when #114 (Serverpod 4.0.4) merges. Once it lands on main:

1. Upgrade locally to Flutter 3.44.4 (Dart 3.12.2) and serverpod_cli 4.0.4. The steps are in docs/operations/serverpod-4-migration.md.
2. Update your branch from main and run dart pub get again. The lockfile moves to Serverpod 4.0.4, and checks run with the new toolchain.
3. Generated code and migrations: if you touch a *.spy.yaml or an endpoint, run serverpod generate with 4.0.4 and commit the output, since CI fails on any generated diff. Create new migrations with 4.0.4 too, so they sort after upgrade-4-0.
4. Two new required CI jobs:
   • docker builds the server image.
   • scripts runs the offline tests in scripts/tests/, including the env inventory: every PULS3_* variable read in code must be listed in .env.example. Secrets stay commented out there, with an entry in docs/infra/secrets.md.
5. Cloudflare Pages builds with scripts/cloudflare-pages-build.sh, which pins the same Flutter version as CI.

Specific to this PR: it reads PULS3_RUNTIME_TIMEOUT_SECONDS and PULS3_ANTHROPIC_API_KEY, so the new scripts job will fail until both are in .env.example. The timeout can have a value there. The API key is a secret: list it commented out and add it to docs/infra/secrets.md, or, better, read it as a Serverpod password (session.passwords), as suggested in the last review. Serverpod Cloud then manages it like the other secrets.

[XxHugheadxX]

@TOMOKI977 thanks for the review. I agree with the three blocking points, and #2 is a real flaw in this PR. Here is what changes. Nothing is pushed yet.

1. The timeout does not cancel the call. Confirmed: Future.timeout only stops waiting, so the request keeps running and is billed. AbortableRequest with abortTrigger is in package:http 1.6.0 (lib/src/abortable.dart).

• The runner will pass a deadline to ModelRuntime.complete.
• The Anthropic adapter will send an AbortableRequest that aborts when the deadline passes.
• A test will check that the request is aborted on timeout, not just abandoned.

2. The API key in PULS3_ANTHROPIC_API_KEY. This one is the flaw. PULS3_* holds public values, and the root .env goes into the web build through --dart-define-from-file. A key there could end up in the published app (docs/infra/secrets.md).

• RuntimeConfig will take the key from the Serverpod password anthropicApiKey: passwords.yaml locally, and scloud password set anthropicApiKey --from-file <file> on Cloud.
• It will read only PULS3_RUNTIME_TIMEOUT_SECONDS from the environment. A test will check that PULS3_ANTHROPIC_API_KEY and ANTHROPIC_API_KEY are ignored.
• .env.example will list PULS3_RUNTIME_TIMEOUT_SECONDS.
• passwords.example.yaml will get a commented anthropicApiKey entry, outside the <generate> placeholders, so setup-local-secrets.sh does not fill it with a random value.
• The LLM key row in secrets.md will say where the key is stored.
• With this, the env inventory in the new scripts job passes.

3. max_tokens is fixed at 16000. It will be derived from maxOutputChars with a ceiling. That caps the cost per hire, and a small output.max_chars will no longer pay for 16000 tokens.

Branch up to date with main. I upgraded locally to Flutter 3.44.4, Dart 3.12.2 and serverpod_cli 4.0.4, and merged main (Serverpod 4.0.4) into the branch. Everything is clean:

• domain: 127 tests pass;
• server unit tests: 328 pass, and serverpod generate leaves no diff;
• app: 104 tests pass.

The push goes out with fixes 1–3.

Follow-ups for the wiring PR (I'll track them in #20):

• model allow-list, with feat: AgentManifest domain model and validation #34;
• timeout bound to expired_at instead of a fixed duration;
• max_chars counted in characters, not UTF-16 code units, with a non-ASCII test;
• pause_turn handling;
• 408/409 retryable, and retry-after honored on 429;
• capturing request-id and usage per hire;
• the relation between ModelRuntime and the domain's AgentRuntimePort;
• ArgumentError instead of FormatException in RuntimeConfig, to match TrackerLoopConfig.