From 2f061edca902efc38dd509ee32d0f1cb5524dffa Mon Sep 17 00:00:00 2001 From: Yesh Yendamuri <1115318+Yeshwanthyk@users.noreply.github.com> Date: Tue, 4 Aug 2026 16:38:35 +0000 Subject: [PATCH] docs: prioritize low-complexity agent primitives --- LOG.md | 4 + docs/plans/automation-scheduler.md | 18 +- docs/plans/cli-polish.md | 49 ++- docs/plans/openclaw-hermes-primitives.md | 262 +++++++++++++++ docs/plans/primitive-status.md | 35 +- docs/plans/profile-lease.md | 60 ---- docs/plans/resident-lease.md | 53 +++ docs/research/eve-flue-ziggy-comparison.md | 318 ++++++++++++++++++ .../openclaw-hermes-current-primitives.md | 210 ++++++++++++ docs/research/pi-sdk-surface.md | 27 ++ 10 files changed, 924 insertions(+), 112 deletions(-) create mode 100644 docs/plans/openclaw-hermes-primitives.md delete mode 100644 docs/plans/profile-lease.md create mode 100644 docs/plans/resident-lease.md create mode 100644 docs/research/eve-flue-ziggy-comparison.md create mode 100644 docs/research/openclaw-hermes-current-primitives.md diff --git a/LOG.md b/LOG.md index 18bde84..b93c4d2 100644 --- a/LOG.md +++ b/LOG.md @@ -92,3 +92,7 @@ Chronological; newest at the bottom. One entry per logical block of work. **Truthful configured automation delivery.** A manual wake still runs the model once and prints its local reply first. When `telegram-chat` is declared, missing or invalid `telegram.json` now fails through `AutomationDeliveryUnavailable` instead of logging a skipped delivery and returning success; Telegram API failures remain typed. Automations without delivery are unchanged. Focused filesystem-and-fake-agent tests cover ordering, one prompt, both configuration failures, no-delivery success, and API error preservation; no retry, receipt, outbox, ledger, scheduler, lease, or registry was added. **Portable package cwd proofs.** Agent Browser's wrapper test now compares the spawned process cwd with the Profile's physical directory while continuing to assert that Ziggy passes the original Profile spelling into package runtime paths. Telephony's default-path test applies the same rule to Python's `Path.cwd()`. Both cover macOS `/var` → `/private/var` normalization without changing Profile identity or production behavior. + +**eve / Flue comparison.** Cloned `vercel/eve@6c5f4fe2` and `withastro/flue@a171cc1b` into a disposable `/tmp` workspace and compared both live implementations with Ziggy at `98988c29`. `docs/research/eve-flue-ziggy-comparison.md` records the shared surfaces, ownership differences, execution paths, and reuse boundary. Flue shares Pi lineage with Ziggy but owns a separate durable conversation protocol over `pi-agent-core`; eve uses AI SDK `ToolLoopAgent` inside Workflow durability. Agent Skills are the direct portable bridge; hosted agents can be delegated to over public HTTP clients, while sessions, extensions, channels, memory, and durability engines should remain separate authorities. + +**OpenClaw / Hermes low-complexity plan.** Refreshed primary-source evidence at `openclaw/openclaw@50a30b73` and `NousResearch/hermes-agent@42708f8b`, corrected the pinned Pi `0.82.0` session surface to use `SessionManager.listAll(customDirectory)` for unfiltered metadata, and reduced the borrow queue to read-only session inventory → narrow doctor → optional same-face resident lease. The plan rejects a Profile-wide lease while channel commands are independent and defers graceful drain, scheduler claims, custom skill metadata, and durable dedupe until their product triggers exist. No runtime code or new authority was added. diff --git a/docs/plans/automation-scheduler.md b/docs/plans/automation-scheduler.md index 1c2fb06..bdd616d 100644 --- a/docs/plans/automation-scheduler.md +++ b/docs/plans/automation-scheduler.md @@ -12,19 +12,21 @@ Pi session only when admitted, prints the answer, and optionally sends it to Tel ## When to build the scheduler -Build this after the Profile lease. The lease gives one resident process authority to claim -scheduled work. +Keep this deferred until automatic triggers are a concrete product requirement. Manual `wake` has +no competing dispatcher, and configured Telegram delivery already fails truthfully after printing +the local result. -Before this slice, fix the current manual-wake false success: when an automation declares -`telegram-chat`, missing or invalid Telegram configuration must return a typed delivery failure -after the local reply is printed. That change needs no retry or delivery state. +Do not couple scheduling to the optional face-scoped channel lease. Choose a scheduler-specific +resident owner when this slice is accepted; that keeps Telegram, Discord, and Slack independently +runnable and avoids a Profile-wide gateway authority. ## Slice 1. Add one optional `cron:` frontmatter field and decode it with Effect `Cron`. 2. Add trigger provenance: `manual` or `schedule`, with scheduled trigger IDs equal to the firing instant in canonical ISO form. -3. Start one scheduler fiber inside whichever channel gateway owns the Profile lease. +3. Run one scheduler-specific resident for the Profile; do not hide scheduling inside an arbitrary + channel gateway. 4. Store the last claimed firing instant per automation in `/.runtime/automation-schedule.json`. 5. Atomically persist the claim before `wake`; restart does not replay a claimed slot. @@ -34,7 +36,7 @@ A scheduled automation without a gate is declined. Manual wake keeps its current ## Invariants -- Exactly one lease-owning resident process schedules a Profile. +- Exactly one scheduler-specific resident process schedules a Profile. - A due slot is claimed before model or delivery work. - Restart never replays a claimed slot. - A declined gate creates no Pi session. @@ -49,7 +51,7 @@ Use a fake clock, fake agent, and temporary Profile: 2. Claim one due slot, run it once, restart the scheduler, and prove no replay. 3. Decline a scheduled automation without a gate before Pi construction. 4. Prove the same automation cannot overlap while a different ID can run. -5. Run the scheduler under each lease-owning gateway seam. +5. Prove a second scheduler owner is refused while channel residents remain independent. Then run: diff --git a/docs/plans/cli-polish.md b/docs/plans/cli-polish.md index df9d2e7..5141472 100644 --- a/docs/plans/cli-polish.md +++ b/docs/plans/cli-polish.md @@ -8,55 +8,48 @@ credentials in the Profile. ## Next 1: sessions -Add: +Implement Chunk 1 from [`openclaw-hermes-primitives.md`](./openclaw-hermes-primitives.md): ```text -ziggy sessions +ziggy sessions [--json] ``` -The command is read-only. It lists root TUI/run sessions plus Telegram, Discord, Slack, and -automation session leaves. Each output row contains: - -```text - -``` - -Keep Pi imports in `src/adapters/pi/sessions.ts`. Use `SessionManager.list` once per known leaf -directory, open each session only for metadata, sort by relative path, and never print transcript -content. +The command recursively discovers session leaf directories without following directory symlinks, +refuses symlinked `.jsonl` files, and invokes pinned Pi `SessionManager.listAll(customDirectory)` +once per leaf. It projects only session ID, relative path, created/modified timestamps, and message +count; Pi's transcript-derived preview fields never leave the adapter. Acceptance: - Missing session directories print `no sessions`. -- Every valid JSONL appears once. -- Output reveals no prompts or replies. -- Malformed or unreadable sessions return a typed failure. +- Every Pi-readable regular JSONL appears once, newest-first with a stable path tie-break, regardless of header-cwd spelling. +- Text and JSON reveal no prompts, replies, or transcript previews. +- Regular files for which Pi cannot build metadata produce a typed failure; tolerated lines remain Pi policy. +- Directory symlinks are ignored and `.jsonl` file symlinks fail before Pi reads the leaf. ## Next 2: doctor -Add: +Implement Chunk 2 from [`openclaw-hermes-primitives.md`](./openclaw-hermes-primitives.md): ```text -ziggy doctor +ziggy doctor [--json] ``` -The command is read-only and reports `ok`, `warn`, or `error` for: - -- `SOUL.md` and Profile readability; -- provider auth availability; -- Telegram, Discord, and Slack config decoding; -- memory files and caps; -- automation parsing; -- installed skill shape. +The command is read-only and reports stable check codes with `ok`, `warn`, or `error` for Profile +initialization/readability, optional channel config decoding, session metadata, and automation +parsing. -Reuse existing decoders and auth status. Do not duplicate validation or repair files. +Reuse existing local config/automation decoders and the session projection. Do not call +`Auth.status` (Pi may create or refresh credential files), call a provider, poll Telegram, open +Discord/Slack, load executable extensions, test skill binaries, migrate config, or repair files. +Skill requirement/shape diagnosis remains out because Ziggy does not own a skill parser. Acceptance: -- Exit 0 when there are no errors. +- Exit 0 when there are no errors; absent optional channels are not errors. - Exit nonzero when any check is an error. - Never print secrets or transcript content. -- Stable output order makes failures easy to diff. +- Text and JSON expose the same stable codes, severities, and order. ## Later diff --git a/docs/plans/openclaw-hermes-primitives.md b/docs/plans/openclaw-hermes-primitives.md new file mode 100644 index 0000000..ddfa668 --- /dev/null +++ b/docs/plans/openclaw-hermes-primitives.md @@ -0,0 +1,262 @@ +# Borrowed primitives without a second system + +Status: proposed implementation packet, 2026-08-04. + +## Orientation + +Ziggy should borrow operator and ownership **invariants** from OpenClaw and Hermes-Agent, not their gateways, schedulers, stores, or control planes. The current-source audit is in +[`openclaw-hermes-current-primitives.md`](../research/openclaw-hermes-current-primitives.md). + +The smallest useful sequence is: + +1. expose Pi-owned session metadata through a read-only Ziggy command; +2. add a narrow, read-only Profile doctor using existing decoders; +3. add a face-scoped resident lease only if duplicate channel consumers are a real operator risk. + +This order improves visibility first. It adds no provider, session, skill, memory, or transcript authority. It also avoids a Profile-wide lease, which would prevent the currently separate Telegram, Discord, and Slack residents from running together. + +## Complexity budget + +An `act-now` slice must satisfy all of these rules: + +- no new long-running process or background fiber; +- no new durable product record, index, registry, or config format; +- no transcript parsing outside Pi and no transcript content in operator output; +- no network call, repair, migration, or secret output from diagnostic commands; +- no new dependency when an existing Pi or Effect boundary can carry the behavior. + +A candidate that violates one of these rules moves to `defer` unless a concrete product requirement justifies the added authority. + +## Settled decisions + +| Area | Decision | Why it fits | +| --- | --- | --- | +| Sessions | **Act now.** Add `ziggy sessions [--json]`. | Pinned Pi `0.82.0` already owns and returns session metadata. Ziggy only combines leaf-directory projections. | +| Doctor | **Act next.** Add `ziggy doctor [--json]`. | It is a read-only composition of Profile-owned decoders and the session projection. | +| Duplicate residents | **Explore after visibility.** Fence `(Profile, face)`, not the whole Profile. | It prevents two copies of one channel consumer without disabling other channels or local faces. | +| Profile-wide lease | **Reject for the current process shape.** | It changes the shipped multi-channel behavior and still provides no attach path for TUI/CLI. | +| Graceful drain | **Defer.** | A correct drain changes signal handling and all three receive loops, but Ziggy has no durable interrupted-turn or recovery promise yet. | +| Automation claims | **Defer until an automatic trigger exists.** | Manual `wake` has no competing dispatcher; a claim store now would be unused durable state. | +| Skill requirements | **Keep Pi-owned.** | Ziggy already supplies ordered roots and Pi parses skills. A Ziggy metadata parser would duplicate ownership. | +| Inbound dedupe | **Keep current behavior.** | All channels already have bounded/process-local suppression or monotonic offsets. | + +## Target flow + +### Read-only operator path + +```text +CLI + -> application Sessions / Doctor + -> adapters/pi SessionManager.listAll metadata + -> existing Profile/channel/automation decoders + -> stable text or JSON projection +``` + +The projection never opens an agent session, calls a model, connects a channel, changes a file, or returns transcript previews. + +### Optional resident guard + +```text +ziggy + -> decode local config + -> acquire scoped (Profile path, channel kind) SQLite lease + -> open transport and Pi chats + -> existing Effect scope closes chats/socket + -> release lease last +``` + +The SQLite transaction is the lock. A JSON owner file, PID polling, heartbeat, stale-time policy, and daemon registry are not part of the design. + +## Chunk 1 — Pi-backed session inventory + +### Behavior delivered + +Add: + +```text +ziggy sessions +ziggy sessions --json +``` + +Text rows are newest-first and contain only: + +```text + +``` + +JSON returns the same four fields plus `created`. It must not expose Pi's `firstMessage` or `allMessagesText` fields. + +### Files and symbols + +- `src/domain/session.ts` + - `SessionListing` inferred from an Effect Schema; + - typed `SessionInventoryError` carrying operation/path/message but no content. +- `src/adapters/pi/sessions.ts` + - recursively discover directories below `/sessions` without following symlinked directories; + - reject a leaf containing a symlinked `.jsonl` before giving that directory to Pi; + - call `SessionManager.listAll(leafDirectory)` once for every directory containing direct regular `.jsonl` children, avoiding `list(cwd, customDir)`'s header-cwd filter; + - immediately map Pi `SessionInfo` to the safe metadata shape; + - compare discovered regular `.jsonl` paths with returned paths so Pi's silent metadata omission becomes a typed failure. +- `src/application/sessions.ts` + - expose one client-neutral `list(target)` Effect; + - normalize paths relative to the Profile and sort by modified time, then relative path. +- `src/main.ts` + - parse only the exact optional `--json` flag; + - print `no sessions` for an absent/empty tree; + - render stable text or JSON. +- Focused tests beside the adapter/application code. + +### Ownership and failure behavior + +Pi remains the only session-format reader. Directory traversal inspects entry types and paths but does not parse content. A symlinked `.jsonl` or a regular file for which Pi cannot build metadata fails the command rather than disappearing. Missing `sessions/` is a successful empty result. Symlinked directories are not followed. Lines Pi intentionally tolerates are not reclassified by Ziggy. + +### Verification + +1. Root local, local-main, Telegram, Discord, Slack, and automation leaf directories each appear once. +2. A Pi-readable session appears even when its header `cwd` is empty, imported, or uses another accepted Profile spelling. +3. Rows are newest-first with a deterministic path tie-break. +4. Text and JSON contain no prompt, reply, `firstMessage`, or `allMessagesText`. +5. Missing session root prints `no sessions` and exits zero. +6. A no-header/unreadable regular `.jsonl` produces the typed metadata failure. +7. A directory symlink is ignored; a `.jsonl` file symlink fails before Pi reads the leaf. + +### Risk + +`SessionManager.listAll(customDirectory)` intentionally swallows files for which it cannot build metadata, while Pi also tolerates malformed non-header lines. The path comparison prevents silent metadata omissions without creating a stricter Ziggy parser. Keep the adapter pinned to documented `0.82.0` behavior and update the Pi surface note when upgrading Pi. + +## Chunk 2 — narrow read-only doctor + +### Behavior delivered + +Add: + +```text +ziggy doctor +ziggy doctor --json +``` + +Every check has a stable code, `ok | warn | error`, one evidence message, and an optional remedy. Exit nonzero when any check is `error`. Stable codes—not prose—are the JSON contract. + +Initial checks: + +| Code | Source | Result policy | +| --- | --- | --- | +| `profile.soul` | Explicit local `readFile` plus existing Profile path rule | missing/unreadable/non-file = error | +| `channel.telegram` | `telegram.json` existence + existing decoder | absent = ok/not configured; invalid = error | +| `channel.discord` | `discord.json` existence + existing decoder | absent = ok/not configured; invalid = error | +| `channel.slack` | `slack.json` existence + existing decoder | absent = ok/not configured; invalid = error | +| `sessions.metadata` | Chunk 1 inventory | absent/empty = ok; invalid/unreadable = error | +| `automation.` | Existing automation decoder for direct `automations/*.md` | absent directory = ok; invalid definition = error | + +### Files and symbols + +- `src/domain/doctor.ts` + - `DoctorSeverity`, `DoctorCheck`, and `DoctorReport` Effect Schemas; + - data only—no check implementation. +- `src/application/doctor.ts` + - read `SOUL.md` to prove local readability, not merely `stat` it; + - compose checks in a fixed order; + - catch expected typed failures into report rows; + - defects still fail the command rather than being mislabeled healthy. +- `src/application/automations.ts` + - expose/reuse automation reading and parsing instead of creating a second frontmatter parser. +- Existing channel application modules + - reuse their config loaders only after a local existence check so an optional absent channel is not an error. +- `src/main.ts` + - render text/JSON and set the exit code from the report. +- Focused tests for aggregation, redaction, order, and exit behavior. + +### Explicit limits + +Doctor is **not** live health. It does not call `Auth.status` (Pi auth checks may create/refresh credential files), call a provider, poll Telegram, connect Discord/Slack, validate remote credentials, execute tools, load executable extensions, test skill binaries, or repair files. It reports local Profile/config readiness only. `ziggy auth` remains the explicit provider-auth surface, and gateway startup retains live credential/network validation. + +### Verification + +1. A minimal readable `SOUL.md` with no optional channels exits zero with explicit not-configured rows. +2. Missing, non-file, and unreadable `SOUL.md` each produce `profile.soul` error. +3. Each invalid optional config produces one stable error without revealing a token. +4. One malformed automation is attributed by ID/path and does not hide other checks. +5. Session inventory failure is rendered once and exits nonzero. +6. Text and JSON have identical check codes, severities, and order. +7. No check creates/refreshes `auth.json`, changes Profile mtimes, or opens a network connection. + +### Risk + +A broad doctor becomes a second policy engine quickly. Keep every check attached to an existing owner and require a separate decision before adding network probes, repair actions, skill/package requirements, security scoring, or compatibility policy. `Profiles.listSkills` intentionally ignores malformed candidates rather than diagnosing them, so skill validation is not part of this slice. + +## Chunk 3 — optional face-scoped resident lease + +### Behavior delivered + +Prevent two concurrent copies of the same resident face for one Profile: + +```text +(Profile, telegram) != (Profile, discord) != (Profile, slack) +``` + +A duplicate exits with a typed, actionable error before channel or Pi work. Different channel faces and local TUI/`run`/`wake` remain allowed. + +### Files and symbols + +- `src/domain/resident.ts` + - `ResidentKind = telegram | discord | slack`; + - `ResidentAlreadyRunning` typed failure. +- `src/adapters/bun/resident-lease.ts` + - create `/.runtime/resident-leases/.sqlite`; + - hold `BEGIN IMMEDIATE` for the Effect scope; + - map `SQLITE_BUSY` to `ResidentAlreadyRunning`; + - rollback and close idempotently on scope release. +- `src/application/resident.ts` + - one `withLease(target, kind, effect)` capability. +- `src/main.ts` + - wrap each resident command after local config decode and before `runLoop`. +- Focused two-process and fake-transport tests. + +### Verification + +1. First same-face owner acquires; second receives `ResidentAlreadyRunning`. +2. Different faces acquire concurrently for the same Profile. +3. TUI, `run`, and `wake` are unchanged. +4. A refused duplicate performs zero transport and Pi calls. +5. Normal interruption and process death both permit the next acquire. +6. Alternate path spelling that resolves through the same Profile directory still reaches the same SQLite file; no realpath identity rule is introduced. + +### Risk and gate + +Do not implement this merely for parity. First reproduce or operationally confirm duplicate provider consumers. If no one runs resident channels unattended, sessions and doctor deliver more value with less lifecycle code. + +## Verification matrix + +| Contract | Unit | Integration | Full gate | +| --- | --- | --- | --- | +| Session metadata only | safe mapper + sort | recursive tree, cwd variants, symlink refusal | `bun test`, `bun run check` | +| Doctor is read-only and stable | report aggregation | temporary configs/automations + no auth-file creation | `bun test`, `bun run check` | +| Same-face exclusion | SQLite lease | child-process death + fake gateway | `bun test`, `bun run check` | +| Existing faces unchanged | existing gateway/application suites | current walking skeletons | `bun test`, `bun run check` | + +## Rollout + +Each chunk is an independent PR. Ship session inventory first and use it to prove the doctor session check. Ship doctor without any fixer. Treat the face lease as a separately accepted behavior change because it can reject a command that currently starts. + +No migration is required. The first two chunks write nothing. The optional lease creates only `.runtime` coordination state; the held SQLite lock, not file contents, is authoritative. + +## Residual risks + +- Pi's listing API may change on upgrade; keep all `SessionInfo` handling inside `src/adapters/pi/`. +- Local channel-config checks cannot promise remote credentials are valid; output must say `configured`, not `healthy`. +- A face lease prevents duplicate consumers but does not make accepted channel work durable. +- Scoped shutdown can still interrupt an in-flight reply; that remains truthful until a delivery/recovery contract is selected. + +## Work kept out + +- unified daemon, attach/RPC, web control plane, or resident event bus; +- Profile-wide lease while channels are separate commands; +- custom session parser/index, transcript search, or resume picker; +- durable gateway ingress/outbound journal, replay, or receipts; +- scheduler, cron claim ledger, retries, run history, or dashboards; +- Ziggy-owned skill metadata parser, tool search bridge, or package marketplace; +- graceful-drain lifecycle state before interrupted work has a promised outcome. + +## Open decisions + +Only one decision gates code after the first two chunks: whether duplicate same-face resident processes are common enough to justify the lease. Session inventory and doctor do not depend on that choice. diff --git a/docs/plans/primitive-status.md b/docs/plans/primitive-status.md index cd7187d..92f76b8 100644 --- a/docs/plans/primitive-status.md +++ b/docs/plans/primitive-status.md @@ -1,6 +1,6 @@ # Primitive status -Current Ziggy status on 2026-07-29. +Current Ziggy status on 2026-08-04. This is a work queue, not a parity checklist. Pi remains authoritative for providers, sessions, compaction, branching, skills, extensions, and the TUI. Reference repositories are evidence only, @@ -20,27 +20,28 @@ not active targets. ## Ordered work -### 1. Decide scheduler ownership +### 1. Session visibility -Choose the process that owns scheduled claims before implementing cron: +Land `ziggy sessions [--json]` as the first read-only slice in +`openclaw-hermes-primitives.md`. Pinned Pi `0.82.0` exposes +`SessionManager.listAll(customDirectory)`; Ziggy can recurse its known Profile tree and project only +ID/path/timestamps/message count without parsing JSONL or exposing transcript previews. -- one Profile-wide resident, which means only one of the current channel commands can run; or -- a scheduler-specific resident owner, leaving channel processes independent. +### 2. Narrow doctor -The current separate Telegram, Discord, and Slack commands make a Profile-wide lease a product -decision, not a mechanical prerequisite. +Land `ziggy doctor [--json]` as a read-only composition of Profile readability, +channel-config, automation, and session checks. It does not call `Auth.status`, because Pi auth +checks may create/refresh credentials, and performs no network calls, repairs, migrations, +extension loading, or skill-requirement parsing. -### 2. Claim-before-wake scheduler +### 3. Duplicate-resident decision -After ownership is settled, implement only the slice in `automation-scheduler.md`: parse `cron`, -derive one deterministic firing ID, atomically claim that firing before model or delivery work, and -prevent overlap for the same automation. Keep definitions as Markdown and every run as a fresh Pi -session. Do not add a general run ledger, retries, dashboards, or lifecycle state machine. +If duplicate channel consumers are a real operator risk, add only a face-scoped lease keyed by +`(Profile, telegram|discord|slack)`. Do not add a Profile-wide lease: the shipped channel commands +are independent and must remain able to run together. TUI, `run`, and `wake` remain unaffected. -### 3. Operator visibility - -Land `ziggy sessions `, then `ziggy doctor `, as separate read-only slices from -`cli-polish.md`. They project Pi and Profile state; they do not create new authorities. +Scheduling stays deferred until an automatic trigger is a concrete product requirement. Its first +slice must still atomically claim a deterministic trigger occurrence before model or delivery work. Live Telegram, Discord, and Slack proofs can run beside this queue whenever disposable credentials are available. @@ -48,6 +49,8 @@ are available. ## Explicitly deferred - Canonical Profile identity or symlink rejection. +- Profile-wide resident ownership while channels are separate commands. +- Graceful-drain lifecycle state before interrupted work has a promised outcome. - Durable gateway ingress/outbound journals and replay. - A daemon attach protocol, RPC layer, or cross-channel event bus. - A general automation run ledger, retries, or delivery receipts. diff --git a/docs/plans/profile-lease.md b/docs/plans/profile-lease.md deleted file mode 100644 index 1932b57..0000000 --- a/docs/plans/profile-lease.md +++ /dev/null @@ -1,60 +0,0 @@ -# Profile lease - -## Why this slice exists - -Telegram, Discord, and Slack can currently run as separate resident processes for the same -Profile. Local TUI and `run` can also open while a gateway is resident. Their session files are -separate, but they share Profile policy, memory, auth, and runtime state. - -The next code slice is one Profile-scoped resident lease. It is an ownership guard, not an attach -protocol. - -## Slice - -Use a stable SQLite lock at `/.runtime/gateway-lease.sqlite`. - -1. Add a `ProfileLease` Effect service backed by `BEGIN IMMEDIATE`. -2. Hold the database connection for the resident scope. Rollback and close on shutdown; process - death releases the OS lock automatically. -3. Write diagnostic owner metadata to `/.runtime/gateway-owner.json` after acquisition. - Ownership comes from the SQLite lock, never from the metadata file. -4. Telegram, Discord, and Slack acquire before any network call or Pi runtime construction. -5. TUI and `run` probe the lease and refuse while held unless `--force` is explicit. -6. `wake` remains allowed because it uses a fresh session and serialized memory writes. - -The owner metadata contains the gateway kind, PID, and acquisition time. A stale metadata file is -cleaned only after Ziggy proves the SQLite lease is free. - -## Invariants - -- At most one resident gateway owns a Profile. -- A refused gateway performs zero channel and Pi work. -- Normal interruption and process death both release ownership. -- `--force` never removes or changes the resident lease. -- A missing or stale metadata file cannot create or revoke ownership. -- Telegram, Discord, and Slack use the same lease path and service. - -## Focused proof - -Tests should use fake gateway transports and a temporary Profile: - -1. Hold the lease; a second hold gets a typed `ProfileLeaseHeld` failure. -2. Close the holder; the next hold succeeds. -3. Kill a child holder; the next hold succeeds without deleting a lock file. -4. Start each gateway while held; assert zero transport and Pi calls. -5. Guard TUI and `run`; assert refusal without `--force` and one warning with it. -6. Confirm `wake` is unchanged. - -Then run: - -```sh -bun test -bun run check -``` - -## Not in this slice - -- Attaching local faces to the resident process. -- Multiple resident channels for one Profile. -- Session migration or RPC. -- Heartbeats or time-based stale takeover. diff --git a/docs/plans/resident-lease.md b/docs/plans/resident-lease.md new file mode 100644 index 0000000..90dd6cf --- /dev/null +++ b/docs/plans/resident-lease.md @@ -0,0 +1,53 @@ +# Resident lease + +## Decision + +Do **not** add one Profile-wide lease to the current process shape. Telegram, Discord, and Slack are +separate shipped residents with isolated session roots; a Profile-wide lease would make those +channels mutually exclusive without providing an attach path for local TUI/CLI. + +If duplicate provider consumers become a demonstrated operator risk, add only the face-scoped +slice in [`openclaw-hermes-primitives.md`](./openclaw-hermes-primitives.md): + +```text +(Profile path, telegram | discord | slack) +``` + +This is an ownership guard, not a daemon, status registry, or attach protocol. + +## Slice + +1. Add a `ResidentLease` Effect service backed by a held SQLite `BEGIN IMMEDIATE` transaction at + `/.runtime/resident-leases/.sqlite`. +2. Acquire after local config decoding but before any network call or Pi runtime construction. +3. Hold the database connection for the resident scope; rollback and close on shutdown. Process + death releases the OS lock. +4. Map a busy lock to a typed `ResidentAlreadyRunning` failure with the Profile and face. +5. Leave different channel faces, TUI, `run`, and `wake` unchanged. + +The SQLite transaction is authoritative. Do not add owner JSON, PID/start-time inspection, +heartbeats, polling, stale-lock deletion, canonical-realpath identity, or `--force` bypass. + +## Invariants + +- At most one resident of each face owns one Profile. +- Telegram, Discord, and Slack may run concurrently for the same Profile. +- A refused duplicate performs zero channel and Pi work. +- Normal interruption and process death release ownership. +- A leftover SQLite file does not imply a live owner. +- Profile path policy remains unchanged. + +## Focused proof + +1. Hold one face lease; a second same-face hold gets `ResidentAlreadyRunning`. +2. Hold Telegram, Discord, and Slack leases concurrently for one Profile. +3. Close or kill a holder; the next same-face hold succeeds without deleting files. +4. Start each gateway while its face is held; assert zero transport and Pi calls. +5. Confirm TUI, `run`, and `wake` are unchanged. +6. Run `bun test` and `bun run check`. + +## Gate + +Do not implement this for reference parity. First confirm that users run unattended channel +residents and can accidentally start a duplicate of the same command. Session inventory and doctor +remain ahead of this slice because they add visibility with no behavior change. diff --git a/docs/research/eve-flue-ziggy-comparison.md b/docs/research/eve-flue-ziggy-comparison.md new file mode 100644 index 0000000..0d18d96 --- /dev/null +++ b/docs/research/eve-flue-ziggy-comparison.md @@ -0,0 +1,318 @@ +# eve, Flue, and Ziggy comparison + +Source-grounded comparison of: + +- Ziggy at `98988c29b7676b9fd0de1cc6c452598134b13fd0` +- [`vercel/eve`](https://github.com/vercel/eve) at + `6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474` (`eve@0.30.0`) +- [`withastro/flue`](https://github.com/withastro/flue) at + `a171cc1bc8a552775a820ae3d343ccd09597cc8c` (`@flue/*@2.0.2`) + +The external repositories were cloned read-only under +`/tmp/ziggy-agent-framework-comparison/{eve,flue}`. Paths and line numbers below refer to +those exact snapshots unless a link points into this repository. + +## Executive finding + +**They overlap in capability vocabulary, not in product boundary.** All three provide an +agent loop, models, tools, skills, conversation state, channels, and local execution. They are +not interchangeable implementations of the same thing: + +- **Ziggy is a local-first assistant product.** A live Profile folder is the assistant's + identity and visible durable world, including human-owned policy files and Pi-owned session + files. Ziggy composes the published Pi coding-agent runtime rather than building an agent + framework. +- **Flue is a deployable TypeScript application framework.** An agent is a re-rendered + function, an instance ID identifies a conversation, Flue owns admission/recovery semantics, + and Pi's lower-level packages own the model/tool loop and provider protocol. Cross-restart + durability requires a durable Node database or the Cloudflare target; the Node production + default is in-memory. +- **eve is a deployable filesystem-authored backend-agent framework.** The agent folder is + compiler input; eve owns a durable outer loop around Vercel AI SDK `ToolLoopAgent` and the + Workflow SDK. + +**Flue is Ziggy's closest technical relative because both use Pi.** It is nevertheless closer +to eve in product shape: Flue and eve compile agent applications, expose HTTP conversation +protocols plus durability machinery, support remote clients, and target hosted runtimes. Ziggy +is a private executable with in-process faces and Profile-local files. + +**There is no direct runtime tie between Ziggy and eve.** There is a shared Pi lineage between +Ziggy and Flue, but not a shared session format, tool API, extension loader, or durability +engine. The clean common portable unit today is a spec-compliant **Agent Skill**. A separately +hosted eve or Flue agent can also be called from Ziggy over its public HTTP protocol, but it +must remain a separate state authority. Flue's SDK is fetch-portable; the official `eve` +package requires Node 24, so a Ziggy/eve client package integration needs compatibility testing +or a small Ziggy-owned HTTP adapter. + +## Where they do not differ + +1. **Agentic capability model.** Each combines instructions, model selection, callable tools, + progressive skills, multi-turn context, and multiple ingress surfaces. +2. **TypeScript host.** All three are TypeScript systems with explicit typed boundaries around + model and tool execution. +3. **Progressive skills.** All advertise compact skill metadata and load full skill instructions + on demand. All accept spec-compliant Agent Skills packages; each also has host-specific + authoring options or looser accepted Markdown shapes. +4. **One core, multiple faces.** Ziggy's TUI/CLI/gateways share `ZiggyAgent`; eve's terminal, + clients, and frontend hooks share its HTTP channel; Flue's CLI, dispatch API, channels, and + HTTP routes share its conversation runtime. +5. **External side effects still need idempotency.** None can make a payment, email, or foreign + database mutation exactly once merely by recording an agent/tool result. + +Those similarities are real, but they sit above substantially different ownership models. + +## Where they differ + +| Dimension | Ziggy | Flue | eve | +| --- | --- | --- | --- | +| Product | Opinionated personal assistant | Agent application/harness framework | Durable backend-agent framework | +| Authoring unit | Live Profile folder, led by `SOUL.md` | Capitalized exported function plus `use*` hooks | Conventional `agent/` source tree plus `define*` files | +| Durable identity | Profile path; sessions below it | Agent function identity + caller-chosen instance ID | Compiled agent + session/continuation identity | +| Filesystem meaning | **Live Profile state and agent cwd**, including human-owned policy files | Application source; optional runtime sandbox is separate | **Compiler input**; the mandatory per-session runtime sandbox is separate | +| Inner loop | Pi coding-agent owns the whole session runtime | Pi `pi-agent-core.Agent` | Vercel AI SDK `ToolLoopAgent` | +| Durable orchestration | Pi JSONL transcript only; Ziggy has no accepted-work recovery layer | Flue conversation records, submission queue, attempts, and recovery; cross-restart durability needs a durable Node store or Cloudflare DO | Workflow SDK journal, steps, hooks, streams, and park/resume | +| Provider layer | Pi coding-agent `ModelRuntime`, Profile-local auth/model files | Pi AI provider objects and built-in provider catalog | AI SDK provider objects or Vercel AI Gateway IDs | +| Long-term memory | Explicit shared/person/group Markdown, separate from transcript | Conversation-scoped persisted hook state; cross-conversation memory is application-owned | Session-scoped `defineState`; cross-session memory is external | +| Conversation ordering | One live handle and semaphore per gateway chat; Pi JSONL persists history | Durable per-conversation admission queue | No durable FIFO for concurrent deliveries; channel/app queues bursts | +| Sandbox | No product sandbox boundary; Pi tools run in the Profile world | Optional `useSandbox`; no filesystem/shell without one | Every agent gets one per-session sandbox and file/shell tools; the backend is defaulted or overridden, separate from Workflow durability | +| Extensibility | Pi extensions and Agent Skills admitted by Ziggy policy | Hooks, Valibot tools, skills, MCP, subagents, persistence/sandbox adapters | Filesystem definitions for tools, skills, hooks, channels, connections, schedules, extensions, subagents | +| Channels | Resident owner-only Telegram, Discord, and Slack processes | Verified inbound Hono channel packages; app owns IDs and outbound behavior | Built-in/custom HTTP/WebSocket channels own continuation and delivery | +| Ingress trust | Static configured owner-ID checks; no public client endpoint | Mounted agents have no built-in auth; application middleware must authenticate and authorize each conversation ID | Channel-level auth helpers; default eve channel rejects production traffic until real auth is configured | +| Client/UI | Pi TUI and CLI; no external client protocol | `@flue/sdk`, `@flue/react`, CLI, private demo UI | `eve/client`, React/Vue/Svelte hooks, Next/Nuxt/SvelteKit integrations, terminal UI | +| Deployment | Bun checkout/process | Node app/server or Cloudflare Worker/DO; scripts via `start()` | Nitro Node self-host or Vercel; pluggable Workflow world and sandbox | +| Reuse contract | Private bin-only package; internal `ZiggyAgentShape` | Published runtime, Vite, SDK, React, adapter, channel, and database packages | Published package with authoring, client, frontend, framework, tool, channel, sandbox, and eval exports | + +The sandbox and trust rows are intentionally asymmetric. eve gives every agent a default +sandbox +([`docs/sandbox.mdx:6-10`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/docs/sandbox.mdx#L6-L10), +[`127-132`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/docs/sandbox.mdx#L127-L132)); +Flue has no implicit sandbox. Flue agent mounts require application authentication **and** +conversation authorization +([`guide/routing.md:133-142`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/apps/docs/src/content/docs/guide/routing.md#L133-L142)), +while eve's default channel falls back to a production-rejecting auth chain +([`docs/channels/eve.mdx:107-116`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/docs/channels/eve.mdx#L107-L116)). + +### The most important filesystem distinction + +The phrase “filesystem-first” can hide three different contracts: + +```text +Ziggy: Profile files = the running assistant's identity and live durable state + (human-owned policy, curated memory, and Pi-owned sessions) +Flue: TypeScript files = app authoring; database records = conversation truth + sandbox files = a separately selected workspace lifetime +eve: agent/ files = compiler input; Workflow state = conversation truth + sandbox files = a separately selected workspace lifetime +``` + +Moving Ziggy onto either framework would therefore be a state-model migration, not a runtime +swap. `SOUL.md`, Pi JSONL trees, scoped Markdown memory, Profile auth, and repository-owned Pi +extensions do not map one-for-one to either framework's deployed conversation records. + +## Runtime lineage and execution paths + +### Ziggy: full Pi composition + +```text +CLI / TUI / gateway + -> ZiggyAgent Effect service + -> PiAgent adapter + -> createAgentSessionRuntime + -> Pi AgentSession / runPrintMode / InteractiveMode + -> Profile-local Pi JSONL + provider +``` + +Ziggy depends on `@earendil-works/pi-coding-agent@0.82.0` and delegates runtime creation, +streaming, tools, provider behavior, JSONL sessions, and terminal UI to it +([`package.json:17`](../../package.json), +[`src/adapters/pi/pi-agent.ts:503-542`](../../src/adapters/pi/pi-agent.ts)). The conversational `ZiggyAgent` seam is only `runOnce`, `openTui`, and `openChat` +([`src/application/agent.ts:9-26`](../../src/application/agent.ts)). + +### Flue: lower-level Pi wrapped in Flue durability + +```text +HTTP / dispatch / CLI admission + -> durable submission + per-conversation queue + -> Flue coordinator claims an attempt + -> render agent function and hooks + -> Flue Session + -> Pi pi-agent-core.Agent + -> Pi AI provider stream + -> Flue canonical conversation records + settlement +``` + +Flue imports `@earendil-works/pi-agent-core` and `@earendil-works/pi-ai` `^0.83.0` +([`packages/runtime/package.json:87-88`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/packages/runtime/package.json#L87-L88)). +It constructs Pi's `Agent`, configuring parallel tools and Flue's per-turn rerender callback +([`packages/runtime/src/session.ts:2211-2235`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/packages/runtime/src/session.ts#L2211-L2235)). +Flue then projects Pi events into its own canonical stream; Pi JSONL and Pi coding-agent +extensions are not involved. + +With durable storage configured, Flue's stronger accepted-work contract is explicit: one +terminal settlement per accepted submission, an ordered queue per conversation, and +at-least-once execution over exactly-once recording +([`guide/durability.md:7-39`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/apps/docs/src/content/docs/guide/durability.md#L7-L39)). +Without `db.ts`, however, Node production uses in-memory SQLite and a restart loses all +conversations, submissions, and state +([`guide/database.md:39-51`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/apps/docs/src/content/docs/guide/database.md#L39-L51)). + +### eve: AI SDK loop wrapped in Workflow durability + +```text +HTTP channel + -> compile/resolve agent graph + -> Workflow runtime and workflowEntry + -> durable session driver + -> one durable turn step + -> AI SDK ToolLoopAgent, limited to one model/tool step + -> Workflow stream/checkpoint + -> park or dispatch the next durable step +``` + +eve's workflow entry is a Workflow SDK `"use workflow"` function +([`execution/workflow-entry.ts:96-169`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/packages/eve/src/execution/workflow-entry.ts#L96-L169)). +Its harness creates Vercel AI SDK `ToolLoopAgent` with `stopWhen: isStepCount(1)`, making the +outer eve/Workflow loop the durable step authority +([`harness/tool-loop.ts:1029-1075`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/packages/eve/src/harness/tool-loop.ts#L1029-L1075)). + +Unlike Flue, eve does not promise an ordered durable input queue for a busy session. Its docs +require clients to wait for `session.waiting` or queue bursts in the channel/application +([`execution-model-and-durability.mdx:71-77`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/docs/concepts/execution-model-and-durability.mdx#L71-L77)). + +## What can be reused elsewhere + +| Surface | Portability | Practical answer | +| --- | --- | --- | +| Agent Skills (`SKILL.md`) | **High** | Ziggy's existing spec-shaped skills can be copied/imported into Flue or placed under eve's `agent/skills/`. Review host-specific tool names, paths, binaries, and credentials. | +| Ziggy Pi extensions | **Low** | They target the full Pi coding-agent extension API. Flue uses Pi core directly and eve uses AI SDK; neither loads them. Port behavior as native Flue/eve tools instead. | +| Flue Pi providers | **Medium at the Pi protocol, low through Flue** | Flue exposes Pi `Provider` objects, but Ziggy currently goes through coding-agent `ModelRuntime` and pins a different Pi version. Share or adapt against Pi's provider protocol, not Flue internals. | +| Flue SDK | **High over HTTP** | `@flue/sdk` is ESM and works anywhere `fetch` exists. It addresses one deployed conversation URL; it does not embed Flue's loop. | +| eve HTTP/client/frontend surfaces | **High over HTTP** | `eve/client` and React/Vue/Svelte integrations are supported exports, but the `eve` package requires Node 24. Ziggy should use the wire protocol through its own fetch adapter or a Node sidecar unless direct Bun compatibility is proven. | +| Flue `start()` | **Medium within supported Node hosts** | Useful for a standalone Node script/test, but one process holds one Flue runtime and `@flue/runtime` requires Node 22.19+. It is not a supported nested runtime inside Ziggy's Bun/Effect composition. | +| eve framework adapters | **Medium inside web apps** | Next/Nuxt/SvelteKit and Nitro/self-hosting surfaces make eve usable in existing applications, but the compiled eve app remains its own runtime. | +| Channel packages | **Low across frameworks** | Flue channels are verified inbound Hono routes; eve channels own eve continuation semantics; Ziggy gateways are resident transport owners with direct outbound replies. Their contracts differ. | +| Persistence/durability engines | **Low as code, high as lessons** | Flue stores and eve Workflow worlds are coupled to their record/step protocols. Borrow invariants and adapter shapes, not implementations. | +| Sandboxes | **Low as framework objects** | Each has framework-specific tool/context contracts. The underlying Docker, Cloudflare, Vercel, or remote sandbox service can be shared through separate adapters. | +| Hosted agents | **High as separate services** | A Ziggy Pi tool can use fetch-portable `@flue/sdk` for Flue or a Ziggy-owned HTTP adapter for eve. Keep remote conversation state separate from the Profile. | + +The runtime-package caveat is contractual, not a claim that Bun must fail: `eve` declares Node +24+ and `@flue/runtime` declares Node 22.19+ +([eve package](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/packages/eve/package.json#L401-L403), +[Flue runtime package](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/packages/runtime/package.json#L77-L79)). +Use the HTTP boundary unless direct Bun execution is deliberately tested and supported. + +### Skills are the real direct bridge + +Flue intentionally parses the Agent Skills format and packages imported `SKILL.md` trees, +while also supporting inline `defineSkill` +([`skill-frontmatter.ts:16-75`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/packages/runtime/src/skill-frontmatter.ts#L16-L75), +[`guide/skills.md:98-146`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/apps/docs/src/content/docs/guide/skills.md#L98-L146)). +eve says standards-compliant Agent Skills port as-is and supports flat Markdown, packaged +`SKILL.md` directories, and TypeScript definitions +([`docs/skills.mdx:6-55`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/docs/skills.mdx#L6-L55)). +Pi likewise discovers `SKILL.md` directories through explicit skill paths +([`docs/research/pi-sdk-surface.md:323-334`](pi-sdk-surface.md)). + +Ziggy's skill text is therefore reusable. Its executable extension code is not. A skill that +says “call `memory_write`” or assumes Ziggy's Profile cwd remains syntactically portable but +behaviorally Ziggy-specific until rewritten for the destination host. + +## Can Ziggy tie into either framework? + +### 1. Remote delegation: yes, with a hard state boundary + +A Ziggy Pi extension can expose a tool such as `call_flue_agent` or `call_eve_agent`. The tool +would use `@flue/sdk` or a Ziggy-owned HTTP adapter and return remote progress/final output. +This is a reasonable fit for a specialized hosted worker. + +Required invariant: + +```text +Ziggy Profile/session/memory authority != remote framework conversation authority +``` + +Do not let both systems write the same Profile files, pretend their transcripts are one +session, or compact one another's history. Persist the remote conversation ID/URL as an +explicit reference if continuity is required. + +### 2. In-process Flue inside Ziggy: unsupported and unverified + +Flue's Node `start()` is public, but `@flue/runtime` requires Node 22.19+, direct Bun +compatibility is unproven, and it allows one Flue runtime per process +([`packages/runtime/src/node/start.ts:101-109`](https://github.com/withastro/flue/blob/a171cc1bc8a552775a820ae3d343ccd09597cc8c/packages/runtime/src/node/start.ts#L101-L109)). +Using it inside Ziggy would introduce a second loop/session/state/provider authority beside Pi +coding-agent, plus Node-target assumptions inside a Bun/Effect product. A separate Flue service +or process is clearer. + +### 3. In-process eve inside Ziggy: no supported seam + +eve publishes authoring definitions and hosted/client integrations, not its internal +`workflowEntry` or `createToolLoopHarness`. Its package export map centers on compiled eve apps, +HTTP clients, web-framework integrations, and framework-specific extension points +([`packages/eve/package.json:58-287`](https://github.com/vercel/eve/blob/6c5f4fe25f659bb21e9e267cfee3c9f9bbfc9474/packages/eve/package.json#L58-L287)). +Run it as its own service if Ziggy needs it. + +### 4. Making Ziggy usable in other places: not yet a supported contract + +Ziggy already has the right internal direction: conversational CLI/TUI, gateways, and +automation execution all call `ZiggyAgent`. But the package is private, bin-only, and has no +export map +([`package.json:2-6`](../../package.json)). `ZiggyAgentShape` is also narrower than the intended +client-neutral surface: it lacks event observation, steering, abort, session identity, and +reconnect semantics. + +The smallest honest reuse move is **not** to adopt eve or Flue. It is to publish a headless +Ziggy application surface around Profile policy and `ZiggyAgent`, then add a transport only when +a second process or external client is actually required. + +## What Ziggy should borrow—and what it should not + +### Borrow now or soon + +1. **Keep Agent Skills as the cross-runtime content format.** Test portable packages against + host-specific tool/path assumptions. +2. **Borrow Flue's accepted-submission vocabulary when gateway work becomes load-bearing:** + admission, attempt, settlement, idempotency key, and canonical recorded outcome are clearer + than treating a live chat handle as durable work. +3. **Borrow both frameworks' separation of conversation durability from workspace durability.** + Ziggy already separates curated memory from Pi transcripts; keep any future remote sandbox + as a third explicit authority. +4. **Add a public headless boundary before adding web clients.** Stabilize prompt/events/abort/ + dispose/session identity in-process first; HTTP can project that contract later. + +### Do not borrow yet + +1. Do not replace Pi JSONL with a second conversation engine merely to match a framework. +2. Do not add a compiler/Vite/Nitro/Workflow layer to a Profile-first assistant. +3. Do not copy Flue/eve channel packages into Ziggy; their ownership and outbound contracts do + not match resident gateways. +4. Do not merge remote eve/Flue state into Profile memory or Pi transcripts. +5. Do not treat shared Pi ancestry as binary compatibility. Ziggy pins full coding-agent + `0.82.0`; Flue uses lower-level Pi packages `^0.83.0` and owns a different session protocol. + +## Decision summary + +- **Replace Ziggy with Flue?** Only if the product goal changes from a local folder-assistant to + a deployable durable agent application framework. Flue is the closer migration target because + of Pi, but Profile state and Pi sessions still require a redesign/migration. +- **Replace Ziggy with eve?** No for the current product. eve is a stronger fit for Vercel/Nitro + backend agents and web application integration, not for preserving Pi/Profile semantics. +- **Use either from Ziggy?** Yes as a separate remote agent service over its public HTTP + boundary; use Flue's fetch-portable SDK and a Ziggy-owned eve HTTP adapter unless Bun support + is proven. +- **Use Ziggy assets elsewhere?** Skills: yes. Pi extensions, memory tools, channels, and + sessions: adapt or rewrite. +- **Make Ziggy itself reusable elsewhere?** Expose and stabilize its headless application API; + do not expose Pi adapter internals or invent a remote protocol before a real external client + needs one. + +## Evidence boundary and caveats + +- Findings describe the exact commits listed above, not every released version. +- External documentation was checked against representative source execution paths. Where docs + and source differed, source was treated as authoritative. +- No claim is made that Workflow durability, Flue database durability, or sandbox persistence + has identical guarantees across every adapter; physical guarantees depend on the selected + world/store/backend. +- Apache-2.0 licenses in eve and Flue permit code reuse subject to their license and notice + obligations. Architectural fit remains the larger constraint. diff --git a/docs/research/openclaw-hermes-current-primitives.md b/docs/research/openclaw-hermes-current-primitives.md new file mode 100644 index 0000000..edbea90 --- /dev/null +++ b/docs/research/openclaw-hermes-current-primitives.md @@ -0,0 +1,210 @@ +# Current OpenClaw and Hermes-Agent primitives Ziggy can borrow + +**Inspected:** 2026-08-04 +**Question:** Which current, proven primitives can Ziggy borrow without substantial code or a second architectural authority? + +**Implementation packet:** [`openclaw-hermes-primitives.md`](../plans/openclaw-hermes-primitives.md) + +## Source snapshots and method + +Only current first-party source was used. The official repositories were freshly cloned into disposable `/tmp` directories and inspected at these exact revisions: + +| Repository | Branch | Exact commit | Commit date | Subject | +|---|---|---|---|---| +| [openclaw/openclaw](https://github.com/openclaw/openclaw) | `main` | [`50a30b7373d78fe8a63e597bd28138b6aa805765`](https://github.com/openclaw/openclaw/commit/50a30b7373d78fe8a63e597bd28138b6aa805765) | 2026-08-04 11:36:03 -04:00 (authored and committed) | `fix(codex): migrate redundant native service tiers (#118738)` | +| [NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent) | `main` | [`42708f8bb39c9c2fc19146956699699bc3ea2da5`](https://github.com/NousResearch/hermes-agent/commit/42708f8bb39c9c2fc19146956699699bc3ea2da5) | 2026-08-04 12:04:42 -04:00 (authored and committed) | `Merge pull request #74864 from bbednarski9/fix/relay-concurrent-turn-scopes` | +| Ziggy baseline | `scotty/ec7028bfd5f4` | [`98988c29b7676b9fd0de1cc6c452598134b13fd0`](https://github.com/Yeshwanthyk/ziggy/commit/98988c29b7676b9fd0de1cc6c452598134b13fd0) | 2026-07-29 18:48:48 -04:00 (authored and committed) | `Merge pull request #2 from scotty/0724a83ee3fb` | + +The comparison follows Ziggy's settled boundary: Pi owns sessions, skill parsing, providers, and the agent loop; Ziggy owns Profile policy and resident-process ownership ([spec](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/docs/research/minimal-ziggy-scout.md#L7-L25)). “Tiny” below means a local invariant that fits an existing Ziggy owner. It does **not** mean copying the source subsystem that demonstrated it. + +### Complexity scale + +- **XS:** under roughly 50 logical lines plus focused tests. +- **S:** roughly 50–150 logical lines plus focused tests. +- **M:** roughly 150–300 logical lines or a persistent-state change. +- Architecture cost overrides line count: a second parser, session index, scheduler, or control plane is “too large” even when its first patch looks short. + +## Decision summary + +| Area | Current Ziggy | Recommendation | Rough complexity | Small invariant worth retaining | +|---|---|---:|---:|---| +| Resident ownership | Separate Telegram, Discord, and Slack residents are shipped; duplicate same-face residents are not fenced | **Explore next, face-scoped** | S | Fence only `(Profile, face)` so duplicate consumers fail before network/Pi work without preventing multiple channels | +| Startup checks / doctor | Per-face config decoders exist; no unified strictly read-only check surface | **Act now, narrowly** | S | Reuse local file/config decoders, report `ok/warn/error` plus a remedy, and exclude auth/runtime checks that may refresh credentials | +| Session visibility | Pi persistence/routing exists; pinned Pi `0.82.0` publicly exposes `SessionManager.listAll(customDir)` | **Act now** | S | Recursively discover leaf directories, project only Pi metadata, and never expose transcript preview fields | +| Graceful shutdown / admission | Scoped cleanup exists; no explicit stop-admission/bounded-drain phase | **Defer** | M across three faces | Preserve the invariant, but add lifecycle state only when interrupted in-flight work has a promised recovery or delivery behavior | +| Automation claims / idempotency | Manual `wake`, gate, and fresh session exist; no scheduler or durable claim | **Defer** | S/M when scheduling lands | Persist a trigger-occurrence claim before effects and consume it without replay; add immutable terminal/`unknown` outcomes only with a later run-history contract | +| Skill requirements / discovery | Profile-first explicit roots and Pi-owned parsing already exist | **Keep** | 0 | Preserve current discovery; if requirements are needed, put namespaced metadata and eligibility diagnostics in Pi, not Ziggy | +| Bounded inbound dedupe | Slack/Discord already have 1,000-entry process-local FIFO caches; Telegram advances an update offset | **Keep** | 0 | Keep bounded process-local suppression; consider two-phase claim/commit only with a demonstrated redelivery bug | + +## 1. Resident ownership — explore a face-scoped lease, not a Profile-wide lease + +### Primary-source evidence + +OpenClaw's gateway lock records a process identity with `pid`, random `ownerId`, creation time, and optional process start time ([payload and identity](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/gateway-lock.ts#L25-L80)). Its stale policy is the important part: when a PID is known, reclaim only if the owner is proved dead; if a lock cannot be inspected, fail closed ([reclaim policy](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/gateway-lock.ts#L236-L270)). Acquisition generates a unique owner token and ultimately fails with an operator-visible “gateway already running” error ([acquisition](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/gateway-lock.ts#L359-L425), [failure](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/gateway-lock.ts#L488-L549)). + +Hermes independently uses the same lease shape for active-session capacity: a UUID lease ID plus PID and process start time prevents PID-reuse errors ([identity and liveness](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/active_sessions.py#L205-L268)); acquire/prune/write occurs under one file lock ([acquire](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/active_sessions.py#L271-L334)); release removes only the matching lease ID and is idempotent ([release](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/active_sessions.py#L337-L350)). + +### Ziggy consequence + +The upstream invariant is useful, but its scope must match Ziggy's shipped product. Telegram, Discord, and Slack are separate commands with separate Pi session roots ([CLI dispatch](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/main.ts#L222-L250), [chat session creation](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/pi/pi-agent.ts#L640-L656)). A Profile-wide lease would make those three channels mutually exclusive before Ziggy has a unified resident or attach protocol. That is a product change, not a small correctness patch. + +The concrete collision today is narrower: two copies of the **same** channel command can consume the same provider stream and target the same session directory. If that failure matters in practice, borrow only this adapted invariant: + +1. Key the lease by `(Profile path, face)` where face is Telegram, Discord, or Slack. +2. Acquire after local config decoding but before opening a channel/socket or Pi session. +3. Use an OS-released lock such as a held SQLite write transaction; process death releases it without PID files or stale takeover logic. +4. Fail a duplicate face clearly and perform zero network or Pi work. +5. Release through the existing Effect scope; local TUI, `run`, `wake`, and other channel faces remain unaffected. + +This is runtime coordination, not durable Profile truth. Do not canonicalize Profile identity, write owner registries, or turn the lease into a daemon protocol. + +### Do not copy + +OpenClaw's dual state/config locks, roles, ports, Windows command-line inspection, polling, and multi-gateway override span hundreds of lines ([full acquisition machinery](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/gateway-lock.ts#L296-L549)). Hermes' cross-process active-session registry and configurable concurrency cap are also not needed. If Ziggy adds a guard, it needs one OS-released lock per resident face—not a process registry. + +## 2. Startup checks, status, and doctor — act now, read-only + +### Primary-source evidence + +Hermes' useful primitive is not its 2,775-line doctor; it is the result grammar. Checks are consistently emitted as `ok`, `warn`, `fail`, or informational detail, and a failure appends a concrete repair instruction ([check/result helpers](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/doctor.py#L204-L226)). The final summary makes remaining actions explicit and numbered ([summary](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/doctor.py#L2748-L2774)). Its deprecated-config policy also deliberately warns without silently migrating; migrations remain with the config owner ([policy](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/doctor.py#L229-L235)). + +OpenClaw contributes a startup invariant: a blocking preflight failure must prevent the process from reporting ready, with the owned reason and remedy shown to the operator ([startup refusal](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/commands/doctor-config-preflight.ts#L164-L185)). + +### Ziggy consequence + +Ziggy already owns useful local checks: + +- gateway config loaders verify `SOUL.md` shape and Schema-decode channel JSON—for example Telegram ([loader](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/gateway.ts#L63-L106)); +- Slack validates credentials with `authTest` at live startup ([startup](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/slack-gateway.ts#L209-L221)); +- Telegram makes an initial API call and establishes its offset at live startup ([startup](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/gateway.ts#L250-L264)); +- `ziggy auth ` already has a separate provider-status command ([auth status](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/main.ts#L150-L173)). + +What is absent is one strictly read-only Profile check surface in the command switch ([current command surface](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/main.ts#L54-L139)). Add a small `doctor` collector that reads `SOUL.md`, reuses **local** config/automation decoders, consumes the safe session projection, and renders severity, evidence, and remedy. Do not call `Auth.status`: Pi auth checks may refresh/persist credentials and malformed auth is not exposed as a hard decoder failure. Existing gateway startup retains live credential/network ownership. Doctor must not poll Telegram, connect sockets, call providers, rewrite files, migrate configs, install tools, repair sessions, or probe every optional integration. + +### Do not copy + +Do not copy OpenClaw's migration preflight or Hermes' broad auto-fixer. They combine config migration, security advisories, service management, profiles, plugins, providers, and repair actions. A Ziggy doctor that mutates Profile files would become a new policy authority and violate the plain-file/human-ownership model. + +## 3. Session visibility — act now through pinned Pi metadata + +### Primary-source evidence + +Both projects keep the user-facing invariant small: + +- OpenClaw caps recent sessions at 10 ([limit](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/status/summary.ts#L35-L38)), selects newest-first without retaining an unbounded sorted result ([selection](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/status/summary.ts#L183-L212)), and reads entries through a read-only listing boundary ([listing](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/status/summary.ts#L214-L227)). Its status model exposes a count and bounded recent list ([shape](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/status/types.ts#L86-L100)). +- Hermes defaults to source-scoped results, excludes the current session, hides unnamed rows unless requested, and enforces a final limit of 10 ([query policy](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/session_listing.py#L45-L88)). Its compact rendering includes a title, stable ID, short preview, and optional source ([rendering](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/hermes_cli/session_listing.py#L91-L117)). + +### Ziggy consequence + +Ziggy already routes local, channel, and automation sessions into explicit Pi-owned directories ([local manager](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/pi/pi-agent.ts#L423-L446), [automation session](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/automations.ts#L191-L209)). The pinned `@earendil-works/pi-coding-agent@0.82.0` exposes `SessionManager.listAll(customDirectory)`, returning stable ID, timestamps, message count, and path without `list(cwd, customDirectory)`'s exact-header-cwd filter ([pinned implementation](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L1631-L1670)). It lists one directory at a time and also returns transcript-derived preview fields; Ziggy must not project those fields. + +The small implementation is therefore available now: recursively discover directories under `/sessions`, reject symlinked `.jsonl` entries before invoking Pi, call `listAll` once per remaining leaf directory, map only path/ID/created/modified/message count, and sort the combined rows. Comparing regular discovered `.jsonl` paths with Pi's returned paths can surface files for which Pi cannot build metadata, but it cannot and should not impose stricter line-validity rules than Pi itself. No Ziggy session index or session-format parser is needed. + +### Do not copy + +Do not copy Hermes' session database/search/resume machinery or OpenClaw's aggregate model/token/task status. Ziggy needs visibility into Pi-owned sessions, not another session store. + +## 4. Graceful shutdown and admission — keep the invariant, defer the machinery + +### Primary-source evidence + +OpenClaw's large admission coordinator demonstrates four compact invariants: + +1. a distinct draining error rejects new work ([error](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/process/gateway-work-admission.ts#L11-L16)); +2. admitted roots are counted and their release is idempotent ([lease/release](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/process/gateway-work-admission.ts#L61-L105)); +3. restart drain is a one-way state transition and new root admission returns no lease once closed ([close and admission](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/process/gateway-work-admission.ts#L194-L211), [admit](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/process/gateway-work-admission.ts#L250-L270)); +4. shutdown waits for active roots with a bound and reports whether work drained ([bounded wait](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/process/gateway-work-admission.ts#L353-L400)). + +Hermes' durable drain marker illustrates machinery Ziggy does **not** need: it exists specifically because a separate dashboard cannot call the gateway and because state survives VM/container restarts ([contract and rationale](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/gateway/drain_control.py#L1-L48)). + +### Ziggy consequence + +Ziggy already has the cleanup half: + +- all gateways are `Effect.scoped` and register finalizers for sockets/chat handles ([Telegram](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/gateway.ts#L203-L212), [Slack](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/slack-gateway.ts#L209-L221), [Discord](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/discord-gateway.ts#L205-L218)); +- Slack and Discord message work is forked inside the gateway scope ([Slack](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/slack-gateway.ts#L260-L268), [Discord](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/discord-gateway.ts#L257-L265)); +- gateway-only teardown maps interrupt-only termination to exit 0 ([entrypoint teardown](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/main.ts#L317-L330)). + +What is missing is the ordering guarantee. Adding it is not yet a tiny shared utility: Telegram is sequential while Slack and Discord fork scoped turn fibers, and the current runtime interruption reaches those scopes before a new drain phase could complete. A correct change would alter signal admission, all three receive loops, in-flight accounting, timeout behavior, and the meaning of an interrupted turn. + +Keep the upstream invariant as a gate for future load-bearing gateways, but defer implementation until Ziggy promises either graceful completion or an explicit interrupted outcome. Scoped finalization already provides honest cancellation and resource cleanup; a partial drain wrapper without recovery semantics would add lifecycle state without making delivery durable. + +### Do not copy + +Do not copy OpenClaw's `AsyncLocalStorage`, nested/subordinate-root semantics, reversible host suspension, restart generations, or global singleton. Do not copy Hermes' dashboard endpoint, watcher, durable marker, VM epoch, or notification policy. Ziggy currently needs one one-way shutdown phase in each resident process. + +## 5. Automation claims and idempotency — defer, but make this a gate for scheduling + +### Primary-source evidence + +Hermes has the clearest failure semantics: + +- finite dispatch is claimed and persisted **before** execution, converting duplicate-prone one-shots to at-most-once dispatch; a crash after claim is made operator-visible rather than silently refired ([finite claim](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/jobs.py#L1839-L1911)); +- claim heartbeats compare the expected owner before refresh, so a stale runner cannot extend somebody else's claim ([heartbeat](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/jobs.py#L1921-L1950)); +- external fire claims happen under the store lock; a recurring job advances its next occurrence in the same critical section; malformed or future-dated timestamps cannot wedge the claim forever ([external fire](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/jobs.py#L2024-L2076)); +- its execution ledger explicitly is not a retry queue, bounds terminal history at 1,000, and makes terminal states immutable ([ledger contract/schema](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/executions.py#L1-L53), [terminal transition](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/executions.py#L175-L196)); +- after restart, only attempts whose exact owner is proved gone become `unknown`, and recovery does not schedule a retry ([recovery](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/cron/executions.py#L199-L233)). + +OpenClaw independently persists `runningAtMs` before execution ([activation](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/cron/service/run-admission.ts#L146-L189)) and converts a surviving running marker into an explicit interrupted failure on startup rather than pretending success or blindly replaying it ([startup repair](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/cron/service/startup-run-repair.ts#L52-L119)). + +### Ziggy consequence + +Current Ziggy has only explicit manual `wake`: it reads one automation, runs a cheap gate, opens a fresh Pi session, prompts, and delivers ([wake path](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/automations.ts#L170-L215)). There is no scheduler, webhook ingress, or competing automation dispatcher to claim. Adding a durable execution database now would be premature machinery and a new state surface. + +Before the first automatic trigger ships, require this minimal contract: + +1. Claim key is the automation ID plus the **trigger occurrence** (scheduled instant or external event ID), not merely the automation ID. +2. Persist the claim atomically before the model call or delivery. +3. A claimed occurrence is consumed and is not automatically replayed after a crash. +4. Bound state by retaining only the latest claimed recurring occurrence per automation when no run history is promised. + +That future work remains a focused claim-before-wake slice. Add `running`/terminal/`unknown` transitions only if Ziggy later promises run history, recovery diagnosis, or retries; those promises would make a ledger and its crash semantics part of the feature. + +### Do not copy + +Do not copy either project's scheduler, recurring-calendar engine, heartbeat service, multi-machine election, task ledger, retry policy, monitoring projection, or auto-disable machinery. Ziggy should add a claim only when an actual trigger source creates concurrency/replay risk. + +## 6. Skill requirement metadata and discovery — keep Ziggy's current boundary + +### Primary-source evidence + +OpenClaw's reusable metadata vocabulary is compact: namespaced skill metadata can declare all-required binaries, any-of binaries, environment variables, config paths, and OS constraints ([metadata shape](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/skills/types.ts#L20-L34)). It normalizes that manifest block at the frontmatter boundary ([metadata decoding](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/skills/loading/frontmatter.ts#L193-L213)), evaluates requirements into both `eligible` and structured `missing` reasons ([evaluation](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/shared/requirements.ts#L133-L218)), and preserves missing requirements in status instead of simply hiding the skill ([status](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/skills/discovery/status.ts#L260-L342)). Its discovery is explicitly bounded by candidate, loaded-skill, prompt, character, and file-size limits ([bounds](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/skills/loading/workspace.ts#L232-L249)). + +Hermes also namespaces conditional metadata under `metadata.hermes` ([conditions](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/agent/skill_utils.py#L665-L679)) and distinguishes `available`, `setup_needed`, and `unsupported` ([readiness states](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/tools/skills_tool.py#L225-L229)). Its discovery scans local before external roots so local names win ([precedence](https://github.com/NousResearch/hermes-agent/blob/42708f8bb39c9c2fc19146956699699bc3ea2da5/tools/skills_tool.py#L670-L723)). + +### Ziggy consequence + +Ziggy already implements the relevant discovery invariant: Profile skills first, sorted package skill roots next, and repository top-level skills last ([resource discovery](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/pi/resources.ts#L39-L78)). It then passes those explicit roots to Pi's resource loader with ambient skill discovery disabled and returns Pi diagnostics ([runtime composition](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/pi/pi-agent.ts#L502-L538)). This exactly matches the ownership rule that Pi parses skills. + +Therefore **keep the current implementation**. If users later need requirement readiness, the worthwhile invariant is namespaced optional metadata plus separate “discovered” and “eligible” states with structured missing reasons. That parsing/evaluation should be added to or exposed by Pi; Ziggy may render Pi's result. Until then, a Ziggy frontmatter parser would be a duplicate authority. + +### Do not copy + +Do not copy OpenClaw install recipes, remote-node capability evaluation, config-path probing, or large recursive scanner. Do not copy Hermes secret capture, setup prompting, skill database/cache, external-root lifecycle, or platform/environment policy. They are reference-scale product machinery, not Profile composition. + +## 7. Bounded inbound dedupe — keep what Ziggy has; defer two-phase semantics + +### Primary-source evidence + +OpenClaw's process-local implementation separates an in-flight claim from recently completed work. Its composite key includes agent/session scope, provider, account, peer/conversation, thread, and provider message ID ([key](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/auto-reply/reply/inbound-dedupe.ts#L38-L76)). Claims distinguish `duplicate`, `inflight`, and `claimed`; commit moves a successful claim into the recent cache, while release permits a safe retry ([claim/commit/release](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/auto-reply/reply/inbound-dedupe.ts#L78-L110)). The recent cache is bounded by both 20-minute TTL and 5,000 entries ([bounds](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/auto-reply/reply/inbound-dedupe.ts#L13-L30)); the underlying cache prunes expired and over-limit entries ([cache](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/infra/dedupe.ts#L26-L55)). Crucially, its dispatch error path releases only when replay is known safe; if a side effect may already have happened, it commits suppression ([error disposition](https://github.com/openclaw/openclaw/blob/50a30b7373d78fe8a63e597bd28138b6aa805765/src/auto-reply/reply/dispatch-from-config.ts#L75-L99)). + +### Ziggy consequence + +Ziggy already has a bounded, process-local FIFO `RecentIds` set ([implementation](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/bun/recent-ids.ts#L1-L24)). Slack records provider event IDs in a 1,000-entry cache before enqueue ([Slack](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/slack/socket.ts#L41-L44), [use](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/slack/socket.ts#L240-L265)); Discord does the same for message IDs ([Discord](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/discord/socket.ts#L47-L52), [use](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/adapters/discord/socket.ts#L282-L311)). Telegram discards startup backlog and then advances a monotonic provider offset ([offset handling](https://github.com/Yeshwanthyk/ziggy/blob/98988c29b7676b9fd0de1cc6c452598134b13fd0/src/application/gateway.ts#L250-L272)). + +That is adequate for current single-owner gateways. Keep it process-local and bounded; do not add a database. Ziggy's cache does mark an ID before the model/delivery result is known, so a failed turn suppresses redelivery. OpenClaw's two-phase design is the right refinement **only after** a concrete provider-redelivery test demonstrates the need, because deciding whether a failed model/delivery path is replay-safe is the real policy. Merely adding TTL is optional; count-bounding already prevents unbounded memory. + +### Do not copy + +Do not add distributed dedupe, durable replay logs, cross-restart suppression, or a generic event bus. A single Profile owner plus provider-native IDs/offsets keeps this a tiny adapter concern. + +## Recommended order + +1. **Session inventory first.** Use Pi's pinned public metadata API and add no writes, process, parser, or index. +2. **Narrow read-only doctor second.** Reuse local decoders; do not fix files or turn readiness into network health. +3. **Face-scoped duplicate-resident lease third, if duplicate consumers are a demonstrated operator risk.** Do not block different channels or local faces. +4. **No work now** on Profile-wide ownership, admission/drain state, automation ledgers, skill parsing, or durable dedupe. Revisit claims with the first scheduler/webhook, drain with a promised interrupted-work outcome, requirements in Pi, and two-phase inbound dedupe with a reproduced retry failure. + +## Bottom line + +The immediate low-complexity borrowing is **read-only visibility**: Pi-backed session inventory and a narrow decoder-backed doctor. A **face-scoped duplicate-resident lease** is the smallest ownership guard worth exploring; a Profile-wide lease would change current multi-channel behavior. Skill-root precedence, scoped cleanup, and bounded inbound suppression are already present. Drain state, session indexes/parsers, skill parsers, scheduler ledgers, and external control planes remain intentionally out. diff --git a/docs/research/pi-sdk-surface.md b/docs/research/pi-sdk-surface.md index 06c20e5..f7639d5 100644 --- a/docs/research/pi-sdk-surface.md +++ b/docs/research/pi-sdk-surface.md @@ -121,6 +121,33 @@ const sessions = SessionManager.create(cwd, "/var/lib/my-wrapper/sessions"); For non-persistent operation use `SessionManager.inMemory(cwd)`; it passes an empty session directory and `persist = false` (`:1567-1569`). +The pinned `0.82.0` package also exposes read-only metadata listing APIs: + +```ts +static list(cwd: string, sessionDir?: string, onProgress?: SessionListProgress): Promise +static listAll(sessionDir?: string, onProgress?: SessionListProgress): Promise +``` + +`SessionInfo` contains `path`, `id`, `cwd`, optional name/parent path, `created`, +`modified`, and `messageCount`, plus transcript-derived `firstMessage` and +`allMessagesText`. A Ziggy operator projection must map only the metadata fields and +must never expose those two transcript fields. + +Both methods inspect only direct `.jsonl` children for a custom directory and silently +omit files for which Pi cannot build metadata. `list(cwd, customDirectory)` additionally +filters by the normalized header `cwd`, so it can hide an otherwise Pi-readable old, +imported, or alternate-path-spelling session. Use `listAll(customDirectory)` for +Profile inventory. Pi intentionally skips malformed non-header lines, so Ziggy must +not claim stricter JSONL validation. Recursive inventory should discover leaf +directories without following directory symlinks, refuse symlinked `.jsonl` files +before invoking Pi for that leaf, call `listAll` for each safe leaf, and compare +regular discovered paths with returned paths to detect metadata omissions. Refs: +pinned source [`SessionInfo`](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L174-L188), +[tolerant line parsing](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L498-L524), +[metadata construction and error omission](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L687-L764), +[direct-directory listing](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L811-L840), +and [`list`/`listAll` behavior](https://github.com/earendil-works/pi/blob/083e61621276bff9f6faefab87ce07fcd98734e2/packages/coding-agent/src/core/session-manager.ts#L1631-L1708). + ## 4. DefaultResourceLoader Exact options type: