diff --git a/.github/workflows/gate.yml b/.github/workflows/gate.yml index c0b1b0c..2b2b6d7 100644 --- a/.github/workflows/gate.yml +++ b/.github/workflows/gate.yml @@ -96,9 +96,12 @@ jobs: - run: mix deps.get # The adapter is chosen at compile time (config/config.exs); this proves the build - # under TRINITY_DB=postgres links the Postgres adapter and nothing SQLite. + # under TRINITY_DB=postgres links the Postgres adapter and nothing SQLite. `--no-start` + # since slice 050: the check needs the compiled module, not a booted application, and + # Oban refuses to start before its table is migrated (run 35711666503, this step booted + # the application before `ecto.reset`). - name: The compiled adapter is Postgres - run: mix run -e 'Ecto.Adapters.Postgres = Trinity.Repo.__adapter__()' + run: mix run --no-start -e 'Ecto.Adapters.Postgres = Trinity.Repo.__adapter__()' - run: mix ecto.reset - run: mix test --exclude sqlite diff --git a/README.md b/README.md index 350ad29..1a726c2 100644 --- a/README.md +++ b/README.md @@ -46,8 +46,15 @@ merged with a merge commit and tagged `slice/NNN` (`git tag -l 'slice/*' | wc -l the assistant's own, and one that needs your approval waits for it on the permissions page while the client carries a sealed state it can retry with. A headless release runs the same tree as a server, in a container or under systemd (`docs/mcp-server.md`). - -Not there yet: scheduled tasks and MCP's authorization roles (M5a), messaging gateways and subagents (M5b), +- **Runs on a schedule.** Tasks on the `/tasks` page: a prompt, a persona, the skills to hint, + and when (a cron expression, a one-shot time, or a phrase like "every weekday at 9am" the + model turns into cron). Each run is a fresh conversation you can open, its result waits on + the page until you have read it, and the work is a durable job (Oban on the app's own + database) that survives a restart and retries a failed turn. The memory observer and a + curator that marks old memories stale and archives the untouched ones (never deleting) + run on the same queues; `/oban` shows the jobs. + +Not there yet: MCP's authorization roles (M5a), messaging gateways and subagents (M5b), the native desktop shell and signed releases (M6), executable skills in a sandbox (M7). `ROADMAP.md` carries the live status of every slice, and the [Milestones](#milestones) section below explains how to read it. @@ -91,7 +98,8 @@ full-text search, `/personas` and `/memory` the persona and its memory (with the the embedding model's download), `/skills` the skills, the changes waiting for your decision and the learn form, `/mcp` the MCP servers you connect to (their health and the tools they contribute), `/permissions` the rules and pending approvals (a server's question to you, when -one of its tools asks for input mid-call, is answered there too), `/settings` the export. `mix trinity.export` and `mix trinity.import` do what `/settings` does from a terminal; +one of its tools asks for input mid-call, is answered there too), `/tasks` the scheduled tasks and +their results, `/oban` the jobs (in development, or when configured), `/settings` the export. `mix trinity.export` and `mix trinity.import` do what `/settings` does from a terminal; `docs/backup.md` explains the archive. Semantic recall needs the local embedding model (91 MB, downloaded from the memory page on your diff --git a/ROADMAP.md b/ROADMAP.md index 82321e2..6220fb8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -51,7 +51,7 @@ standards register names the rows that ask for them. | 034 | Export, import, restore | 3 Memory | S | 030, 031 | approved | | 040 | Skills registry + agentskills.io format + progressive disclosure | 4 Skills | M | 020 | approved | | 041 | Skill self-management with staged approval + scanner | 4 Skills | M | 040, 021 | approved | -| 050 | Scheduler: Oban cron agent tasks with delivery targets | 5 Automation | M | 012 | planned | +| 050 | Scheduler: Oban cron agent tasks with delivery targets | 5 Automation | M | 012 | done | | 059 | MCP capability gap against beam_mcp, and the server seam probe | 6 MCP | S/M | 020 | approved | | 060 | MCP client: Trinity's thin driver (2026-07-28 preferred, 2025-11-25 compat, MRTR, Tasks) | 6 MCP | L | 059, 021 | approved | | 061 | MCP server (stateless 2026-07-28 + compat, MRTR approvals, headless profile) | 6 MCP | M | 060, 024 | approved | diff --git a/VERSIONS.md b/VERSIONS.md index 36d47a7..7af0aed 100644 --- a/VERSIONS.md +++ b/VERSIONS.md @@ -88,7 +88,8 @@ never pin a version hex marks as retired or vulnerable. | `ecto_sqlite3` | >= 0.0.0 | ✅ in `mix.lock` | Primary DB. FTS5 available. | | `postgrex` | >= 0.0.0 (optional) | ✅ in `mix.lock` | Secondary DB driver, `optional: true` so the desktop build carries none of it; compiled in only under `TRINITY_DB=postgres`, which the CI job proves. Added at Slice 010. Was one row with pgvector; pgvector keeps its own row below. | | `pgvector` | ~> 0.4.1 | ✅ in `mix.lock` | Vectors on the Postgres path: `memories.embedding_vector vector(384)` under an HNSW cosine index, searched by `Trinity.Memory.VectorStores.Pgvector` (Slice 032). The postgres job runs on the `pgvector/pgvector:pg17` image. Split from the postgrex row at Slice 010. | -| `oban` | ~> 2.24 | 🔍 not yet a dependency | Uses `Oban.Engines.Lite` on SQLite. ⚠️ Oban Pro Workflows/Smart engine are Postgres-only. Added at Slice 050. | +| `oban` | ~> 2.24 | ✅ in `mix.lock` | Uses `Oban.Engines.Lite` on SQLite and `Oban.Engines.Basic` on Postgres (the adapter is compile time, ADR-0002, so the engine is too). ⚠️ Oban Pro Workflows/Smart engine are Postgres-only. Added at Slice 050 (2.24.1 measured on hex.pm 2026-09-22, Apache-2.0). | +| `oban_web` | ~> 2.13 | ✅ in `mix.lock` | The Oban dashboard, a LiveView mounted at /oban (dev, and behind `config :trinity, :oban_web, true` elsewhere). Apache-2.0 on hex.pm since the 2.12 line (2.13.0 measured 2026-09-22); the plan of 2026-09-05 wrote it in when it was commercial. Added at Slice 050. | | `req` | ~> 0.5 | ✅ in `mix.lock` | HTTP client. | | `req_llm` | ~> 1.22 | ✅ in `mix.lock` | Provider layer (streaming, tools, structured output, usage). ⚠️ The pin was `~> 1.10` against a recorded latest of 1.10.0; the real latest was twelve minors ahead. Check event shapes against the current version at Slice 011, not against this file's prose. Added at Slice 011. | | `beam_mcp` | ~> 0.9 | ✅ in `mix.lock` | MCP server core, Apache-2.0, ADR-0007 decision 5 (owner decision 2026-09-08, recorded 2026-09-20). 0.9.0 on hex.pm (2026-09-22): the :server seam on both transports and the core server as a named behaviour, which slice 061's wrapper implements; standing before 1.0.0. Server side only: the client, MRTR and OAuth are Trinity's, above it. Added at Slice 059, bumped at 061. The earlier candidate list (anubis_mcp, fastest_mcp, gen_mcp) is history. | diff --git a/config/config.exs b/config/config.exs index efec080..99b5214 100644 --- a/config/config.exs +++ b/config/config.exs @@ -109,6 +109,30 @@ config :trinity, :fs, roots: [] # without evaluating the environment-specific imports below. import_config "llm.exs" +# Slice 050: Oban on the app's repo. The engine follows the compile-time adapter (Lite on +# SQLite, Basic on Postgres); the queues are small because the machine's model is one; the +# tick is the scheduler's one Cron entry and the curator its second; the pruner keeps the jobs +# table to a week; the lifeline rescues a job orphaned by a crash. config/test.exs sets +# `testing: :manual` so nothing runs on its own in the suite. +config :trinity, Oban, + repo: Trinity.Repo, + engine: + if(System.get_env("TRINITY_DB", "sqlite") == "postgres", + do: Oban.Engines.Basic, + else: Oban.Engines.Lite + ), + notifier: Oban.Notifiers.PG, + queues: [agent_tasks: 1, memory: 2, maintenance: 1], + plugins: [ + {Oban.Plugins.Cron, + crontab: [ + {"* * * * *", Trinity.Scheduler.Workers.Tick}, + {"0 3 * * *", Trinity.Memory.Curator} + ]}, + {Oban.Plugins.Pruner, max_age: 7 * 24 * 60 * 60}, + {Oban.Plugins.Lifeline, rescue_after: :timer.minutes(30)} + ] + # Slice 010, every environment, SQLite only (the Postgres adapter ignores keys it does not # know, and the CI matrix proves that). One writer: the pool has exactly one connection, so the # single-writer rule SQLite imposes is the pool's shape rather than a hope. Each pragma is named diff --git a/config/test.exs b/config/test.exs index 37c9965..f4e1934 100644 --- a/config/test.exs +++ b/config/test.exs @@ -127,9 +127,16 @@ if System.get_env("TRINITY_DB") == "postgres" do queue_target: 5_000, queue_interval: 30_000 else + # Slice 050: two connections, not one. At boot the sandbox is still in auto mode and the + # first long-lived process to query holds a connection until it exits; Oban verifies its + # migration at start through a raw checkout (`unboxed_run`) and waited 90 s on the one + # connection (run of 2026-09-22). Every test still shares its owner's single connection + # with every process it starts (shared mode), so 010's "writers queue on one connection" + # holds within a test; the second serves the boot-time check alone. config :trinity, Trinity.Repo, database: Path.expand("../trinity_test.db", __DIR__), pool: Ecto.Adapters.SQL.Sandbox, + pool_size: 2, queue_target: 5_000, queue_interval: 30_000 @@ -196,3 +203,8 @@ config :trinity, :skills, # needs); a short backoff so the reconnect test runs in seconds. config :trinity, :mcp_boot, false config :trinity, :mcp_client, backoff_ms: 50, max_backoff_ms: 400, connect_timeout: 5_000 + +# Slice 050: Oban runs nothing on its own in the suite; each test drives its jobs. +config :trinity, Oban, testing: :manual +# Slice 050: the dashboard route is mounted in the suite so its mount is a test. +config :trinity, :oban_web, true diff --git a/coverage.tsv b/coverage.tsv index 384a6a0..465a8a1 100644 --- a/coverage.tsv +++ b/coverage.tsv @@ -21,3 +21,4 @@ slice_id percent sha date 059 80.54 53c9091 2026-09-21 060 80.32 49f40a4 2026-09-21 061 80.05 c6b6faa 2026-09-22 +050 80.31 c01466d 2026-09-22 diff --git a/docs/01-architecture.md b/docs/01-architecture.md index a2528b3..75e9b95 100644 --- a/docs/01-architecture.md +++ b/docs/01-architecture.md @@ -89,7 +89,7 @@ without anything failing. | `Trinity.Receipts` | Local chain (one supervised writer per scope, ADR-0013), the signer seam (Ed25519, P-384, ML-DSA-87), key custody and the registry, checkpoints, the verifier, the alarm | Repo (`Repo.Receipts`) | | `Trinity.Memory` | Always-on tiers with their budget and consolidator (030), search (031), semantic store and retrieval (032), compaction (023) | LLM (summaries/embeddings), Repo | | `Trinity.Skills` | SKILL.md parsing, registry, loader, manager, scanner (as built at 040: parser, sources, registry, index, the three tools; at 041: staging, promotion, manager, scanner, diff, learn, `skill_manage` and `learn`) | Repo, Permissions, **Effects**, **Receipts**, Sandbox (as built at 041: Tools, Memory, Permissions, Receipts and LLM; Effects is not a dependency: the promotion is not a tool call, it writes its own effect receipt; Tools never depends on Skills) | -| `Trinity.Scheduler` | Oban workers for agent tasks, delivery | Sessions, Gateways, **Repo** | +| `Trinity.Scheduler` | Oban workers for agent tasks, delivery (as built at 050: the `tasks` and `task_runs` rows, `next_run_at` computed from a cron expression or a one-shot datetime, `Workers.Tick` (the one Cron plugin entry, every minute, enqueuing `Workers.RunTask` for what is due), `RunTask` (one turn in a fresh `origin: "cron"` session, the summary, the delivery), `Delivery` with `Delivery.Desktop`, `Parse.human/2` (a phrase to cron through the model, checked by Oban's parser); Oban itself on the app's repo, Lite on SQLite and Basic on Postgres, queues `agent_tasks`, `memory`, `maintenance`; the observer as `Trinity.Memory.ObserverWorker` and the curator as `Trinity.Memory.Curator` live in Memory) | Sessions, LLM, **Repo** (Gateways at 070) | | `Trinity.MCP` | Client manager, tool bridge, server (as built at 059: the boundary alone, holding the core's version, its JSON depth and its telemetry event names; at 060: `Servers` and `ServerConfig` (the rows), `Supervisor` and `Boot` (one `Client` per enabled row), `Client` with its `Wire` (the outbound request, the headers, the core's decoder and validator) and two transports (`Transport.Stdio`, a child on a Port; `Transport.HTTP`, one POST per request), `Bridge` (the one tool module every MCP tool runs through) and `Client.Auth` (062's seam, a static token at 060); at 061: `Server` (the module above the core, `@behaviour BeamMCP.Server`, handed to both transports through `:server`; `tools/call` answered through the membrane, everything else the core's), `Server.Catalog` (the exported tools, sorted), `Server.Exports` (the configured names, `:catalog` refused), `Server.Session` (the `origin: "mcp"` session with its own persona), `Server.Envelope` and `Server.Replay` (the sealed `requestState` and the nonce table), `Server.Auth.Local` (the static bearer, 062's seam on the server side), `Server.Plug` (the `POST /mcp` endpoint, mounted in `TrinityWeb.Endpoint` ahead of the parsers) and `Server.Stdio` with `mix trinity.mcp.stdio`) | Tools, **Effects**, **Permissions**, Memory (as built at 059: a top-level boundary, like `Trinity.Smoke`, with `deps: [Trinity, BeamMCP.JSON]`; the boundary compiler checks every call into the `beam_mcp` application and this boundary alone lists its modules; at 060 the deps are `[Trinity, BeamMCP.JSON, BeamMCP.Schema]`, Trinity's tools, permissions and receipts reached through `Trinity`'s exports, and `Trinity.Application` and `TrinityWeb` list `Trinity.MCP`; at 061 the core's `Server`, `Catalog`, `ToolSpec` and both transports are listed too, and `Trinity` exports `Tools.Registry` and `Effects.Runner` to it) | | `Trinity.Gateways` | Adapter behaviour, router, allowlists, pairing | Sessions, **Permissions**, PubSub | | `Trinity.Subagents` | Delegation, result collection | Sessions, Tools | diff --git a/docs/05-data-model.md b/docs/05-data-model.md index 7ce636d..1214e51 100644 --- a/docs/05-data-model.md +++ b/docs/05-data-model.md @@ -164,18 +164,35 @@ One row per request, the audit trail this slice owns; slice 024 reads it for dec | request | map, nullable | slice 060: a server's input request as sent (`kind`, `server`, `inputRequests`), when the approval is a multi-round-trip question and not a yes or no; the server's `requestState` is never here (it lives in the client process) | | answer | map, nullable | slice 060: the answer the decision carried, the revision's `inputResponses` map keyed as `inputRequests` were | -### tasks (Slice 050) +### tasks (Slice 050, as built) | column | type | notes | |---|---|---| | name | string | | -| schedule | string | cron expr or ISO one-shot | -| prompt | text | | -| persona_id | fk | | -| skill_names | {array, string} | | -| deliver_to | map | `{"gateway": "telegram", "ref": {...}}` or desktop | -| enabled | boolean | | -| last_run_at, next_run_at | | mirrors Oban state | -Execution history is in `oban_jobs` + a `task_runs` table (status, session_id, summary). +| kind | string | "cron" \| "once" | +| schedule | string | a five-field cron expression Oban's parser accepts (`@daily` and its kin too), or an ISO 8601 datetime for `once` | +| prompt | text | the user message of the run's turn | +| persona_id | fk personas, nullable | the default persona when unset | +| skill_names | {array, string} | hinted to the model in the prompt | +| deliver_to | map | `%{"kind" => "desktop"}` at 050; gateways name their kind at 070 | +| enabled | boolean | the tick enqueues enabled tasks only; a `once` task is disabled once enqueued | +| timeout_ms | integer | one run's bound (600 000 by default) | +| last_run_at, next_run_at | timestamp | `next_run_at` is computed by `Trinity.Scheduler` from the schedule, not mirrored from Oban: the tick (one Cron plugin entry a minute) enqueues what is due and advances it | + +### task_runs (Slice 050, as built) +| column | type | notes | +|---|---|---| +| task_id | fk tasks | | +| scheduled_at | timestamp | unique with `task_id`: a tick that fires twice enqueues once | +| session_id | fk sessions, nullable | the `origin: "cron"` session the turn ran in | +| status | string | "queued" \| "running" \| "retrying" \| "ok" \| "failed" | +| attempt | integer | Oban's attempt number | +| summary | text | the assistant's answer, its first 2 000 bytes | +| error | text | on `retrying` and `failed` | +| started_at, finished_at, delivered_at, seen_at | timestamp | `seen_at` is set from the tasks page | +Oban's own `oban_jobs` (and `oban_peers`) carry the jobs; the pruner keeps a week of them. + +The curator (slice 050) adds `stale_at` and `archived_at` to `memories` (docs above): stale is still +recalled and shown as such; archived leaves recall and stays in the row. ### usage_events (Slice 011; the ledger and budgets that read it are Slice 090) One row per completed call, as built at slice 011: diff --git a/docs/07-security-model.md b/docs/07-security-model.md index d06dda8..4365d85 100644 --- a/docs/07-security-model.md +++ b/docs/07-security-model.md @@ -137,6 +137,20 @@ root loads as any other and is not scanned (a follow-up in the slice's NOTES). - Luerl with reduction limits, no `os`/`io`/`require`, no filesystem; explicit host functions only. - Native/shell code is never "sandboxed" by the BEAM: the UI says so plainly when approving `:exec`. +## Scheduled tasks (Slice 050, as built) + +- **A run is an ordinary turn** in a fresh session with `origin: "cron"`, the persona the task names, the + task's prompt as the user message. It reaches tools through the same gate and the same membrane, and + every call it makes leaves the same receipts; the session's history is the run's record and the tasks + page links to it. +- **Nobody is at the desk.** A tool call that asks for approval in a cron session waits its expiry on the + permissions page (021, ten minutes by default) and is denied when it passes; the turn goes on and the + run's summary shows the denial. A rule the owner writes beforehand is what lets a scheduled task write. +- **The curator deletes nothing.** Marking stale is a query receipt; archiving is an effect receipt written + by the curator itself (as 041's promotion writes its own); both on the persona's memory scope. +- **Oban's dashboard** at `/oban` is in the browser pipeline with no authentication, as every page is + until 062; it is mounted in development and where `config :trinity, :oban_web` says so. + ## MCP client (Slice 060, as built) - **Every tool a server lists is a dynamic tool under `mcp::`**, registered through one bridge diff --git a/lib/trinity.ex b/lib/trinity.ex index 46042bd..98ff881 100644 --- a/lib/trinity.ex +++ b/lib/trinity.ex @@ -69,6 +69,11 @@ defmodule Trinity do Skills.Promotion, Skills.Manager, Skills.Learn, + # Slice 050: the tasks page reads and writes the scheduler. + Scheduler, + Scheduler.Task, + Scheduler.Run, + Scheduler.Parse, CorePolicy, Receipts, Receipts.Receipt, diff --git a/lib/trinity/application.ex b/lib/trinity/application.ex index 373e142..79e5cec 100644 --- a/lib/trinity/application.ex +++ b/lib/trinity/application.ex @@ -66,7 +66,10 @@ defmodule Trinity.Application do # Slice 061: the server's replay table, before the endpoint that serves /mcp. Trinity.MCP.Server.Replay, Trinity.MCP.Boot, - Trinity.Sessions.Supervisor + Trinity.Sessions.Supervisor, + # Slice 050: Oban after the sessions its workers drive (a run is a turn in a session), + # the engine chosen by the adapter (config.exs). + {Oban, Application.fetch_env!(:trinity, Oban)} ] ++ Trinity.Smoke.probe(Trinity.Smoke.argv()) ++ [ diff --git a/lib/trinity/memory/always_on.ex b/lib/trinity/memory/always_on.ex index 4f1c607..41a4242 100644 --- a/lib/trinity/memory/always_on.ex +++ b/lib/trinity/memory/always_on.ex @@ -50,9 +50,11 @@ defmodule Trinity.Memory.AlwaysOn do def entries(persona_id, session_id) do scopes = chain(persona_id, session_id) + # Slice 050: an archived entry (the curator's mark) is not part of what a session sees. from(e in Entry, where: - e.persona_id == ^persona_id and e.scope in ^scopes and e.tier in ^Entry.always_on_tiers(), + e.persona_id == ^persona_id and e.scope in ^scopes and e.tier in ^Entry.always_on_tiers() and + is_nil(e.archived_at), order_by: [e.tier, e.key] ) |> Repo.all() diff --git a/lib/trinity/memory/curator.ex b/lib/trinity/memory/curator.ex new file mode 100644 index 0000000..62c3fe9 --- /dev/null +++ b/lib/trinity/memory/curator.ex @@ -0,0 +1,120 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Memory.Curator do + @moduledoc """ + The memory curator (slice 050), a job on the `maintenance` queue the Cron plugin runs daily: + an entry untouched for `stale_days` (30) is marked stale (`stale_at`; it is still recalled, + and the pages show it as stale), one untouched for `archive_days` (90) is archived + (`archived_at`; it leaves recall and stays in the row). Nothing is ever deleted here. + "Untouched" is `last_used_at`, or `updated_at` when the entry was never used. + + Marking stale is a read the record keeps as one query receipt per persona on the memory + scope; archiving is a write, and each entry archived is an effect receipt written by the + curator itself (phase `done`, subject the entry), as slice 041's promotion writes its own: + the membrane's runner is for tool calls, and this is not one. The thresholds are + `config :trinity, :curator, stale_days:` and `archive_days:`. + """ + use Oban.Worker, queue: :maintenance, max_attempts: 3, unique: [period: 3_600] + + import Ecto.Query, only: [from: 2] + + alias Trinity.Memory.Entry + alias Trinity.Receipts + alias Trinity.Repo + + @default_stale_days 30 + @default_archive_days 90 + + @impl Oban.Worker + def perform(%Oban.Job{args: args}) do + now = + case Map.get(args, "now") do + nil -> + DateTime.utc_now() + + iso -> + {:ok, at, _} = DateTime.from_iso8601(iso) + at + end + + {:ok, curate(now)} + end + + @doc "Runs the curator once at `now`; returns the counts of entries marked stale and archived." + @spec curate(DateTime.t()) :: %{stale: non_neg_integer(), archived: non_neg_integer()} + def curate(now \\ DateTime.utc_now()) do + stale_before = DateTime.add(now, -days(:stale_days, @default_stale_days), :day) + archive_before = DateTime.add(now, -days(:archive_days, @default_archive_days), :day) + + archived = + for entry <- untouched_before(archive_before), is_nil(entry.archived_at), reduce: 0 do + n -> + {:ok, _} = Repo.update(Ecto.Changeset.change(entry, archived_at: now)) + archive_receipt(entry, now) + n + 1 + end + + stale = + untouched_before(stale_before) + |> Enum.filter(&(is_nil(&1.stale_at) and is_nil(&1.archived_at))) + + for entry <- stale, do: {:ok, _} = Repo.update(Ecto.Changeset.change(entry, stale_at: now)) + + stale + |> Enum.group_by(& &1.persona_id) + |> Enum.each(fn {persona_id, entries} -> stale_receipt(persona_id, entries, now) end) + + %{stale: length(stale), archived: archived} + end + + @doc "The chain scope the curator's receipts go to for a persona." + @spec scope(String.t()) :: String.t() + def scope(persona_id), do: "memory:" <> persona_id + + defp days(key, default), + do: Application.get_env(:trinity, :curator, []) |> Keyword.get(key, default) + + defp untouched_before(cutoff) do + Repo.all( + from(e in Entry, + where: + (not is_nil(e.last_used_at) and e.last_used_at < ^cutoff) or + (is_nil(e.last_used_at) and e.updated_at < ^cutoff) + ) + ) + end + + defp stale_receipt(persona_id, entries, now) do + Receipts.append(scope(persona_id), %{ + kind: "query", + subject: %{ + "persona_id" => persona_id, + "curator" => "stale", + "entries" => Enum.map(entries, & &1.id), + "at" => DateTime.to_iso8601(now) + }, + decision: %{"ok" => true, "count" => length(entries)}, + subject_ref: "curator:stale:#{persona_id}:#{DateTime.to_iso8601(now)}", + meta: %{} + }) + end + + defp archive_receipt(entry, now) do + Receipts.append(scope(entry.persona_id), %{ + kind: "effect", + subject: %{ + "persona_id" => entry.persona_id, + "entry_id" => entry.id, + "tier" => entry.tier, + "scope" => entry.scope, + "key" => entry.key, + "phase" => "done", + "curator" => "archive", + "at" => DateTime.to_iso8601(now) + }, + decision: %{"outcome" => "archived", "by" => "curator"}, + subject_ref: "curator:archive:#{entry.id}", + meta: %{} + }) + end +end diff --git a/lib/trinity/memory/entry.ex b/lib/trinity/memory/entry.ex index f782a20..bc2c187 100644 --- a/lib/trinity/memory/entry.ex +++ b/lib/trinity/memory/entry.ex @@ -31,6 +31,10 @@ defmodule Trinity.Memory.Entry do field :embedding, :binary field :embedding_model, :string field :embedding_dim, :integer + # Slice 050: the curator's marks. Stale is untouched for a while and still recalled; + # archived leaves recall and stays in the row (nothing is deleted by the curator). + field :stale_at, :utc_datetime_usec + field :archived_at, :utc_datetime_usec timestamps(type: :utc_datetime_usec) end diff --git a/lib/trinity/memory/observer.ex b/lib/trinity/memory/observer.ex index 6b61b20..c840b65 100644 --- a/lib/trinity/memory/observer.ex +++ b/lib/trinity/memory/observer.ex @@ -3,8 +3,9 @@ defmodule Trinity.Memory.Observer do @moduledoc """ Fills the semantic tier after a turn (slice 032). The session hands the completed turn's - messages to `observe/2`, which runs `run/2` under `Trinity.Memory.TaskSupervisor` so the - session is idle at once and a crash here is this task's alone. `run/2` asks the session's + messages to `observe/2`, which enqueues `Trinity.Memory.ObserverWorker` on Oban's `memory` + queue (slice 050; a task under `Trinity.Memory.TaskSupervisor` before it) so the session is + idle at once, a crash here is the job's alone, and a model error is retried. `run/2` asks the session's own model (`Trinity.LLM.generate_object/3`, the one the operator chose for the conversation: no text goes anywhere new, NOTES decision 5) for 0 to 3 durable facts, preferences or decisions, embeds them in one batch, drops each whose cosine to a memory already in the @@ -67,9 +68,21 @@ defmodule Trinity.Memory.Observer do @doc "Runs `run/2` under the memory task supervisor; `:off` when the observer is off." @spec observe(turn(), [message()]) :: {:ok, pid()} | :off + # Slice 050: a job on the `memory` queue rather than a task under the supervisor, so the + # extraction survives a restart and is retried on a model error (`Trinity.Memory.ObserverWorker`). + # The job carries the message ids, never their text. def observe(turn, messages) do if on?(turn) do - Task.Supervisor.start_child(Trinity.Memory.TaskSupervisor, fn -> run(turn, messages) end) + %{ + "turn" => %{ + "session_id" => turn.session_id, + "persona_id" => turn.persona_id, + "model" => turn.model + }, + "message_ids" => Enum.map(messages, & &1.id) + } + |> Trinity.Memory.ObserverWorker.new() + |> Oban.insert() else :off end diff --git a/lib/trinity/memory/observer_worker.ex b/lib/trinity/memory/observer_worker.ex new file mode 100644 index 0000000..b6f1f70 --- /dev/null +++ b/lib/trinity/memory/observer_worker.ex @@ -0,0 +1,39 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Memory.ObserverWorker do + @moduledoc """ + The observer as a job on the `memory` queue (slice 050): `Trinity.Memory.Observer.observe/2` + enqueues one per finished turn with the turn and the ids of its messages; this worker reloads + those messages from the session's history (the rows, not a copy of their text in the job) and + runs `Trinity.Memory.Observer.run/2` as the task supervisor did at 032. A model error is + `{:error, _}` and Oban retries under `max_attempts` (3) with its backoff; `:off` is a + discard, since nothing changes by waiting. + """ + use Oban.Worker, queue: :memory, max_attempts: 3 + + alias Trinity.Memory.Observer + alias Trinity.Sessions + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"turn" => turn, "message_ids" => ids}}) do + turn = %{ + session_id: turn["session_id"], + persona_id: turn["persona_id"], + model: turn["model"] + } + + wanted = MapSet.new(ids) + + messages = + turn.session_id + |> Sessions.history() + |> Enum.filter(&MapSet.member?(wanted, &1.id)) + |> Enum.map(&%{id: &1.id, role: &1.role, content: &1.content}) + + case Observer.run(turn, messages) do + {:ok, _entries} -> :ok + :off -> {:cancel, :off} + {:error, reason} -> {:error, reason} + end + end +end diff --git a/lib/trinity/memory/semantic.ex b/lib/trinity/memory/semantic.ex index c2a05a7..7602e7c 100644 --- a/lib/trinity/memory/semantic.ex +++ b/lib/trinity/memory/semantic.ex @@ -146,7 +146,9 @@ defmodule Trinity.Memory.Semantic do @spec entries(String.t(), [String.t()], keyword()) :: [Entry.t()] def entries(persona_id, scopes, opts \\ []) do from(e in Entry, - where: e.persona_id == ^persona_id and e.tier == "semantic" and e.scope in ^scopes, + where: + e.persona_id == ^persona_id and e.tier == "semantic" and e.scope in ^scopes and + is_nil(e.archived_at), order_by: [desc: e.inserted_at], limit: ^Keyword.get(opts, :limit, 200) ) diff --git a/lib/trinity/memory/vector_stores/brute.ex b/lib/trinity/memory/vector_stores/brute.ex index 96eee25..bc232fb 100644 --- a/lib/trinity/memory/vector_stores/brute.ex +++ b/lib/trinity/memory/vector_stores/brute.ex @@ -38,7 +38,7 @@ defmodule Trinity.Memory.VectorStores.Brute do from(e in Entry, where: e.persona_id == ^persona_id and e.tier == "semantic" and e.scope in ^scopes and - e.embedding_model == ^model and not is_nil(e.embedding) + e.embedding_model == ^model and not is_nil(e.embedding) and is_nil(e.archived_at) ) |> Repo.all() |> score(query) @@ -105,7 +105,7 @@ defmodule Trinity.Memory.VectorStores.Brute do from(e in Entry, where: e.persona_id == ^persona_id and e.tier == "semantic" and e.scope in ^scopes and - e.embedding_model == ^model and not is_nil(e.embedding) + e.embedding_model == ^model and not is_nil(e.embedding) and is_nil(e.archived_at) ), :count ) diff --git a/lib/trinity/memory/vector_stores/pgvector.ex b/lib/trinity/memory/vector_stores/pgvector.ex index 6a5ef21..9426eb4 100644 --- a/lib/trinity/memory/vector_stores/pgvector.ex +++ b/lib/trinity/memory/vector_stores/pgvector.ex @@ -47,7 +47,7 @@ defmodule Trinity.Memory.VectorStores.Pgvector do from(e in Entry, where: e.persona_id == ^persona_id and e.tier == "semantic" and e.scope in ^scopes and - e.embedding_model == ^model and not is_nil(e.embedding), + e.embedding_model == ^model and not is_nil(e.embedding) and is_nil(e.archived_at), order_by: fragment("embedding_vector <=> ?", ^vec), limit: ^k, select: {e, fragment("1 - (embedding_vector <=> ?)", ^vec)} diff --git a/lib/trinity/scheduler.ex b/lib/trinity/scheduler.ex new file mode 100644 index 0000000..e282bee --- /dev/null +++ b/lib/trinity/scheduler.ex @@ -0,0 +1,233 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler do + @moduledoc """ + Scheduled agent tasks (slice 050): a `tasks` row is a prompt on a schedule, a `task_runs` row one + execution of it, and Oban carries the work. The schedule is the tree's, not Oban's cron plugin's + (whose table is static): every task carries the time it next runs, computed here from its cron + expression or its one-shot datetime, and one plugin entry a minute (`Trinity.Scheduler.Workers.Tick`) + enqueues a `RunTask` job for each task that is due, unique on the task and the scheduled time, + then advances the task. `run_now/1` enqueues one at once. + + What this context offers: the rows (`list_tasks/0`, `get_task/1`, `create_task/1`, `update_task/2`, + `delete_task/1`), the schedule (`next_run_at/2`), the runs (`runs/2`, `recent_runs/1`, + `unseen_runs/0`, `mark_seen/1`) and the enqueueing (`enqueue_due/1`, `run_now/1`). The topic + `tasks` carries `{:task_run, %Run{}}` when a run is delivered. + """ + use Boundary, + deps: [Trinity, Trinity.Sessions, Trinity.LLM], + exports: [Task, Run, Delivery, Delivery.Desktop, Parse, Workers.Tick, Workers.RunTask] + + import Ecto.Query, only: [from: 2] + + alias Trinity.Repo + alias Trinity.Scheduler.{Run, Task} + alias Trinity.Scheduler.Workers.RunTask + + @topic "tasks" + + ## Tasks + + @doc "Every task, by name." + @spec list_tasks() :: [Task.t()] + def list_tasks, do: Repo.all(from(t in Task, order_by: t.name)) + + @doc "A task by id." + @spec get_task(String.t()) :: Task.t() | nil + def get_task(id), do: Repo.get(Task, id) + + @doc "Creates a task; `next_run_at` is computed from the schedule." + @spec create_task(map()) :: {:ok, Task.t()} | {:error, Ecto.Changeset.t()} + def create_task(attrs) do + %Task{} + |> Task.changeset(attrs) + |> put_next_run() + |> Repo.insert() + end + + @doc "Updates a task; a changed schedule recomputes `next_run_at`." + @spec update_task(Task.t(), map()) :: {:ok, Task.t()} | {:error, Ecto.Changeset.t()} + def update_task(%Task{} = task, attrs) do + task + |> Task.changeset(attrs) + |> put_next_run() + |> Repo.update() + end + + @doc "Removes a task and its runs." + @spec delete_task(Task.t()) :: {:ok, Task.t()} | {:error, Ecto.Changeset.t()} + def delete_task(%Task{} = task), do: Repo.delete(task) + + @doc "A changeset for a form." + @spec change_task(Task.t(), map()) :: Ecto.Changeset.t() + def change_task(%Task{} = task, attrs \\ %{}), do: Task.changeset(task, attrs) + + defp put_next_run(changeset) do + if changeset.valid? and + (Ecto.Changeset.changed?(changeset, :schedule) or + Ecto.Changeset.changed?(changeset, :kind) or + Ecto.Changeset.get_field(changeset, :next_run_at) == nil) do + kind = Ecto.Changeset.get_field(changeset, :kind) + schedule = Ecto.Changeset.get_field(changeset, :schedule) + Ecto.Changeset.put_change(changeset, :next_run_at, next_run_at(kind, schedule)) + else + changeset + end + end + + @doc """ + When a schedule next fires after `from` (now by default): the next matching minute of a cron + expression, or the one-shot datetime itself (nil once it is past). + """ + @spec next_run_at(String.t(), String.t(), DateTime.t()) :: DateTime.t() | nil + def next_run_at(kind, schedule, from \\ DateTime.utc_now()) + + def next_run_at("cron", schedule, from) do + case Task.parse("cron", schedule) do + {:ok, expr} -> + case Oban.Cron.Expression.next_at(expr, from) do + %DateTime{} = at -> usec(at) + :unknown -> nil + end + + {:error, _} -> + nil + end + end + + def next_run_at("once", schedule, from) do + case Task.parse("once", schedule) do + {:ok, at} -> if DateTime.compare(at, from) == :gt, do: usec(at), else: nil + {:error, _} -> nil + end + end + + # The columns carry microseconds; Oban's parser answers whole minutes and an ISO string + # whatever it carried. + defp usec(%DateTime{} = at), do: DateTime.add(at, 0, :microsecond) + + ## Enqueueing + + @doc """ + Enqueues a run for every enabled task whose `next_run_at` is at or before `now`, unique on the + task and that time (a tick that fires twice enqueues once), and advances each task: a cron task + to its next minute after `now`, a one-shot task to nothing (disabled). Returns the runs enqueued. + """ + @spec enqueue_due(DateTime.t()) :: [Run.t()] + def enqueue_due(now \\ DateTime.utc_now()) do + due = + Repo.all( + from(t in Task, where: t.enabled and not is_nil(t.next_run_at) and t.next_run_at <= ^now) + ) + + for task <- due, {:ok, run} <- [enqueue(task, task.next_run_at)] do + advance(task, now) + run + end + end + + @doc "Enqueues one run of a task now (the page's button); the scheduled time is now." + @spec run_now(Task.t()) :: {:ok, Run.t()} | {:error, term()} + def run_now(%Task{} = task), do: enqueue(task, usec(DateTime.utc_now())) + + # The run row first (unique on task and time: a second enqueue for the same time is refused + # here, and the job's own uniqueness is the second lock), then the job carrying its id. + defp enqueue(%Task{} = task, %DateTime{} = at) do + at = usec(at) + + with {:ok, run} <- Repo.insert(Run.changeset(%Run{}, %{task_id: task.id, scheduled_at: at})), + {:ok, _job} <- + %{"run_id" => run.id, "task_id" => task.id, "scheduled_at" => DateTime.to_iso8601(at)} + |> RunTask.new() + |> Oban.insert() do + {:ok, run} + else + {:error, %Ecto.Changeset{errors: [task_id: {_, [constraint: :unique, constraint_name: _]}]}} -> + {:error, :already_scheduled} + + {:error, %Ecto.Changeset{} = cs} -> + if Keyword.has_key?(cs.errors, :task_id) or Keyword.has_key?(cs.errors, :scheduled_at), + do: {:error, :already_scheduled}, + else: {:error, cs} + + {:error, reason} -> + {:error, reason} + end + end + + defp advance(%Task{kind: "cron"} = task, now), + do: + Repo.update!( + Ecto.Changeset.change(task, next_run_at: next_run_at("cron", task.schedule, now)) + ) + + defp advance(%Task{kind: "once"} = task, _now), + do: Repo.update!(Ecto.Changeset.change(task, next_run_at: nil, enabled: false)) + + ## Runs + + @doc "A task's runs, newest first (`limit:`)." + @spec runs(Task.t() | String.t(), keyword()) :: [Run.t()] + def runs(task_or_id, opts \\ []) do + id = if is_binary(task_or_id), do: task_or_id, else: task_or_id.id + limit = Keyword.get(opts, :limit, 50) + + Repo.all( + from(r in Run, where: r.task_id == ^id, order_by: [desc: r.scheduled_at], limit: ^limit) + ) + end + + @doc "A run by id." + @spec get_run(String.t()) :: Run.t() | nil + def get_run(id), do: Repo.get(Run, id) + + @doc "The latest runs across every task, newest first, with their tasks." + @spec recent_runs(keyword()) :: [Run.t()] + def recent_runs(opts \\ []) do + limit = Keyword.get(opts, :limit, 50) + Repo.all(from(r in Run, order_by: [desc: r.scheduled_at], limit: ^limit, preload: :task)) + end + + @doc "The finished runs the owner has not seen, oldest first (the notifications list)." + @spec unseen_runs() :: [Run.t()] + def unseen_runs do + Repo.all( + from(r in Run, + where: r.status in ["ok", "failed"] and is_nil(r.seen_at), + order_by: r.finished_at, + preload: :task + ) + ) + end + + @doc "Marks a run seen." + @spec mark_seen(Run.t() | String.t()) :: {:ok, Run.t()} | {:error, term()} + def mark_seen(%Run{} = run), + do: run |> Run.changeset(%{seen_at: DateTime.utc_now()}) |> Repo.update() + + def mark_seen(id) when is_binary(id) do + case get_run(id) do + nil -> {:error, :not_found} + run -> mark_seen(run) + end + end + + @doc "Updates a run (the workers' path)." + @spec update_run(Run.t(), map()) :: {:ok, Run.t()} | {:error, Ecto.Changeset.t()} + def update_run(%Run{} = run, attrs), do: run |> Run.changeset(attrs) |> Repo.update() + + ## Topic + + @doc "The PubSub topic a delivered run is broadcast on." + @spec topic() :: String.t() + def topic, do: @topic + + @doc "Subscribes the caller to `{:task_run, %Run{}}`." + @spec subscribe() :: :ok | {:error, term()} + def subscribe, do: Phoenix.PubSub.subscribe(Trinity.PubSub, @topic) + + @doc "Broadcasts a delivered run." + @spec broadcast(Run.t()) :: :ok + def broadcast(%Run{} = run), + do: Phoenix.PubSub.broadcast(Trinity.PubSub, @topic, {:task_run, run}) +end diff --git a/lib/trinity/scheduler/delivery.ex b/lib/trinity/scheduler/delivery.ex new file mode 100644 index 0000000..24fc05d --- /dev/null +++ b/lib/trinity/scheduler/delivery.ex @@ -0,0 +1,24 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Delivery do + @moduledoc """ + Where a finished run goes (slice 050). One implementation ships here, `Desktop`: the run is + marked delivered and broadcast, and the tasks page shows it until the owner marks it seen. + Gateways add implementations at 070; a task's `deliver_to` names one by `"kind"`, and the + worker resolves it through `for/1`. + """ + + alias Trinity.Scheduler.{Run, Task} + + @doc "Delivers a finished run of a task; the run comes back as updated." + @callback deliver(Run.t(), Task.t()) :: {:ok, Run.t()} | {:error, term()} + + @doc "The implementation for a task's `deliver_to`; the desktop for anything unknown." + @spec for(Task.t()) :: module() + def for(%Task{deliver_to: %{"kind" => kind}}) do + Application.get_env(:trinity, :deliveries, %{}) + |> Map.get(kind, __MODULE__.Desktop) + end + + def for(%Task{}), do: __MODULE__.Desktop +end diff --git a/lib/trinity/scheduler/delivery/desktop.ex b/lib/trinity/scheduler/delivery/desktop.ex new file mode 100644 index 0000000..d139d5c --- /dev/null +++ b/lib/trinity/scheduler/delivery/desktop.ex @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Delivery.Desktop do + @moduledoc """ + The desktop delivery (slice 050): the run is marked delivered and broadcast on the `tasks` + topic; the tasks page lists it among the runs not yet seen, and the bar shows their count. + """ + @behaviour Trinity.Scheduler.Delivery + + alias Trinity.Scheduler + + @impl true + def deliver(run, _task) do + with {:ok, run} <- Scheduler.update_run(run, %{delivered_at: DateTime.utc_now()}) do + Scheduler.broadcast(run) + {:ok, run} + end + end +end diff --git a/lib/trinity/scheduler/parse.ex b/lib/trinity/scheduler/parse.ex new file mode 100644 index 0000000..1628cd1 --- /dev/null +++ b/lib/trinity/scheduler/parse.ex @@ -0,0 +1,70 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Parse do + @moduledoc """ + A schedule from a phrase (slice 050): "every weekday at 9am" to `0 9 * * 1-5`. A phrase that + already parses as a cron expression is returned as it is; otherwise the model is asked for one + (`Trinity.LLM.generate_object/3` with a one-field schema) and its answer is accepted only when + Oban's parser accepts it, so what the page saves is always a schedule the tick can run. + """ + + alias Trinity.LLM + alias Trinity.LLM.Request + alias Trinity.Scheduler.Task + + @schema %{ + "type" => "object", + "properties" => %{ + "cron" => %{ + "type" => "string", + "description" => + "a five-field cron expression (minute hour day-of-month month day-of-week), UTC" + } + }, + "required" => ["cron"] + } + + @doc "The JSON Schema the model answers with." + @spec schema() :: map() + def schema, do: @schema + + @doc "A cron expression for a phrase, or why not; `model:` names the model (the default otherwise)." + @spec human(String.t(), keyword()) :: {:ok, String.t()} | {:error, term()} + def human(phrase, opts \\ []) when is_binary(phrase) do + phrase = String.trim(phrase) + + case Task.parse("cron", phrase) do + {:ok, _} -> {:ok, phrase} + {:error, _} -> ask(phrase, opts) + end + end + + defp ask("", _opts), do: {:error, :empty} + + defp ask(phrase, opts) do + request = + Request.new!(%{ + model: Keyword.get(opts, :model) || LLM.default_model(), + system: + "Turn the schedule the user describes into one five-field cron expression in UTC " <> + "(minute hour day-of-month month day-of-week). Answer with the expression alone in the cron field.", + messages: [%{role: "user", content: phrase}] + }) + + case LLM.generate_object(request, @schema, Keyword.take(opts, [:session_id])) do + {:ok, %{"cron" => cron}} when is_binary(cron) -> + cron = String.trim(cron) + + case Task.parse("cron", cron) do + {:ok, _} -> {:ok, cron} + {:error, reason} -> {:error, {:not_cron, cron, reason}} + end + + {:ok, other} -> + {:error, {:unreadable, other}} + + {:error, reason} -> + {:error, reason} + end + end +end diff --git a/lib/trinity/scheduler/run.ex b/lib/trinity/scheduler/run.ex new file mode 100644 index 0000000..0f21b15 --- /dev/null +++ b/lib/trinity/scheduler/run.ex @@ -0,0 +1,63 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Run do + @moduledoc """ + One row of `task_runs` (slice 050, docs/05): one scheduled execution of a task, unique on the + task and the time it was scheduled for. `status` goes `queued`, `running`, then `ok` or, through + `retrying`, `failed`; `session_id` is the `cron` session the turn ran in; `summary` the assistant's + answer (its head); `delivered_at` when the delivery ran and `seen_at` when the owner saw it on the + tasks page. + """ + use Ecto.Schema + import Ecto.Changeset + + @primary_key {:id, Trinity.UUID, autogenerate: true} + @foreign_key_type Trinity.UUID + @timestamps_opts [type: :utc_datetime_usec] + + @type t :: %__MODULE__{} + + @statuses ~w(queued running retrying ok failed) + + schema "task_runs" do + field :scheduled_at, :utc_datetime_usec + field :status, :string, default: "queued" + field :attempt, :integer, default: 0 + field :summary, :string + field :error, :string + field :started_at, :utc_datetime_usec + field :finished_at, :utc_datetime_usec + field :delivered_at, :utc_datetime_usec + field :seen_at, :utc_datetime_usec + belongs_to :task, Trinity.Scheduler.Task + belongs_to :session, Trinity.Sessions.SessionRow + timestamps() + end + + @doc "The statuses." + @spec statuses() :: [String.t()] + def statuses, do: @statuses + + @doc false + @spec changeset(t(), map()) :: Ecto.Changeset.t() + def changeset(run, attrs) do + run + |> cast(attrs, [ + :task_id, + :scheduled_at, + :session_id, + :status, + :attempt, + :summary, + :error, + :started_at, + :finished_at, + :delivered_at, + :seen_at + ]) + |> validate_required([:task_id, :scheduled_at, :status]) + |> validate_inclusion(:status, @statuses) + |> unique_constraint([:task_id, :scheduled_at]) + |> foreign_key_constraint(:task_id) + end +end diff --git a/lib/trinity/scheduler/task.ex b/lib/trinity/scheduler/task.ex new file mode 100644 index 0000000..64da931 --- /dev/null +++ b/lib/trinity/scheduler/task.ex @@ -0,0 +1,101 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Task do + @moduledoc """ + One row of `tasks` (slice 050, docs/05): a prompt to run on a schedule. `kind` is `cron` (the + `schedule` a five-field cron expression Oban's parser accepts, `@daily` and its kin included) or + `once` (the `schedule` an ISO 8601 datetime in UTC). `next_run_at` is the tick's cue, computed + by `Trinity.Scheduler` from the schedule; `skill_names` are hinted to the model; `deliver_to` + names where the result goes (`%{"kind" => "desktop"}` at this slice; gateways at 070); + `timeout_ms` bounds one run. + """ + use Ecto.Schema + import Ecto.Changeset + + @primary_key {:id, Trinity.UUID, autogenerate: true} + @foreign_key_type Trinity.UUID + @timestamps_opts [type: :utc_datetime_usec] + + @type t :: %__MODULE__{} + + @kinds ~w(cron once) + @name_pattern ~r/^\S.{0,119}$/ + + schema "tasks" do + field :name, :string + field :kind, :string, default: "cron" + field :schedule, :string + field :prompt, :string + field :skill_names, {:array, :string}, default: [] + field :deliver_to, :map, default: %{"kind" => "desktop"} + field :enabled, :boolean, default: true + field :timeout_ms, :integer, default: 600_000 + field :last_run_at, :utc_datetime_usec + field :next_run_at, :utc_datetime_usec + belongs_to :persona, Trinity.Sessions.Persona + has_many :runs, Trinity.Scheduler.Run + timestamps() + end + + @doc "The kinds." + @spec kinds() :: [String.t()] + def kinds, do: @kinds + + @doc false + @spec changeset(t(), map()) :: Ecto.Changeset.t() + def changeset(task, attrs) do + task + |> cast(attrs, [ + :name, + :kind, + :schedule, + :prompt, + :persona_id, + :skill_names, + :deliver_to, + :enabled, + :timeout_ms, + :last_run_at, + :next_run_at + ]) + |> validate_required([:name, :kind, :schedule, :prompt]) + |> validate_format(:name, @name_pattern, message: "must be 1 to 120 characters, not blank") + |> validate_inclusion(:kind, @kinds) + |> validate_number(:timeout_ms, greater_than: 0, less_than_or_equal_to: 3_600_000) + |> validate_schedule() + |> foreign_key_constraint(:persona_id) + end + + # A cron schedule must parse (Oban's parser, the one the tick uses); a one-shot must be an + # ISO 8601 datetime with an offset. + defp validate_schedule(changeset) do + kind = get_field(changeset, :kind) + + validate_change(changeset, :schedule, fn :schedule, schedule -> + case {kind, parse(kind, schedule)} do + {_, {:ok, _}} -> [] + {"cron", {:error, reason}} -> [schedule: "is not a cron expression: #{reason}"] + {"once", {:error, reason}} -> [schedule: "is not an ISO 8601 datetime: #{reason}"] + _ -> [] + end + end) + end + + @doc "Parses a schedule for its kind: a cron expression, or a datetime." + @spec parse(String.t(), String.t()) :: {:ok, term()} | {:error, String.t()} + def parse("cron", schedule) when is_binary(schedule) do + case Oban.Cron.Expression.parse(schedule) do + {:ok, expr} -> {:ok, expr} + {:error, exception} -> {:error, Exception.message(exception)} + end + end + + def parse("once", schedule) when is_binary(schedule) do + case DateTime.from_iso8601(schedule) do + {:ok, at, _offset} -> {:ok, at} + {:error, reason} -> {:error, Atom.to_string(reason)} + end + end + + def parse(_kind, _schedule), do: {:error, "unknown kind"} +end diff --git a/lib/trinity/scheduler/workers/run_task.ex b/lib/trinity/scheduler/workers/run_task.ex new file mode 100644 index 0000000..0fdc950 --- /dev/null +++ b/lib/trinity/scheduler/workers/run_task.ex @@ -0,0 +1,181 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Workers.RunTask do + @moduledoc """ + One run of a task (slice 050): a fresh session with `origin: "cron"` titled after the task and + the persona the task names, one turn with the task's prompt (and a line naming the skills it + hints), the wait for the turn to end on the session's topic (the task's `timeout_ms`), the + assistant's answer as the run's summary, the delivery. A turn that ends in the session's error + state, times out or raises is `{:error, reason}`: Oban retries under `max_attempts` (3) with its + backoff, and the run is `retrying` until the last attempt marks it `failed` with the error. + + A tool call that asks for approval in a cron session has nobody at the desk: the request waits + its expiry on the permissions page (021, ten minutes by default) and the turn goes on with the + denial, which the summary shows. That is the honest outcome, recorded in docs/07. + """ + use Oban.Worker, + queue: :agent_tasks, + max_attempts: 3, + unique: [fields: [:args], keys: [:run_id]] + + require Logger + + alias Trinity.Scheduler + alias Trinity.Scheduler.{Delivery, Run, Task} + alias Trinity.Sessions + + @summary_bytes 2_000 + + @impl Oban.Worker + def perform(%Oban.Job{args: %{"run_id" => run_id}, attempt: attempt, max_attempts: max}) do + with %Run{} = run <- Scheduler.get_run(run_id) || {:error, :run_missing}, + %Task{} = task <- Scheduler.get_task(run.task_id) || {:error, :task_missing} do + {:ok, run} = + Scheduler.update_run(run, %{ + status: "running", + attempt: attempt, + started_at: DateTime.utc_now() + }) + + case execute(run, task) do + {:ok, session_id, summary} -> + finish(run, task, session_id, summary) + + {:error, reason} -> + fail(run, reason, attempt, max) + end + else + {:error, reason} -> {:cancel, reason} + end + end + + @impl Oban.Worker + def timeout(%Oban.Job{args: %{"run_id" => run_id}}) do + case Scheduler.get_run(run_id) do + %Run{task_id: task_id} -> + case Scheduler.get_task(task_id) do + %Task{timeout_ms: ms} -> ms + 5_000 + nil -> :infinity + end + + nil -> + :infinity + end + end + + # The session's process broadcasts `:idle` once when it starts; that one is drained before + # the message goes, so the wait below is for the turn's own. + defp execute(run, task) do + with {:ok, session} <- session(run, task), + :ok <- Sessions.subscribe(session.id), + {:ok, _pid} <- Sessions.ensure_started(session.id), + :ok <- drain_start(session.id), + {:ok, _message} <- Sessions.send_user_message(session.id, prompt(task)), + :ok <- await_idle(session.id, task.timeout_ms) do + {:ok, session.id, summary(session.id)} + end + end + + defp drain_start(session_id) do + receive do + {:session, ^session_id, {:state, :idle}} -> :ok + after + 2_000 -> :ok + end + end + + # One session per run, so the history a run leaves is its own; the row records the task and + # the run in `origin_ref`. + defp session(run, task) do + Sessions.create_session(%{ + persona_id: task.persona_id || Sessions.default_persona().id, + origin: "cron", + title: task.name, + origin_ref: %{"task_id" => task.id, "run_id" => run.id} + }) + end + + defp prompt(%Task{prompt: prompt, skill_names: []}), do: prompt + + defp prompt(%Task{prompt: prompt, skill_names: names}), + do: prompt <> "\n\n(Use the skills " <> Enum.join(names, ", ") <> " where they apply.)" + + # The turn ends at :idle; the session's :error state is a failure; the timeout is the task's. + defp await_idle(session_id, timeout_ms) do + receive do + {:session, ^session_id, {:state, :idle}} -> :ok + {:session, ^session_id, {:state, :error}} -> {:error, :session_error} + {:session, ^session_id, {:error, reason}} -> {:error, {:turn, reason}} + {:session, ^session_id, _other} -> await_idle(session_id, timeout_ms) + after + timeout_ms -> + _ = Sessions.cancel_turn(session_id) + {:error, :timeout} + end + end + + defp summary(session_id) do + session_id + |> Sessions.history() + |> Enum.filter(&(&1.role == "assistant")) + |> List.last() + |> case do + nil -> + "" + + %{content: content} when is_binary(content) -> + binary_part(content, 0, min(byte_size(content), @summary_bytes)) + end + end + + defp finish(run, task, session_id, summary) do + {:ok, run} = + Scheduler.update_run(run, %{ + status: "ok", + session_id: session_id, + summary: summary, + error: nil, + finished_at: DateTime.utc_now() + }) + + _ = Trinity.Repo.update(Ecto.Changeset.change(task, last_run_at: DateTime.utc_now())) + + case Delivery.for(task).deliver(run, task) do + {:ok, _} -> :ok + {:error, reason} -> Logger.warning("task #{task.name}: delivery failed: #{inspect(reason)}") + end + + :ok + end + + defp fail(run, reason, attempt, max) do + last? = attempt >= max + text = describe(reason) + + {:ok, run} = + Scheduler.update_run(run, %{ + status: if(last?, do: "failed", else: "retrying"), + error: text, + finished_at: if(last?, do: DateTime.utc_now()) + }) + + if last?, do: deliver_failure(run) + {:error, text} + end + + defp deliver_failure(run) do + case Scheduler.get_task(run.task_id) do + %Task{} = task -> Delivery.for(task).deliver(run, task) + nil -> :ok + end + end + + defp describe(:timeout), do: "the turn did not finish within the task's timeout" + defp describe(:session_error), do: "the session ended the turn in error" + defp describe({:turn, reason}), do: "the turn failed: " <> inspect(reason) + + defp describe(%Ecto.Changeset{} = cs), + do: "the session could not be created: " <> inspect(cs.errors) + + defp describe(other), do: inspect(other) +end diff --git a/lib/trinity/scheduler/workers/tick.ex b/lib/trinity/scheduler/workers/tick.ex new file mode 100644 index 0000000..014a889 --- /dev/null +++ b/lib/trinity/scheduler/workers/tick.ex @@ -0,0 +1,29 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Scheduler.Workers.Tick do + @moduledoc """ + The minute tick (slice 050): the one Oban Cron plugin entry the scheduler needs. Every minute + it enqueues a `RunTask` for each task whose `next_run_at` has passed and advances the task; + a tick that runs twice for the same minute enqueues nothing the second time (the run is unique + on the task and the scheduled time). Cheap enough that a missed minute (the VM was down) + catches up on the next: a task due while Trinity was off runs once when it returns, at the + time it was due for. + """ + use Oban.Worker, queue: :maintenance, max_attempts: 1, unique: [period: 30] + + @impl Oban.Worker + def perform(%Oban.Job{args: args}) do + now = + case Map.get(args, "now") do + nil -> + DateTime.utc_now() + + iso -> + {:ok, at, _} = DateTime.from_iso8601(iso) + at + end + + runs = Trinity.Scheduler.enqueue_due(now) + {:ok, %{enqueued: length(runs)}} + end +end diff --git a/lib/trinity/versions.ex b/lib/trinity/versions.ex index b4a99b4..63e24c6 100644 --- a/lib/trinity/versions.ex +++ b/lib/trinity/versions.ex @@ -151,7 +151,14 @@ defmodule Trinity.Versions do pin: "~> 2.24", lock: "oban", note: - "Uses `Oban.Engines.Lite` on SQLite. ⚠️ Oban Pro Workflows/Smart engine are Postgres-only. Added at Slice 050." + "Uses `Oban.Engines.Lite` on SQLite and `Oban.Engines.Basic` on Postgres (the adapter is compile time, ADR-0002, so the engine is too). ⚠️ Oban Pro Workflows/Smart engine are Postgres-only. Added at Slice 050 (2.24.1 measured on hex.pm 2026-09-22, Apache-2.0)." + }, + %{ + name: "oban_web", + pin: "~> 2.13", + lock: "oban_web", + note: + "The Oban dashboard, a LiveView mounted at /oban (dev, and behind `config :trinity, :oban_web, true` elsewhere). Apache-2.0 on hex.pm since the 2.12 line (2.13.0 measured 2026-09-22); the plan of 2026-09-05 wrote it in when it was commercial. Added at Slice 050." }, %{name: "req", pin: "~> 0.5", lock: "req", note: "HTTP client."}, %{ diff --git a/lib/trinity_web/live/session_live/index.ex b/lib/trinity_web/live/session_live/index.ex index 6545ee9..fc3560d 100644 --- a/lib/trinity_web/live/session_live/index.ex +++ b/lib/trinity_web/live/session_live/index.ex @@ -48,6 +48,9 @@ defmodule TrinityWeb.SessionLive.Index do <.link id="mcp-link" navigate={~p"/mcp"} class="text-meta opacity-70 hover:opacity-100"> {gettext("mcp")} + <.link id="tasks-link" navigate={~p"/tasks"} class="text-meta opacity-70 hover:opacity-100"> + {gettext("tasks")} +
assign( + page_title: gettext("Tasks"), + editing: nil, + form: new_form(), + viewing: nil, + suggestion: nil + ) + |> load()} + end + + defp new_form(attrs \\ %{"kind" => "cron", "timeout_ms" => "600000", "enabled" => "true"}), + do: to_form(attrs, as: :task) + + defp load(socket) do + assign(socket, + tasks: Scheduler.list_tasks(), + unseen: Scheduler.unseen_runs(), + personas: Trinity.Personas.list(), + runs: if(socket.assigns[:viewing], do: Scheduler.runs(socket.assigns.viewing), else: []) + ) + end + + @impl true + def handle_info({:task_run, _run}, socket), do: {:noreply, load(socket)} + def handle_info(_other, socket), do: {:noreply, socket} + + @impl true + def handle_event("save", %{"task" => params}, socket) do + attrs = attrs(params) + + result = + case socket.assigns.editing do + nil -> Scheduler.create_task(attrs) + %Task{} = task -> Scheduler.update_task(task, attrs) + end + + case result do + {:ok, task} -> + {:noreply, + socket + |> assign(editing: nil, form: new_form(), suggestion: nil) + |> put_flash(:info, gettext("Task %{name} saved.", name: task.name)) + |> load()} + + {:error, changeset} -> + {:noreply, assign(socket, form: to_form(params, as: :task, errors: errors(changeset)))} + end + end + + def handle_event("change", %{"task" => params}, socket), + do: {:noreply, assign(socket, form: to_form(params, as: :task))} + + def handle_event("suggest", %{"phrase" => phrase}, socket) do + case Parse.human(phrase) do + {:ok, cron} -> + params = + socket.assigns.form.params |> Map.put("schedule", cron) |> Map.put("kind", "cron") + + {:noreply, assign(socket, form: to_form(params, as: :task), suggestion: cron)} + + {:error, reason} -> + {:noreply, + put_flash( + socket, + :error, + gettext("No schedule from that: %{reason}", reason: inspect(reason)) + )} + end + end + + def handle_event("edit", %{"id" => id}, socket) do + case Scheduler.get_task(id) do + nil -> + {:noreply, socket} + + task -> + params = %{ + "name" => task.name, + "kind" => task.kind, + "schedule" => task.schedule, + "prompt" => task.prompt, + "persona_id" => task.persona_id, + "skill_names" => Enum.join(task.skill_names, ", "), + "timeout_ms" => Integer.to_string(task.timeout_ms), + "enabled" => to_string(task.enabled) + } + + {:noreply, assign(socket, editing: task, form: to_form(params, as: :task))} + end + end + + def handle_event("cancel_edit", _params, socket), + do: {:noreply, assign(socket, editing: nil, form: new_form())} + + def handle_event("run_now", %{"id" => id}, socket) do + with %Task{} = task <- Scheduler.get_task(id), {:ok, _run} <- Scheduler.run_now(task) do + {:noreply, socket |> put_flash(:info, gettext("Run queued.")) |> load()} + else + {:error, :already_scheduled} -> + {:noreply, put_flash(socket, :error, gettext("A run for this moment is already queued."))} + + _ -> + {:noreply, put_flash(socket, :error, gettext("Not queued."))} + end + end + + def handle_event("toggle", %{"id" => id}, socket) do + with %Task{} = task <- Scheduler.get_task(id), + {:ok, _} <- Scheduler.update_task(task, %{enabled: !task.enabled}) do + {:noreply, load(socket)} + else + _ -> {:noreply, put_flash(socket, :error, gettext("Not changed."))} + end + end + + def handle_event("remove", %{"id" => id}, socket) do + with %Task{} = task <- Scheduler.get_task(id), {:ok, _} <- Scheduler.delete_task(task) do + {:noreply, socket |> assign(viewing: nil) |> load()} + else + _ -> {:noreply, put_flash(socket, :error, gettext("Not removed."))} + end + end + + def handle_event("view", %{"id" => id}, socket), + do: {:noreply, socket |> assign(viewing: id) |> load()} + + def handle_event("hide", _params, socket), + do: {:noreply, socket |> assign(viewing: nil) |> load()} + + def handle_event("seen", %{"id" => id}, socket) do + _ = Scheduler.mark_seen(id) + {:noreply, load(socket)} + end + + def handle_event("seen_all", _params, socket) do + for run <- socket.assigns.unseen, do: Scheduler.mark_seen(run) + {:noreply, load(socket)} + end + + # The form's strings to the row's attributes: skills split on commas, blanks dropped, the + # persona nil when unset. + defp attrs(params) do + %{ + name: String.trim(params["name"] || ""), + kind: params["kind"] || "cron", + schedule: String.trim(params["schedule"] || ""), + prompt: params["prompt"] || "", + persona_id: blank_to_nil(params["persona_id"]), + skill_names: (params["skill_names"] || "") |> String.split(~r/[\s,]+/, trim: true), + timeout_ms: to_int(params["timeout_ms"], 600_000), + enabled: params["enabled"] in ["true", "on", true] + } + end + + defp blank_to_nil(nil), do: nil + defp blank_to_nil(s), do: if(String.trim(s) == "", do: nil, else: s) + + defp to_int(nil, default), do: default + + defp to_int(s, default) do + case Integer.parse(String.trim(s)) do + {i, ""} -> i + _ -> default + end + end + + defp errors(changeset), do: for({field, {msg, _}} <- changeset.errors, do: {field, {msg, []}}) + + defp stamp(nil), do: gettext("never") + defp stamp(%DateTime{} = at), do: Calendar.strftime(at, "%Y-%m-%d %H:%M UTC") + + @impl true + def render(assigns) do + ~H""" + + <:bar> + {gettext("Tasks")} + {length(@tasks)} + + {ngettext("%{count} result to read", "%{count} results to read", length(@unseen))} + + +
+
+
+

{gettext("Results")}

+ +
+
    +
  • +
    + {r.task.name} + {stamp(r.finished_at)} + {r.status} + + <.link + :if={r.session_id} + navigate={~p"/s/#{r.session_id}"} + class="text-meta opacity-70 hover:opacity-100" + >{gettext("open the conversation")} + +
    +

    {r.summary || r.error}

    +
  • +
+
+ +
+

{gettext("Scheduled tasks")}

+

+ {gettext("No task yet. Add one below: a prompt, and when to run it.")} +

+
    +
  • +
    + + {t.kind} · {t.schedule} + {gettext( + "disabled" + )} + + + + + +
    +

    + {gettext("next")}: {stamp(t.next_run_at)} · {gettext("last")}: {stamp(t.last_run_at)} +

    +
    +
    + {gettext("Runs")} + +
    +

    {gettext("None yet.")}

    + + + + + + + + + +
    + {stamp(r.scheduled_at)} + + {r.status} + +
    + {r.summary || r.error} +
    +
    + <.link + :if={r.session_id} + navigate={~p"/s/#{r.session_id}"} + class="opacity-70 hover:opacity-100" + >{gettext("conversation")} +
    +
    +
  • +
+
+ +
+

+ {if @editing, + do: gettext("Edit %{name}", name: @editing.name), + else: gettext("Add a task")} +

+ <.form + for={@form} + id="task-form" + phx-change="change" + phx-submit="save" + class="flex flex-col gap-2" + > +
+ + +
+
+ + +
+ +
+ + + +
+

+ {field}: {msg} +

+
+ + +
+ +
+ + + {@suggestion} +
+
+
+
+ """ + end +end diff --git a/lib/trinity_web/router.ex b/lib/trinity_web/router.ex index 7398fea..7e70441 100644 --- a/lib/trinity_web/router.ex +++ b/lib/trinity_web/router.ex @@ -40,6 +40,8 @@ defmodule TrinityWeb.Router do live "/skills", SkillsLive, :index # Slice 060: the MCP servers and their health. live "/mcp", MCPLive, :index + # Slice 050: scheduled tasks, their runs and the results to read. + live "/tasks", TasksLive, :index # Slice 024: a session's receipt chain, and the boot receipt of this run. live "/s/:id/receipts", ReceiptsLive, :session live "/receipts/boot", ReceiptsLive, :boot @@ -51,6 +53,18 @@ defmodule TrinityWeb.Router do # pipe_through :api # end + # Slice 050: Oban's dashboard, in development and wherever `config :trinity, :oban_web` is + # set (the pages carry no authentication yet; the same rule as every other page). + if Application.compile_env(:trinity, :dev_routes) || + Application.compile_env(:trinity, :oban_web, false) do + import Oban.Web.Router + + scope "/" do + pipe_through :browser + oban_dashboard("/oban", csp_nonce_assign_key: :csp_nonce) + end + end + # Enable LiveDashboard in development if Application.compile_env(:trinity, :dev_routes) do # If you want to use the LiveDashboard in production, you should put diff --git a/mix.exs b/mix.exs index 5d2173b..48754be 100644 --- a/mix.exs +++ b/mix.exs @@ -172,6 +172,10 @@ defmodule Trinity.MixProject do # Slice 059: the MCP server core (ADR-0007 decision 5), reached only through the # Trinity.MCP boundary; the slice measures its gap, 060 and 061 build on it. {:beam_mcp, "~> 0.9"}, + # Slice 050: durable scheduled work. Oban's Lite engine on SQLite, the Basic engine on + # Postgres; Oban Web is the dashboard, Apache-2.0 on hex since its 2.12 line. + {:oban, "~> 2.24"}, + {:oban_web, "~> 2.13"}, # Slice 013 (owner decision, 2026-09-20): the linux package builds mdex's NIF from # source for musl (MDEX_NATIVE_BUILD=1 and TRINITY_NIF_TARGET in config/config.exs), # because neither precompiled artifact loads in Burrito's musl ERTS (NOTES finding 13). diff --git a/mix.lock b/mix.lock index c160927..0915b4d 100644 --- a/mix.lock +++ b/mix.lock @@ -60,6 +60,9 @@ "nx": {:hex, :nx, "0.13.1", "f96311a573dcd57d11ff889958ee518f52f7c16dbf6d22191020d322887856fc", [:mix], [{:complex, "~> 0.7", [hex: :complex, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "20556ea36c1e9732c1d87eebbf7c0773517e876e92af4ff592b974f80db0b9d9"}, "nx_image": {:hex, :nx_image, "0.1.2", "0c6e3453c1dc30fc80c723a54861204304cebc8a89ed3b806b972c73ee5d119d", [:mix], [{:nx, "~> 0.4", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "9161863c42405ddccb6dbbbeae078ad23e30201509cc804b3b3a7c9e98764b81"}, "nx_signal": {:hex, :nx_signal, "0.2.0", "e1ca0318877b17c81ce8906329f5125f1e2361e4c4235a5baac8a95ee88ea98e", [:mix], [{:nx, "~> 0.6", [hex: :nx, repo: "hexpm", optional: false]}], "hexpm", "7247e5e18a177a59c4cb5355952900c62fdeadeb2bad02a9a34237b68744e2bb"}, + "oban": {:hex, :oban, "2.24.1", "2a609c54697ad2c44ba339df30491df2a40eda0758c95b5b879426e4e478bd1f", [:mix], [{:ecto_sql, "~> 3.10", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:ecto_sqlite3, "~> 0.9", [hex: :ecto_sqlite3, repo: "hexpm", optional: true]}, {:igniter, "~> 0.5", [hex: :igniter, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:myxql, "~> 0.7", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.20", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 1.3", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ef8482472cf198554400b7f8e36a0ffee75c3a64de425d5c7ee625d271925ac7"}, + "oban_met": {:hex, :oban_met, "1.3.1", "1d8f5ab69e94802c8fbc154dfb3de346528f55f08724f591d3c327bf4fca2ef8", [:mix], [{:oban, "~> 2.21", [hex: :oban, repo: "hexpm", optional: false]}], "hexpm", "abf49524a305bad413fef7cad798fcd3a2d42ec739a0736475582c5dc8cbca1c"}, + "oban_web": {:hex, :oban_web, "2.13.0", "9c37ab5c2b976681a647dead0de8a67139620e9ff636d041b5324f90c34467b3", [:mix], [{:jason, "~> 1.2", [hex: :jason, repo: "hexpm", optional: false]}, {:oban, "~> 2.24", [hex: :oban, repo: "hexpm", optional: false]}, {:oban_met, "~> 1.1", [hex: :oban_met, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.7", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 3.3 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:phoenix_live_view, "~> 1.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}], "hexpm", "78af08760ca455ab7d19a311c77de07a0e7fb512d7d68043185cee04d51fbee2"}, "owl": {:hex, :owl, "0.13.1", "1ec4a5dea170465f0e90c502c203079224516bc0cbd599281c8667b3c6ef8848", [:mix], [{:ucwidth, "~> 0.2", [hex: :ucwidth, repo: "hexpm", optional: true]}], "hexpm", "351e768af8f2edc575cdaab1a5a2f6d6381be591758a026c701c703145508a0c"}, "pgvector": {:hex, :pgvector, "0.4.1", "f45b3be1ad37efa944b418757da5414eb59085997c6de85ea83ac5b5b82701af", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: true]}, {:nx, "~> 0.5 or ~> 1.0", [hex: :nx, repo: "hexpm", optional: true]}, {:postgrex, ">= 0.0.0", [hex: :postgrex, repo: "hexpm", optional: false]}], "hexpm", "2bb5c468ff6c435941ca326b408a0a01a1526dcd8f2c3c478be44be091ad88e4"}, "phoenix": {:hex, :phoenix, "1.8.14", "9279cbbcd755ac8d5f42d206915e7df13b9ab49096b98114f1c99ff908e5b42e", [:mix], [{:bandit, "~> 1.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.7", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 2.2", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "2782ff375824b2b5e41561fbae4764ee7b875af6898483bca49f24a9d1e37816"}, diff --git a/priv/repo/migrations/20260922100000_add_oban.exs b/priv/repo/migrations/20260922100000_add_oban.exs new file mode 100644 index 0000000..f814259 --- /dev/null +++ b/priv/repo/migrations/20260922100000_add_oban.exs @@ -0,0 +1,10 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Repo.Migrations.AddOban do + use Ecto.Migration + + # Slice 050: Oban's jobs table. `Oban.Migration` picks the adapter's migration (SQLite or + # Postgres) from the repo, so one file serves both legs (ADR-0002). + def up, do: Oban.Migration.up() + def down, do: Oban.Migration.down() +end diff --git a/priv/repo/migrations/20260922100100_create_tasks.exs b/priv/repo/migrations/20260922100100_create_tasks.exs new file mode 100644 index 0000000..045d7a0 --- /dev/null +++ b/priv/repo/migrations/20260922100100_create_tasks.exs @@ -0,0 +1,55 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Repo.Migrations.CreateTasks do + use Ecto.Migration + + # Slice 050 (docs/05): scheduled agent tasks and their runs. A run is unique on the task and + # the time it was scheduled for, so a tick that fires twice enqueues once. + def change do + create table(:tasks, primary_key: false) do + add :id, :binary_id, primary_key: true + add :name, :string, null: false + add :kind, :string, null: false, default: "cron" + add :schedule, :string, null: false + add :prompt, :text, null: false + add :persona_id, references(:personas, type: :binary_id, on_delete: :nilify_all) + add :skill_names, {:array, :string}, null: false, default: [] + add :deliver_to, :map, null: false, default: %{} + add :enabled, :boolean, null: false, default: true + add :timeout_ms, :integer, null: false, default: 600_000 + add :last_run_at, :utc_datetime_usec + add :next_run_at, :utc_datetime_usec + timestamps(type: :utc_datetime_usec) + end + + create index(:tasks, [:enabled, :next_run_at]) + + create table(:task_runs, primary_key: false) do + add :id, :binary_id, primary_key: true + add :task_id, references(:tasks, type: :binary_id, on_delete: :delete_all), null: false + add :scheduled_at, :utc_datetime_usec, null: false + add :session_id, references(:sessions, type: :binary_id, on_delete: :nilify_all) + add :status, :string, null: false, default: "queued" + add :attempt, :integer, null: false, default: 0 + add :summary, :text + add :error, :text + add :started_at, :utc_datetime_usec + add :finished_at, :utc_datetime_usec + add :delivered_at, :utc_datetime_usec + add :seen_at, :utc_datetime_usec + timestamps(type: :utc_datetime_usec) + end + + create unique_index(:task_runs, [:task_id, :scheduled_at]) + create index(:task_runs, [:status]) + create index(:task_runs, [:seen_at]) + + # The curator's marks (never a delete). + alter table(:memories) do + add :stale_at, :utc_datetime_usec + add :archived_at, :utc_datetime_usec + end + + create index(:memories, [:archived_at]) + end +end diff --git a/scripts/dev_tasks.sh b/scripts/dev_tasks.sh new file mode 100755 index 0000000..1e5c090 --- /dev/null +++ b/scripts/dev_tasks.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +# Slice 050's screenshots and GIF (AC2, AC6, AC7): the test environment's application on its own +# databases (both repos off the sandbox pool; the receipts database migrated before boot), Oban +# with its queues and plugins running (the suite's manual mode overridden), the observer on, and +# the fake provider scripted with the summary a "daily summary of my notes dir" task would +# produce, then a plain turn. Prints PORT=. `mix assets.build` first; remove stale +# priv/static/assets/**/*.gz. +cd "$(dirname "$0")/.." +export MIX_ENV=test +exec systemd-run --user --scope -p MemoryMax=32G --quiet -- mix run --no-start --no-halt -e ' +repo = Application.get_env(:trinity, Trinity.Repo) |> Keyword.delete(:pool) |> Keyword.put(:database, "trinity_screenshots.db") +Application.put_env(:trinity, Trinity.Repo, repo) +rrepo = Application.get_env(:trinity, Trinity.Repo.Receipts) |> Keyword.delete(:pool) |> Keyword.put(:database, "trinity_screenshots_receipts.db") +Application.put_env(:trinity, Trinity.Repo.Receipts, rrepo) +endpoint = Application.get_env(:trinity, TrinityWeb.Endpoint) |> Keyword.put(:check_origin, false) +Application.put_env(:trinity, TrinityWeb.Endpoint, endpoint) +Application.put_env(:trinity, :mcp_boot, false) +Application.put_env(:trinity, Oban, Application.get_env(:trinity, Oban) |> Keyword.delete(:testing)) +Application.put_env(:trinity, :memory, Keyword.put(Application.get_env(:trinity, :memory, []), :observer, true)) +{:ok, _} = Application.ensure_all_started(:ecto_sql) +for r <- [Trinity.Repo, Trinity.Repo.Receipts] do + {:ok, pid} = r.start_link() + Ecto.Migrator.run(r, :up, all: true) + GenServer.stop(pid) +end +{:ok, _} = Application.ensure_all_started(:trinity) +summary = "Your notes directory has 14 files, 3 changed since yesterday. New: **hiring.md** (two candidates to call back), **oban-notes.md** (the Lite engine needs its own migration). Changed: **todo.md** lost four items and gained one: renew the domain before Friday." +final = fn text -> Enum.flat_map(String.split(text), &[{:text_delta, &1 <> " "}, {:sleep, 25}]) ++ [{:usage, %{input_tokens: 40, output_tokens: 60}}, {:done, :stop}] end +Trinity.LLM.Providers.Fake.scripts([final.(summary)]) +Trinity.LLM.Providers.Fake.object(%{"memories" => [%{"kind" => "fact", "body" => "The domain renewal is due before Friday.", "confidence" => 0.8}]}) +{:ok, {_, port}} = TrinityWeb.Endpoint.server_info(:http) +IO.puts("PORT=#{port}") +' diff --git a/slices/050-scheduler-oban/NOTES.md b/slices/050-scheduler-oban/NOTES.md new file mode 100644 index 0000000..772c89d --- /dev/null +++ b/slices/050-scheduler-oban/NOTES.md @@ -0,0 +1,161 @@ +# Slice 050: NOTES + +## Read before code, 2026-09-22 + +Tree at `eecb7b8` on `main` (061 approved). What this slice joins: 012's Session (a session row with an +`origin` from the closed vocabulary, `cron` among them; `send_user_message/2`; the `{:state, :idle}` +broadcast on its topic when a turn ends), 032's observer (`Trinity.Memory.Observer.observe/2` runs `run/2` +under a task supervisor after each turn; off in the suite), 024's receipts (a query receipt for a read, an +effect receipt written by the promotion path itself at 041: the pattern a non-tool write takes), 041's +skills (the prompt's skills index; a task names skills to hint), and the compile-time adapter choice +(ADR-0002: SQLite primary with a pool of one writer, Postgres on the CI leg). + +Measured today: oban 2.24.1 on hex.pm (Apache-2.0), oban_web 2.13.0 (Apache-2.0: the 2026-09-05 plan wrote +it in as a commercial package and it is not one any more), oban_met 1.3.1 (Apache-2.0, oban_web's +dependency). `Oban.Engines.Lite` is the SQLite engine and `Oban.Migration` picks `Oban.Migrations.SQLite` +by the repo's adapter; `Oban.Cron.Expression.parse/1` is Oban's own cron parser, so no crontab package is +added. Oban's open-source Cron plugin takes a static table at configuration time: a task's own schedule +cannot be a plugin entry, and Oban Pro's dynamic cron is out of scope, so the design below ticks. + +Decisions at G1, open to the owner's veto in this turn: + +- **Schedules are the tree's, not the plugin's.** A `tasks` row carries a cron expression or a one-shot + ISO datetime and a `next_run_at` the context computes with Oban's parser. One Cron plugin entry, every + minute, runs `Trinity.Scheduler.Workers.Tick`, which enqueues a `RunTask` job for every enabled task + whose `next_run_at` has passed, unique on `(task_id, scheduled_at)` (the SLICE's idempotency key, as + Oban's `unique` option on the job's args and as the `task_runs` unique index), and advances + `next_run_at`; a one-shot task is disabled once enqueued. AC1's "inserts a cron entry" is read as: the + task carries its cron and the tick enqueues it when due; the test drives the tick at a chosen time. +- **A run is one turn in a fresh `origin: "cron"` session** titled after the task, the persona the task + names (the default when none), the prompt as the user message with a line naming the task's skills + (the skills index already shows them; the line asks the model to use them). The worker subscribes to + the session's topic, sends the message, waits for `:idle` (the task's `timeout_ms`, 10 minutes by + default), reads the last assistant message as the summary (its first 2,000 bytes) and marks the run. + A tool call that asks in a cron session has nobody at the desk: the approval waits its expiry (021, + ten minutes by default) and the turn goes on with the denial, which the run's summary shows. +- **Failure is Oban's retry, then a failed run.** A turn that raises, times out or ends in the session's + error state returns `{:error, reason}`; Oban retries under the worker's `max_attempts` (3) with its + backoff; the run row is `failed` with the error text on the last attempt (`attempt == max_attempts`), + `retrying` before that. +- **Delivery is a behaviour with one implementation.** `Trinity.Scheduler.Delivery` (`deliver/2`); + `Trinity.Scheduler.Delivery.Desktop` marks the run `delivered_at`, broadcasts `{:task_run, run}` on the + `tasks` topic, and the `/tasks` page shows the runs not yet seen as the notifications list with a count + in the bar (the pending-approvals badge's shape); `seen_at` is set from the page. Gateways add + implementations at 070; `deliver_to` on the row is `%{"kind" => "desktop"}` until then. +- **The observer becomes a job on the `memory` queue** (032 is approved, so AC6 stands): `observe/2` + enqueues `Trinity.Memory.ObserverWorker` with the turn and the message ids; the worker reloads the + messages from the session's history and calls `run/2` as before; `max_attempts` 3. `run/2` keeps its + contract (032 AC3's test calls it), and `observer: false` keeps the suite quiet. +- **The curator on the `maintenance` queue**, a Cron plugin entry daily at 03:00: a `memories` row + untouched (`last_used_at`, else `updated_at`) for 30 days is marked stale (`stale_at`, a query receipt on + the persona's memory scope), one untouched for 90 days is archived (`archived_at`, an effect receipt + written by the curator as 041's promotion writes its own, subject the entry, phase `done`); nothing is + deleted; the thresholds are `config :trinity, :curator` with those defaults. Archived entries leave + recall (the always-on chain and the semantic search filter them); stale ones stay and are shown as such. + Two new columns on `memories`, no new table. +- **Oban runs on the app's repo with its pool of one writer.** Oban Lite polls its queues (the producers' + `poll_interval`, 1 s by default) through the same pool as the app; the SLICE's risk (contention with the + app's writes under the 010 stress test) is measured at G3 and the interval set from the measurement. + Queues: `agent_tasks: 1` (one turn at a time: the machine's model is one), `memory: 2`, `maintenance: 1`. + `Oban.Plugins.Pruner` keeps the jobs table to a week; `Oban.Plugins.Lifeline` rescues an orphaned + executing job after a crash. +- **Oban Web at `/oban`** in dev, and elsewhere behind `config :trinity, :oban_web, true`, in the browser + pipeline (the pages carry no authentication yet; the same rule as the rest of the pages). +- **The human schedule helper** asks the session's default model for `%{"cron" => …}` with + `Trinity.LLM.generate_object/3` and accepts the answer only when `Oban.Cron.Expression.parse/1` does; a + phrase that already parses as cron is returned as is, and the page's form offers "suggest". + +## G1 plan, 2026-09-22 + +Branch `slice/050-scheduler-oban`; ROADMAP row 050 to `in_progress` in this commit; `oban ~> 2.24`, +`oban_web ~> 2.13` in mix.exs and `VERSIONS.md`. Each line names its test. + +1. Migrations: Oban's (`Oban.Migration.up/1`, both adapters), `tasks` (docs/05: `name`, `schedule`, + `kind` (`cron | once`), `prompt`, `persona_id`, `skill_names`, `deliver_to`, `enabled`, `timeout_ms`, + `last_run_at`, `next_run_at`), `task_runs` (`task_id`, `scheduled_at`, `session_id`, `status` (`queued | + running | retrying | ok | failed`), `attempt`, `summary`, `error`, `started_at`, `finished_at`, + `delivered_at`, `seen_at`; unique on `(task_id, scheduled_at)`), and `memories` gains `stale_at` and + `archived_at`. Schemas `Trinity.Scheduler.Task` and `Trinity.Scheduler.Run`. +2. Oban in `Trinity.Application` (engine by adapter, the queues, the plugins: Cron with the tick and the + curator, Pruner, Lifeline), `testing: :manual` in the suite; `Trinity.Scheduler` context (`tasks` CRUD, + `schedule/1` computing `next_run_at`, `run_now/1` enqueuing a `RunTask` at once, `runs/1`). Tests AC1 + (a task with `*/5 * * * *` carries the next five-minute boundary; the tick at that time enqueues + `RunTask` with the task and the time, asserted with `Oban.Testing`; `perform_job` runs it). +3. `Trinity.Scheduler.Workers.Tick` and `Trinity.Scheduler.Workers.RunTask` (the session, the turn, the + wait, the run row, the delivery). Tests AC2 (the automatic half: a `FakeProvider` turn, the `cron` + session, the run row's summary, the delivery's broadcast and `delivered_at`) and AC3 (a turn whose + provider raises: `{:error, _}`, the run `retrying`, then `failed` with the error on the last attempt). +4. `Trinity.Scheduler.Delivery` and `Delivery.Desktop`; the `tasks` topic. Tested under AC2. +5. `Trinity.Scheduler.Parse.human/2`: the LLM call and the parser check. Test AC4 with the fake's + scripted object (`0 9 * * 1-5` for "every weekday at 9am"; a scripted answer that does not parse is + refused; a phrase that is already cron passes through). +6. AC5: a test with its own Oban instance (`testing: :disabled`, the Lite engine on the sandbox + connection, `agent_tasks: 1`): a `RunTask` job inserted, the instance stopped, started again, the job + runs and its run row is `ok`. +7. `Trinity.Memory.ObserverWorker` on the `memory` queue and `observe/2` enqueuing it. Test: `observe/2` + enqueues the job with the turn and the ids; `perform_job` on it runs `run/2` (the 032 test's fake). +8. `Trinity.Memory.Curator` on the `maintenance` queue. Tests: an entry 31 days old gets `stale_at` and a + query receipt; one 91 days old gets `archived_at` and an effect receipt; a fresh one is untouched; + nothing is deleted; an archived entry is absent from `AlwaysOn.entries/2` and from the semantic search. +9. `/tasks` LiveView: the list, the form (name, schedule with "suggest" from a phrase, prompt, persona, + skills, enabled), run now, the run history with session links, the notifications list and the bar's + count; `/oban` mounted in dev and behind the flag. LiveView tests for the list, the form and run now. +10. docs/05 as built, docs/01 (the `Trinity.Scheduler` boundary), docs/07 (a cron session's approvals + expire unanswered), README (`/tasks`, `/oban`, the Automates bullet's first half); PROOF; the ROADMAP + row; the stress measurement with Oban running (the SLICE's risk). + +Manual verification queue: **AC2** (a screenshot of the notifications list after a run), **AC6** (the +observer's job in Oban Web, a screenshot), **AC7** (a "daily summary of my notes dir" task created on the +page, run now, the result: a GIF). All three are taken here with the dev-run scripts and the headless +browser, as 060 and 061's were; the owner's queue is to look. + +Not built here: Oban Pro, multi-step workflows, gateway deliveries (070), a task's own approval channel +(a cron session's approvals expire, recorded above), Oban Web behind authentication (the pages have none +yet; 062). + +## Deviations while building, 2026-09-22 + +- **The suite's SQLite pool has two connections, not one.** Oban verifies its migration at start through + a raw checkout (`Oban.Migration.verify_migrated!/1`, `Sandbox.unboxed_run`), and at that moment the + sandbox is still in auto mode with the first long-lived boot process holding the one connection until + it exits: Oban waited 90 s and the application failed to start. `config/test.exs` gives the test pool a + second connection for that check; every test still shares its owner's single connection with every + process it starts, which is 010's property, and 010's pool-size test now reads the shipped + configuration from `config/config.exs` under the production environment (one connection, as before). +- **AC1's "inserts a cron entry" is the tick's design**, decided at G1: the task carries its cron and its + next time; the tick enqueues it when due. The test drives the tick at the task's boundary. +- **032's observer-through-session test drains the memory queue** instead of waiting for a task's row, + since the observer is a job now and the suite runs Oban manually. +- **Oban Web's LiveView cannot mount in the suite** (it awaits `Oban.Met`, which Oban does not start in + manual testing mode); the route's mount is asserted on the router, and the page is AC6's screenshot. +- **The dashboard route is mounted in the suite** through `config :trinity, :oban_web, true` in + config/test.exs, so the mount is a test and not a dev-only line. + +## Findings at G3, 2026-09-22 + +1. **The SLICE's risk, measured: Oban's polling against the app's single writer.** `scripts/stress_010.exs` + in the dev environment, Oban's three queues polling at the Lite engine's default interval (1 s) and the + plugins running: `appends=4000 errors=0 wall_ms=1559 appends_per_s=2565.7 integrity=ok + wal_bytes=4165352 gapless=true sqlite=3.53.4`, against 3012.1 and 3420.3 appends per second at 010 + (its two runs, on that tree, without Oban). No error, no lost append, the WAL the same size; a + throughput a fifth lower under a write storm no session produces. The poll interval stays at the + default; the number is here to set it from if a later slice needs to. +2. **Three workers, one dashboard.** After one run of the "daily summary" task with the observer on, + `/oban/jobs` lists the completed `Tick` (maintenance), `RunTask` (agent_tasks) and `ObserverWorker` + (memory, the message ids in its args and no text), each once (`proof/ac6-2-oban-jobs.png`). +3. **The session's first `:idle` is the process's, not the turn's.** A fresh session broadcasts `:idle` + when it starts; the worker waiting for the turn's end read that one and finished with no summary. + `RunTask` drains the start's idle before it sends the message (the SessionCase's `start_drained` + is the same lesson from 012). +4. **`Oban.Cron.Expression.next_at/2` answers whole minutes** and an ISO string carries whatever it + carried; the columns are microsecond, so the context normalises every datetime it stores. +5. **Two flakes recorded on 061's branch did not recur here**: the suite ran clean three times locally + (500, 501 with the new tests); CI's numbers are in the closing correction. + +## Follow-ups + +- 070: gateway deliveries implement `Trinity.Scheduler.Delivery` and name their kind in `deliver_to`. +- 062: Oban Web and the tasks page behind authentication with the rest of the pages. +- A task's approvals: a cron session's tool call that asks is denied at expiry; a delivery that carries + the pending approval to a gateway (070) or a longer expiry per task is the next step if it bites. +- The poll interval (finding 1) is a knob without a need yet. diff --git a/slices/050-scheduler-oban/PROOF.md b/slices/050-scheduler-oban/PROOF.md new file mode 100644 index 0000000..fbf812b --- /dev/null +++ b/slices/050-scheduler-oban/PROOF.md @@ -0,0 +1,173 @@ +# Proof for slice 050: Scheduler: Oban cron agent tasks with delivery targets + +Agent: Trinity · Coding Agent · Date: 2026-09-22 · Branch: slice/050-scheduler-oban · Final commit: (the commit carrying this file; named in the closing correction) + +## Summary +Oban on the app's own database (the Lite engine on SQLite, Basic on Postgres), a `tasks` row per +scheduled prompt with the time it next runs computed by `Trinity.Scheduler` from a cron expression or a +one-shot datetime, one Cron plugin entry a minute (`Workers.Tick`) enqueuing a `RunTask` for what is due +(unique on the task and the scheduled time), each run a turn in a fresh `origin: "cron"` session with the +run row carrying the summary, Oban's retry then a failed run, the desktop delivery (a results list on +`/tasks` until read), a phrase-to-cron helper checked by Oban's parser, the observer as a job on the +`memory` queue, a curator on the `maintenance` queue that marks and archives and never deletes, the +`/tasks` page and Oban Web at `/oban`. The SLICE's risk measured (Oban polling against the single +writer: a fifth off the write storm's throughput, no error). Five deviations and five findings in NOTES.md. + +## Gate +``` +$ mix gate (tree c01466d, before this file and the coverage row were added, this machine, OTP 28.5.0.5, Elixir 1.20.4, under a 32 GiB cgroup) +2657 mods/funs, found no issues. +... SCAN COMPLETE ... +No retired or security advisory packages found +No vulnerabilities found. +versions.verify: OK. 106 locked packages, none disagreeing with 54 pins +versions.gen: VERSIONS.md already matches Trinity.Versions and mix.lock +trinity.version_form: OK +trinity.names: OK over 662 tracked files +trinity.secrets.scan: OK over 662 files +trinity.reuse: OK. Every commentable tracked file carries an SPDX header +Result: 501 passed, 18 excluded +trinity.coverage: 061 80.05% vs 060 80.32%: OK +plan_check: PASS +exit=0 +``` +CI: named in the closing correction. + +## Tests +``` +$ mix test --cover (tree c01466d) +Result: 501 passed, 18 excluded +| 77.19% | Trinity.Scheduler | +| 77.97% | Trinity.Scheduler.Workers.RunTask | +| 83.33% | Trinity.Scheduler.Workers.Tick | +| 88.24% | Trinity.Scheduler.Task | +| 81.25% | Trinity.Scheduler.Parse | +| 100.00% | Trinity.Scheduler.Delivery.Desktop | +| 93.75% | Trinity.Memory.Curator | +| 80.00% | Trinity.Memory.ObserverWorker | +| 80.31% | Total | +``` +`coverage.tsv` row: `050 80.31 c01466d 2026-09-22` (from 80.05 at 061: up a quarter) + +The gate line `trinity.coverage: 061 80.05% vs 060 80.32%` is the gate run before this row was appended; `mix trinity.coverage` on the row reads `050 80.31% vs 061 80.05%: OK``. + +The slice's tests (`mix test test/trinity/scheduler test/trinity_web/live/tasks_live_test.exs --trace`): +``` +* test the form adds a task; the list shows it with its next run; a bad schedule shows its error; edit, disable, remove + * test run now queues a run; when it finishes the result is listed until marked seen; the run history shows it with its conversation + * test Oban Web is mounted at /oban + * test the suggest form fills the schedule from the model's answer + * test stale at 30 days with a query receipt, archived at 90 with an effect receipt, fresh untouched, nothing deleted; the archived entry leaves recall + * test an archived semantic entry leaves the semantic entries and the brute store's search + * test the observer enqueues a memory job carrying the message ids and not their text; the job runs run/2 + * test AC1: creating a task with */5 * * * * computes the next boundary; the tick at that time enqueues one RunTask, unique on task and time; perform_job runs it + * test AC2 (automatic half): the run is a turn in a fresh cron session titled after the task; the run row carries the summary; the desktop delivery marks it and broadcasts + * test AC4: the human schedule helper turns a phrase into cron through the model and refuses an answer that does not parse; a cron phrase passes through + * test a one-shot task runs once at its time and is then disabled; a schedule that does not parse is refused + * test AC5: a job inserted before Oban stops runs after Oban starts again + * test AC3: a failing turn is an error Oban retries; the run reads retrying, then failed with the error on the last attempt +Result: 13 passed +``` + +## Acceptance criteria evidence + +### AC1 [auto]: Creating a task with `*/5 * * * *` inserts a cron entry; `Oban.Testing.assert_enqueued` / `perform_job` runs it +`scheduler_test.exs` "AC1": the task's `next_run_at` is the next five-minute boundary; a tick a second +before it enqueues nothing; `perform_job(Tick, %{"now" => at})` enqueues one `RunTask` with the task and the +time (`assert_enqueued`), writes one `queued` run, and advances the task by five minutes; a second tick for +the same minute enqueues nothing more (one run, one job); `perform_job(RunTask, …)` runs the FakeProvider +turn and the run reads `ok` with its summary. The design (the tick, not the plugin's static table) is in +NOTES.md "Read before code". + +### AC2 [manual]: `RunTask` creates a session with `origin: "cron"`, completes a FakeProvider turn, records a `task_runs` row with summary, and delivers a desktop notification (test + screenshot) +The test: `scheduler_test.exs` "AC2 (automatic half)": the session row (`origin: "cron"`, the task's name as +its title, the task's persona, `origin_ref` with the task and run ids), the history (the prompt with the +skills line, the assistant's answer), the run (`ok`, the summary, `delivered_at`), the `{:task_run, run}` +broadcast, the run among `unseen_runs/0` until `mark_seen/1`, the task's `last_run_at`. The screenshot: +`proof/ac2-results-list.png` (the "Results" list on `/tasks` after "run now": the task's name, the time, +`ok`, the summary, "open the conversation", "seen"; the bar's "1 result to read"), taken with +`scripts/dev_tasks.sh` and a headless browser. + +### AC3 [auto]: A failing turn retries per policy then marks the run failed with the error +`scheduler_test.exs` "AC3": with the provider failing, `perform_job(RunTask, args, attempt: 1)` returns +`{:error, text}` and the run reads `retrying` with the error; `attempt: 3` (the worker's `max_attempts`) +marks it `failed` with `finished_at` and delivers the failure (the results list shows it in red). + +### AC4 [auto]: Human schedule "every weekday at 9am" → `0 9 * * 1-5` +`scheduler_test.exs` "AC4": with the fake's object scripted to `0 9 * * 1-5`, `Parse.human/1` answers it; +scripted to "at nine on weekdays", the answer is refused as `{:not_cron, …}` (Oban's parser is the check); +`@daily` and a padded expression pass through without the model; a blank is `:empty`. The page's "suggest" +form fills the schedule (`tasks_live_test.exs`). + +### AC5 [auto]: App restart with a due job pending → job runs after restart +`scheduler_test.exs` "AC5": a `RunTask` job inserted; an Oban instance with a live `agent_tasks` queue +(`testing: :disabled`, the Lite engine, on the test's connection) started and stopped while the run is +still `queued`; started again, the job runs and the run reads `ok` within the wait. The job is a row; the +restart is Oban's supervisor's. + +### AC6 [manual]: Memory observer runs as an Oban job and is visible in Oban Web (screenshot) +The test half: `curator_test.exs` "the observer enqueues a memory job": `observe/2` enqueues +`Trinity.Memory.ObserverWorker` on the `memory` queue with the turn and the message ids and none of the +text; `observer_test.exs` (032) drains the queue and the extraction runs. The screenshots: +`proof/ac6-2-oban-jobs.png` (Oban Web's completed jobs after one task run with the observer on: +`Trinity.Memory.ObserverWorker` on `memory` with `message_ids` in its args, `RunTask` on `agent_tasks`, +`Tick` on `maintenance`) and `proof/ac6-1-oban-queues.png` (the three queues). + +### AC7 [manual]: Create a "daily summary of my notes dir" task, run now, see the result (GIF) +`proof/ac7-daily-summary-task.gif`: the task created on `/tasks` (name, `0 9 * * *`, the prompt), "run now", +the result appearing in the list, the run history opened, the conversation opened; the frames +`proof/ac7-1-task-created.png`, `proof/ac7-2-run-history.png`, `proof/ac7-3-conversation.png`. The model +is the fake provider scripted with a summary (`scripts/dev_tasks.sh`): the flow is the criterion, the +words are the script's, and a real provider run is the same page with a slower answer. + +### The curator (scope, added 2026-09-20) +`curator_test.exs`: at 31 days `stale_at` and one query receipt on the persona's memory scope; at 91 days +`archived_at` and one effect receipt (phase `done`, the entry); a fresh entry untouched; three rows before +and after (nothing deleted); the archived entry absent from the always-on chain and from the semantic +entries and the brute store's search; a second run changes nothing. + +## Manual verification for the reviewer +AC2, AC6 and AC7 are the screenshots and the GIF above; `scripts/dev_tasks.sh` reruns the page with Oban's +queues live (`PORT=` printed) for a look at `/tasks` and `/oban`. + +## Deviations from SLICE.md +NOTES.md: the G1 decisions (the tick; a run's approvals expire unanswered) and five found building (the +test pool of two; AC1 read as the tick; the 032 test draining the queue; Oban Web's mount asserted on the +router; the dashboard route mounted in the suite). + +## Versions touched +`VERSIONS.md` updated: yes, by `mix versions.gen`: `oban` from 🔍 to ✅ in `mix.lock` (2.24.1) and a new row +`oban_web` ✅ (2.13.0; Apache-2.0 on hex.pm, which the 2026-09-05 plan did not know); `oban_met` 1.3.1 comes +with it (Apache-2.0). `mix versions.verify`: named in the gate output above. + +## Git +``` +$ git log --oneline main..HEAD +(named in the closing correction, after the final commit) +``` + +## Closing correction, 2026-09-22 + +Supersedes "named in the closing correction" above. The tree the PR is merged from is `abcfcd0`; the code +is `e382fc0` (`feat(s050): complete slice 050`, the commit carrying this file) plus `abcfcd0` (the postgres +job's adapter check run with `--no-start`: its first run, 35711666503, booted the application before +`ecto.reset` and Oban refused to start without its table; the SQLite suite never hits that because its +`mix test` alias migrates first). On `abcfcd0`, CI run 35712404770: `gate` success (501 passed, 18 +excluded), `postgres` success (481 passed, 38 excluded; the Basic engine and Oban's Postgres migration +on that leg), `fips-tag` and `fips` success (506 passed, 13 excluded; the six FIPS tests by name); the +`push` event's run 35712400033 the same numbers. The coverage row stays at `c01466d` (80.31%): the +commits after it change this file, the workflow, ROADMAP.md and coverage.tsv only. + +``` +$ git log --oneline main..HEAD +abcfcd0 ci(s050): the adapter check runs without booting the application (Oban needs its table first) +e382fc0 feat(s050): complete slice 050 (scheduler with Oban cron agent tasks) +c01466d chore(s050): VERSIONS.md regenerated (oban, oban_web rows in mix.lock) +228e361 docs(s050): deviations, findings at G3 (the stress measurement with Oban polling), follow-ups +4d7dd97 docs(s050): AC2, AC6 and AC7 proof (the results list, Oban Web with the three workers, the task's GIF); the dev script +89f5249 docs(s050): docs 01, 05, 07 and the README as built +9be8540 test(s010): the pool-size property reads the shipped configuration (the suite's pool is two for Oban's boot check) +384d2c0 feat(s050): the /tasks page (list, form with suggest, run now, results to read, run history), Oban Web at /oban; the curator and observer tests +2152a5d feat(s050): Oban on the app's repo; tasks and runs; the tick, the run worker, the desktop delivery, the schedule helper; the observer as a job; the curator; AC1 to AC5 green +b73184a docs(s050): G1 plan; oban and oban_web pinned and fetched +``` diff --git a/slices/050-scheduler-oban/proof/ac2-results-list.png b/slices/050-scheduler-oban/proof/ac2-results-list.png new file mode 100644 index 0000000..d705951 Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac2-results-list.png differ diff --git a/slices/050-scheduler-oban/proof/ac6-1-oban-queues.png b/slices/050-scheduler-oban/proof/ac6-1-oban-queues.png new file mode 100644 index 0000000..d11d7fa Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac6-1-oban-queues.png differ diff --git a/slices/050-scheduler-oban/proof/ac6-2-oban-jobs.png b/slices/050-scheduler-oban/proof/ac6-2-oban-jobs.png new file mode 100644 index 0000000..8c2ca13 Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac6-2-oban-jobs.png differ diff --git a/slices/050-scheduler-oban/proof/ac7-1-task-created.png b/slices/050-scheduler-oban/proof/ac7-1-task-created.png new file mode 100644 index 0000000..1a6f3a8 Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac7-1-task-created.png differ diff --git a/slices/050-scheduler-oban/proof/ac7-2-run-history.png b/slices/050-scheduler-oban/proof/ac7-2-run-history.png new file mode 100644 index 0000000..9f82a84 Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac7-2-run-history.png differ diff --git a/slices/050-scheduler-oban/proof/ac7-3-conversation.png b/slices/050-scheduler-oban/proof/ac7-3-conversation.png new file mode 100644 index 0000000..88a72ca Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac7-3-conversation.png differ diff --git a/slices/050-scheduler-oban/proof/ac7-daily-summary-task.gif b/slices/050-scheduler-oban/proof/ac7-daily-summary-task.gif new file mode 100644 index 0000000..689ddec Binary files /dev/null and b/slices/050-scheduler-oban/proof/ac7-daily-summary-task.gif differ diff --git a/test/trinity/memory/observer_test.exs b/test/trinity/memory/observer_test.exs index eac36cc..2b4bee0 100644 --- a/test/trinity/memory/observer_test.exs +++ b/test/trinity/memory/observer_test.exs @@ -184,17 +184,14 @@ defmodule Trinity.Memory.ObserverTest do {:ok, _} = Session.send_user_message(pid, "I live in Lisbon") _ = collect(row.id, &match?({:state, :idle}, &1)) - # The observer's task is not the session's: wait for the row, not for the session. - entries = - Enum.find_value(1..50, fn _ -> - case Semantic.entries(persona.id, [pscope]) do - [] -> Process.sleep(20) && nil - found -> found - end - end) + # Slice 050: the observer is a job on the memory queue, enqueued off the session's path; + # the suite runs Oban manually, so the queue is drained here and the job runs in this + # process (the 032 version waited for a task's row). + assert %{success: 1, failure: 0} = Oban.drain_queue(queue: :memory, with_safety: false) + entries = Semantic.entries(persona.id, [pscope]) assert [%{body: "Lives in Lisbon.", by_session: nil} = e] = - Enum.map(entries || [], &Map.put(&1, :by_session, nil)) + Enum.map(entries, &Map.put(&1, :by_session, nil)) assert e.source_message_id != nil [log | _] = AlwaysOn.changes(persona.id) diff --git a/test/trinity/repo_config_test.exs b/test/trinity/repo_config_test.exs index 8d3ad7a..aae0603 100644 --- a/test/trinity/repo_config_test.exs +++ b/test/trinity/repo_config_test.exs @@ -11,8 +11,15 @@ defmodule Trinity.RepoConfigTest do @moduletag :sqlite describe "the write pool" do - test "has exactly one connection" do - assert Trinity.Repo.config()[:pool_size] == 1 + # Slice 050: the shipped configuration is read from the file under the production + # environment, since the suite's own pool has two connections (config/test.exs says why: + # Oban verifies its migration at boot through a raw checkout while a boot process holds + # the sandbox's first connection in auto mode). What every test still gets is one + # connection shared with every process it starts. + test "has exactly one connection in the shipped configuration" do + config = Config.Reader.read!("config/config.exs", env: :prod) + assert config[:trinity][Trinity.Repo][:pool_size] == 1 + assert config[:trinity][Trinity.Repo.Receipts][:pool_size] == 1 end end diff --git a/test/trinity/scheduler/curator_test.exs b/test/trinity/scheduler/curator_test.exs new file mode 100644 index 0000000..e5b6ff9 --- /dev/null +++ b/test/trinity/scheduler/curator_test.exs @@ -0,0 +1,171 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.Memory.CuratorTest do + @moduledoc """ + Slice 050: the curator marks an entry untouched for 30 days stale (a query receipt) and one + untouched for 90 days archived (an effect receipt); a fresh one is untouched; nothing is + deleted; an archived entry leaves the always-on chain and the semantic entries; the observer + is a job on the memory queue with the message ids, never the text. + """ + use Trinity.DataCase, async: false + use Oban.Testing, repo: Trinity.Repo + @moduletag :capture_log + + import Ecto.Query, only: [from: 2] + + alias Trinity.Memory.{AlwaysOn, Curator, Entry, Observer, ObserverWorker, Semantic} + alias Trinity.Receipts + alias Trinity.Repo + + setup do + persona = Trinity.Factory.persona!() + scope = AlwaysOn.persona_scope(persona.id) + on_exit(fn -> Receipts.stop_writer(Curator.scope(persona.id)) end) + {:ok, persona: persona, scope: scope} + end + + defp aged!(persona, scope, key, days) do + {:ok, e} = + AlwaysOn.add( + %{persona_id: persona.id, tier: "always_on", scope: scope, key: key, body: key}, + by: "test" + ) + + at = DateTime.utc_now() |> DateTime.add(-days, :day) + + Repo.update_all(from(x in Entry, where: x.id == ^e.id), + set: [last_used_at: at, updated_at: at] + ) + + Repo.get!(Entry, e.id) + end + + test "stale at 30 days with a query receipt, archived at 90 with an effect receipt, fresh untouched, nothing deleted; the archived entry leaves recall", + %{persona: persona, scope: scope} do + fresh = aged!(persona, scope, "fresh", 1) + stale = aged!(persona, scope, "stale", 31) + old = aged!(persona, scope, "old", 91) + now = DateTime.utc_now() + + assert {:ok, %{stale: 1, archived: 1}} = + perform_job(Curator, %{"now" => DateTime.to_iso8601(now)}) + + assert %Entry{stale_at: nil, archived_at: nil} = Repo.get!(Entry, fresh.id) + assert %Entry{stale_at: %DateTime{}, archived_at: nil} = Repo.get!(Entry, stale.id) + assert %Entry{archived_at: %DateTime{}} = Repo.get!(Entry, old.id) + assert Repo.aggregate(from(e in Entry, where: e.persona_id == ^persona.id), :count) == 3 + + receipts = Receipts.list(Curator.scope(persona.id)) + + assert [ + %{ + kind: "effect", + subject: %{"entry_id" => oid, "phase" => "done", "curator" => "archive"} + } + ] = Enum.filter(receipts, &(&1.kind == "effect")) + + assert oid == old.id + + assert [%{kind: "query", subject: %{"curator" => "stale", "entries" => [sid]}}] = + Enum.filter(receipts, &(&1.kind == "query")) + + assert sid == stale.id + + # Recall: the archived entry is gone from the chain a session sees; the stale one stays. + keys = AlwaysOn.entries(persona.id, nil) |> Enum.map(& &1.key) |> Enum.sort() + assert keys == ["fresh", "stale"] + + # A second run changes nothing more. + assert {:ok, %{stale: 0, archived: 0}} = + perform_job(Curator, %{"now" => DateTime.to_iso8601(now)}) + end + + test "an archived semantic entry leaves the semantic entries and the brute store's search", %{ + persona: persona, + scope: scope + } do + vector = Enum.map(1..8, fn _ -> 0.1 end) + + {:ok, e} = + %Entry{} + |> Entry.semantic_changeset(%{ + persona_id: persona.id, + tier: "semantic", + scope: scope, + key: "cat", + body: "the cat is grey", + confidence: 0.9, + embedding: Trinity.Memory.Embedder.to_binary(vector), + embedding_model: "fake:embed", + embedding_dim: 8 + }) + |> Repo.insert() + + assert [_] = Semantic.entries(persona.id, [scope]) + + assert [_] = + Trinity.Memory.VectorStores.Brute.search(vector, 5, %{ + persona_id: persona.id, + scopes: [scope], + model: "fake:embed" + }) + + Repo.update_all(from(x in Entry, where: x.id == ^e.id), + set: [archived_at: DateTime.utc_now()] + ) + + assert [] = Semantic.entries(persona.id, [scope]) + + assert [] = + Trinity.Memory.VectorStores.Brute.search(vector, 5, %{ + persona_id: persona.id, + scopes: [scope], + model: "fake:embed" + }) + end + + test "the observer enqueues a memory job carrying the message ids and not their text; the job runs run/2", + %{persona: persona} do + session = Trinity.Factory.session!(%{persona_id: persona.id}) + m1 = Trinity.Factory.message!(session.id, %{role: "user", content: "I live in Lisbon"}) + m2 = Trinity.Factory.message!(session.id, %{role: "assistant", content: "Noted."}) + turn = %{session_id: session.id, persona_id: persona.id, model: "fake:chat"} + + Application.put_env( + :trinity, + :memory, + Keyword.put(Application.get_env(:trinity, :memory, []), :observer, true) + ) + + on_exit(fn -> + Application.put_env( + :trinity, + :memory, + Keyword.put(Application.get_env(:trinity, :memory, []), :observer, false) + ) + end) + + case Observer.observe(turn, [ + %{id: m1.id, role: "user", content: m1.content}, + %{id: m2.id, role: "assistant", content: m2.content} + ]) do + {:ok, %Oban.Job{args: args}} -> + assert args == %{ + "turn" => %{ + "session_id" => session.id, + "persona_id" => persona.id, + "model" => "fake:chat" + }, + "message_ids" => [m1.id, m2.id] + } + + refute inspect(args) =~ "Lisbon" + assert_enqueued(worker: ObserverWorker, queue: :memory) + + :off -> + # The semantic tier is off in this environment (no store or model): the job is not + # enqueued and the assertion is that nothing else happened. + refute_enqueued(worker: ObserverWorker) + end + end +end diff --git a/test/trinity/scheduler/scheduler_test.exs b/test/trinity/scheduler/scheduler_test.exs new file mode 100644 index 0000000..1a54cb7 --- /dev/null +++ b/test/trinity/scheduler/scheduler_test.exs @@ -0,0 +1,227 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule Trinity.SchedulerTest do + @moduledoc """ + Slice 050 AC1 (a task with `*/5 * * * *` carries its next five-minute boundary; the tick at that + time enqueues `RunTask` for it, once; `perform_job` runs it), AC2's automatic half (the run in a + `cron` session, the summary, the delivery), AC3 (a failing turn retries, then the run is failed + with the error), AC4 (the human schedule helper) and AC5 (a job survives a restart of Oban). + """ + use Trinity.SessionCase + use Oban.Testing, repo: Trinity.Repo + @moduletag :capture_log + + alias Trinity.LLM.Providers.Fake + alias Trinity.Scheduler + alias Trinity.Scheduler.{Parse, Run, Task} + alias Trinity.Scheduler.Workers.{RunTask, Tick} + + defp task!(attrs \\ %{}) do + {:ok, task} = + Scheduler.create_task( + Map.merge( + %{ + name: "every five", + kind: "cron", + schedule: "*/5 * * * *", + prompt: "Say hello.", + timeout_ms: 5_000 + }, + attrs + ) + ) + + task + end + + test "AC1: creating a task with */5 * * * * computes the next boundary; the tick at that time enqueues one RunTask, unique on task and time; perform_job runs it" do + task = task!() + assert %DateTime{minute: m, second: 0} = task.next_run_at + assert rem(m, 5) == 0 + assert DateTime.compare(task.next_run_at, DateTime.utc_now()) == :gt + + # Not due yet: nothing enqueued. + assert [] = Scheduler.enqueue_due(DateTime.add(task.next_run_at, -1, :second)) + refute_enqueued(worker: RunTask) + + # Due: the tick enqueues one run for that time and advances the task; a second tick for the + # same minute enqueues nothing more. + at = task.next_run_at + assert {:ok, _} = perform_job(Tick, %{"now" => DateTime.to_iso8601(at)}) + + assert_enqueued( + worker: RunTask, + args: %{"task_id" => task.id, "scheduled_at" => DateTime.to_iso8601(at)} + ) + + assert [%Run{status: "queued", scheduled_at: ^at}] = Scheduler.runs(task) + assert {:ok, _} = perform_job(Tick, %{"now" => DateTime.to_iso8601(at)}) + assert [_one] = Scheduler.runs(task) + assert [_one] = all_enqueued(worker: RunTask) + + advanced = Scheduler.get_task(task.id) + assert DateTime.compare(advanced.next_run_at, at) == :gt + assert advanced.next_run_at.minute == rem(at.minute + 5, 60) + + # The job runs: a FakeProvider turn. + Fake.scripts([script_deltas(2, "hello ")]) + [%Run{id: run_id}] = Scheduler.runs(task) + + assert :ok = + perform_job(RunTask, %{ + "run_id" => run_id, + "task_id" => task.id, + "scheduled_at" => DateTime.to_iso8601(at) + }) + + assert %Run{status: "ok", summary: "hello hello "} = Scheduler.get_run(run_id) + end + + test "AC2 (automatic half): the run is a turn in a fresh cron session titled after the task; the run row carries the summary; the desktop delivery marks it and broadcasts" do + :ok = Scheduler.subscribe() + persona = Trinity.Factory.persona!() + task = task!(%{name: "morning note", persona_id: persona.id, skill_names: ["git-workflow"]}) + Fake.scripts([script_deltas(3, "done ")]) + + {:ok, run} = Scheduler.run_now(task) + + assert :ok = + perform_job(RunTask, %{ + "run_id" => run.id, + "task_id" => task.id, + "scheduled_at" => DateTime.to_iso8601(run.scheduled_at) + }) + + run = Scheduler.get_run(run.id) + + assert %Run{status: "ok", summary: "done done done ", delivered_at: %DateTime{}, seen_at: nil} = + run + + assert_receive {:task_run, %Run{id: rid}}, 1_000 + assert rid == run.id + + session = Sessions.get_session(run.session_id) + + assert %{ + origin: "cron", + title: "morning note", + persona_id: pid, + origin_ref: %{"task_id" => tid} + } = session + + assert pid == persona.id and tid == task.id + + assert [%{role: "user", content: content}, %{role: "assistant"}] = + Sessions.history(run.session_id) + + assert content =~ "Say hello." and content =~ "git-workflow" + + assert [%Run{id: ^rid}] = Scheduler.unseen_runs() + {:ok, _} = Scheduler.mark_seen(run) + assert [] = Scheduler.unseen_runs() + assert %{last_run_at: %DateTime{}} = Scheduler.get_task(task.id) + end + + test "AC3: a failing turn is an error Oban retries; the run reads retrying, then failed with the error on the last attempt" do + task = task!(%{name: "doomed"}) + {:ok, run} = Scheduler.run_now(task) + + args = %{ + "run_id" => run.id, + "task_id" => task.id, + "scheduled_at" => DateTime.to_iso8601(run.scheduled_at) + } + + Fake.fail(10, Trinity.LLM.Error.permanent(:model_down)) + + assert {:error, text} = perform_job(RunTask, args, attempt: 1) + assert text =~ "the turn failed" or text =~ "error" + assert %Run{status: "retrying", error: error} = Scheduler.get_run(run.id) + assert is_binary(error) + + assert {:error, _} = perform_job(RunTask, args, attempt: 3) + + assert %Run{status: "failed", finished_at: %DateTime{}, delivered_at: %DateTime{}} = + Scheduler.get_run(run.id) + end + + test "AC4: the human schedule helper turns a phrase into cron through the model and refuses an answer that does not parse; a cron phrase passes through" do + Fake.object(%{"cron" => "0 9 * * 1-5"}) + assert {:ok, "0 9 * * 1-5"} = Parse.human("every weekday at 9am") + + Fake.object(%{"cron" => "at nine on weekdays"}) + assert {:error, {:not_cron, "at nine on weekdays", _}} = Parse.human("every weekday at 9am") + + assert {:ok, "@daily"} = Parse.human("@daily") + assert {:ok, "30 6 * * *"} = Parse.human(" 30 6 * * * ") + assert {:error, :empty} = Parse.human(" ") + end + + test "a one-shot task runs once at its time and is then disabled; a schedule that does not parse is refused" do + at = DateTime.utc_now() |> DateTime.add(90, :second) |> DateTime.truncate(:second) + task = task!(%{name: "once", kind: "once", schedule: DateTime.to_iso8601(at)}) + assert DateTime.compare(task.next_run_at, at) == :eq + + assert [] = Scheduler.enqueue_due(DateTime.add(at, -1, :second)) + assert [%Run{}] = Scheduler.enqueue_due(at) + assert %Task{enabled: false, next_run_at: nil} = Scheduler.get_task(task.id) + assert [] = Scheduler.enqueue_due(DateTime.add(at, 60, :second)) + + assert {:error, cs} = + Scheduler.create_task(%{ + name: "bad", + kind: "cron", + schedule: "every day", + prompt: "x" + }) + + assert %{schedule: [msg]} = errors_on(cs) + assert msg =~ "not a cron expression" + + assert {:error, cs} = + Scheduler.create_task(%{ + name: "bad", + kind: "once", + schedule: "tomorrow", + prompt: "x" + }) + + assert %{schedule: [_]} = errors_on(cs) + end + + test "AC5: a job inserted before Oban stops runs after Oban starts again" do + task = task!(%{name: "survivor"}) + Fake.scripts([script_deltas(1, "back ")]) + {:ok, run} = Scheduler.run_now(task) + + # The job is a row; a second Oban instance with a live queue picks it up after a restart. + opts = [ + name: :oban_restart, + repo: Trinity.Repo, + engine: Application.fetch_env!(:trinity, Oban)[:engine], + notifier: Oban.Notifiers.PG, + testing: :disabled, + queues: [agent_tasks: 1], + plugins: false, + stage_interval: 50, + poll_interval: 50 + ] + + {:ok, pid} = Oban.start_link(opts) + Ecto.Adapters.SQL.Sandbox.allow(Trinity.Repo, self(), pid) + :ok = Supervisor.stop(pid) + assert %Run{status: "queued"} = Scheduler.get_run(run.id) + + {:ok, pid} = Oban.start_link(opts) + Ecto.Adapters.SQL.Sandbox.allow(Trinity.Repo, self(), pid) + + assert Enum.find_value(1..100, fn _ -> + case Scheduler.get_run(run.id) do + %Run{status: "ok"} = r -> r + _ -> Process.sleep(50) && nil + end + end) + + Supervisor.stop(pid) + end +end diff --git a/test/trinity_web/live/tasks_live_test.exs b/test/trinity_web/live/tasks_live_test.exs new file mode 100644 index 0000000..866c1a8 --- /dev/null +++ b/test/trinity_web/live/tasks_live_test.exs @@ -0,0 +1,135 @@ +# SPDX-FileCopyrightText: Sudo Apt Holdings LLC +# SPDX-License-Identifier: Apache-2.0 +defmodule TrinityWeb.TasksLiveTest do + @moduledoc "Slice 050: the tasks page lists, adds, edits, runs and removes tasks; the results list marks runs seen; /oban answers in dev." + use TrinityWeb.ConnCase, async: false + use Oban.Testing, repo: Trinity.Repo + @moduletag :capture_log + + import Phoenix.LiveViewTest + import Trinity.SessionCase, only: [script_deltas: 2] + + alias Trinity.LLM.Providers.Fake + alias Trinity.Scheduler + alias Trinity.Scheduler.Workers.RunTask + + setup do + Fake.clear() + on_exit(fn -> Trinity.SessionCase.stop_all_sessions() end) + :ok + end + + test "the form adds a task; the list shows it with its next run; a bad schedule shows its error; edit, disable, remove", + %{conn: conn} do + {:ok, view, _} = live(conn, ~p"/tasks") + + view + |> form("#task-form", %{ + "task" => %{ + "name" => "nightly", + "kind" => "cron", + "schedule" => "every night", + "prompt" => "Summarise." + } + }) + |> render_submit() + + assert has_element?(view, "#task-form p", "schedule:") + + view + |> form("#task-form", %{ + "task" => %{ + "name" => "nightly", + "kind" => "cron", + "schedule" => "0 2 * * *", + "prompt" => "Summarise.", + "skill_names" => "git-workflow, web-research" + } + }) + |> render_submit() + + assert [task] = Scheduler.list_tasks() + assert task.skill_names == ["git-workflow", "web-research"] + assert has_element?(view, "#task-#{task.id}", "nightly") + assert has_element?(view, "#task-#{task.id}", "cron · 0 2 * * *") + + view |> element("#task-#{task.id} button", "edit") |> render_click() + + view + |> form("#task-form", %{ + "task" => %{ + "name" => "nightly summary", + "kind" => "cron", + "schedule" => "0 2 * * *", + "prompt" => "Summarise." + } + }) + |> render_submit() + + assert %{name: "nightly summary"} = Scheduler.get_task(task.id) + + view |> element("#task-#{task.id} button", "disable") |> render_click() + assert has_element?(view, "#task-#{task.id}", "disabled") + assert %{enabled: false} = Scheduler.get_task(task.id) + + view |> element("#task-#{task.id} button", "remove") |> render_click() + refute has_element?(view, "#task-#{task.id}") + assert [] = Scheduler.list_tasks() + end + + test "run now queues a run; when it finishes the result is listed until marked seen; the run history shows it with its conversation", + %{conn: conn} do + {:ok, task} = + Scheduler.create_task(%{ + name: "hello", + kind: "cron", + schedule: "@daily", + prompt: "Say hello.", + timeout_ms: 5_000 + }) + + {:ok, view, _} = live(conn, ~p"/tasks") + + view |> element("#task-#{task.id} button", "run now") |> render_click() + assert [run] = Scheduler.runs(task) + assert_enqueued(worker: RunTask, args: %{"run_id" => run.id}) + + Fake.scripts([script_deltas(2, "hi ")]) + + assert :ok = + perform_job(RunTask, %{ + "run_id" => run.id, + "task_id" => task.id, + "scheduled_at" => DateTime.to_iso8601(run.scheduled_at) + }) + + # The delivery's broadcast reloads the page. + assert render(view) =~ "hi hi" + assert has_element?(view, "#notification-#{run.id}", "hello") + assert has_element?(view, "#unseen-runs", "1 result to read") + + view |> element("#notification-#{run.id} button", "seen") |> render_click() + refute has_element?(view, "#notification-#{run.id}") + refute has_element?(view, "#unseen-runs") + + view |> element("#task-#{task.id} button", "hello") |> render_click() + assert has_element?(view, "#runs-#{task.id} td", "ok") + assert has_element?(view, "#runs-#{task.id} a", "conversation") + end + + test "the suggest form fills the schedule from the model's answer", %{conn: conn} do + Fake.object(%{"cron" => "0 9 * * 1-5"}) + {:ok, view, _} = live(conn, ~p"/tasks") + view |> form("#suggest-form", %{"phrase" => "every weekday at 9am"}) |> render_submit() + assert has_element?(view, "#task-form input[name='task[schedule]'][value='0 9 * * 1-5']") + assert has_element?(view, "#suggest-form span", "0 9 * * 1-5") + end + + # The dashboard's LiveView waits for Oban.Met, which Oban does not start in the suite's + # manual testing mode, so the mount is asserted on the router (the page itself is AC6's + # screenshot, taken against the dev run). + test "Oban Web is mounted at /oban" do + assert %{plug: Phoenix.LiveView.Plug, phoenix_live_view: {Oban.Web.DashboardLive, _, _, _}} = + Phoenix.Router.route_info(TrinityWeb.Router, "GET", "/oban", "localhost") + end +end