Skip to content

agent-forge.md

Latest commit

 

History

History
379 lines (317 loc) · 19.9 KB

agent-forge.md

File metadata and controls

379 lines (317 loc) · 19.9 KB
Raw

Agent Forge

Agent Forge (apps/agent-forge) is this SDK's companion visual dashboard: build an agent's graph on a canvas, run it, watch it execute in a live debug console, chat with it, and author sandboxed pre/post hooks - all reading and writing the exact same AgentSpec YAML that lousho dev and lousho build use. It's launched with one command, lousho studio, and ships as part of this SDK's npm package.

This doc covers: installation, the lousho studio quickstart, a first-agent walkthrough, hook authoring, time travel (replaying a run from a past step), and how settings/secrets/deploy wiring works.

Installation

Agent Forge ships inside @lousho/build-ai-agent itself - there's no separate package to install:

npm install @lousho/build-ai-agent

That's it. lousho studio (below) runs a pre-built copy of the app; you don't need apps/agent-forge's own source or its devDependencies (Vite, tsx, etc.) to use it.

If you're working inside this SDK's own monorepo instead (contributing to Agent Forge itself), see Dev mode below.

Quickstart

npx lousho studio

This starts one local server and prints its URL (default http://127.0.0.1:4750/?token=<random>). Open that exact URL in a browser - you'll see the canvas, the left rail (your saved agents + a node/hook palette), the Inspector (right), and a bottom drawer with Chat/Logs/Trace/Output/Settings tabs. Below 860px wide the rail and the Inspector become slide-over panels you open from the top bar, and Debug, Import, Export and Save move into its More (⋯) menu.

Useful flags:

npx lousho studio --port 5000       # pick a different port
npx lousho studio --host 0.0.0.0    # bind to all interfaces
npx lousho studio --prod            # force production mode (see below)
npx lousho studio --dev             # force dev mode (monorepo only)

Access token

The studio can store provider keys, start runs and execute hook code, so its API is locked to the browser tab you opened from the printed URL:

  • Each launch mints a random token and prints it in the URL (?token=...). The page keeps it for that tab (it survives a reload) and sends it on every API call (x-lousho-studio-token header) and on the WebSocket (?token= query). A request without it gets 401. Opening the bare http://127.0.0.1:4750 in a new tab loads the page, but its API calls fail until you use the printed URL.
  • The server sends no CORS headers and refuses (403) a request whose Origin is not the studio's own, or whose Host is not a loopback name (localhost, 127.0.0.1, [::1]) or the --host you bound to. With --host 0.0.0.0 any Host is accepted and the token is the only gate.
  • GET /health and the static client need no token.

From curl, pass the token: curl -H "x-lousho-studio-token: $TOKEN" localhost:4750/agents.

lousho studio hands these to the processes it starts through environment variables; you only set them yourself when you start the studio server without the CLI:

Variable Read by Meaning
LOUSHO_STUDIO_TOKEN the studio API server The access token. Unset, the server mints a random one per launch.
LOUSHO_STUDIO_ALLOWED_ORIGINS the studio API server Comma-separated extra origins to accept besides the studio's own (dev mode sets the Vite dev server's).
LOUSHO_STUDIO_API_URL the Vite dev server (dev mode) Where to proxy API calls (default http://127.0.0.1:4750).

Where agents live

Agent specs and run state are stored under .lousho/ in the directory you ran lousho studio from (agent YAML files under .lousho/agents/, checkpoints and approvals alongside them) - the same .lousho/ layout lousho dev/lousho build use.

The left rail lists exactly the agents in that .lousho/agents/: a YAML file you add there shows up, and an agent you create in the studio is written there at once (autosave and Save both write the file), so it can be chatted with straight away. Another project, opened in its own lousho studio, shows only its own agents. The browser keeps nothing but a draft of an edit that could not reach the server (it is pushed on the next load), stored per workspace directory. A file in .lousho/agents/ that no longer parses is skipped with a warning in the server log.

Production vs. dev mode

lousho studio has two modes, and picks the right one automatically:

  • Production (the default once Agent Forge has been built): a single Express server serves both the REST/WebSocket API and the pre-built client (React app) as static files, on one port. This is what runs when you npm install the published package - there's no separate Vite dev server and no TypeScript loader involved.
  • Dev (falls back to this if the build hasn't been run yet, i.e. inside a source checkout of this monorepo): the API server runs straight off its TypeScript source via tsx, alongside a real Vite dev server (with hot-module reload) that proxies API requests to it. Two processes, two ports internally, one command.

--prod/--dev force one or the other; run without either flag and lousho studio auto-detects based on whether apps/agent-forge/dist-server exists.

First-agent walkthrough

  1. Create an agent. In the left rail's "Agents" tab, click New, give it a name, and pick a starting template - Blank graph (a single LLM node using the built-in mock provider) is the fastest way to try things out without any API keys. Agent Forge creates the agent under that name (.lousho/agents/<name>.yaml, with name: <name> in the spec) and opens it on the canvas. The open agent's card has Rename and Delete. Rename moves the spec to the new name; earlier chats, traces and checkpoints stay under the old one.

  2. Look at the graph. A blank agent is one llm node. Drag more nodes in from the left rail's palette (Trigger, LLM step, Tool call, Approval gate, Response / output), or click (or press Enter on) a palette item to add it at the centre of the canvas, and connect them by dragging between their handles. Click a node to edit its settings (prompt, provider/model, tools, breakpoints) in the Inspector on the right.

  3. Run it. Click Run in the top bar. With the mock provider there's nothing to configure - it returns deterministic canned output, which is exactly the point for trying the rest of the UI without needing a real LLM API key. The status pill in the top bar tracks the run (running → done, stopped after you press Stop, error or awaiting approval). Run sends a fixed prompt, which the Chat tab shows as a Run input bubble instead of a message from you. If the studio server goes away, a banner says the connection was lost and the studio reconnects when it is back.

  4. Watch it in the debug console. Open the bottom drawer's Logs tab for a live, filterable log feed of the run (trigger/llm/tool/ sandbox/checkpoint/approval events), or Trace for a span waterfall. Turn on Debug in the top bar (or set a breakpoint on a node in the Inspector) to pause the run at LLM/tool boundaries and step through it with the debug bar's Step/Continue controls; expand "Live message array" there to inspect the in-flight message list. Output shows the full ExecutionResult (messages, tool calls, usage, steps) as a collapsible JSON tree once the run finishes or pauses.

    Trace history. Every run is also saved as a trace file, so the Trace tab keeps the runs of the agent after the page is reloaded or the server restarted. A list on the left shows the agent's past runs, newest first (start time, duration, model calls, tokens in/out, cost, status), with a Live entry on top while a run is active; choose one to open its spans in the same waterfall, with each span's kind (internal or client) and failed spans in the error colour. The list refreshes when a run ends. The files are the ones lousho traces reads, in .lousho/agents/<agent id>/traces/<day>/<trace id>.jsonl under the directory lousho studio was started in (see Local traces): npx lousho traces --dir .lousho/agents/<agent id>/traces shows the same runs in the terminal, and deleting the agent's folder deletes its traces. The tab shows what the spans hold: model and tool names, token counts, cost, durations and error messages, and with the span's tooltip the attributes, which carry prompt and tool content. The server reads traces only from that folder (an id with a path separator is a 400) and never sends the file path to the browser. Keep .lousho/ out of version control.

  5. Chat with it. The Chat tab is a separate conversational transport (POST /agents/:id/message, streamed back over the same WebSocket as everything else) - send it a message and it replies in the thread, independent of the graph "Run" button above. Each agent keeps its own chat history (+ New chat starts a fresh session; old ones stay browsable).

  6. Approve a paused tool call. Add a Tool call node, open it in the Inspector, and pick a tool - some tools (or an agent's own policy) can require human approval before running. When a run or chat hits one, an inline approval card appears (in the top bar for a graph run, or as a message bubble in the Chat thread) showing the tool name and its arguments. Click Approve to let it proceed or Reject to abort that call; the run resumes automatically either way.

Hooks

Hooks are small, sandboxed functions that run immediately before or after a tool call or an LLM generate - the same mechanism the SDK core exposes as HookRegistry/AgentHook (see API Overview), but authored visually and attached to a specific canvas node.

To attach one:

  1. Select an llm or tool node on the canvas (hooks only apply to these - they're the two points AgentExecutor actually calls out to).
  2. Open the Inspector's Hooks section. Drag a Pre-hook or Post-hook entry from the left rail's palette onto the node (or drop it directly in the Hooks section), or click the palette entry while the node is selected, to attach a new one; you'll get a choice of starter templates (redact-pii, rate-limit, audit-log, inject-context) as an editable starting point.
  3. Click the hook's chip to select it, then edit its code in the CodeMirror editor. The code is the body of an async function invoked as hook(ctx) inside a sandboxed subprocess (server/hookSandbox.ts) - never in the browser or the API server process itself:
    • For a tool-call hook, ctx is { toolName, args, result?, error? }. A pre-hook can mutate ctx.args; a post-hook can inspect/mutate ctx.result/ctx.error. Return ctx.
    • For an LLM-generate hook, ctx is { messages, model }. Mutate/push onto ctx.messages and return ctx.
    • Throwing aborts the step - a rate-limit hook, for example, throws to block the tool call outright rather than letting it run.
  4. Toggle the switch on a hook's chip to enable/disable it without removing it, or use its x to remove it entirely. Only enabled hooks are compiled into the run.

Example: a redact-PII pre-hook on a tool node (this is the redact-pii starter template) -

const PII = /[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}/gi;
for (const key of Object.keys(ctx.args || {})) {
  if (typeof ctx.args[key] === 'string') {
    ctx.args[key] = ctx.args[key].replace(PII, '[REDACTED]');
  }
}
return ctx;

Saving the agent persists these as spec.policy.hooks in the agent's YAML; the server compiles enabled hooks into a real HookRegistry for the run (server/compileHooks.ts), routed through the same SandboxAdapter a sandboxed tool uses.

Time travel

Agent Forge keeps every run's checkpoints, not just the latest: its file store (server/checkpointStore.ts) keeps the same bounded history as the SDK's LocalStorageCheckpointStore (the newest 50 saves per run, under .lousho/agents/<id>/checkpoint-history/; see Checkpoint history). From that history you can fork a run at any step, change what happened there, and replay it next to the original, with AgentExecutor.fork() and compareTrajectories() doing the work.

The History tab

  1. Run the agent (Topbar Run, or a Chat message).
  2. Open the bottom drawer's History tab. It lists the run's steps: step number, status, the model's finish reason, the tool calls made, and the tokens and cost of the step's model call when known.
  3. Click Edit and replay from here on a step. Choose Append a user message and type one, or pick one of the step's tool calls to edit its result (prefilled with the recorded result; text that parses as JSON is sent as JSON).
  4. Click Fork and replay. The run is forked at that step with your edit and started as a new run, <id>.fork-<n>.
  5. The fork opens next to the original as a side-by-side trajectory: each model turn of both runs (text, tool calls and results), the first turn where they differ highlighted as diverged, and the drift entries (tool order, arguments, step count, finish reason) listed below. It refreshes when the fork finishes.

Routes

A run id is the run's checkpoint session id: the agent id for the agent's own runs, <id>.fork-<n> for a fork of run <id>. The runtime control server has three routes for it (the History tab uses them), and two more for an agent's saved traces (the Trace tab uses them):

Route What it does
GET /runs/:id/history { runId, steps }: one entry per step of the run's latest execution (the newest checkpoint saved at that step), oldest first, with status, savedAt, finishReason, toolCalls ({ id, name, args, result? }), and tokens / costUsd of the step's model call when known. 404 when the run has no checkpoints.
POST /runs/:id/fork Body { fromStep, patch? }, where patch is { toolResult?: { toolCallId, result }, appendInput?, businessState? }. Forks the run at fromStep, then starts the fork through the run registry with the original run's spec. Returns 202 { runId, fromStep, status }; the fork streams on WS /agents/<runId>/stream and reports on GET /agents/<runId>/status like any run. 400 for a bad body or an unknown toolCallId, 404 for an unknown run or a step with no checkpoint.
GET /agents/:id/traces?limit=N { traces }: the agent's saved runs from .lousho/agents/<id>/traces, newest first (limit defaults to 20, at most 200), each with traceId, name, startTime, durationMs, status, modelCalls, toolCalls, inputTokens, outputTokens and costUsd when known. An agent with no runs gives an empty list; 404 for an unknown agent.
GET /agents/:id/traces/:traceId { traceId, spans }: the spans of one saved run by start time, each with id, parentId, name, kind, status, startTime, endTime and attributes. traceId may be a unique prefix. 400 for an id that is not letters, digits and dashes or a prefix that matches several runs, 404 for an unknown agent or trace.
GET /runs/compare?a=&b= compareTrajectories(a, b) of two runs' latest checkpoints: each run's model turns, divergedAt (the first turn that differs) and the drift entries. 404 when either run has no checkpoint.

The flow, for an agent weather that has run once:

# $TOKEN is the ?token= value `lousho studio` printed
curl -H "x-lousho-studio-token: $TOKEN" localhost:4750/runs/weather/history
# {"runId":"weather","steps":[{"step":1,"status":"finished","finishReason":"stop","toolCalls":[],"tokens":...}]}

curl -X POST localhost:4750/runs/weather/fork -H "x-lousho-studio-token: $TOKEN" -H 'content-type: application/json' \
  -d '{"fromStep":1,"patch":{"appendInput":"And in Celsius?"}}'
# {"runId":"weather.fork-1","fromStep":1,"status":{"status":"running",...}}

curl -H "x-lousho-studio-token: $TOKEN" 'localhost:4750/runs/compare?a=weather&b=weather.fork-1'
# {"a":[...],"b":[...],"divergedAt":2,"drift":[{"field":"steps","committed":"1","current":"2"}]}

The fork is a run of its own: the original run's checkpoints, status and chat are unchanged.

Settings, secrets and deploy wiring

The bottom drawer has a Settings tab with three parts: provider API keys (OpenAI and Anthropic, stored encrypted under .lousho/ and never shown again), named settings profiles (provider, deploy adapter, hook timeout, OpenTelemetry toggle; kept in .lousho/settings.json, which holds no secrets), and a Deploy section that picks an adapter (node-server, docker or cloudflare-worker) and runs lousho build for the selected agent. Other providers still read their environment variables, the same way lousho dev/lousho build do (see Configuration), and the mock provider needs no credentials.

Stored keys are encrypted with AES-256-GCM in .lousho/secrets.json. The encryption key is generated on first use and kept next to it in .lousho/secrets.key (both 0600 on POSIX). This keeps keys out of anything that copies the ciphertext alone - a backup, a zip of the project, a shared agent YAML - but anyone who can read both files as your user can decrypt them. It is not an OS keychain. Keep .lousho/ out of version control and out of shared archives, and prefer environment variables on shared machines.

An agent whose provider has no key (no stored key and no environment variable) does not run. Run and Chat fail with LOUSHO_PROVIDER_MISSING_API_KEY and a message naming the fix: add the key in Settings, set the environment variable, or switch the agent's provider to mock. The studio never swaps in the mock provider on its own. The top bar's provider pill shows the provider/model the agent actually runs on, and a mock agent carries a MOCK badge.

Dev mode

If you're working inside this SDK's own monorepo (contributing to Agent Forge itself, not just using it), lousho studio falls back to dev mode automatically as long as apps/agent-forge/dist-server hasn't been built yet:

git clone https://github.com/LinuxDevil/agent-sdk.git
cd agent-sdk
npm install
npx lousho studio --dev   # or just `npx lousho studio` before building

This runs the API server straight from TypeScript (tsx) and a real Vite dev server with hot-module reload for apps/agent-forge/src/**, as two sibling processes. Open the printed http://localhost:5173/?token=... URL; Vite is pinned to port 5173 and the API server accepts that origin. To build the production bundle used by everyone else (and to verify what actually ships):

npm run build:studio   # builds apps/agent-forge/dist (client) and
                        # apps/agent-forge/dist-server (bundled server)
npx lousho studio       # now runs in production mode

npm run build:studio also runs automatically as part of the root prepublishOnly script, so a published npm publish can never ship a stale or unbuilt studio.

E2E smoke test

apps/agent-forge/e2e/studio.spec.ts is a headless Playwright test that drives the real, built lousho studio (the same dist-server/index.cjs production server, not a dev-mode Vite server) through a browser: it creates an agent from the "Support bot" template, retargets its tool node at a server-local demo-approval tool (always needsApproval: true - see server/buildAgent.ts's doc comment for why this exists, since none of the SDK's spec-resolvable built-in tools require approval), sends it a chat message that triggers the mock provider's tool-call heuristic, waits for the run to pause on the approval gate, approves it via the Chat tab's inline approval card, and asserts the run completes. Run it with:

npm run test:e2e:studio   # from the repo root, or apps/agent-forge/

This builds Agent Forge first (build:studio), then runs Playwright against it.