Add the gates demo: marimo narrative, policy, and one-box launch - #81
Conversation
…run in hexkit A marimo showcase (deploy/gates-demo/) that proves "hexgate plugs into any app": the notebook is the agent's definition (code + MCP + gate diagram + a complex constraint policy) and the same gated agent runs live in hexkit, with the dashboard as the single source of truth for the policy. - notebook.py: gate diagram, agent code, the docs_agent policy, a live in-kernel gate over a fake Google-Docs MCP server (no key needed), plus BYOK + links to hexkit and the policy console. - gdocs_mcp_server.py: a fake Google-Docs MCP server (the third-party surface). - policy.yaml: default-deny, 3 roles (analyst<editor<admin), argument constraints exercising startswith/endswith/every/count/matches/consts. - run-integrated.sh: brings the whole one-box stack up (platform + marimo + gdocs backend + proxy + front-app). - provision.py: seeds the docs_agent policy so the dashboard shows/edits it and the hexkit agent binds it (seeded before the API starts so its bundle compiles). - boot.py: configurable HEXGATE_NOTEBOOK + publishes the hexkit URL. - Makefile: `make demo-gates` opens the notebook standalone (no key, no snapshot).
665c8fe to
0255891
Compare
guillaume-hexamind
left a comment
guillaume-hexamind
left a comment
There was a problem hiding this comment.
I approve of the logic, and did not found any breaking issues within the PR.
For safety, if you can also check these auto-review comments:
- Provision now runs before the API spawns — a seed failure kills the whole boot — 85/100
deploy/boot.py:115 · correctness · CONFIRMED
provision_serve_token() (now including the new docs_agent seed) runs before _spawn(uvicorn …). Previously the API was already up and past _wait_healthy, so even if minting failed the API stayed serving the notebook. Now any error in _seed_gdocs_agent (reading/parsing policy.yaml, DB/seed error) raises before uvicorn starts — taking down even the working BYOK notebook path.
- SERVE_KEY_FILE written before API is healthy → backend binds against a dead API — 75/100
deploy/boot.py:115 · correctness · PLAUSIBLE
Same reorder. run-integrated.sh:50 treats the presence of SERVE_KEY_FILE as the "platform ready" signal and immediately launches the hexkit gdocs backend binding docs_agent to http://127.0.0.1:8000. The key used to appear only after _wait_healthy; now it appears before uvicorn even starts, so on any non-instant boot the backend's policy bind fails with a connection error — breaking the "run it in a different app" half of the demo.
- Uncaught exception in the interactive "Try it yourself" cell — 70/100
deploy/gates-demo/notebook.py:426 · correctness · CONFIRMED
The cell only catches json.JSONDecodeError. If a user switches the tool dropdown without rewriting the default share_doc args, or enters valid-but-non-object JSON (5, []), json.loads succeeds, then wrapped[tool].ainvoke(args) raises a TypeError/validation error that escapes — rendering a raw traceback instead of an allow/deny result.
- gdocs-backend readiness loop has no failure guard — 65/100
deploy/gates-demo/run-integrated.sh:67 · correctness · CONFIRMED
The loop breaks on success but never fails on timeout (unlike the serve-key wait). If the backend never answers /agents within 60s (crash, port conflict, bad venv), the loop just exhausts and set -e continues to launch the proxy + front-app. hexkit comes up pointing at a dead backend → empty/broken chat UI with no pointer to the real failure in /tmp/gates-agent.log.
…t docs_agent seed Reverts the provision-before-uvicorn reorder (review #81 comments 1 & 2): - run-integrated.sh treats SERVE_KEY_FILE appearing as "platform ready" and binds the hexkit backend to the API, so the key must not exist before uvicorn answers — restore minting after _wait_healthy. - A seed error (bad policy.yaml, DB) must not kill the BYOK notebook path, so the docs_agent seed is now best-effort (logged, non-fatal). docs_agent binds via the SDK's pydantic policy_yaml fallback (no compiled bundle); fine here — the demo never requires signed bundles.
…of a traceback Review #81 comment 3: the cell only caught JSONDecodeError, so valid-but-wrong args (a non-object like 5/[], or keys that don't fit the tool) escaped as a raw traceback. Guard that args is a JSON object and catch any run error, rendering it as a warning callout.
Review #81 comment 4: the backend-readiness loop only broke on success and never signalled a timeout, so a dead backend let the proxy + front-app start pointing at nothing — a broken chat UI with no pointer to the cause. Track readiness and print a clear warning to /tmp/gates-agent.log on timeout (non-fatal: the notebook + dashboard are still worth bringing up).
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Thanks Guillaume — all four addressed (3 commits appended). 1 & 2 (provision reorder) — reverted. You're right; that reorder traded a non-problem for two real ones. I'd moved provision before uvicorn only to get
3 (try-it cell) — fixed. Now guards that args is a JSON object and catches any run error, rendering it as a warning callout instead of a traceback (covers 4 (backend readiness) — fixed. The loop now tracks readiness and prints a clear pointer to Verified: ruff + compile clean, |
Sections 1 and 3 interpolated column-0 file content (gdocs server, policy.yaml) into an indented f-string. That defeats marimo's dedent — the common indent collapses to 0, so the prose renders as an indented code block and the file's leading `#` lines leak as <h1> headings. Dedent the prose separately, then append the fenced code block flush-left.
…e gates README Section 2 now walks the full hexgate+hexkit demo: build/spawn commands, the hexkit logins (ana/ed/adah, password hexademo), and a per-role script of exact prompts with the expected allow/deny and the policy reason, plus editing the docs_agent policy live in the dashboard. The Daytona build/spawn steps are flagged interim (local scripts, pinned to feat/gates-demo until #81 merges).
…o-end launcher)
deploy/daytona_full_{snapshot,spawn}.py build + run the whole hexgate+hexkit
gates demo in one Daytona sandbox — a manually-triggerable end-to-end path in
this branch. Refs default to main (override HEXGATE_REF/HEXKIT_REF); until #81
is on main, build with HEXGATE_REF=feat/gates-demo (see gates-demo/README.md).
Wiring this into CI + a one-click launch is the post-#81 follow-up. The pure
experiment scripts (daytona_phase_a/spawn) stay out.
Rewrote the markdown for skim-while-you-talk: one takeaway per section, shorter headlines and one-liners, less meta-framing. Dropped the 7-row DSL grammar table for 3 plain-English rules (the grammar lives in policy_dsl_demo.py), and trimmed the live decision table from 13 rows to 3 sharp pairs — the same call split only by role or argument. Logic, widgets, and the runnable gate are unchanged.
…im) + clearer spawn diagnostics The gates code isn't on main until #81 merges, so default HEXGATE_REF to feat/gates-demo — the demo builds with no override for now (flip to main at merge; the productionization follow-up does this). Also: daytona_full_spawn now dumps /tmp/gates.log when nothing comes up and only greps logs that exist, so a "snapshot missing the gates code" failure reads clearly instead of as a fake OOM.
The cooldown returned "One launch at a time" and blocked opening a second demo from the same machine within 20s — a problem for a live showcase (and easy to mistake for a Modal endpoint limit). Default it to 0 (off); the concurrent cap (20) + daily budget (200) remain the real cost guards, and the knob can be set > 0 for a public/abuse-prone deploy.
… count The concurrent-cap check failed CLOSED on any daytona.list() error and swallowed the exception, so a list() failure showed as a permanent "Demo is at capacity" with no clue why (even with zero sandboxes). Now: log the error, fail OPEN (the daily budget stays the hard cap so a blip can't brick a live demo), and add GET /debug that returns the daytona SDK version + live count or the real error.
… len() daytona SDK 0.197 changed list() to return a generator, so len(daytona.list()) raised "object of type 'generator' has no len()" on every count read — which (failing closed) showed as a permanent "Demo is at capacity" even with zero sandboxes. Materialize with list() in _running_count and /debug.
…ank 500 The /launch handler had no try/except around daytona.create()+boot, so a failure (commonly a Daytona concurrent-sandbox/resource quota when one demo is already live) became an unhandled "Internal Server Error". Wrap it: log the exception and return a readable page with the error type + message.
Each hexgate-demo sandbox was 4 vCPU / 8 GB, so only a couple ran at once within the Daytona account quota. The demo peaks at ~0.76 GB, so 1 GB is plenty; drop to 1 vCPU / 1 GB / 5 GB disk to fit ~10 concurrent. Bump memory to 2 if a live run OOMs. Requires a snapshot rebuild (--force).
… sandboxes Review #81 findings: - provision: _seed_gdocs_agent now opens its OWN session and runs after the mint transaction, so a seed commit failure can't poison the token-mint session (which the demo actually depends on). Still best-effort/logged. - daytona_snapshot: bump per-sandbox memory 1 -> 2 GB — headroom over the ~0.76 GB idle peak for a live LLM turn + the dashboard build, still ~5x concurrency vs the old 8 GB.
…ive-demo notebook The main (Playground) demo notebook now opens with a mermaid schema of the gate (agent → PolicyEnforcer → allow/deny/approval → tool, approvals in the Playground), and adds a block showing a sample policy.yaml (default-deny + allow/approval_required + an argument constraint) mapped to the demo's own tools, so the audience sees what a policy looks like before editing it in the Playground.
…GATE_REF A committed default pointing at a feature branch would break on main after the branch merges (and blocks a clean merge). Default HEXGATE_REF/HEXKIT_REF to main; pass HEXGATE_REF=feat/gates-demo to build the full demo until #81 lands.
guillaume-hexamind
left a comment
guillaume-hexamind
left a comment
There was a problem hiding this comment.
Logic still looks good. Below two /code-review potential findings; you check them and ship then!
-
deploy/spawner/modal_app.py:232 — concurrent-cap flipped from fail-closed to fail-open.
The guard changed from count is None or count >= MAX to count is not None and count >= MAX. When _running_count() can't read Daytona (API outage, auth/SDK breakage), launches now proceed instead of being blocked. The old code explicitly guaranteed "an API blip can't turn the cap into unlimited launches" — that guarantee is gone. Up to the 200/day budget can spin up at once, blowing past the concurrent-sandbox quota and running up compute cost. -
deploy/spawner/modal_app.py:45 — per-IP rate limit disabled entirely.
PER_IP_COOLDOWN_SECONDS lowered 20 → 0, and the cooldown check short-circuits on 0. A single IP can open sandboxes back-to-back with no throttle, draining the shared concurrent cap / daily budget by itself — legitimate visitors then hit "at capacity."
Review #81 (Guillaume): the cap had flipped to fail-open, dropping the "an API blip can't turn the cap into unlimited launches" guarantee. The reason for the flip was the daytona.list() generator bug (count always None → permanent "at capacity") — now fixed by the list() wrap, so fail-closed no longer false-bricks. Restore `count is None or count >= MAX`; logging + /debug stay for diagnosis.
|
Thanks Guillaume, both handled: 1 (fail-open cap) — fixed (331835f). Restored the fail-closed guard ( 2 (per-IP cooldown off) — intentional, keeping it off. Two reasons:
Cost stays bounded without it: the concurrent cap (20, fail-closed) and the daily budget (200) are the hard guards. For a truly public/unattended endpoint the right protection is the Turnstile hook (already wired, just needs the secret), not the per-IP timer. Left Thanks for the review! |
2 participants
This is the hexgate half of a one-click, end to end demo.
The point of the demo is that hexgate plugs into whatever app you already run your agents in. Three surfaces:
docs_agentpolicy and edits hot-reload into hexkit.What's in it:
deploy/gates-demo/: the marimo notebook, a fake Google Docs MCP server, the policy (default deny, three roles, per-argument constraints), andrun-integrated.shto bring the whole box upprovision.py: seeds thedocs_agentpolicy so the dashboard shows it and hexkit binds itboot.py: can point marimo at any notebook and publishes the hexkit url for the notebook to link tomake demo-gates: opens the notebook on its own, no key and no sandboxPairs with the hexkit PR above. Both are needed to run the full demo.
Stacked on #78 for now, since it builds on that branch.