Skip to content

feat(journeys): the diagnosis in CI and the HTML report, plus a fix prompt for coding agents - #13

Merged
pyjeebz merged 2 commits into
mainfrom
feat/diagnosis-everywhere
Nov 3, 2026
Merged

pyjeebz merged 2 commits into
mainfrom
feat/diagnosis-everywhere

Conversation

@pyjeebz

@pyjeebz pyjeebz commented Nov 3, 2026

Copy link
Copy Markdown
Owner

#12 made investigate say why a journey step breaks. This PR adds two things:

  • The diagnosis where people look: the PR comment and the HTML report.
  • A fix prompt: plain text for a coding agent, so the next step is a paste.

Fix prompt (fix_prompt, new Result field). Every diagnosis, for URL runs and journey runs, now carries a prompt like this, from a real run on the example shop:

Where: journey "checkout", step 3 "place order" (POST /api/carts/{{cart_id}}/order)
When: It survives ~20 concurrent users (likely 20–30); errors crossed 2% at ~30.
Likely cause (connection_pool, high confidence): 5xx under load while the journey's other steps held — …
Evidence: … · Suggested fixes: … · Detected: server gunicorn/21.2.0
Find the code that handles POST /api/carts/{{cart_id}}/order … make the smallest change that fixes the
cause … Don't change the test setup to make the number look better. Then re-run
`prescale investigate http://127.0.0.1:8412 --journey checkout` to confirm the capacity went up.
  • Built from the Result (fixprompt.py), with no model.
  • The URL loses user:password@, the query and the fragment.
  • Query values in routes and evidence become *** (a preview's x-vercel-protection-bypass, ?token=).
  • The Server header is cut down to a short product token, so the app can't put instructions into the prompt.
  • The re-run command reproduces the diagnosis:
    • --journey NAME
    • every journey that ran together, when it broke only together
    • --urls --path /route for URL runs

Where it shows:

  • Terminal: investigate prints it as plain text, with no box, so it copies cleanly.
  • --json and saved runs: in the fix_prompt field.
  • HTML report (show --html): a new Diagnosis section (evidence, fixes, the prompt) with a Copy button. If the browser refuses the clipboard, the button says "Select and copy".
  • MCP: investigate returns it.
  • PR comment: see below.

prescale ci now diagnoses journey runs, as it already did URL runs. The PR comment shows the diagnosed step, Why, Evidence (journeys) and Fix, with the prompt folded in <details>. A break that only happens together is shown under "Held" as "Breaks together". All of this text is escaped: it can't @mention anyone, render HTML, break the table or close the fence. Query values are redacted across the whole comment.

How it was verified

  • The real prescale ci against the example shop, with local stand-ins for the dashboard upload and GitHub's comment API.
    • The comment has the diagnosis, evidence, fixes and the folded prompt.
    • SHOP_PASSWORD appears 0 times.
  • The real investigate prints the prompt as above.
  • The HTML report from that run, checked in Chromium:
    • the Diagnosis section renders
    • Copy puts the prompt on the clipboard and says "Copied"
    • with the clipboard refused it says "Select and copy"
    • no page errors
  • Tests:
    • ci journey runs pass diagnose=True
    • the comment's sections, the together case, and escaping (@, HTML, backticks, a ``` in evidence)
    • query redaction in the prompt and the comment
    • re-run commands: journey, together, URL with --path
    • "errors crossed 5%" and "p95 latency crossed the 1.5s wall"
    • a hostile Server header
    • credentials and IPv6 in the URL
    • the URL-mode comment
    • the HTML section and its escaping
  • ruff check cli/src cli/tests: clean · pytest cli/tests -q: 361 passed (15 new).

Risk

Tier: risky.

Public surface:

  • the new Result field fix_prompt (additive; schema_version stays 1)
  • the PR comment format: a new diagnosis block, and the URL "Why" text is now escaped
  • prescale ci journey runs now diagnose

Diagnosis in CI costs:

  • a 1s pause
  • about 2s of 1-user replays of the culprit journey (5 to 50 walks, within max-rps), with writes for journeys that change data
  • up to ~50s if the app stays down

The Action README says so, and it's under the same host and write rules.

Rollback: revert. Nothing stored changes meaning.

Fresh-context review: no blockers. Three should-fixes, all done:

  1. Query values in the culprit route reached the prompt, e.g. a Vercel bypass token on a preview URL.
  2. The re-run for a break that only happens together named one journey, so it couldn't reproduce the break.
  3. URL re-runs lacked --path and --urls, and "When" read "latency at ~30".

Also done from the review:

  • the Copy button only says "Copied" when the copy worked
  • the Server header is cleaned
  • the Action README notes the extra writes

Notes for review

  • MCP can't run journeys yet. Its investigate is still URL-only. Adding journey inputs to MCP tools is a one-way door that belongs with Journeys 3 (the MCP mapping driver).
  • Next slice (J1.5c): named classes for "everyone hits the same row" and "N+1 at scale", each with a demo app that PreScale must catch.

Generated by Claude Code

@mintlify

mintlify Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
prescale 🟢 Ready View Preview Nov 3, 2026, 3:43 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@pyjeebz
pyjeebz merged commit 85e5606 into main Nov 3, 2026
4 checks passed

This branch was successfully deployed

1 active deployment
staging - docs — 2a9eaa75 Deployed Nov 3, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants