Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,16 @@
pinned older `prescale-version` keeps working). Until the dashboard compares journey runs only
with journey baselines, a journey run's PR comment shows no change instead of a possibly
misleading one, and never fails on it.
- **A fix prompt for your coding agent.** Every diagnosis now comes with `fix_prompt`: plain
text that says where it broke, at how many users, the likely cause, the evidence and the
fixes, and asks the agent for the smallest fix and to re-run PreScale to confirm. It's built
from the run itself, with no model, and never includes response bodies, secrets or
credentials from the URL. `investigate` prints it, `--json` and saved runs carry it, the
HTML report (`show --html`) has a Diagnosis section with a Copy button, MCP's `investigate`
returns it, and the PR comment folds it under the diagnosis.
- **Journey runs in CI are diagnosed.** `prescale ci` and the GitHub Action now probe the
journey step that broke first (as URL runs already were), so the PR comment says why it broke
and how to fix it, not just where.
- **`investigate` diagnoses the journey step that breaks first.** With journeys in
`prescale/journeys/`, `prescale investigate` ramps them like `run`, then probes the step that
broke first with the journey's own state (its 1-user latency comes from replaying the journey
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ Try this

With journeys in `prescale/journeys/`, `investigate` diagnoses the journey step that breaks first instead, reaching it the way a user does (logged in, cart filled): `culprit checkout, step 3 place order`. The journey's other steps play the part of the static assets. `--urls` investigates URLs.

Every diagnosis ends with a **fix prompt**, plain text to paste into your coding agent (`--json | jq -r .fix_prompt`); the PR comment folds it under the diagnosis.

### `journeys` — your users' real paths

A journey is a path a user takes through your app (log in, add to cart, place an order), saved as the requests it makes. Journeys live in your repo as `prescale/journeys/<name>.yaml`:
Expand Down
2 changes: 1 addition & 1 deletion action/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ jobs:

## Journeys

If the checkout has journey files (`prescale/journeys/*.yaml`, see the main README), the action replays them at load instead of testing URLs: each journey alone, then all together. The PR comment shows each journey's verdict, any interference, and journeys that no longer replay against the build. Give journeys their secrets through the step's `env:` (for `{{env.NAME}}`):
If the checkout has journey files (`prescale/journeys/*.yaml`, see the main README), the action replays them at load instead of testing URLs: each journey alone, then all together. The PR comment shows each journey's verdict, any interference, and journeys that no longer replay against the build. When a journey breaks, the step that broke first is diagnosed (the bottleneck, the evidence, the fix), with a fix prompt to paste into your coding agent folded underneath. Diagnosing replays that journey at one user for a few more seconds (5 to 50 walks, within `max-rps`), so a journey that changes data makes a few more writes: run those only against staging or a replica. Give journeys their secrets through the step's `env:` (for `{{env.NAME}}`):

```yaml
- uses: pyjeebz/prescale/action@v1
Expand Down
41 changes: 35 additions & 6 deletions cli/src/prescale_cli/ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import httpx

from prescale_cli.bars import bar_rows, route_rows
from prescale_cli.fixprompt import redact_query
from prescale_cli.journeys.load import LABELS
from prescale_cli.throughput import VISITOR_NOTE, capacity_rps, visitors_for

Expand Down Expand Up @@ -261,6 +262,33 @@ def render_stale_only(runs, *, target: str) -> str:
return "\n".join(lines) + "\n"


def _fenced(text: str) -> list[str]:
"""TEXT as a fenced block that nothing inside can close early or @mention from."""
safe = text.replace("```", "'''").replace("@", "@\u200b")
return ["```text", *safe.splitlines(), "```"]


def _diagnosis(result: dict, inv: dict, *, together: bool = False) -> list[str]:
"""Why it broke, the fixes, and the fix prompt (folded), all escaped."""
lines = []
step = inv.get("culprit_step")
if step:
lead = "Breaks together" if together else "Diagnosed step"
lines.append(f"**{lead}:** {_cell(inv.get('culprit_journey') or '')}, step "
f"{step['index']} {_cell(step['name'])} {_code(step['route'])}")
lines.append(f"**Why:** {_code(inv['bottleneck_class'])} ({_cell(inv['confidence'])} "
f"confidence) · {_cell(inv['summary'])}")
if step and inv.get("evidence"):
lines += ["", "**Evidence:**"] + [f"- {_cell(e)}" for e in inv["evidence"]]
if inv.get("remediation"):
lines += ["", "**Fix:**"] + [f"- {_cell(fix)}" for fix in inv["remediation"]]
prompt = result.get("fix_prompt")
if prompt:
lines += ["", "<details><summary>Fix prompt for your coding agent</summary>", "",
*_fenced(prompt), "", "</details>"]
return lines


def render_comment(result: dict, upload: dict, *, target: str, sha: str | None) -> str:
"""The PR comment: the change vs baseline, what breaks first, why, the fix."""
verdict = result["verdict"]
Expand Down Expand Up @@ -337,18 +365,18 @@ def render_comment(result: dict, upload: dict, *, target: str, sha: str | None)
investigation = result.get("investigation")
lines.append("")
if verdict.get("onset_users") is None:
lines.append(f"**Held:** {_onset(verdict)} without crossing a failure threshold.")
lines.append(f"**Held:** {_onset(verdict)} without crossing a failure threshold"
+ (" (each journey on its own)." if investigation else "."))
if investigation: # the journeys broke only when ramped together
lines += _diagnosis(result, investigation, together=True)
else:
culprit = verdict.get("culprit_route")
journey = verdict.get("culprit_journey")
where = f"{_cell(journey)} › {_code(culprit)}" if journey and culprit else _code(culprit)
lines.append(f"**Breaks first:** {where} · {_onset(verdict)}" if culprit
else f"**Breaks first:** {_onset(verdict)}")
if investigation:
lines.append(f"**Why:** `{investigation['bottleneck_class']}` "
f"({investigation['confidence']} confidence) · {investigation['summary']}")
if investigation.get("remediation"):
lines += ["", "**Fix:**"] + [f"- {fix}" for fix in investigation["remediation"]]
lines += _diagnosis(result, investigation)
# The routes at the stage that broke, as bars toward the latency wall (│).
stages = result.get("stages", [])
stage = next((s for s in stages if s.get("users") == verdict["onset_users"]), None)
Expand All @@ -375,4 +403,5 @@ def render_comment(result: dict, upload: dict, *, target: str, sha: str | None)
if target_info.get("verified") is False:
notes.append(f"{target} isn't verified yet, so runs can't be signed off.")
lines += ["", f"<sub>PreScale{' · ' + ' '.join(notes) if notes else ''}</sub>"]
return "\n".join(lines) + "\n"
# Routes can carry tokens in their query (a preview's bypass, ?token=): never on a PR.
return redact_query("\n".join(lines)) + "\n"
2 changes: 1 addition & 1 deletion cli/src/prescale_cli/commands/ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ def _ci_journeys(journeys, url: str, *, target: str, ctx, creds, headers: dict[s
try:
_, result, _ = run_journeys(journeys, url, config, latency_wall=2.0,
error_threshold=0.02, timeout=10.0, headers=headers,
replica=replica, as_json=False)
replica=replica, as_json=False, diagnose=True)
except NoJourneyReplaysError as exc:
body = render_stale_only(exc.runs, target=target)
_publish(body, target=target, ctx=ctx, comment=comment, pr_number=pr_number,
Expand Down
3 changes: 2 additions & 1 deletion cli/src/prescale_cli/commands/investigate.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from prescale_cli.live import LiveRamp, ramp_progress
from prescale_cli.loadtest import LoadError
from prescale_cli.profiles import PROFILES, lookup, scenario_block
from prescale_cli.render import render_investigation, render_terminal
from prescale_cli.render import render_fix_prompt, render_investigation, render_terminal
from prescale_cli.result import latest_id, load_result, write_result

console = Console()
Expand Down Expand Up @@ -214,6 +214,7 @@ def _show(result: dict, *, as_json: bool, live_mode: bool, upload: bool,
else:
render_terminal(result, show_ramp=not live_mode)
render_investigation(result)
render_fix_prompt(result)
if result.get("audit"):
console.print("\n[bold]HTTP hygiene[/bold]")
for finding in result["audit"]:
Expand Down
2 changes: 2 additions & 0 deletions cli/src/prescale_cli/commands/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
from prescale_cli.commands.journeys import dir_option, refuse_missing_env, refuse_unallowed_writes
from prescale_cli.commands.replicaopts import app_option, app_timeout_option, start_replica
from prescale_cli.commands.upload import header_option, upload_after_run, upload_option
from prescale_cli.fixprompt import fix_prompt
from prescale_cli.journeys.diagnose import diagnose_run
from prescale_cli.journeys.load import (
LABELS,
Expand Down Expand Up @@ -474,6 +475,7 @@ def run_journeys(journeys: list[Journey], url: str, config: dict, *, latency_wal
headers=headers, max_rps=config["max_rps"]))
if investigation is not None:
result["investigation"] = investigation
result["fix_prompt"] = fix_prompt(result)
return worst.report, result, live_mode


Expand Down
126 changes: 126 additions & 0 deletions cli/src/prescale_cli/fixprompt.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
"""A ready-to-paste prompt for a coding agent, from an investigated Result.

Built from the Result's own fields (where it broke, at how many users, the
diagnosis, its evidence and fixes), so it says nothing the run didn't measure.
No model writes it. It never includes response bodies, headers sent, or secrets:
only what's already in the Result, with any credentials in the URL removed.
"""

from __future__ import annotations

import re
import shlex
from urllib.parse import urlsplit, urlunsplit


def _safe_url(url: str) -> str:
"""URL without user:password@, query or fragment."""
parts = urlsplit(url)
host = parts.hostname or ""
if ":" in host: # IPv6
host = f"[{host}]"
if parts.port:
host = f"{host}:{parts.port}"
return urlunsplit((parts.scheme, host, parts.path, "", ""))


# key=value in a query string: the value may be a token (e.g. a preview's
# protection bypass), so it never leaves in a prompt or a PR comment.
_QUERY_VALUE = re.compile(r"([?&][^\s=&?#]+)=[^\s&#)`'\"|<>]*")


def redact_query(text: str) -> str:
"""TEXT with every query-string value replaced by ***."""
return _QUERY_VALUE.sub(r"\1=***", str(text))


def _server(value: str) -> str:
"""A Server header as a short product token: the app controls it, so it gets
no room to carry instructions into an agent's prompt."""
return re.sub(r"[^\w./()+ -]", "", value)[:60].strip()


def _when(result: dict) -> str:
verdict = result["verdict"]
config = result.get("config") or {}
together = result.get("combined") or {}
conf = verdict.get("confidence") or {}
band = ""
if conf.get("survives_low") is not None and conf.get("survives_high") is not None:
band = f" (likely {conf['survives_low']}–{conf['survives_high']})"
if verdict.get("onset_users") is not None:
reason = {
"errors": f"errors crossed {config.get('error_threshold', 0.02):.0%}",
"latency": f"p95 latency crossed the {config.get('latency_wall_s', 2.0):g}s wall",
}.get(verdict.get("onset_reason"), verdict.get("onset_reason") or "it failed")
return (f"It survives ~{verdict['survives_users']} concurrent users{band}; "
f"{reason} at ~{verdict['onset_users']}.")
if together.get("onset_users") is not None:
return (f"Each journey held on its own, but together they survive "
f"~{together['survives_users']} concurrent users in total.")
return "It degraded under load."


def _rerun(result: dict, url: str) -> str:
"""The command that reproduces this run's diagnosis."""
inv = result["investigation"]
verdict = result["verdict"]
together = result.get("combined") or {}
cmd = f"prescale investigate {shlex.quote(url)}"
if (result.get("config") or {}).get("mode") == "journeys" or inv.get("culprit_journey"):
if verdict.get("onset_users") is None and together.get("onset_users") is not None:
# It broke only with the journeys together: rerun all of them.
names = together.get("journeys") or []
else:
names = [inv["culprit_journey"]] if inv.get("culprit_journey") else []
return cmd + "".join(f" --journey {shlex.quote(n)}" for n in names)
cmd += " --urls"
route = urlsplit(inv["culprit_route"]).path
if route and route != "/":
cmd += f" --path {shlex.quote(route)}"
return cmd


def fix_prompt(result: dict) -> str | None:
"""The prompt, or None when there's no diagnosis to act on."""
inv = result.get("investigation")
if not inv:
return None
url = _safe_url(result["target"]["url"])
step = inv.get("culprit_step")
journey = inv.get("culprit_journey")
route = redact_query(inv["culprit_route"])

if step and journey:
where = (f'journey "{journey}", step {step["index"]} "{step["name"]}" '
f"({redact_query(step['route'])})")
else:
where = route

lines = [
f"PreScale load-tested {url} and found what breaks first under load.",
"",
f"Where: {where}",
f"When: {_when(result)}",
f"Likely cause ({inv['bottleneck_class']}, {inv['confidence']} confidence): "
f"{inv['summary']}",
]
evidence = [(f"server: {_server(e[8:])}" if e.startswith("server: ") else redact_query(e))
for e in inv.get("evidence") or []]
if evidence:
lines += ["", "Evidence:"] + [f"- {e}" for e in evidence]
if inv.get("remediation"):
lines += ["", "Suggested fixes:"] + [f"- {r}" for r in inv["remediation"]]
stack = {k: _server(v) for k, v in (inv.get("stack") or {}).items()}
if stack:
lines += ["", "Detected: " + ", ".join(f"{k} {v}" for k, v in stack.items())]

lines += [
"",
f"Find the code that handles {route} (and what it depends on: database pool, "
"workers, locks, upstream calls). Check it against the diagnosis above, make the "
"smallest change that fixes the cause, and explain it. Don't change the test "
f"setup to make the number look better. Then re-run `{_rerun(result, url)}` to "
"confirm the capacity went up.",
]
return "\n".join(lines)
2 changes: 2 additions & 0 deletions cli/src/prescale_cli/investigate.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import httpx

from prescale_cli.audit import _CDN_MARKERS, AuditError, extract_assets, run_audit
from prescale_cli.fixprompt import fix_prompt
from prescale_cli.loadtest import (
analyze,
build_targets,
Expand Down Expand Up @@ -347,6 +348,7 @@ async def investigate(url, *, max_users=200, paths=(), stage_seconds=5.0,
"remediation": remediate(diag, stack),
"stack": stack,
}
result["fix_prompt"] = fix_prompt(result)

if profile is not None:
result["profile"] = scenario_block(profile, report.survives_users)
Expand Down
1 change: 1 addition & 0 deletions cli/src/prescale_cli/mcp_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ async def investigate_summary(url: str, *, allow: set[str],
host_permission=permission_from_env())
summary = summarize_result(result)
summary["investigation"] = result.get("investigation")
summary["fix_prompt"] = result.get("fix_prompt")
return summary


Expand Down
10 changes: 10 additions & 0 deletions cli/src/prescale_cli/render.py
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,16 @@ def render_investigation(result: dict) -> None:
console.print(Panel("\n".join(lines), title="🔬 Diagnosis", border_style=color))


def render_fix_prompt(result: dict) -> None:
"""Print the fix prompt as plain text (no box), so it copies cleanly."""
prompt = result.get("fix_prompt")
if not prompt:
return
console.print("\n[bold]📋 Fix prompt[/bold] [dim](paste into Claude Code, Cursor or "
"your coding agent; also in --json as fix_prompt)[/dim]\n")
console.print(escape(prompt), soft_wrap=True, highlight=False)


def _band(verdict: dict) -> str:
conf = verdict.get("confidence") or {}
lo, hi = conf.get("survives_low"), conf.get("survives_high")
Expand Down
40 changes: 40 additions & 0 deletions cli/src/prescale_cli/report.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,12 @@
td.n{white-space:nowrap;}
.cause p+p{margin-top:8px;}
.cause b{color:var(--ink);}
.prompt{position:relative;margin-top:10px;}
.prompt pre{white-space:pre-wrap;font-family:var(--mono);font-size:12px;color:var(--muted);
background:var(--canvas);border:1px solid var(--hair);border-radius:8px;padding:14px;}
.prompt button{position:absolute;top:8px;right:8px;font:500 11px var(--sans);
color:var(--ink);background:var(--s2);border:1px solid var(--hair);border-radius:6px;
padding:4px 10px;cursor:pointer;}
footer{margin-top:44px;padding-top:20px;border-top:1px solid var(--hair);display:flex;
justify-content:space-between;font-family:var(--mono);font-size:11.5px;color:var(--tertiary);}
"""
Expand Down Expand Up @@ -268,6 +274,39 @@ def _cause(result: dict) -> str:
return f'<div class="cause"><div class="lbl">Likely cause</div>{"".join(paras)}</div>'


def _diagnosis_html(result: dict) -> str:
"""`investigate`: why it broke, the evidence, the fixes, and the fix prompt."""
inv = result.get("investigation")
if not inv:
return ""
step = inv.get("culprit_step")
if step:
where = (f"{_esc(inv.get('culprit_journey') or '')}, step {step['index']} "
f"<b>{_esc(step['name'])}</b> <code>{_esc(step['route'])}</code>")
else:
where = f"<code>{_esc(inv['culprit_route'])}</code>"
parts = [f"<p>{_esc(inv['summary'])}</p>",
f"<p><b>{_esc(inv['bottleneck_class'])}</b> ({_esc(inv['confidence'])} "
f"confidence) {MIDDOT} {where}</p>"]
if inv.get("evidence"):
parts.append("<div class=\"lbl\" style=\"margin-top:12px\">Evidence</div><ul>"
+ "".join(f"<li>{_esc(e)}</li>" for e in inv["evidence"]) + "</ul>")
if inv.get("remediation"):
parts.append("<div class=\"lbl\" style=\"margin-top:12px\">Try this</div><ul>"
+ "".join(f"<li>{_esc(r)}</li>" for r in inv["remediation"]) + "</ul>")
prompt = result.get("fix_prompt")
if prompt:
parts.append(
'<div class="lbl" style="margin-top:12px">Fix prompt for your coding agent</div>'
'<div class="prompt"><button type="button" onclick="var b=this;'
"navigator.clipboard.writeText(b.nextElementSibling.textContent).then("
"function(){b.textContent='Copied'},function(){b.textContent='Select and copy'})"
'">Copy</button>'
f"<pre>{_esc(prompt)}</pre></div>")
return ('<div class="section"><h3>Diagnosis</h3><div class="cause">'
+ "".join(parts) + "</div></div>")


def _journeys_html(result: dict) -> str:
"""Journey runs: each journey's verdict, the combined pass, and --expect."""
journeys = result.get("journeys")
Expand Down Expand Up @@ -389,6 +428,7 @@ def render_html(result: dict) -> str:
</div>
{_route_table(result)}
{_journeys_html(result)}
{_diagnosis_html(result)}
<footer><span>Generated by prescale {result["tool_version"]}</span>
<span>{meta}</span></footer>
</div></body></html>
Expand Down
4 changes: 4 additions & 0 deletions cli/src/prescale_cli/schema/result.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@
"untested": { "type": "array", "items": { "type": "string" }, "description": "Journeys that weren't load-tested; their share of N is unknown." }
}
},
"fix_prompt": {
"type": ["string", "null"],
"description": "Present with `investigation`: a plain-text prompt for a coding agent (where it broke, the diagnosis, evidence and fixes). Built from the Result itself; never includes response bodies or secrets."
},
"investigation": {
"type": "object",
"description": "Present when `prescale investigate` diagnosed a failure.",
Expand Down
Loading
Loading