Skip to content

feat(journeys): journeys in the HTML report, compare and MCP summaries - #10

Merged
pyjeebz merged 10 commits into
mainfrom
feat/journeys-everywhere
Nov 3, 2026
Merged

pyjeebz merged 10 commits into
mainfrom
feat/journeys-everywhere

Conversation

@pyjeebz

@pyjeebz pyjeebz commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Stacked on #9. The base is #9's branch, so the diff shows only this slice. It gets retargeted to main once #9 merges.

Journey results now show up wherever results show up:

  • HTML report (--html, show --html): a Journeys section with each journey's verdict (survives, band, the step that breaks first, why), the combined run with any interference, and the --expect answer.
  • prescale compare (terminal, --json, --markdown) diffs each journey by name. In a journey run, a drop past the threshold in any journey is a regression, even when it isn't the weakest journey. That matches the settled M4 rule: confidence labels a regression, it doesn't gate it. A journey that held everything it was tested to, below the old level, is inconclusive, not a drop. New, removed and stale journeys are listed, never counted as regressions.
  • MCP run summaries list each journey compactly (status, survives, the step that breaks, why), plus the combined run's interference and the --expect answer.

How it was verified

Two real runs of browse + checkout: one against the example shop, one against a copy with a 10-connection pool instead of 25 (like a bad deploy).

$ prescale compare --markdown
| journey | old | new | change | breaks at |
| browse | 200+ | 200+ | +0% | — |
| checkout | ~20 | ~10 | -50% 🔴 | place order |
$ prescale compare --fail-on-regression    → exit 1
  • The HTML report from a real two-journey run with interference shows the journeys table, the "Together" box and "Expected ~60 users: no". I checked a screenshot at 1000px.
  • The MCP summary of that run: culprit_journey: checkout, breaks_at_step: "3. place order (POST /api/carts/{{cart_id}}/order)", together.interference: {users_at_break: 15, held_alone: 20}, expect.meets: false. About 1.6 KB.
  • Unit tests cover:
    • a drop in a journey that isn't the weakest still regresses
    • a confident overall gain doesn't make a noisy journey drop confident
    • inconclusive and held rows
    • | in names escaped in markdown
    • HTML escaping of every user-controlled string
    • URL runs unchanged (no journeys section or field)
  • ruff check cli/src: clean · pytest cli/tests -q: 314 passed.

Risk

Tier: risky · Public surface: compare --json gains journeys rows; --fail-on-regression fails on a drop past the threshold in any journey of a journey run; the MCP summary gains journeys, together and expect; the HTML report gains a section. Rollback: revert. These are renderers and comparison logic only; nothing stored or uploaded changes.

Fresh-context review: ok to ship. All its should-fixes are taken:

  • confidence now comes only from drops
  • the docs said "confident drops" gate, but the code (correctly, per M4) gates any drop past the threshold, so the docs were fixed
  • inconclusive and held journeys are labelled as such
  • | is escaped in markdown cells
  • it injected HTML into every journey field: all escaped

Notes for review

prescale ci and the GitHub Action still run URL mode with their own comparison. Journeys in CI (with --stale warn|fail and a per-journey PR comment) is the next PR, J1.4b.



Generated by Claude Code

Copilot AI balanced review requested due to automatic review settings November 2, 2026 16:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@pyjeebz
pyjeebz changed the base branch from feat/journeys-combined to main November 2, 2026 21:29
@pyjeebz
pyjeebz merged commit c81ccb4 into main Nov 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants