Skip to content

Add live gh and gh stack end-to-end suite - #41

Merged
tcamise-gpsw merged 12 commits into
tcamise-gpsw/support-non-stacked-mode-a-single-pr-as-a-one-lafrom
tcamise-gpsw/end-to-end-validation-of-gh-and-gh-stack-behavio
Oct 15, 2026
Merged

tcamise-gpsw merged 12 commits into
tcamise-gpsw/support-non-stacked-mode-a-single-pr-as-a-one-lafrom
tcamise-gpsw/end-to-end-validation-of-gh-and-gh-stack-behavio

Conversation

@cam1se

@cam1se cam1se commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Summary

Stacked on #39 (single-PR and no-PR mode, stack #46): merge #39 first. This PR's live tests expect the one-row single-mode view that #39 introduces.

Adds an end-to-end suite that runs the real gh and gh stack against live fixture PRs (issue #23), and fixes three classifier bugs the real binaries exposed:

  • gh 2.101.0 installs a missing stack extension on first use and exits 0 without running the command. The adapter now retries once after seeing the install message; before, the action silently did nothing.
  • gh stack exit 2 also means "not a git repository". Push/sync now map to LocalTrackingMissing only when stderr says the branch is not part of a stack, so a non-repo failure no longer offers to link.
  • gh exit 4 (no token) now maps to Auth; the old text match missed gh's wording.

The fixture lives in this repo under fixture/* (stacks s1 three-PR, s2 conflict, s3 merged, s4 unstacked, plus fork PRs sharing a head branch name), with fixture-start/* reset tags, single-mode fixtures (s4 single PR on main, s5 single PR on a non-default base, s6 no-PR branch, s7 closed, s8 merged), scripts/live-fixture.sh reset|conflict|status, and rulesets that block deleting fixture branches and updating the tags. CI and commit-lint now skip fixture/* pull requests. make live-test runs the suite; a nightly (06:17 UTC) and dispatch-only workflow runs it in CI against a pinned known-good gh/gh-stack pair and the latest releases, so a latest-only failure signals upstream drift. A failed nightly run opens or updates an issue.

Testing

  • cargo fmt --all -- --check: clean (cargo fmt --all run).

  • cargo clippy --workspace --all-targets --locked -- -D warnings: clean.

  • cargo test --workspace --locked --no-fail-fast: all pass (live tests skip without STACK_STAT_LIVE=1).

  • Live: STACK_STAT_LIVE=1 cargo test -p stack-stat-core --test live_gh against this repo: 15 passed in ~150-225s, after fixing two failures the run found (exit-4 auth, and a fork test that did not prove owner disambiguation).

  • TUI smoke via make demo: N/A because dashboard rendering is unchanged; the change is at the gh adapter boundary, which the live suite exercises directly.

  • scripts/live-fixture.sh check passes; rebuild verified by deleting fixture/s6-nopr and rebuilding it. The PR-creation paths of rebuild (stacks s1/s2, PR s5) are untested.

Docs and architecture

  • Living docs: new 07 Live validation; 05, docs index, and AGENTS.md updated.
  • ADR-0004 (Proposed). The no-PTY rule in AGENTS.md is kept.
  • cargo doc --workspace --no-deps --locked: builds.

Notes for reviewers

Closes: #23

@cam1se
cam1se changed the base branch from main to tcamise-gpsw/support-non-stacked-mode-a-single-pr-as-a-one-la October 14, 2026 04:39
@cam1se
cam1se force-pushed the tcamise-gpsw/end-to-end-validation-of-gh-and-gh-stack-behavio branch from 977156d to c4e1de1 Compare October 14, 2026 04:39
@cam1se
cam1se added this pull request to stack #46 October 14, 2026 04:39
@tcamise-gpsw
tcamise-gpsw marked this pull request as ready for review October 15, 2026 20:11
Exit 2 from gh stack also means 'not a git repository', so only 'is not part of a stack' maps to LocalTrackingMissing. gh exit 4 (no token) now maps to Auth. gh installs a missing stack extension on first use and exits 0 without running the command, so the adapter retries once after seeing the install message. Classifier tests now use output observed from gh 2.101.0 and gh-stack v0.1.1.

Refs: #23
Runs the real binaries against fixture PRs (stacked, merged, unstacked, fork with same head branch, conflict). Opt-in with STACK_STAT_LIVE=1 via make live-test; scripts/live-fixture.sh resets fixture branches to their start tags.

Refs: #23
The live workflow runs daily and on dispatch with a serializing concurrency group. CI and commit-lint skip pull requests whose head branch starts with fixture/.

Refs: #23
Records the live fixture approach, keeps the PTY rule, and lists coverage gaps.

Refs: #23
Matrix of known-good (gh 2.101.0, gh-stack v0.1.1) and latest releases so upstream changes that break stack-stat show up as a latest-only failure. Uses the existing GH_ACCESS_TOKEN secret, opens or updates an issue when a scheduled run fails, and carries a temporary push trigger to exercise the workflow from this branch.

Refs: #23
Documents the version matrix, GH_ACCESS_TOKEN usage and failure issue; removes the temporary branch push trigger from the live workflow.

Refs: #23
Adds fixtures for a single PR on the default base (s4), a single PR on a non-default base (s5), a pushed branch without a PR (s6), a closed PR (s7) and a merged PR (s8), and replaces the NotInStack expectations with the one-row single-mode view. The reset script restores s5 and s6; s7 and s8 are read-only.

Refs: #23
Read-only tests share a lock and mutating tests hold it exclusively, resetting the fixture before and, via Drop, after the test. Verified by forcing a failing assertion after the conflict step: the fixture was clean afterwards.

Refs: #23
Fixture PRs stay open: closing and reopening them sends notification emails and the local TUI tests need them open. live-fixture.sh gains check (tags, branches, PR states, fixture and do not merge labels), up (reopen after an accidental close) and rebuild (recreate missing branches from tags and missing s1, s2 and s5 PRs). The live tests run check once per process. Verified by deleting fixture/s6-nopr and rebuilding it.

Refs: #23
A failed fixture check opens a 'Live gh fixture broken' issue instead of a drift issue, so fixture damage is not mistaken for a gh change.

Refs: #23
@tcamise-gpsw
tcamise-gpsw force-pushed the tcamise-gpsw/end-to-end-validation-of-gh-and-gh-stack-behavio branch from 23d96e8 to e55d689 Compare October 15, 2026 20:26
Pushes a no-PR branch with plain git push without local stack tracking and checks a non-fast-forward push is reported as a Git rejection.

Refs: #23
@tcamise-gpsw
tcamise-gpsw merged commit 1f04dad into main Oct 15, 2026
5 checks passed
@tcamise-gpsw
tcamise-gpsw deleted the tcamise-gpsw/end-to-end-validation-of-gh-and-gh-stack-behavio branch October 15, 2026 21:08
@tcamise-gpsw tcamise-gpsw mentioned this pull request Oct 15, 2026
3 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

End-to-end validation of gh and gh stack behavior, runnable in CI

2 participants