From 31ea80fc8c5ac528870a7104c901cdb51a1abf6c Mon Sep 17 00:00:00 2001 From: raiseCatError <315733358+raiseCatError@users.noreply.github.com> Date: Sun, 18 Oct 2026 21:27:47 +0530 Subject: [PATCH 1/3] v0.21 release audit: integrated state, verification, QA walkthrough and runbook MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The ledger records the merged state (#411–#424 on dev at 433af8fd, each merge commit), CI on that exact commit, the local gates, benchmarks, the code review's findings (#425, #426, #427), the issue and PR disposition (#403 and #394 closed) and the remaining release gates. The physical QA checklist gains 3.9 (#425) and a walkthrough: three sittings over the twelve passes, with what is unavailable on the QA Mac and the coverage that exists instead. scripts/qa/v021-fixtures.sh builds every fixture in one new folder, with a launcher that gives the QA build its own configuration and session service. The runbook follows the v0.20 release: release branch, merge into master, tag on the merge commit, GitHub release, a manual tap PR, website sync and the approved manual publish, then the post-release smoke and rollback. --- docs/development/v021-ledger.md | 154 ++++++++++++++++++----- docs/development/v021-release-runbook.md | 126 +++++++++++++++++++ docs/testing/v021-physical-qa.md | 96 +++++++++++++- scripts/qa/v021-fixtures.sh | 140 +++++++++++++++++++++ 4 files changed, 477 insertions(+), 39 deletions(-) create mode 100644 docs/development/v021-release-runbook.md create mode 100755 scripts/qa/v021-fixtures.sh diff --git a/docs/development/v021-ledger.md b/docs/development/v021-ledger.md index 69a3cc08..11872072 100644 --- a/docs/development/v021-ledger.md +++ b/docs/development/v021-ledger.md @@ -7,27 +7,28 @@ Automated results here are not physical-terminal validation. ## Baseline (verified 2026-10-18) - `master` = `dev` = `6af5cbab`; latest release v0.20.0 (`65e4de55`); CI green on both. -- Open PRs: draft #394 (`/track` first slice, based on an older line; reconciled by hand in slice 5, not merged). +- Open PRs: draft #394 (`/track` first slice, based on an older line; reconciled by hand in slice 5, not merged; + closed as superseded by #416 in the release audit). - Work happens in disposable worktrees under `/Volumes/T7Tmp/ai-dev/worktrees/notMyShell/`; scratch on the T7. ## Slices | # | Slice | Issues | Branch / PR | Merged commit | State | | --- | --- | --- | --- | --- | --- | -| 0 | Design and ledger | #410 | `docs/v021-design` / [#411](https://github.com/raiseCatError/notMyShell/pull/411) → `dev` | | PR open, CI green | -| 1 | Reliability gates | #403, #408, #388 | `fix/v021-reliability` / [#412](https://github.com/raiseCatError/notMyShell/pull/412) → #411 | | PR open, CI green | -| 2 | `/view` | #397, #402 | `feature/v021-view` / [#413](https://github.com/raiseCatError/notMyShell/pull/413) → #412 | | PR open, CI green | -| 3 | `/edit` | #402 | `feature/v021-edit` / [#414](https://github.com/raiseCatError/notMyShell/pull/414) → #413 | | PR open, CI green | -| 4 | Semantic Output | #386 | `feature/v021-semantic` / [#415](https://github.com/raiseCatError/notMyShell/pull/415) → #414 | | PR open, CI green (one rerun) | -| 5 | `/track` | #393 (#394) | `feature/v021-track` / [#416](https://github.com/raiseCatError/notMyShell/pull/416) → #415 | | PR open, CI green | -| 6 | Modules and diagnostics | #364, #365 | `feature/v021-modules` / [#417](https://github.com/raiseCatError/notMyShell/pull/417) → #416 | | PR open, CI green (one rerun) | -| 7 | Themes and controls | #401, #336 | `feature/v021-themes` / [#418](https://github.com/raiseCatError/notMyShell/pull/418) → #417 | | PR open, CI green | -| 8 | Checkpoint ledger | #410 | `docs/v021-checkpoint` / [#419](https://github.com/raiseCatError/notMyShell/pull/419) → #418 | | PR open, CI green | -| 9 | `/track` refresh and finalization lifecycle | #393 | `fix/v021-track-review-hardening` / [#420](https://github.com/raiseCatError/notMyShell/pull/420) → #419 | | PR open, CI green | -| 10 | `/track` review: repetition safety gate, process cleanup, presentation | #393 | `fix/v021-stabilize` / [#421](https://github.com/raiseCatError/notMyShell/pull/421) → #420 | | PR open, CI green | -| 11 | Flaky CI at the cause | #410 | `fix/v021-flaky-tests` / [#422](https://github.com/raiseCatError/notMyShell/pull/422) → #421 | | PR open, CI green | -| 12 | Bugs: #388 wrapped rows, command colors after a slow zshrc, #365 width | #388, #408, #365 | `fix/v021-bugs` / [#423](https://github.com/raiseCatError/notMyShell/pull/423) → #422 | | PR open, CI green | -| 13 | Stabilization ledger (this) | #410 | `docs/v021-stabilization` → #423 | | PR open | +| 0 | Design and ledger | #410 | `docs/v021-design` / [#411](https://github.com/raiseCatError/notMyShell/pull/411) → `dev` | `a412fe53` | Merged into `dev` | +| 1 | Reliability gates | #403, #408, #388 | `fix/v021-reliability` / [#412](https://github.com/raiseCatError/notMyShell/pull/412) → #411 | `3282ad40` | Merged into `dev` | +| 2 | `/view` | #397, #402 | `feature/v021-view` / [#413](https://github.com/raiseCatError/notMyShell/pull/413) → #412 | `e945c75a` | Merged into `dev` | +| 3 | `/edit` | #402 | `feature/v021-edit` / [#414](https://github.com/raiseCatError/notMyShell/pull/414) → #413 | `934ba7ce` | Merged into `dev` | +| 4 | Semantic Output | #386 | `feature/v021-semantic` / [#415](https://github.com/raiseCatError/notMyShell/pull/415) → #414 | `a97b3f77` | Merged into `dev` (one rerun) | +| 5 | `/track` | #393 (#394) | `feature/v021-track` / [#416](https://github.com/raiseCatError/notMyShell/pull/416) → #415 | `a5f2fd69` | Merged into `dev` | +| 6 | Modules and diagnostics | #364, #365 | `feature/v021-modules` / [#417](https://github.com/raiseCatError/notMyShell/pull/417) → #416 | `ac543bd0` | Merged into `dev` (one rerun) | +| 7 | Themes and controls | #401, #336 | `feature/v021-themes` / [#418](https://github.com/raiseCatError/notMyShell/pull/418) → #417 | `0ea7ca39` | Merged into `dev` | +| 8 | Checkpoint ledger | #410 | `docs/v021-checkpoint` / [#419](https://github.com/raiseCatError/notMyShell/pull/419) → #418 | `f8e53541` | Merged into `dev` | +| 9 | `/track` refresh and finalization lifecycle | #393 | `fix/v021-track-review-hardening` / [#420](https://github.com/raiseCatError/notMyShell/pull/420) → #419 | `d612f120` | Merged into `dev` | +| 10 | `/track` review: repetition safety gate, process cleanup, presentation | #393 | `fix/v021-stabilize` / [#421](https://github.com/raiseCatError/notMyShell/pull/421) → #420 | `0eca52d6` | Merged into `dev` | +| 11 | Flaky CI at the cause | #410 | `fix/v021-flaky-tests` / [#422](https://github.com/raiseCatError/notMyShell/pull/422) → #421 | `ac0f780a` | Merged into `dev` | +| 12 | Bugs: #388 wrapped rows, command colors after a slow zshrc, #365 width | #388, #408, #365 | `fix/v021-bugs` / [#423](https://github.com/raiseCatError/notMyShell/pull/423) → #422 | `7cb857f4` | Merged into `dev` | +| 13 | Stabilization ledger | #410 | `docs/v021-stabilization` / [#424](https://github.com/raiseCatError/notMyShell/pull/424) → #423 | `433af8fd` | Merged into `dev` | ## Decisions @@ -63,7 +64,9 @@ toggle, copy flags, Off). Slice 5: section 5 (`/track` block, refresh, copy, sto ## Open risks -- Merging into `dev`: see "Ordered integration" below. Use regular merge commits, in order. +- Merging into `dev`: done (see "Ordered integration" below). +- #426 (pre-existing): under heavy load, a reply typed right after a prompt-less read starts can be queued. Its live + test can fail on a loaded host; the test's own race is fixed in #427. - #388: absolute rows (`CSI H/f/d`) are deliberately not mapped and scroll regions are not emulated (a program's status row below its region is not stored); the contract is explicit and tested (stabilization). - Nothing in v0.21 has had a physical-terminal pass yet. Every slice issue stays open until its pass is recorded. @@ -165,27 +168,110 @@ runs the maintainer's containers); `docker system df` stays the one Docker verb ## Ordered integration -`dev` is unprotected and allows merge commits; merged branches are not deleted automatically. Merge with regular merge -commits, in order: #411 into `dev`; then for each of #412 … #423 and this ledger PR, retarget it to `dev` -(`gh pr edit N --base dev`), check that its diff shows only its own changes, and merge it (`gh pr merge N --merge`). -Then let `dev` CI pass (push runs include Fedora and both timing smokes) and run `npm run verify:release` on `dev`. +Done on 2026-10-18. #411 to #424 were merged into `dev` in order with regular merge commits (table above). `dev` is +`433af8fd`, whose tree is identical to #424's head `06bc2034`. Every PR head is an ancestor of `dev`, and so is +`master` (`6af5cbab`), so the release needs no conflict resolution. + +## Release audit (2026-10-18) + +An independent pass over the integrated `dev` at `433af8fd`, from a clean detached worktree on the T7 (`npm ci`: 25 +packages, 0 vulnerabilities). Machine: macOS 27 arm64, 8 GB, Node 26.10.0, zsh 5.9, Bash 5.3.20, Fish 4.9.3, with +`NMSH_TEST_TMPDIR=/Volumes/T7Tmp/ai-dev`. It was shared throughout (a browser, ten Docker containers in a VM, another +agent session), with 5.4–6.7 GB of swap in use. + +**CI on the exact commit.** Push run [39135227830](https://github.com/raiseCatError/notMyShell/actions/runs/39135227830) +on `433af8fd`: all 13 jobs green. That covers Quality (Node 26: build, benchmark typecheck, fast suite, diff check), +the full suite on macOS and Ubuntu (Node 26, two shards each), Node 22 compatibility, the Fedora portability subset, +timing smoke on macOS and Ubuntu, and the `CI` and `Verify (22.x, 26.x)` aggregates. + +**Local gates on `433af8fd`.** + +| Gate | Result | +| --- | --- | +| `npm ci` | PASS, 0 vulnerabilities | +| `npm run build` | PASS | +| `npm test` (canonical suite, 4 workers) | FAIL: 3,142 tests, 3,139 pass, 2 skipped, 1 fail, 553 s, load 3 rising to 16 | +| `npm run typecheck:bench` | PASS (run on its own: `verify:release` stops at its first failing step) | +| `npm run bench:smoke` | PASS, every budget, at load 4–5 | +| `git diff --check` | PASS | + +The failure is `inputOwnershipLive` → `live zsh: a waiting read gets the answer typed at once …`. The answer to a +prompt-less `read` was queued, so the read never got it. The file alone passed 3/3 at load 5–8. Four copies at once +(load rising from 8 to 27–33) failed in 2 of 4 copies on `dev` and 3 of 4 on `master` `6af5cbab`, so this is +pre-existing, not from v0.21. Two mechanisms, told apart by a diagnostic run that recorded the queue entries: + +- **Product, pre-existing, open as #426.** A key typed before NMSh has evidence that the command reads input is + held for at most 800 ms while the session looks. On macOS a prompt-less read needs two process-table samples, and + when the look misses the deadline, Enter queues. This was the full-suite failure, and it reappeared at load 20–33 + after the test fix. It was not seen at ordinary load or in CI. +- **Test race, fixed in #427.** The test typed its follow-up `echo BACK-…` as soon as the answer's output appeared + and checked the queue at once. Before the shell's prompt for the answered command arrives, Enter queues, as designed. + This was the CI failure on #425 (Ubuntu) and most of the parallel failures; the recorded entry was the test's own + follow-up. + +**Benchmarks** (`bench:smoke` at `433af8fd`, load 4–5), p50: pty-ready 24 ms, service-ready 233 ms, cold context +collection 50 ms, every-module render 40 ms, configured completion warm 8.8 ms, `transcript/wrap-present-10000` 41 ms, +`transcript/wrap-present-100000` 444 ms. Every one is within budget and within noise of the stack-head numbers above. + +**Code review,** highest impact first: `/edit` saving, `/view` loading, the `/track` lifecycle (`WatchTasks`, +`repeatSafety`, `trackBlock` and the `TerminalApp` wiring), the parser's live region, Semantic Output bounds and caching, +`SemanticService` startup timing, the new module capabilities, and what the Homebrew formula installs. + +- Fixed in #425: a new file in a folder reached through a symlink (macOS `/tmp`, `/var/folders`) hit a false "was + replaced on disk" conflict on its second save. The new file kept its unresolved path as its real path. The + regression fails on `dev`. Physical check 3.9. +- Fixed in #427 (test only): the race above. +- Filed as #426: the product behavior above. +- Noted, not changed: + - The reattach notice says `zsh pid` for every shell (`TerminalApp.finishReplay`, since `b0bd6878`, before shell + backends). Cosmetic. + - `clear` removes `/track` blocks with every other block. The tracker keeps running and its summary row stays, and + `/track stop` writes the final result, which matches `clear`'s contract. + - Stopped trackers keep their last two results until 128 records accumulate. That is bounded, but not small. +- Checked, no defect found: + - `/view` refuses FIFOs and devices before anything blocks, reads in bounded chunks and cuts UTF-8 on a character + boundary. + - `/edit` uses an exclusive temporary file, syncs it, checks the original's identity before renaming, refuses hard + links and keeps symlinks. + - `/track` runs each sample in its own process group, escalates TERM to KILL, ends stragglers after every sample, + refuses overlapping refreshes and ends everything when the window closes. + - `SemanticService` resolves pending lookups when it is killed during startup. + - Semantic readings are cached per record in a `WeakMap`. + - The built-in context packs are copied into `dist`, and the Homebrew formula's install list still covers every + file v0.21 reads at runtime: all new code compiles into `dist`, and no new assets were added. + +**Issues and PRs.** + +- Closed: #403 (a test fixture; verified on `dev` and in CI). Draft PR #394, as superseded by #416 (its scheduler, + panel and all 8 tests are on `dev`). +- Open until their physical passes are recorded: #388, #393, #397, #401, #402, #408 (pass 1.7), #364, #365, #386 and + #336. The work still out of scope stays on each issue: + - #386: external providers, a fully collapsed raw view and a separate semantic search scope. + - #401: four families deferred. + - #402: the provider registry (#366). + - #364: Kotlin/Scala, Lua and Haskell as candidates. + - #365: no per-segment truncation trace for the Main Prompt. + - #336: no other surface needed a new control. +- #356 stays open: it was not reproduced, and passing its reproducer does not close it. +- #410 tracks the release. ## Release readiness -Not ready to release; ready for integration. Exact-head CI on the stack head `e8411e1a` is green in every job, including -Fedora portability and both timing smokes. Remaining gates, in order: +**Not released, and not ready to ship until physical QA passes.** Integration, exact-commit CI and the code review are +done. Remaining gates, in order: -1. **Integration** (maintainer, or the agent where its permission policy allows): the ordered merges above, #411 to - #423 and this ledger PR, with regular merge commits; then `dev` CI green on push. -2. **Local `npm run verify:release` on `dev`** at normal machine load (the in-session runs were overloaded; see above). -3. **Physical QA, mandatory and not run**: every pass in the [checklist](../testing/v021-physical-qa.md), in Ghostty - and Terminal.app, recorded in its results table. Headless PTY tests and CI do not count. -4. Then close the issues whose acceptance is met (#388, #393, #397, #401, #402 viewer and editor phases, #403, #408, - #364, #365, #386 first phase, #336 where applicable), move `[Unreleased]` to `[0.21.0]` in the changelog, and - prepare the tag, release notes and distribution. Tagging, the GitHub release, the Homebrew tap and the website - deploy stay with the maintainer. +1. Merge #425 and #427 into `dev` once their CI is green. This ledger's PR follows. +2. **Release candidate:** a `release/v0.21.0` branch from `dev` with one commit holding the version (`0.21.0`), the + changelog's `[0.21.0]` candidate section and the status pointers, opened as a draft PR to `master`, as v0.20 did + (#392). Then dispatched CI on its exact head (Fedora and both timing smokes) and a local `verify:release` on it. +3. **Physical QA, mandatory and not run:** every pass in the [checklist](../testing/v021-physical-qa.md), on the + candidate build, recorded in its results table. Headless PTY tests and CI do not count. +4. Then, with the maintainer's go-ahead: date the changelog section, merge the release PR into `master`, then the tag, + GitHub release, Homebrew tap and website, as in the [release runbook](v021-release-runbook.md). Close the issues + whose acceptance is met. -Known limitations shipped deliberately: `/track` is window-owned (stops on detach); Semantic Output is its first +Known limitations shipped deliberately: `/track` is window-owned (it stops on detach); Semantic Output is its first phase (no `jc`/`delta` providers); inline absolute cursor positioning is not mapped and scroll regions are not -emulated (#388 contract); the Main Prompt has no per-segment truncation trace (#365); four theme families deferred -(#401); type-ahead *during* a Fish read stays subject to Fish's own editor queries (#356). +emulated (#388 contract); the Main Prompt has no per-segment truncation trace (#365); four theme families are deferred +(#401); type-ahead *during* a Fish read stays subject to Fish's own editor queries (#356); on a heavily loaded machine +a reply typed within about a second of starting a prompt-less read can be queued (#426, pre-existing). diff --git a/docs/development/v021-release-runbook.md b/docs/development/v021-release-runbook.md new file mode 100644 index 00000000..211f6e7f --- /dev/null +++ b/docs/development/v021-release-runbook.md @@ -0,0 +1,126 @@ +# v0.21.0 release runbook + +The exact steps from the v0.21 release candidate to a published v0.21.0, following the v0.20.0 release (PR #392, +tag `v0.20.0` on its merge commit, Homebrew tap PR #3, website sync). State and evidence are in the +[ledger](v021-ledger.md). **Nothing here runs without the maintainer's go-ahead after physical QA.** Tagging, the +GitHub release, the tap and the website deploy are public and hard to undo. + +## How a release moves + +`dev` → `release/v0.21.0` (one release-metadata commit) → PR into `master`, merged with a regular merge commit → +annotated tag `v0.21.0` on that merge commit → GitHub release (no uploaded assets; the source archive is GitHub's tag +archive) → Homebrew tap PR (source formula: `url` and `sha256` of the tag archive) → website sync PR into `master` → +manual **Publish NMSh website** dispatch, approved in the `nmsh-production` environment. No npm package is published +(`package.json` is `private`). `master` needs the `Verify (22.x)` and `Verify (26.x)` checks; the old GitHub Pages +deploy (`deploy.yml`) is disabled. + +## 0. Before starting + +- The physical QA results table has a recorded result for every row on the candidate build (or the maintainer has + decided, in writing, which rows a release may go without, as v0.20 did). +- Dispatched CI is green on the **exact** head of `release/v0.21.0`. Code changes need the full dispatch: Fedora and + both timing smokes run only on dispatch and push. +- `npm run verify:release` has run on that head locally, and any failure is explained by evidence. + +```sh +git fetch origin +RC=$(git rev-parse origin/release/v0.21.0) +gh run list --workflow CI --commit "$RC" --json event,conclusion,url # a workflow_dispatch run, success +``` + +## 1. Final release commit (on `release/v0.21.0`) + +1. `CHANGELOG.md`: `## [0.21.0] - Unreleased (release candidate)` becomes `## [0.21.0] - YYYY-MM-DD` (the + publication date); remove the candidate sentence; keep Compatibility, Upgrade notes and Known limitations accurate + to the QA results. Link references at the bottom: `[Unreleased]: …/compare/v0.21.0...HEAD` and + `[0.21.0]: …/compare/v0.20.0...v0.21.0`. +2. `docs/testing/v021-physical-qa.md`: the recorded results and the release decision. +3. README, ROADMAP, AGENTS, SECURITY, CONTRIBUTING and `llms.txt`: v0.21.0 is the current stable release. +4. Commit, push, dispatch CI on the new head and wait for it: + +```sh +gh workflow run CI --ref release/v0.21.0 +FINAL=$(git rev-parse origin/release/v0.21.0) +gh run list --workflow CI --commit "$FINAL" --json event,conclusion,url +``` + +## 2. Merge into `master` + +```sh +gh pr ready +gh pr merge --merge # a regular merge commit, as #392 +git fetch origin +MERGE=$(git rev-parse origin/master) +test "$(git rev-parse "$MERGE^{tree}")" = "$(git rev-parse "$FINAL^{tree}")" && echo "tree matches the verified commit" +``` + +## 3. Tag and GitHub release + +```sh +git tag -a v0.21.0 -m "v0.21.0 — " "$MERGE" +git push origin v0.21.0 +gh release create v0.21.0 --verify-tag --latest --title "v0.21.0 — <title>" --notes-file <notes.md> +``` + +The release body becomes the website's release page at the next sync. Write it from the changelog section (a shorter +summary is fine) and link to the full CHANGELOG. + +## 4. Verify the published source + +In a shell started with `t7run zsh` (so `$T7_RUN_DIR` is a scratch directory on the T7): + +```sh +mkdir -p "$T7_RUN_DIR/v021-archive" && cd "$T7_RUN_DIR/v021-archive" +curl -fsSL -o v0.21.0.tar.gz https://github.com/raiseCatError/notMyShell/archive/refs/tags/v0.21.0.tar.gz +shasum -a 256 v0.21.0.tar.gz # the tap's sha256 +tar -xOzf v0.21.0.tar.gz notMyShell-0.21.0/package.json | grep '"version"' # "0.21.0" +mkdir src && tar -xzf v0.21.0.tar.gz -C src && cd src/notMyShell-0.21.0 +npm ci && npm run build && node dist/index.js --version # notMyShell 0.21.0, build <MERGE short sha> +``` + +## 5. Homebrew tap (`raiseCatError/homebrew-tap`) + +The scheduled `brew bump` failed daily from 2026-10-13 to 10-17 (`depends_on "python"`); v0.20.0 was a manual PR (#3). +Do the same unless an autobump PR for 0.21.0 has already appeared and is correct: + +1. Branch `nmsh-0.21.0`; in `Formula/nmsh.rb` set `url` to `…/archive/refs/tags/v0.21.0.tar.gz` and `sha256` to the + value from step 4. Nothing else needs changing: every file v0.21 reads at runtime is in `dist`, and the formula's + install list is unchanged since v0.20. +2. Open the PR; `tests.yml` (`brew test-bot`) must pass on macOS and Linux; merge (source-only formula, no bottles). +3. Locally: `brew update && brew upgrade raiseCatError/tap/nmsh && nmsh --version`. + +## 6. Website + +After the tap PR is merged (the sync records the tap's version, and the deploy checks it again): + +```sh +cd website && npm ci +npm run sync:upstream -- --published v0.21.0 --version 0.21.0 +npm test && npm run check && npm run build:launch && npm run test:browser +``` + +Open a PR into `master` with the result. Once it is merged, dispatch **Publish NMSh website** from `master` with the +confirmation `nmsh.raisecaterror.com`, approve the `nmsh-production` deployment, and check the live site. + +## 7. After the release + +- Smoke on the published build: a Homebrew install (`nmsh --version`, start in Ghostty, run a command, `/view + README.md`, `/track df -h`, `/track stop`, `exit`) and a source install's `/update` (offers v0.21.0 as the latest + stable release). +- Bring `dev` to `master`: `git switch dev && git pull --ff-only origin dev && git merge --ff-only origin/master && + git push origin dev` (`master` = `dev` after v0.20 too). +- Close the issues whose acceptance is met, with the physical results; keep partial ones open with the remaining + scope (ledger, "Issues and PRs"). + +## Rollback + +- **Before the tag:** nothing is public yet. A bad merge into `master` is reverted through a PR. +- **A bad release after publication:** fix forward with v0.21.1. A pushed tag may already be cached by Homebrew, + GitHub's archive and users, so do not move or recreate it. Meanwhile: + - `gh release edit v0.20.0 --latest` makes `/update` (it reads `releases/latest`) offer v0.20.0 again. + - Mark v0.21.0 as a prerelease (`gh release edit v0.21.0 --prerelease`) so it is no longer offered. + - Revert the tap PR (formula back to the v0.20.0 `url` and `sha256`) through a tap PR. + - Website: revert its sync PR on `master` and dispatch the publish again (approval required), or roll the Pages + project back to its previous deployment. +- **The tag points at the wrong commit and nothing has consumed it yet** (no release, tap or sync): delete it + (`git push --delete origin v0.21.0`) and tag the verified merge commit. Otherwise fix forward. diff --git a/docs/testing/v021-physical-qa.md b/docs/testing/v021-physical-qa.md index 42d2a7e1..2206075d 100644 --- a/docs/testing/v021-physical-qa.md +++ b/docs/testing/v021-physical-qa.md @@ -2,13 +2,17 @@ ## Status -- **Candidate, not released.** Items are added as each v0.21 slice lands ([ledger](../development/v021-ledger.md)). - Nothing below has been run yet. +- **Candidate, not released.** Every v0.21 slice is merged into `dev` (#411–#424, `433af8fd`); see the + [ledger](../development/v021-ledger.md). Nothing below has been run yet. The [walkthrough](#walkthrough) groups the + passes into three sittings with ready-made fixtures. - **What does not count:** headless PTY fixtures, synthetic keys, captures of a window driven that way, and green CI. Each item needs a person at a real terminal with a real keyboard and pointer. -- **Build under test:** record the exact commit (`node dist/index.js --version`) built from a clean checkout - (`npm ci && npm run build`). -- **Isolation:** a throwaway `HOME` (or `XDG_CONFIG_HOME`) and a temporary working directory. +- **Build under test:** the release candidate commit, built from a clean checkout (`npm ci && npm run build`); record + `node dist/index.js --version` (version and build commit) in every row. +- **Isolation:** `scripts/qa/v021-fixtures.sh` makes a throwaway folder with its own NMSh configuration + (`XDG_CONFIG_HOME`) and session service (`NMSH_RUNTIME_DIR`), so the QA build neither reads your settings nor + attaches to your running sessions. `HOME` is unchanged: your shell's own startup files run as usual, and copy + items replace your clipboard. ## Results @@ -27,6 +31,85 @@ | 7 Themes | Ghostty | macOS | zsh | | Not run | | | 7 Themes | Terminal.app | macOS | zsh | | Not run | | +Not available on the QA Mac, with the coverage that exists instead: 1.5b (`apt` progress; Linux only: parser and live +PTY tests, Ubuntu and Fedora CI), 4.3 `cargo build` (not installed; Rust diagnostics recognizer tests), 6.2 +(`nix-shell`; the module reads only `IN_NIX_SHELL`, unit-tested), 6.3 inside a container (marker files and host +variables, unit-tested; the host half is checked here). Each stays *Not run* in the table, never *Pass*. + +## Walkthrough + +Three sittings, about 90 minutes in all. Every command below is typed in NMSh unless it says "second tab". + +**Prepare once.** In a plain terminal window (not inside NMSh): + +```sh +cd /Volumes/T7Tmp/ai-dev/worktrees/notMyShell/v021-qa # the release candidate build +node dist/index.js --version # record version and build commit +scripts/qa/v021-fixtures.sh /Volumes/T7Tmp/ai-dev/q21 +``` + +Each sitting starts from a plain window: `/Volumes/T7Tmp/ai-dev/q21/nmsh` (zsh), `… nmsh fish` or `… nmsh bash`. NMSh +opens in `q21/work`; paths below are relative to it. Quit with `exit`. If a launch offers to resume a detached +session (after 5.8), press R and `exit` it. + +**Reporting.** One line per pass, for example `Pass 2 · Ghostty 1.3.1 · zsh 5.9: PASS` or `Pass 2: FAIL 2.7 (the FIFO +hung)`; a screenshot helps. Each result is recorded in the table with the build commit. A skipped item is *Not run*. + +### Sitting A: Ghostty, zsh (passes 1–7, about 60 minutes) + +1. **Progress.** 1.1 as listed, then at 40 columns. 1.2 `node progress/multirow.mjs`: two `layer` rows count up in + place; `LOG pulling two layers`, `LOG checkpoint 1`–`4` and `LOG done` all stay, in order. 1.3 `node + progress/multirow.mjs 120` (30 s) while you resize. 1.4 `/copy`, then `pbpaste`: the final rows once. 1.5 as listed. + 1.5a at 40 columns `node progress/wrapped.mjs`: both `LOG kept line` lines and `LOG done` stay, with one `Downloading + … 100%` and one `eta 0s`. 1.6 `vim view/tabs.txt`, `less view/README.md`, `top` (htop is not installed). 1.7 `sleep + 5`, then `ls` Ctrl+Q and `pwd` Ctrl+Q while it runs. +2. **`/view`.** 2.1 `/view view/README.md`. 2.2 `/view view/ViewerPanel.ts`. 2.3 in either. 2.4 as listed. 2.5 `/view + view/tabs.txt`, `c`, Esc, `pbpaste | cat -vet`: `^I` between columns, `$` only at line ends. 2.6 `/view + view/hostile.txt`. 2.7 `/view view/folder`, `view/missing.txt`, `/dev/zero`, `view/fifo` (returns at once), + `/bin/ls`, `view/big.log` (20 MB: the first 8 MiB with a note). 2.8 as listed, with `node progress/multirow.mjs 120` + running. 2.9 in step 8. +3. **`/edit`.** 3.1 `cd edit/empty`, then as listed, with `cat -vet notes.txt` (macOS `cat` has no `-A`): `^I` and a + final `$`. 3.2–3.4 on that file. 3.5 second tab: `echo x >> /Volumes/T7Tmp/ai-dev/q21/work/edit/empty/notes.txt`. + `cd ../..`. 3.6 `/edit edit/crlf-bom.txt`, change a word, save; `xxd edit/crlf-bom.txt | head -1` starts `efbb bf` + and lines end `0d0a`. `/edit edit/mode640.txt`, save, `ls -l edit/mode640.txt` is `-rw-r-----`. `/edit + edit/link.txt`, save, `ls -l edit/link.txt` still points to `target.txt`. 3.7 `/edit /bin/ls`, `/etc/hosts`, + `edit/big.txt` (2 MB), `edit/readonly.txt`, then `/view` each. 3.8 with `view/README.md` and `edit/target.txt`. + 3.9 `/edit edit/alias/new.txt` (`alias` links to `real`). +4. **Semantic Output.** 4.1 `cd semantic/pass && npm test` (40 passing tests stand in for this repository's suite). + 4.2 `cd ../fail && npm test` (2 failing). 4.3 `cd ../ts && npx --no-install tsc --noEmit bad.ts` (2 errors); `cd + ../c && make` (1 error). 4.4 `cd ../../git && git status`, `ls`, `cat tracked.txt`. 4.5 back in `semantic/fail`, + `npm test`, then as listed, with `pbpaste` after each copy. 4.6 as listed. 4.7 in step 8. `cd ../..` when done. +5. **`/track`.** 5.1 `/track df -h`; `/track --every 5s -- docker system df` is optional (read-only, but it asks your + Docker daemon every 5 s). 5.2 `/track --every 2s -- cat track/f.txt`; second tab: `seq 1 2 > + /Volumes/T7Tmp/ai-dev/q21/work/track/f.txt`, then `seq 1 40 > …`. 5.3–5.7 as listed (5.7's `/tmp/t` commands are + refused and never run; answer No to the ones that ask). 5.8 `/track --every 3s -- sh -c 'sleep 2; echo QA-TRACK'` + (Yes), close the window while it runs, then in a new window `pgrep -fl QA-TRACK` prints nothing. 5.9 and 5.10 as + listed. 5.11 in step 8. +6. **Modules.** 6.1 `cd modules/bun-app`, `/modules`: Bun is recommended; enabled, typing `bun` shows `1.1.30` + requested (Bun is not installed, so nothing active). 6.2 Not run (no Nix). 6.3 on this Mac the container module shows + nothing; the in-container half is Not run. 6.4 and 6.5 as listed. `cd ../..`. +7. **Themes.** 7.1–7.3 as listed. 7.4 in step 8. 7.5 second tab, before and after switching themes: `shasum + "$HOME/Library/Application Support/com.mitchellh.ghostty/config"*`: unchanged. Older configuration: `exit`, then in + a plain window `mkdir -p /Volumes/T7Tmp/ai-dev/q21/config-copy/nmsh && cp "$HOME/Library/Application + Support/notMyShell/config.json" /Volumes/T7Tmp/ai-dev/q21/config-copy/nmsh/` and `/Volumes/T7Tmp/ai-dev/q21/nmsh + copy`: the theme and prompt saved by your v0.20 install load unchanged (only the copy can change). +8. **No color and Safe glyphs** (2.9, 4.7, 5.11, 7.4). `exit`, relaunch as `NO_COLOR=1 /Volumes/T7Tmp/ai-dev/q21/nmsh`, + Settings → Appearance → Safe glyphs, then 2.9 on `view/README.md`, 4.7 in `semantic/fail` at 40 columns, 5.11 with + `/track df -h`. Then 7.4's second half: `exit`, relaunch as `env -u COLORTERM TERM=xterm-256color + /Volumes/T7Tmp/ai-dev/q21/nmsh` and switch between two new themes. + +### Sitting B: Terminal.app, Fish (passes 1, 3 and 5, about 20 minutes) + +`/Volumes/T7Tmp/ai-dev/q21/nmsh fish`, then 1.1–1.7 (not 1.5b), 3.1–3.9 and 5.1–5.10 as in sitting A, and 5.11 with +`NO_COLOR=1 /Volumes/T7Tmp/ai-dev/q21/nmsh fish` as in step 8. Terminal.app is a 256-color terminal: note anything that +reads differently from Ghostty. + +### Sitting C: Terminal.app, zsh (passes 2 and 7, about 15 minutes) + +`/Volumes/T7Tmp/ai-dev/q21/nmsh`, then 2.1–2.8 and 7.1–7.5 as in sitting A, and 2.9 and 7.4 as in step 8. + +When every result is recorded: `rm -rf /Volumes/T7Tmp/ai-dev/q21`. + ## 1. Multi-row progress redraws (#388) - [ ] 1.1 `node -e 'for (let i=0;i<=10;i++){process.stdout.write((i?"\x1b[1A\x1b[2K":"")+"Downloading "+i*10+"%\n")}'` @@ -80,6 +163,9 @@ (`chmod 444`) each explain why, and `/view` still opens them. - [ ] 3.8 `e` in `/view README.md` opens the editor on the same file; `vim notes.txt` and `nano notes.txt` still run as usual from the composer. +- [ ] 3.9 A new file in a folder reached through a symlink (macOS `/tmp`, or `alias -> real`): `/edit alias/new.txt`, + type a line, Ctrl+S, type another, Ctrl+S: *Saved* both times and no conflict question (#425); `real/new.txt` has + both lines. ## 4. Semantic Output diff --git a/scripts/qa/v021-fixtures.sh b/scripts/qa/v021-fixtures.sh new file mode 100755 index 00000000..be209355 --- /dev/null +++ b/scripts/qa/v021-fixtures.sh @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# Fixtures for the v0.21 physical QA walkthrough (docs/testing/v021-physical-qa.md). +# +# scripts/qa/v021-fixtures.sh <new-directory> +# +# Creates, inside <new-directory> (which must not exist yet), a launcher for this checkout's build with an NMSh +# configuration and session service of its own, so the QA build never reads your settings or attaches to your running +# sessions, and the small files the checklist uses. Nothing outside that directory is changed, nothing is downloaded +# or installed, and nothing runs as root. Delete the directory once the results are recorded. +set -euo pipefail + +target=${1:-} +if [ -z "$target" ]; then echo 'usage: scripts/qa/v021-fixtures.sh <new-directory>' >&2; exit 2; fi +case $target in /*) ;; *) target="$PWD/$target" ;; esac +if [ -e "$target" ]; then echo "refusing: $target already exists" >&2; exit 2; fi +repo=$(cd "$(dirname "$0")/../.." && pwd -P) +if [ ! -f "$repo/dist/index.js" ]; then echo "build first: (cd $repo && npm ci && npm run build)" >&2; exit 2; fi +# The session service's socket lives under the directory; Unix socket paths are limited to about 100 bytes. +if [ ${#target} -gt 60 ]; then echo "refusing: $target is longer than 60 characters (session sockets need a short path)" >&2; exit 2; fi + +mkdir -p "$target" +root=$(cd "$target" && pwd -P) +work="$root/work" + +for shell in zsh bash fish; do + mkdir -p "$root/config-$shell/nmsh" "$root/run-$shell" + chmod 700 "$root/run-$shell" + printf '{"onboardingComplete":true,"toolsSetupComplete":true,"shellBackend":"%s"}\n' "$shell" > "$root/config-$shell/nmsh/config.json" +done + +cat > "$root/nmsh" <<EOF +#!/usr/bin/env bash +# The QA build ($repo) with this folder's configuration and session service: ./nmsh [zsh|bash|fish|copy] +# "copy" uses config-copy/nmsh, a copy of an existing configuration you make yourself (see the walkthrough). +profile=\${1:-zsh} +case \$profile in zsh|bash|fish|copy) ;; *) echo 'usage: ./nmsh [zsh|bash|fish|copy]' >&2; exit 2 ;; esac +if [ ! -d "$root/config-\$profile/nmsh" ]; then echo "missing $root/config-\$profile/nmsh" >&2; exit 2; fi +mkdir -p "$root/run-\$profile" && chmod 700 "$root/run-\$profile" +cd "$work" || exit 1 +exec env XDG_CONFIG_HOME="$root/config-\$profile" NMSH_RUNTIME_DIR="$root/run-\$profile" NMSH_DISABLE_UPDATES=1 node "$repo/dist/index.js" +EOF +chmod +x "$root/nmsh" + +# 1. Progress redraws (#388) ------------------------------------------------------------------------------------- +mkdir -p "$work/progress" +cat > "$work/progress/multirow.mjs" <<'EOF' +// Two rows redrawn in place (cursor up, erase line), with log lines printed between frames, as concurrent +// downloads and container pulls draw them. node multirow.mjs [steps] (default 24, a quarter second each). +const steps = Number(process.argv[2] ?? 24); +const sleep = ms => new Promise(done => setTimeout(done, ms)); +const bar = (done, width = 20) => `[${'='.repeat(Math.round(done * width)).padEnd(width)}]`; +const layers = ['layer 1/2 sha256:9f86d081884c', 'layer 2/2 sha256:60303ae22b99']; +process.stdout.write('LOG pulling two layers\n'); +let drawn = 0; +for (let step = 0; step <= steps; step += 1) { + let out = drawn ? `\x1b[${drawn}A` : ''; + if (step > 0 && step % 6 === 0) out += `\x1b[2KLOG checkpoint ${step / 6}\n`; + layers.forEach((name, index) => { + const done = Math.min(1, (step / steps) * (index ? 0.8 : 1.2)); + out += `\x1b[2K${name} ${bar(done)} ${Math.round(done * 100)}%\n`; + }); + process.stdout.write(out); + drawn = layers.length; + await sleep(250); +} +process.stdout.write('LOG done\n'); +EOF +cat > "$work/progress/wrapped.mjs" <<'EOF' +// A progress display that counts the terminal rows it drew (its line is wider than a 40-column window), moves up by +// them, erases below and redraws, as indicatif, pip and cargo do. The LOG lines must all stay. +const sleep = ms => new Promise(done => setTimeout(done, ms)); +const rows = text => Math.max(1, Math.ceil(text.length / (process.stdout.columns || 80))); +process.stdout.write('LOG kept line one\nLOG kept line two\n'); +let drawn = 0; +for (let n = 0; n <= 10; n += 1) { + if (drawn) process.stdout.write(`\r\x1b[${drawn}A\x1b[J`); + const frame = [`Downloading big-archive-name.tar.gz [${'='.repeat(n)}${' '.repeat(10 - n)}] ${n * 10}%`, `eta ${10 - n}s`]; + process.stdout.write(`${frame.join('\n')}\n`); + drawn = frame.reduce((sum, line) => sum + rows(line), 0); + await sleep(300); +} +process.stdout.write('LOG done\n'); +EOF + +# 2. /view -------------------------------------------------------------------------------------------------------- +mkdir -p "$work/view/folder" +cp "$repo/README.md" "$work/view/README.md" +cp "$repo/src/viewer/ViewerPanel.ts" "$work/view/ViewerPanel.ts" +printf '\033]8;;https://example.com\a\033[31mred\033[2J\n' > "$work/view/hostile.txt" +printf 'name\tsize\tnote\nalpha\t1\tfirst\nbeta\t22\tsecond\n' > "$work/view/tabs.txt" +mkfifo "$work/view/fifo" +yes 'a log line for the 8 MiB viewer limit check' | head -c 20971520 > "$work/view/big.log" || true + +# 3. /edit -------------------------------------------------------------------------------------------------------- +mkdir -p "$work/edit/empty" "$work/edit/real" +ln -s real "$work/edit/alias" +printf '\357\273\277line one\r\nline two\r\n' > "$work/edit/crlf-bom.txt" +printf 'keep my mode\n' > "$work/edit/mode640.txt" && chmod 640 "$work/edit/mode640.txt" +printf 'the target\n' > "$work/edit/target.txt" && ln -s target.txt "$work/edit/link.txt" +printf 'read only\n' > "$work/edit/readonly.txt" && chmod 444 "$work/edit/readonly.txt" +yes 'a line for the 1 MiB editor limit check' | head -c 2097152 > "$work/edit/big.txt" || true + +# 4. Semantic Output ---------------------------------------------------------------------------------------------- +mkdir -p "$work/semantic/pass/test" "$work/semantic/fail/test" "$work/semantic/ts/node_modules/.bin" "$work/semantic/c" +for kind in pass fail; do + printf '{"name":"qa-%s","private":true,"scripts":{"test":"node --test"}}\n' "$kind" > "$work/semantic/$kind/package.json" + { + echo "import test from 'node:test';" + echo "import assert from 'node:assert/strict';" + for n in $(seq 1 40); do echo "test('adds case $n', () => assert.equal($n + $n, $((n * 2))));"; done + if [ "$kind" = fail ]; then + echo "test('rounds a price', () => assert.equal(Math.round(2.5), 2));" + echo "test('parses a date', () => assert.equal(new Date('2026-10-18').getUTCDate(), 19));" + fi + } > "$work/semantic/$kind/test/math.test.mjs" +done +printf '{"name":"qa-ts","private":true}\n' > "$work/semantic/ts/package.json" +printf 'const count: number = "three";\nexport const total = count + missingValue;\n' > "$work/semantic/ts/bad.ts" +ln -s "$repo/node_modules/typescript/bin/tsc" "$work/semantic/ts/node_modules/.bin/tsc" +printf 'int main(void) {\n return undefined_name;\n}\n' > "$work/semantic/c/bad.c" +printf 'bad.o: bad.c\n\tcc -c bad.c -o bad.o\n' > "$work/semantic/c/Makefile" + +# A small repository for `git status` (no remote, no hooks, a local identity for its one commit). +mkdir -p "$work/git" +printf 'tracked\n' > "$work/git/tracked.txt" +git -C "$work/git" init -q +git -C "$work/git" add tracked.txt +git -C "$work/git" -c user.name=QA -c user.email=qa@example.invalid -c commit.gpgsign=false commit -q -m 'QA fixture' +printf 'untracked\n' > "$work/git/untracked.txt" + +# 5. /track ------------------------------------------------------------------------------------------------------- +mkdir -p "$work/track" +printf 'one\ntwo\nthree\n' > "$work/track/f.txt" + +# 6. Modules ------------------------------------------------------------------------------------------------------ +mkdir -p "$work/modules/bun-app" +printf '{"name":"bun-app","private":true,"packageManager":"bun@1.1.30"}\n' > "$work/modules/bun-app/package.json" +printf '{"lockfileVersion": 1}\n' > "$work/modules/bun-app/bun.lock" + +printf 'QA fixtures ready in %s\nStart the QA build with: %s/nmsh (or: %s/nmsh fish, %s/nmsh bash)\n' "$root" "$root" "$root" "$root" From debbffbe52c5afe98be091cc16e73aafb02360ab Mon Sep 17 00:00:00 2001 From: raiseCatError <315733358+raiseCatError@users.noreply.github.com> Date: Sun, 18 Oct 2026 21:29:37 +0530 Subject: [PATCH 2/3] llms.txt: the latest published stable release is v0.20.0, not v0.18.0 The v0.20.0 release updated the other release pointers (README, ROADMAP, AGENTS, SECURITY, CONTRIBUTING) but not this one. --- llms.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/llms.txt b/llms.txt index 65fb4d87..784db687 100644 --- a/llms.txt +++ b/llms.txt @@ -1,6 +1,6 @@ # notMyShell (NMSh) -Latest published stable release: v0.18.0 (see CHANGELOG.md). Homebrew is the recommended macOS install: brew install raiseCatError/tap/nmsh. Source installation remains available. +Latest published stable release: v0.20.0 (see CHANGELOG.md). Homebrew is the recommended macOS install: brew install raiseCatError/tap/nmsh. Source installation remains available. notMyShell is a terminal frontend that runs on top of a real persistent shell instead of replacing the shell. From 6b3a06e89ca5796ef276a9357a01e148ce7e9f09 Mon Sep 17 00:00:00 2001 From: raiseCatError <315733358+raiseCatError@users.noreply.github.com> Date: Sun, 18 Oct 2026 21:39:32 +0530 Subject: [PATCH 3/3] v0.21 QA walkthrough: name the folder for 4.5 and the no-color 4.7 --- docs/testing/v021-physical-qa.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/testing/v021-physical-qa.md b/docs/testing/v021-physical-qa.md index 2206075d..f6f4b2bd 100644 --- a/docs/testing/v021-physical-qa.md +++ b/docs/testing/v021-physical-qa.md @@ -77,8 +77,8 @@ hung)`; a screenshot helps. Each result is recorded in the table with the build 3.9 `/edit edit/alias/new.txt` (`alias` links to `real`). 4. **Semantic Output.** 4.1 `cd semantic/pass && npm test` (40 passing tests stand in for this repository's suite). 4.2 `cd ../fail && npm test` (2 failing). 4.3 `cd ../ts && npx --no-install tsc --noEmit bad.ts` (2 errors); `cd - ../c && make` (1 error). 4.4 `cd ../../git && git status`, `ls`, `cat tracked.txt`. 4.5 back in `semantic/fail`, - `npm test`, then as listed, with `pbpaste` after each copy. 4.6 as listed. 4.7 in step 8. `cd ../..` when done. + ../c && make` (1 error). 4.4 `cd ../../git && git status`, `ls`, `cat tracked.txt`. 4.5 `cd ../semantic/fail && + npm test`, then as listed, with `pbpaste` after each copy. 4.6 as listed. 4.7 in step 8. `cd ../..` when done. 5. **`/track`.** 5.1 `/track df -h`; `/track --every 5s -- docker system df` is optional (read-only, but it asks your Docker daemon every 5 s). 5.2 `/track --every 2s -- cat track/f.txt`; second tab: `seq 1 2 > /Volumes/T7Tmp/ai-dev/q21/work/track/f.txt`, then `seq 1 40 > …`. 5.3–5.7 as listed (5.7's `/tmp/t` commands are @@ -94,8 +94,8 @@ hung)`; a screenshot helps. Each result is recorded in the table with the build Support/notMyShell/config.json" /Volumes/T7Tmp/ai-dev/q21/config-copy/nmsh/` and `/Volumes/T7Tmp/ai-dev/q21/nmsh copy`: the theme and prompt saved by your v0.20 install load unchanged (only the copy can change). 8. **No color and Safe glyphs** (2.9, 4.7, 5.11, 7.4). `exit`, relaunch as `NO_COLOR=1 /Volumes/T7Tmp/ai-dev/q21/nmsh`, - Settings → Appearance → Safe glyphs, then 2.9 on `view/README.md`, 4.7 in `semantic/fail` at 40 columns, 5.11 with - `/track df -h`. Then 7.4's second half: `exit`, relaunch as `env -u COLORTERM TERM=xterm-256color + Settings → Appearance → Safe glyphs, then 2.9 on `view/README.md`, 4.7 with `cd semantic/fail && npm test` at 40 + columns (`cd ../..` after), 5.11 with `/track df -h`. Then 7.4's second half: `exit`, relaunch as `env -u COLORTERM TERM=xterm-256color /Volumes/T7Tmp/ai-dev/q21/nmsh` and switch between two new themes. ### Sitting B: Terminal.app, Fish (passes 1, 3 and 5, about 20 minutes)