Skip to content

Layer 4.2: Registry cleanup, docs, and full validation (FEAT-008)#3706

Merged
Hmbown merged 18 commits into
Hmbown:mainfrom
aboimpinto:feat/FEAT-008-layer-4-2-registry-cleanup-docs-validation
Jun 27, 2026
Merged

Layer 4.2: Registry cleanup, docs, and full validation (FEAT-008)#3706
Hmbown merged 18 commits into
Hmbown:mainfrom
aboimpinto:feat/FEAT-008-layer-4-2-registry-cleanup-docs-validation

Conversation

@aboimpinto

Copy link
Copy Markdown
Contributor

Refs #2870.
Follows #3652.

Summary

Layer 4.2 / FEAT-008 completes the registry cleanup, source-verified command architecture docs, and final validation evidence for the staged command-boundary work.

This branch was replayed onto current upstream main so the PR contains only the FEAT-008 surface after the already-merged Layer 4.1 work from #3652.

What changed

  • Added registry guard coverage for duplicate built-in command names and aliases.
  • Added an ownership-contract test for the final 9 group-owned command areas.
  • Added a Gherkin eval smoke test proving the TUI binary loads and the eval harness reports step-level shell execution success.
  • Updated docs/architecture/command-dispatch.md with the finalized command dispatch flow, command group boundaries, and permanent exceptions.
  • Updated docs/architecture/pr-issue-evidence-prep.md with Layer 4.2 closure evidence and validation notes.

Gherkin E2E examples

New in this PR:

Feature: Eval smoke test (binary load and eval step reporting)

  Scenario: Binary loads and reports step-level success via eval
    Given a clean CodeWhale evaluation workspace
    When the evaluation harness runs a shell command
    Then the binary exits without crashing
    And the JSON report contains execution steps

Layer 4.x evidence also includes the previously-added plugin E2E coverage from #3652:

Feature: Plugin discovery and listing

  Scenario: Plugin scripts are discovered from the configured plugin directory
    Given an offline CodeWhale workspace with a configured plugin directory
    And the plugin directory contains:
      | name       | description                | approval |
      | greet      | Say hello to the user      | auto     |
      | audit      | Run a security audit       | required |
      | summarizer | Summarize the given input  | suggest  |
    When the plugin scanner discovers plugins
    Then the scanner should report 3 plugins

Validation

  • cargo fmt --all -- --check
  • cargo check -p codewhale-tui
  • cargo test -p codewhale-tui command_registry_has_unique_names_and_aliases
  • cargo test -p codewhale-tui command_ownership_contract_is_enforced
  • cargo test -p codewhale-tui --test eval_smoke_acceptance
  • git diff --check origin/main...HEAD

Paulo Aboim Pinto

Paulo Aboim Pinto added 18 commits June 27, 2026 21:55
- Annotate existing command_registry_has_unique_names_and_aliases test
  with AT-008 doc comment referencing EPIC acceptance criteria
- Add new command_ownership_contract_is_enforced test (AT-009):
  - Validates at least 9 command groups
  - Each group has at least one command
  - Every command has valid CommandInfo metadata
  - Total command count matches registry
  - Documents config/debug group-local metadata as permanent exceptions
- Run cargo fmt to satisfy formatting hygiene
…t_is_enforced

REQUIRED (test-coverage): The AT-009 ownership contract test was too loose:
- Used  instead of exact  for group count
- Counted config/debug commands by name across ALL groups instead of
  verifying them within their specific groups (via content identity
  matching on first command)
- Circular comparison against command_infos() from same source
  (demoted to consistency check)

Fix: exact 9-group assertion, config/debug located by their command
content, per-group count verified within the identified group, circular
check retained as secondary consistency guard.

All 60 command tests pass (cargo test commands::tests -- --test-threads=1).
cargo fmt --all -- --check flagged two assert! calls with long
string arguments in command_ownership_contract_is_enforced.
They were on single lines and rustfmt wraps them to multi-line format.

No behavioral changes.
The Built-In Command Groups table listed 8 groups but the live source
registers 9 (including plugins::PluginsCommands). Added the /plugins
entry and removed the stale 'plugins' mention from utility scope.

Found during FEAT-008 Phase 4 documentation verification (Task 6).
The architecture doc only listed /jihua and /zidong as legacy/permanent
aliases, but live dispatch (commands/mod.rs:157-168) also has /slop and
/canzha dispatching directly to /debt in the same pre-registry match block.

- Updated dispatch flow Step 2 to list all four permanent aliases.
- Updated Permanent Exceptions first row to include all four aliases.

Refs: FEAT-008 Phase 4 code review finding #1
The plugins row claimed 'list, install, and configure tools' but live
source only supports plugin discovery/listing and per-plugin metadata
detail display. No install/configure command behavior exists.

Discovered by FEAT-008 Phase 4 code review.

Refs: FEAT-008
F1: Remap command_surfaces acceptance test as eval smoke test.
Stop counting it as AT-004 command-surface evidence — rename
feature/scenario/step text from 'command surface acceptance' to
'eval smoke test' and update runner doc comments.

F2: Rewrite public docs with draft/Phase 6 in-progress status.
command-dispatch.md: EPIC-002 Completion Status → Draft —
Phase 6 In Progress with neutral table status.
pr-issue-evidence-prep.md: Add draft header, replace ✅ with ⬜
neutral placeholders, mark all evidence as draft Phase 6.

F3: Update FeatureTasks.md status line to include Phase 6
AWAITING_REVIEW state with code review NEEDS_CHANGES note.

Also update phase-6-integration.md AT-004 row and
planning-analysis-report.md to remove command_surfaces from
AT-004 evidence, adjust count from 40 to 39.
… neutral draft wording

F1 [REQUIRED/test-coverage]: Rename command_surfaces.feature and
command_surfaces_acceptance.rs to eval_smoke.feature and
eval_smoke_acceptance.rs to stop implying AT-004 command-surface coverage.
Update world struct name, file path constant, and feature file header to
match. Feature file now includes disclaimer that this is not AT-004 evidence.

F2 [REQUIRED/docs-status]: command-dispatch.md evidence items use '— draft'
instead of '— pass'. pr-issue-evidence-prep.md result cells use
'⬜ Draft (Phase 6 current evidence)' instead of '⬜ Pass (draft Phase 6
evidence)'.

F3 [REQUIRED/docs-status]: FeatureTasks.md status line updated from
'Phase 6 COMPLETED' to 'Phase 6 NEEDS_CHANGES' reflecting latest code review.

Review Finding Decision Ledger recorded in phase-6-integration.md.
…_smoke_acceptance in pr-issue-evidence-prep.md

Code review REQUIRED finding F1: The public validation list still referenced
the deleted command_surfaces_acceptance target. Replaced with the current
eval_smoke_acceptance test and noted it is eval smoke, not AT-004
command-surface evidence.

Ref: FEAT-008 Phase 6 code review 2026-06-27T14:09:26.829Z
…. to FEAT-008 draft

Phase 7 code review (F1) found the FEAT-008 PR Summary Draft did not
follow the required PR Hmbown#3652 pattern: it jumped directly from the heading
to ## Summary without the PR title or Refs Hmbown#2870. opening line.

Added the title line (Layer 4.4: Registry cleanup, docs, and full
validation (FEAT-008)) as a reference outside the markdown fence, and
Refs Hmbown#2870. as the first line of the PR body markdown.

This is a code-review recovery fix for FEAT-008 Phase 7.
…nce command-dispatch.md to Phase 7

- eval_smoke_acceptance.rs: collapse multi-line string constant to single line (cargo fmt fix)
- command-dispatch.md: advance EPIC-002 status from Phase 6 to Phase 7,
  replace detailed Phase 6 evidence list with concise 'Current Evidence' summary
…ts_successfully step

The eval harness (run_eval in main.rs:1393) exits with code 1 when
metrics.success is false. This is expected: the offline multi-step
scenario (List, Read, Search, Edit, ApplyPatch, ExecShell) may report
overall failure while individual steps succeed. The ExecShell step itself
succeeds (verified by json_report_contains_execution_steps).

Changes:
- Store output.status in EvalSmokeWorld
- binary_exits_successfully now verifies: no crash (no signal on Unix),
  known exit code (0 or 1), documented eval-harness exit semantics
- json_report_contains_execution_steps continues to verify step-level
  success (ExecShell step succeeds with expected output)

F1 recovery: code review 2026-06-27T15:27:10 required exit status assertion.
Applied FIXED_WITH_NUANCE: stronger than blind status.success() which
would falsely fail — checks crash safety + known codes + step success.
…ary exits without crashing'

The eval harness exits with code 1 when metrics.success is false (expected
offline multi-step evaluation behavior). The step name claimed 'successfully'
but the assertion accepts exit code 0 or 1. Renamed step to match what is
actually verified: no crash (no signal on Unix), known exit code (0 or 1),
step-level success validated by the next step.

Addresses code review 2026-06-27T15:38:36 F1.
…ot Phase 7) awaiting approval

F3-recovery-2: Reverted Phase 7 wording to 'Phase 6; awaiting code review
approval for Phase 7' in command-dispatch.md and pr-issue-evidence-prep.md.
…oke_acceptance.rs

The cfg!() macro does not conditionally compile its block, so
use of std::os::unix::process::ExitStatusExt inside an if cfg!(unix)
failed to compile on non-Unix targets. Refactored to separate
#[cfg(unix)] and #[cfg(not(unix))] helper functions.

Addresses code review 2026-06-27T15:59:21 F1 (REQUIRED/runtime-code).
…e docs

Phase 6 code review is APPROVED; advance documentation status to Phase 7.
Addresses code review 2026-06-27T16:06:38 F2 (REQUIRED/docs-status).
- command-dispatch.md: update EPIC-002 status to Phase 8 complete
- pr-issue-evidence-prep.md: replace ⬜ placeholders with ✅ final results
- Record all validation: 489 command tests passed, 2 acceptance tests passed,
  5344/5345 workspace tests passed (1 known-flaky papercut)
- Sign off as Paulo Aboim Pinto
@Hmbown

Hmbown commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Thank you @aboimpinto — this is a really strong validation/documentation slice. I rechecked the live state and the branch is clean against main, all required checks are green, and the scope is nicely focused on the Layer 4.2 registry cleanup/evidence surface.

I am holding off on merging only because the PR is still marked draft. Once you mark it ready for review, this looks like a good candidate for the 0.8.66+ lane, with the usual final maintainer read-through of the docs wording and validation claims before merge.

@aboimpinto
aboimpinto marked this pull request as ready for review June 27, 2026 20:48
@aboimpinto
aboimpinto requested a review from Hmbown as a code owner June 27, 2026 20:48
@aboimpinto

Copy link
Copy Markdown
Contributor Author

I was waiting to check for conflicts. It's all ready now.

@Hmbown
Hmbown merged commit 5e9f766 into Hmbown:main Jun 27, 2026
15 checks passed
@Hmbown

Hmbown commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Merged into main as 5e9f7662ff7f9ec4766e35636634974a6e5e7544. Thank you again @aboimpinto — this was a careful, well-scoped validation/docs layer, and I really appreciate you checking conflicts and carrying it through to ready state so cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants