tracking: next-release plan - test bed, governance, agent loop, MCP/RAG (Oct 2026 - Jan 2027) #2555

Goal

Track the next-release plan for Base: harden the test bed, issue and PR governance, build a deterministic agent-loop harness (first for the Base ecosystem, then for inheriting projects), and add safe MCP documentation lookup and retrieval. Validate the resulting product on two maintainer-owned pilots and obtain independent-adopter evidence before making a January 2027 public claim.

Background

This is the revised plan after the detailed review of the original release plan and all 40 linked issues on 2026-10-16. The review confirmed the direction, but found that the original sequence allowed pilots to claim merged loop work before L2 existed, scheduled replay fixtures after the controllers that need them, put MCP security after the server, and left security, optional-package installation, evidence binding, crash recovery, and independent-adopter validation implicit.

The plan now has 45 implementation issues: #2556–#2595 plus the five additions #2599–#2603. The five additions are deliberately contract-sized; they are prerequisites for the claims made by the later controller, MCP, pilot, and scorecard issues.

Decisions retained from the original plan:

• macOS is the primary platform and Ubuntu continues. Native Windows is deferred to Future.
• Feature freeze is the end of December 2026. The last feature release is v1.13.0 on 2026-12-19; 2026-12-19 through 2026-12-31 is fix-only.
• Pilot projects remain bankbuddy (retrofit) and blend (greenfield), but they are not independent-adoption evidence.
• MCP uses the official mcp Python package as an optional dependency; lexical FTS5 retrieval comes first, and embeddings are considered only if the evaluation shows lexical misses.
• The test matrix remains four local macOS accounts: base-source-bash, base-source-zsh, base-brew-bash, and base-brew-zsh, with Ubuntu coverage continuing where supported.
• The loop is a deterministic harness (state machine, ledger, budgets, gates, and evidence), not a model runtime.
• No automatic merge, release, workflow/ruleset change, trust change, installer change, or security change is permitted merely because a loop run succeeded.

Revised sequencing

The important ordering constraints are:

1. docs+schema: issue and PR contract v2 with machine-readable Agent Assignment #2573 defines the machine-readable assignment contract before the loop consumes it. design: agent loop contract - state machine, event schema, autonomy levels #2576 defines the state machine and authority boundaries before a controller exists.
2. feat: agent loop ledger (JSONL events with run IDs) on history and logs #2577 and test: replayable loop fixtures and evals (extends #2499) #2582 establish durable events and offline replay before feat: agent controller L1 - select, claim, and prepare #2578–feat: agent controller L2 - observe CI and bounded repair with stop conditions #2581 are implemented. cli: add crash-safe, resumable agent-loop ledger #2602 adds crash-safe resume/readback before L2 and pilots can claim recoverability.
3. security: define agent-loop authority, trust boundaries, and audit contract #2599 defines agent authority, untrusted-input handling, credentials, approvals, and audit events before controller execution or any future merge policy.
4. test: documentation question eval set with token baselines #2585 measures retrieval before feat: heading-aware chunker and SQLite FTS5 index with git-SHA provenance #2586–feat: MCP search tool with index_sha and staleness reporting #2588 build the index and tools. security: read-only MCP/index threat model, root allowlist, and untrusted-content framing #2589 is designed and tested before feat: basectl mcp serve (stdio) with deterministic lookup tools #2587 exposes the server, and packaging: make optional MCP SDK installation deterministic and supportable #2600 makes the optional SDK installation supportable before it is a release claim.
5. ci: readiness gate for issues created through the API or CLI #2571/ci: validate the PR body contract #2572 enforce the issue and PR contracts, and docs: review and auto-merge tier policy for allow-listed PR classes #2574 defines the allow-list, before any autonomy beyond draft-PR preparation is considered.
6. validate: bankbuddy pilot - retrofit on current Base with loop, MCP, and friction log #2592/validate: blend pilot - greenfield project from first commit through Base and the loop #2593 do not complete until feat: agent controller L2 - verify and propose (draft PR) #2580/feat: agent controller L2 - observe CI and bounded repair with stop conditions #2581 are available and the required evidence is accepted by ci: verify and bind matrix evidence to release and scorecard claims #2601. adoption: run an independent design-partner validation sprint #2603 is the separate gate for any claim about adoption beyond the two maintainer-owned pilots.

Scope

Housekeeping and v1.11.0 foundations

• docs: retarget native Windows from v1.12.0 to Future #2556 docs: retarget native Windows from v1.12.0 to Future
• ci: exercise real Zsh login and interactive startup on the macOS runner #2557–test: matrix scenarios - source route clean install and base-demo loop (Bash and Zsh) #2561 test-bed, shell, install-matrix, precondition, and source-route foundations
• ci: alert on nightly base-demo e2e failures #2564 nightly base-demo e2e alerting
• ci: explain why non-conforming PR head branches exist under the active branch-name ruleset #2569 branch-rule explanation
• feat(gh): PR preflight for branch name, issue link, and body contract #2570 PR preflight for branch, issue, and body contracts
• process: dogfood-friction label and weekly triage cadence #2594 dogfood-friction label and weekly triage cadence
• release: publish the stable release-train contract and AI harness in Base 1.11.0 #2499 existing release-train contract and AI-harness work

v1.12.0 matrix and prerequisite contracts

• test: matrix scenarios - Homebrew consumer path, N-1 to N upgrade, state preservation #2562–test: matrix scenarios - agent shell modes, idempotency, and uninstall residue #2563 Brew-route, upgrade, state, shell-mode, idempotency, and residue scenarios
• test: permanent extreme-value option sweep (no 'Unexpected internal error') #2565–docs: rebuild macOS install validation and testing docs around the matrix #2568 extreme-value coverage, known-bad canaries, sandbox publish rehearsal, and matrix-centered docs
• Support Bash/Zsh-aware project activation #747 Bash/Zsh-aware project activation, Make command metadata canonical across help, dispatch, and completions #1606 canonical command metadata, and Publish a tested JSON Schema for base_manifest.yaml #1613 tested manifest JSON Schema
• docs+schema: issue and PR contract v2 with machine-readable Agent Assignment #2573 issue/PR contract v2 and structured Agent Assignment
• design: agent loop contract - state machine, event schema, autonomy levels #2576 loop state machine, event schema, autonomy levels, and stop conditions
• feat: agent loop ledger (JSONL events with run IDs) on history and logs #2577 redacted loop ledger
• test: replayable loop fixtures and evals (extends #2499) #2582 replayable loop fixtures and offline evals; this must land before the controllers
• test: documentation question eval set with token baselines #2585 documentation retrieval evaluation and token baselines
• feat: heading-aware chunker and SQLite FTS5 index with git-SHA provenance #2586 heading-aware FTS5 index with provenance
• security: read-only MCP/index threat model, root allowlist, and untrusted-content framing #2589 MCP/index threat model and enforced read-only/root/redaction boundaries
• feat: basectl mcp serve (stdio) with deterministic lookup tools #2587 deterministic stdio MCP server, after security: read-only MCP/index threat model, root allowlist, and untrusted-content framing #2589 and the canonical prerequisites
• feat: MCP search tool with index_sha and staleness reporting #2588 MCP search with index SHA and staleness reporting
• security: define agent-loop authority, trust boundaries, and audit contract #2599 agent-loop authority, trust boundary, credential, approval, and audit contract
• packaging: make optional MCP SDK installation deterministic and supportable #2600 optional MCP SDK installation, pinning, diagnostics, and recovery contract
• ci: verify and bind matrix evidence to release and scorecard claims #2601 matrix-evidence verifier binding exact results to commit, scenario, cell, release, and scorecard claims

v1.13.0 execution, pilots, and scorecard

• ci: readiness gate for issues created through the API or CLI #2571 API/CLI issue readiness gate and ci: validate the PR body contract #2572 PR body gate
• docs: review and auto-merge tier policy for allow-listed PR classes #2574 review/auto-merge tier policy and feat(gh): surface and automate worktree hygiene #2575 worktree hygiene integration
• feat: agent controller L1 - select, claim, and prepare #2578 controller L1 select/prepare, using feat: issue claim and lease so two agents cannot work one issue #2579 for claim semantics
• feat: issue claim and lease so two agents cannot work one issue #2579 claim/lease with explicit local-versus-remote concurrency semantics
• feat: agent controller L2 - verify and propose (draft PR) #2580 controller L2 verify/propose draft PR
• feat: agent controller L2 - observe CI and bounded repair with stop conditions #2581 bounded CI observation/repair and handoff
• cli: add crash-safe, resumable agent-loop ledger #2602 crash-safe, resumable ledger and interruption recovery
• feat: loop metrics report (superseded-PR rate, first-pass green, interventions) #2583 loop metrics and feat(manifest): agent block - validation, path allow/deny, size limits, autonomy ceiling #2584 manifest agent: block
• feat(manifest): docs block and repo configure --ai writes the project MCP config #2590 per-project docs block and MCP configuration
• feat: workspace-level aggregate MCP server (optional) #2591 optional workspace aggregate MCP
• validate: bankbuddy pilot - retrofit on current Base with loop, MCP, and friction log #2592 bankbuddy retrofit pilot and validate: blend pilot - greenfield project from first commit through Base and the loop #2593 blend greenfield pilot
• docs+test: January readiness scorecard - every public claim mapped to evidence #2595 January readiness scorecard
• adoption: run an independent design-partner validation sprint #2603 independent-adopter design-partner validation sprint

Prerequisite and release gates

• Support Bash/Zsh-aware project activation #747, Make command metadata canonical across help, dispatch, and completions #1606, and Publish a tested JSON Schema for base_manifest.yaml #1613 are explicit prerequisites, not historical footnotes. They must be closed or their scope must be consciously removed before the dependent MCP/manifest work is accepted.
• A milestone is releasable only when every implementation issue in it is closed or explicitly moved with a recorded reason, all blocked-by relationships are satisfied, hosted checks are complete, and the exact evidence is attached to the issue/PR it supports.
• ci: verify and bind matrix evidence to release and scorecard claims #2601 verifies matrix evidence freshness, commit identity, scenario/cell identity, preconditions, redaction, and completeness. It complements and does not replace the immutable release BOM and release-train gate in release: publish the stable release-train contract and AI harness in Base 1.11.0 #2499.
• A controller or pilot issue cannot be marked complete because a happy-path fixture passed while its security, packaging, replay, recovery, or evidence prerequisite remains open.
• The scorecard may state only what its executable checks and recorded artifacts support. bankbuddy and blend are internal reference projects, not independent adoption evidence; that claim requires adoption: run an independent design-partner validation sprint #2603.

Acceptance Criteria

• Every listed issue is closed or explicitly moved with a recorded reason; no issue remains in a release milestone merely because it is a tracking dependency.
• v1.11.0, v1.12.0, and v1.13.0 each pass their own empty-milestone/release-train gate with no queued required checks.
• Matrix results are accepted by ci: verify and bind matrix evidence to release and scorecard claims #2601 and remain traceable to an immutable Base commit and release BOM.
• The loop has documented authority and untrusted-input boundaries (security: define agent-loop authority, trust boundaries, and audit contract #2599), deterministic optional-MCP installation (packaging: make optional MCP SDK installation deterministic and supportable #2600), replay fixtures (test: replayable loop fixtures and evals (extends #2499) #2582), and crash-safe resume/readback (cli: add crash-safe, resumable agent-loop ledger #2602) before corresponding claims are made.
• Both maintainer-owned pilots have the required loop evidence, sensitive-data checks, and triaged friction log only after the L2 controller is available.
• The January scorecard maps every public claim to an executable check or recorded result and states known limitations.
• No public independent-adoption claim is made unless adoption: run an independent design-partner validation sprint #2603 meets its evidence threshold.

Cut line if December slips

Cut in this order: #2591 workspace aggregate server, #2583 loop metrics, #2584 manifest automation, then optional pilot breadth. Ship only draft-PR behavior for L2 if necessary. Do not cut the matrix core, PR preflight, assignment/authority contracts, MCP security and packaging contracts, replay fixtures, evidence verifier, or the safety/recovery prerequisites for any claim that remains.

Validation

Progress is the checklist and dependency graph above. Release readiness is decided by each milestone's release-train gate, immutable BOM, and #2601 evidence verification.

Non-Goals

• Native Windows work in this train.
• Model-runtime implementation or provider-specific agent integration.
• Auto-merge, auto-release, or automatic workflow/ruleset/trust/security changes.
• Embedding retrieval before test: documentation question eval set with token baselines #2585 shows lexical misses.
• Calling maintainer-owned pilots independent adoption.

Project Fields

• Status: Backlog
• Priority: P1
• Area: Product
• Initiative: Agentic Coding Platform
• Size: L
• Milestone: Future
• Category: enhancement

Agent Assignment

Assignee: codeforester. Tracking issue only; implement the linked issues, not this one.