diff --git a/.codex/README.md b/.codex/README.md index 70fd4d5..9ad9704 100644 --- a/.codex/README.md +++ b/.codex/README.md @@ -14,10 +14,33 @@ In this framework repository, worked product artifacts live under `examples/even - `skills/`: repository-local maintenance skills for Codex. They are not mirrored into adopter repositories. +## Ownership and routing + +| Surface | Canonical owner | Use it for | +| --- | --- | --- | +| `FRAMEWORK.md` | Framework method | Gates, lifecycle, authority, and architecture. | +| `framework/skills/` | Distributed framework skills | Product-facing specialist and orchestrator contracts. | +| `.codex/skills/` | Repository maintenance skills | Reviewing, testing, packaging, and preserving this repository. | +| `starter/` | Clean adopter skeleton | New-product content only. | +| `examples/events/` | Worked product fixture | Learning and validation; never a starter source. | +| Git history | Maintenance record | Evolution of the framework method and contracts. | + +Use `framework-change-reviewer` before or after a method-level change; use `distribution-impact-analyzer` and `upgrade-compatibility` whenever embedded, starter, installer, or packaging assets change. Use `runtime-contract-tester` for CLI or runtime changes, `agent-target-smoke` for harness output, `fixture-governor` for starter/example boundaries, `skill-contract-auditor` for skill contracts, and `ci-matrix-maintainer` to select and maintain mechanical gates. + +## Maintenance rules + +- Read `FRAMEWORK.md` before changing framework behavior. +- Update every affected contract, validator, template, starter asset, documentation surface, and test in the same change. +- Do not edit generated agent trees as canonical sources or create product artifacts under `.codex/`. +- Preserve adopter-owned content and approval records during `upgrade`; never use an overwrite to repair a migration. +- Keep optional capabilities explicit, disabled by default when they change execution behavior, and covered by combination tests. +- Report unavailable environment gates as limitations, never as passing evidence. +- Record framework method evolution directly in `FRAMEWORK.md`, affected contracts, validators, and tests; Git is the maintenance record. + ## Responsible Skill -Primary owner: Documentation Orchestrator. +Primary owner: Documentation Orchestrator. Use the specialist maintenance skill matching the changed surface before handoff. ## Next Step -When a skill changes, validate that its `SKILL.md` still has frontmatter with `name` and `description`, and that the instructions resolve product paths through the active product root instead of assuming product folders at the framework repository root. +When a skill changes, run `skill-contract-auditor` or at minimum validate that its `SKILL.md` has frontmatter with `name` and `description`, that its instructions resolve product paths through the active product root, and that its guidance remains consistent with `FRAMEWORK.md`. diff --git a/.codex/skills/agent-target-smoke/SKILL.md b/.codex/skills/agent-target-smoke/SKILL.md new file mode 100644 index 0000000..e53d6e6 --- /dev/null +++ b/.codex/skills/agent-target-smoke/SKILL.md @@ -0,0 +1,29 @@ +--- +name: agent-target-smoke +description: Smoke-test Spec Framework initialization and upgrade output for Codex, Cursor, and Claude Code agent targets and their Guide-first dispatchers. +--- + +# Agent Target Smoke + +## Purpose + +Verify the generated harness entry points without treating any target-specific tree as the source of truth. + +## Required reading + +- `FRAMEWORK.md` +- `internal/install/` +- `internal/dispatcher/` +- relevant init, upgrade, and dispatcher tests + +## Workflow + +1. Initialize an isolated temporary product for each requested target: Codex, Cursor, and Claude Code. +2. Verify the target receives only its supported dispatcher and that the dispatcher requires a valid `product/.product/framework.json`. +3. Verify that framework skills and templates resolve from the versioned runtime, not from repository-local copied trees. +4. Upgrade the fixture and verify dispatcher refresh while product content remains unchanged. +5. Check target instructions for portable paths and absence of unsupported harness-specific instructions. + +## Output + +Report target, generated paths, activation result, upgrade result, unsupported-target findings, and exact test evidence. diff --git a/.codex/skills/ci-matrix-maintainer/SKILL.md b/.codex/skills/ci-matrix-maintainer/SKILL.md new file mode 100644 index 0000000..10349b7 --- /dev/null +++ b/.codex/skills/ci-matrix-maintainer/SKILL.md @@ -0,0 +1,29 @@ +--- +name: ci-matrix-maintainer +description: Maintain the Spec Framework verification matrix by mapping changed surfaces to unit, integration, race, validation, packaging, and release-smoke gates. +--- + +# CI Matrix Maintainer + +## Purpose + +Keep mechanical confidence proportional to the change. Do not suppress a failing, skipped, or environment-blocked gate. + +## Required reading + +- `AGENTS.md` +- `.github/workflows/` +- `.codex/skills/verify/SKILL.md` +- changed code, assets, and tests + +## Workflow + +1. Map each changed surface to its required checks: formatting, unit, integration, example validation, race, target smoke, init/upgrade, packaging, and release smoke. +2. Verify the workflow contains the required gate or document a justified local-only check. +3. Add combination tests when optional runtime capabilities interact with imports, dispatch, reviews, approvals, or upgrade. +4. Run the applicable checks and preserve exact output for failures. +5. Report App Control, unavailable binaries, or other environment limits as blocked evidence, never as passing evidence. + +## Output + +Return a check matrix with status, command or workflow, coverage rationale, blockers, and recommended next gate owner. diff --git a/.codex/skills/distribution-impact-analyzer/SKILL.md b/.codex/skills/distribution-impact-analyzer/SKILL.md new file mode 100644 index 0000000..ea1b9b1 --- /dev/null +++ b/.codex/skills/distribution-impact-analyzer/SKILL.md @@ -0,0 +1,29 @@ +--- +name: distribution-impact-analyzer +description: Analyze the distribution impact of a Spec Framework asset or packaging change across embedding, install, upgrade, documentation, agent targets, and release verification. +--- + +# Distribution Impact Analyzer + +## Purpose + +Make distribution effects explicit before a framework asset or packaging change is committed. + +## Required reading + +- `FRAMEWORK.md` +- `assets.go` +- affected `starter/` and `framework/` assets +- installer, release, and smoke-test code + +## Workflow + +1. Determine whether the change is embedded, generated, copied at init, materialized at runtime, or documentation-only. +2. Complete an impact matrix for assets, init, upgrade, Codex/Cursor/Claude targets, documentation, CI, archives, checksums, and release smoke. +3. Identify managed versus adopter-owned paths and verify that removals remain safe after upgrade. +4. Require the smallest relevant install, upgrade, target, and release tests. +5. State whether a version bump, migration note, or rollback plan is required. + +## Output + +Return the impact matrix, affected paths, required verification, release impact, compatibility verdict, and migration/rollback notes. diff --git a/.codex/skills/fixture-governor/SKILL.md b/.codex/skills/fixture-governor/SKILL.md new file mode 100644 index 0000000..00b410c --- /dev/null +++ b/.codex/skills/fixture-governor/SKILL.md @@ -0,0 +1,28 @@ +--- +name: fixture-governor +description: Govern changes across Spec Framework starter assets, the Events worked-product fixture, and embedded runtime assets without mixing their ownership boundaries. +--- + +# Fixture Governor + +## Purpose + +Protect the distinction between reusable starter content and the Events validation fixture. + +## Required reading + +- `FRAMEWORK.md` +- `AGENTS.md` +- affected `starter/`, `examples/events/`, `framework/`, and `assets.go` files + +## Workflow + +1. Classify every changed artifact as framework-owned, clean starter content, generated target output, or Events product-owned fixture content. +2. Reject copying Events scope, approvals, decisions, or narratives into reusable framework or starter assets. +3. When a starter contract changes, verify initialization, upgrade preservation, embedded assets, and the matching fixture assertions. +4. When Events changes, read its relevant context and preserve its product lifecycle, approvals, and evidence. +5. Check whether empty directories can be created declaratively rather than retained through placeholder documentation. + +## Output + +Return the ownership map, required synchronization, preservation evidence, validation fixture impact, and unresolved boundary risks. diff --git a/.codex/skills/framework-change-reviewer/SKILL.md b/.codex/skills/framework-change-reviewer/SKILL.md new file mode 100644 index 0000000..36d61c4 --- /dev/null +++ b/.codex/skills/framework-change-reviewer/SKILL.md @@ -0,0 +1,28 @@ +--- +name: framework-change-reviewer +description: Review a proposed Spec Framework maintenance change for method, contract, compatibility, distribution, and test impact before it is committed or released. +--- + +# Framework Change Reviewer + +## Purpose + +Review framework-maintenance diffs. Do not author product artifacts, create approvals, or replace the implementation owner. + +## Required reading + +- `FRAMEWORK.md` +- `AGENTS.md` +- affected contracts, templates, validators, tests, and installer code + +## Workflow + +1. Classify the change: method, skill contract, template, CLI/runtime, starter, validator, or documentation. +2. Map affected surfaces: `FRAMEWORK.md`, `framework/`, `starter/`, `examples/events/`, `assets.go`, init, upgrade, agent targets, documentation, CI, and release packaging. +3. Check that the default behavior and existing adopter content remain compatible, or that a migration is explicit and reversible. +4. Check that product decisions and approval history are not edited by framework maintenance. +5. Require tests for each affected executable boundary and report omitted surfaces as findings. + +## Output + +Return a concise verdict: `ready`, `needs_changes`, or `blocked`; the impact matrix; compatibility and migration notes; required tests; residual risks; and the next owner. diff --git a/.codex/skills/new-framework-skill/SKILL.md b/.codex/skills/new-framework-skill/SKILL.md index 98e5c53..ca9b963 100644 --- a/.codex/skills/new-framework-skill/SKILL.md +++ b/.codex/skills/new-framework-skill/SKILL.md @@ -16,7 +16,7 @@ Answer these first; if any answer is unclear, stop and ask: - **Specialist or orchestrator?** Specialists own exactly one canonical artifact's content. Orchestrators own flow, gates, sequencing, and handoff across artifacts — they never author artifact content themselves. - **Which artifact does it own?** A specialist without an owned artifact, or an artifact already owned by another skill, means the skill should not exist — extend the existing owner instead. - **Does the owned artifact have a template?** If the artifact is new, a matching template in `framework/template/` must be created in the same change. -- **Does this change the method?** Adding or reshaping a skill usually alters the framework contract — record an FDR (use the `fdr` skill) unless the change is purely editorial. +- **Does this change the method?** Adding or reshaping a skill usually alters the framework contract — update `FRAMEWORK.md`, affected contracts, validators, and tests in the same change. Git history is the maintenance record. ## Canonical SKILL.md structure @@ -55,7 +55,7 @@ description: " Skill. Use when Codex needs to + ## Quality checklist - [ ] Preserves traceability to affected artifacts. @@ -86,7 +86,7 @@ After writing the SKILL.md: 2. Check `FRAMEWORK.md` for skill rosters, flow diagrams, or step tables that must mention it, and `AGENTS.md` / `framework/AGENTS.framework.md` for flow references. 3. If it owns a new artifact: create the template in `framework/template/`, add the artifact to the canonical flow documentation, and update the readiness/validator rules if the artifact is gate-relevant. 4. Update the `Handoff` sections of neighboring skills whose flow now includes the new skill. -5. Record the FDR if the method changed. +5. When the method changed, verify that `FRAMEWORK.md`, affected contracts, validators, and tests express the new behavior. No package manifest changes are needed: `assets.go` embeds `framework/skills/` wholesale. Update installer and rendering tests when target-specific metadata changes. diff --git a/.codex/skills/runtime-contract-tester/SKILL.md b/.codex/skills/runtime-contract-tester/SKILL.md new file mode 100644 index 0000000..e617acb --- /dev/null +++ b/.codex/skills/runtime-contract-tester/SKILL.md @@ -0,0 +1,28 @@ +--- +name: runtime-contract-tester +description: Test CLI and runtime contract changes in the Spec Framework, including optional capabilities, dispatch, imports, persistence, concurrency, and failure behavior. +--- + +# Runtime Contract Tester + +## Purpose + +Turn runtime changes into explicit, compatible contracts. Do not declare an unavailable external or sandboxed command as passing. + +## Required reading + +- `FRAMEWORK.md` +- changed `cmd/` and `internal/` packages +- related CLI help, JSON contracts, and tests + +## Workflow + +1. List changed commands, flags, persisted files, defaults, and external process boundaries. +2. Verify new optional capabilities are disabled by default and can be enabled, disabled, and removed safely. +3. Test success, invalid input, interrupted execution, resume/idempotency, and backward-compatible absence of configuration. +4. For concurrent work, test lease ownership, dependency ordering, write scopes, and deterministic reconciliation. +5. Combine changed features with import, dispatch, review, approval, and upgrade boundaries when applicable. + +## Output + +Return the contract matrix, commands run, compatible defaults, combination-test evidence, failures, and residual runtime risks. diff --git a/.codex/skills/skill-contract-auditor/SKILL.md b/.codex/skills/skill-contract-auditor/SKILL.md new file mode 100644 index 0000000..55e524d --- /dev/null +++ b/.codex/skills/skill-contract-auditor/SKILL.md @@ -0,0 +1,29 @@ +--- +name: skill-contract-auditor +description: Audit repository-maintenance and shipped Spec Framework skills for valid metadata, ownership, portable paths, handoffs, and consistency with the canonical method. +--- + +# Skill Contract Auditor + +## Purpose + +Keep skills actionable and non-overlapping. This skill audits contracts; it does not silently change a skill's product ownership or workflow. + +## Required reading + +- `FRAMEWORK.md` +- `AGENTS.md` +- `.codex/README.md` for maintenance skills +- `framework/skills/README.md` for shipped skills + +## Workflow + +1. Validate each `SKILL.md` has only `name` and `description` frontmatter and a precise activation description. +2. Verify the skill has one clear responsibility, explicit exclusions, required reading, and a safe handoff. +3. Check paths work from the framework runtime and active product root; reject lab-specific or generated-tree source assumptions. +4. Check for stale concepts, nonexistent commands, duplicated ownership, bypassed gates, or approval authority violations. +5. Route method-level corrections to `FRAMEWORK.md`, contracts, validators, and tests in the same change. + +## Output + +Report compliant skills, findings by severity, affected owners, exact corrective surface, and verification required after correction. diff --git a/.codex/skills/upgrade-compatibility/SKILL.md b/.codex/skills/upgrade-compatibility/SKILL.md new file mode 100644 index 0000000..c54379f --- /dev/null +++ b/.codex/skills/upgrade-compatibility/SKILL.md @@ -0,0 +1,29 @@ +--- +name: upgrade-compatibility +description: Verify that a Spec Framework distribution change can initialize new products and upgrade existing products without overwriting adopter-owned content or approvals. +--- + +# Upgrade Compatibility + +## Purpose + +Protect adopter-owned product content during distribution changes. Do not change an adopter product or approval record merely to make an upgrade pass. + +## Required reading + +- `FRAMEWORK.md` +- `AGENTS.md` +- `internal/install/` +- affected starter assets and upgrade tests + +## Workflow + +1. Identify changed embedded assets and whether they are framework runtime assets or product-owned starter assets. +2. Exercise a fresh `init` for each affected starting point and agent target. +3. Exercise `upgrade` over an initialized fixture containing changed product files, decisions, and approval history. +4. Verify upgrade refreshes only the runtime, manifest, and selected dispatchers; it must not replay initialization or overwrite adopter content. +5. For removed assets, verify no stale reference remains and document whether the removal is safe, retained, or requires an explicit migration. + +## Output + +Report init and upgrade evidence, preserved paths, changed managed paths, compatibility status, rollback path, and any human migration decision required. diff --git a/FRAMEWORK.md b/FRAMEWORK.md index 8fdcb3f..e5b930a 100644 --- a/FRAMEWORK.md +++ b/FRAMEWORK.md @@ -535,6 +535,16 @@ Runtime v2 makes execution resumable and safely parallel. The complete workspace Runtime commands include `runtime`, `resume`, `handoff`, `checkpoint`, `lease`, `commands`, `schedule`, `integrate`, and local `reviews import`. Imported review findings are immutable, provider-neutral evidence with provenance and an advisory owner route; import cannot resolve a remote review, alter code, or advance an approval. +`dispatch` is an optional supervised orchestration layer. It assigns only ready +units through persisted envelopes and leases. Independent QA, Code Review and +Security Review envelopes pin the same returned diff hash and remain read-only. +Dispatch execution and waves require explicit per-run enablement; they cannot +approve, commit, push, merge, release, or resolve external reviews. + +`dispatch-orchestrator` owns assignment and observation. `subagent-return-reviewer` +owns the structured return record and routes gaps without changing product +lifecycle or approval state. + Skills are specialists. They can operate in modes such as `create`, `update`, `audit`, `evolve`, `explain`, `compare`, and `refactor`, but each must have a clear responsibility. Definition and planning skills follow the shared Discovery and Challenge contract before substantive creation or material revision. They inspect repository and CLI evidence first, then use the harness-native structured question capability for human choices that cannot be discovered. Each round asks one to three focused questions; meaningful choices present concrete options, trade-offs, a recommendation, and a free-form path. Skills proactively warn about material scope, dependency, usability, security, operability, reversibility, approval, and delivery risks and propose safer alternatives. They must not finalize or hand off while a blocking question is unanswered, and conversational answers never grant formal approval. Harness adapters map the canonical `native_user_question` capability to their default question tool; a concise conversational question is the explicit fallback only when no structured tool is exposed. @@ -720,7 +730,7 @@ The runtime behavior shared by all agents is defined in the pinned `AGENTS.frame - `init` resolves one strict declarative starting-point contract, validates its complete materialization plan, stages it, and atomically publishes `product/`. Data contracts cannot execute arbitrary commands or escape the product root. - A starting point changes the initial evidence, registry, bootstrap, and first gate; it never removes later rigor or approval requirements. Existing code and documents remain evidence, not approved truth. `audit-only` remains read-only until an explicit supported transition. -- `existing-documents` creates an analysis-only import run with `traceability.json` as the dedicated per-source ledger. The Artifact Importer agent reads each source, records evidence, extracted claims, candidate destinations, and unmapped gaps there, then proposes mappings. Human review of inventory, traceability, conflicts, and selected mappings is required before explicit draft materialization; imported artifacts retain their normal owners, parents, and individual approval gates. +- `existing-documents` creates an analysis-only import run with a dedicated per-source ledger. Scalable runs page their inventory, apply explicit file/byte budgets and filters before copying, and divide analysis into leased resumable chunks. The Artifact Importer records evidence, extracted claims, candidate destinations, and unmapped gaps for every non-excluded source, then proposes mappings. Human review of inventory, traceability, conflicts, and selected mappings is required before explicit draft materialization; imported artifacts retain their normal owners, parents, and individual approval gates. - Materialized import artifacts carry machine-readable `provenance.kind: import-draft` and `provenance.import_run`. The owning normalization skill must replace this with `skill-normalized` and identify itself before approval; import provenance is lineage, not canonical readiness. - `init` never overwrites an existing `product/`. `upgrade` refreshes only the pinned runtime, manifest, and selected dispatchers; it never replays initialization over adopter-owned content. - Starting-point details belong to `docs/starting-points.md`, `framework/init/`, and generated `BOOTSTRAP.md` rather than this operational summary. diff --git a/README.md b/README.md index 2b7feff..2c29bb5 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ The [Framework Guide skill](framework/skills/framework-guide/SKILL.md) is the de | Intent | Commands | | --- | --- | -| Start or import a product | `init`, `import materialize` | +| Start or import a product | `init`, `import create/status/resume`, `import materialize` | | Navigate and see state | `work`, `status`, `next`, `dashboard`, `guide` | | Approve explicitly | `review`, `approve`, `approve-batch`, `approve-stage`, `gates` | | Design workflows | `design init/import/register/map/inspect/audit`, `design-system init/inspect/validate/migrate` | diff --git a/docs/scalable-imports.md b/docs/scalable-imports.md new file mode 100644 index 0000000..41b0ec1 --- /dev/null +++ b/docs/scalable-imports.md @@ -0,0 +1,110 @@ +# Importação escalável de documentos + +Use este fluxo para importar PRDs, wikis, Jira, Confluence, pesquisas e acervo +legado sem transformar a fonte em verdade aprovada do produto. + +## Como funciona + +1. O CLI encontra os arquivos permitidos e valida limites antes de copiar. +2. Cada fonte recebe hash, tamanho e identificador; o inventário é paginado. +3. As fontes são separadas em lotes (`CHUNK-NNNN`) que podem ser retomados. +4. Um revisor reclama um lote, registra evidências por fonte e suas lacunas. +5. Somente depois da revisão de todos os lotes uma pessoa pode materializar + os mapeamentos selecionados como rascunhos. + +Importar nunca aprova escopo, decisão, arquitetura ou artefato. Rascunhos +continuam sujeitos aos donos e gates normais do framework. + +## Casos de uso + +| Caso | Como usar | +| --- | --- | +| Produto novo com muitos documentos | Inicialize com `existing-documents` e defina orçamento. | +| Importação interrompida | Consulte o status e retome o próximo lote. | +| Arquivo grande/binário | Limite tamanho; inventarie ou rejeite binários. | +| Novas fontes após a inicialização | Crie um novo import run, separado por assunto. | + +## Inicializar a partir de documentos + +```powershell +spec-framework init --target ./meu-produto --agents codex ` + --starting-point existing-documents --sources ../documentos ` + --import-max-files 1000 --import-max-total-bytes 500MB ` + --import-max-file-bytes 20MB --import-chunk-size 25 ` + --import-binary-policy inventory_only --yes +``` + +Os padrões são 500 arquivos, 200 MB totais, 10 MB por arquivo e 25 fontes por +lote. `.git` e `node_modules` são excluídos por padrão. Se o orçamento for +excedido, o run falha antes de publicar uma cópia parcial. + +## Criar um run adicional + +```powershell +spec-framework import create --product-root ./meu-produto/product ` + --include "**/*.md" --exclude "**/archive/**" ` + --max-files 500 --max-total-bytes 200MB --chunk-size 25 ../novas-fontes +``` + +Prefira vários runs coerentes, por exemplo produto/regras, pesquisas e +arquitetura, a um único acervo difícil de revisar. + +## Revisar e retomar + +```powershell +spec-framework import status --product-root ./meu-produto/product --run IMPORT-001 +spec-framework import resume --product-root ./meu-produto/product ` + --run IMPORT-001 --agent artifact-importer +``` + +`resume` retorna um lote, como `CHUNK-0003`. A lease impede revisão concorrente +do mesmo lote e expira para permitir retomada segura. + +Depois de analisar o lote, forneça ao menos uma evidência para cada fonte: + +```json +{ + "source_evidence": { + "SRC-000051": [{ "locator": "Seção 2.3", "claim": "Clientes exportam relatórios." }] + }, + "gaps": { "SRC-000051": ["Formato de exportação não definido."] } +} +``` + +```powershell +spec-framework import record-review --product-root ./meu-produto/product ` + --run IMPORT-001 --chunk CHUNK-0003 --agent artifact-importer ` + --input ./review-chunk-0003.json --yes +``` + +Isso registra evidência e lacunas; não cria aprovações nem altera artefatos do +produto. + +## Materializar rascunhos + +Após revisar conflitos, lacunas e `mapping.json`, uma pessoa autoriza: + +```powershell +spec-framework import materialize --product-root ./meu-produto/product ` + --run IMPORT-001 --approved-by "Product Owner" --yes +``` + +O comando bloqueia se existir lote pendente, em revisão ou bloqueado. Também +recusa destino duplicado, caminho fora do produto e sobrescrita de conteúdo. + +## Onde ficam os dados + +```text +product/knowledge/imports/ +├── sources/IMPORT-001/ # cópias preservadas +└── runs/IMPORT-001/ + ├── import-config.json # filtros e limites + ├── inventory/index.json # índice compacto + ├── inventory/pages/PAGE-0001.jsonl # páginas do inventário + ├── chunks/CHUNK-0001.json # status e lease + ├── traceability/CHUNK-0001.json # evidências por fonte + └── mapping.json # rascunhos propostos +``` + +Para dados sensíveis, decida previamente a retenção e quem pode ler as cópias +preservadas. Não apague import runs como limpeza: eles são evidência auditável. diff --git a/docs/subagent-dispatch.md b/docs/subagent-dispatch.md new file mode 100644 index 0000000..5c6548f --- /dev/null +++ b/docs/subagent-dispatch.md @@ -0,0 +1,33 @@ +# Despacho supervisionado de subagents + +`dispatch` coordena subagents por envelopes persistidos. Ele não aprova, +commita, envia, faz merge ou publica. + +```powershell +spec-framework dispatch plan --product-root product --graph +spec-framework dispatch assign --product-root product --work WORK-001 --graph --task TK-001 --agent runner --yes +spec-framework dispatch return --product-root product --work WORK-001 --id DISPATCH-... --agent runner --summary "feito" --diff-hash --evidence "test log" --yes +spec-framework dispatch assign --product-root product --work WORK-001 --role qa --parent DISPATCH-... --agent qa-1 --yes +spec-framework dispatch reconcile --product-root product --work WORK-001 +``` + +QA, Code Review e Security Review recebem envelopes read-only presos ao mesmo +`diff_hash` retornado pelo Code Runner. A execução local é experimental e exige +`dispatch run --enable --yes`; ondas usam envelopes já atribuídos e +`dispatch wave --enable --yes`. + +Antes de executar, habilite a capability no produto com `dispatch configure` e +liste os harnesses permitidos. `dispatch wave` recebe uma `--wave` persistida +do scheduler, não IDs arbitrários. Transcripts locais seguem a retenção definida +em `--transcript-retention`. + +Também são suportados `--role artifact-importer --run IMPORT-NNN --chunk +CHUNK-NNNN`, `--role threat-modeler --task ` e `--role +technical-discovery --task `. Esses três papéis são read-only: +retornam evidência, opções, ameaças ou lacunas, sem materializar drafts, criar +decisões, aceitar risco residual ou editar Engineering Proposals. + +Para concluir um chunk despachado, use `dispatch return --review-input + --yes`; o JSON é validado e registrado como evidência por fonte. +O executor também confere os caminhos alterados pelo Git contra o `writeScope` +da task e falha quando detectar escrita fora do contrato. diff --git a/docs/todo/large-imports-feature-plan.md b/docs/todo/large-imports-feature-plan.md new file mode 100644 index 0000000..dab6b39 --- /dev/null +++ b/docs/todo/large-imports-feature-plan.md @@ -0,0 +1,177 @@ +# Plano de feature — Importação escalável e retomável + +## Objetivo + +Permitir que `existing-documents` trate acervos grandes com previsibilidade de +volume, lotes retomáveis e rastreabilidade por arquivo e por trecho, sem mudar +a tese do framework: a fonte importada é evidência, rascunhos continuam sendo +do produto, e nenhuma aprovação ou materialização ocorre automaticamente. + +## Estado atual e lacuna + +Hoje `sourceimport.CreateRun` expande toda a árvore, copia cada arquivo, +calcula hash e grava um inventário completo. A cópia e o hash usam streaming, +mas a lista de arquivos e os ledgers são montados integralmente na memória. +Não há filtros, orçamento, exclusões padrão, lote, checkpoint ou retomada. +O Artefact Importer deve revisar cada fonte, o que torna um arquivo muito longo +ou milhares de documentos operacionalmente difícil. + +## Princípios e limites invioláveis + +- Nunca alterar, aprovar ou sobrescrever conteúdo adotante durante inventário, + análise, retomada ou compactação. +- Tratar fontes como evidência; somente mapeamentos humanos selecionados podem + gerar artefatos `draft` com `provenance.import_run`. +- Preservar hash, caminho original, tamanho, formato e evidência de cada fonte. +- Não usar configuração ambiente para semântica; o plano do lote é persistido + dentro do próprio import run e flags explícitas apenas o criam. +- Não enviar conteúdo para serviços externos. Extratores e adaptadores devem + ser declarados, opcionais e sem autoridade de aprovação. + +## Proposta de arquitetura + +### Módulo 1 — Plano de ingestão + +Novo `import create` recebe fontes e cria `import-config.json` imutável após o +início: + +```json +{ + "schema_version": 2, + "include": ["docs/**/*.md", "research/**/*.pdf"], + "exclude": ["**/node_modules/**", "**/.git/**", "**/*.zip"], + "max_files": 500, + "max_total_bytes": 209715200, + "max_file_bytes": 10485760, + "chunk_size": 25, + "binary_policy": "inventory_only" +} +``` + +Padrão seguro: excluir `.git`, dependências e diretórios de build conhecidos; +inventariar binários sem tentar extrair conteúdo; falhar antes de copiar quando +um orçamento é excedido. `--allow-large-file` e `--allow-total-overflow` apenas +criam uma exceção explicitamente registrada, nunca silenciosa. + +### Módulo 2 — Inventário paginado + +Substituir o array único em memória por arquivos ordenados e paginados em +`runs/IMPORT-NNN/inventory/pages/PAGE-0001.jsonl`. Um índice compacto mantém +contagem, bytes, hash do plano e páginas concluídas. O inventário original v1 +continua legível; novos runs usam v2. Cada linha contém `source_id`, caminho +original, cópia preservada, hash, tamanho, formato, classificação e motivo de +exclusão quando aplicável. + +### Módulo 3 — Lotes de análise + +`chunks/CHUNK-0001.json` referencia no máximo `chunk_size` source IDs, com +status `queued | analysing | reviewed | blocked | excluded`. A análise não +altera o inventário; produz `traceability/chunks/CHUNK-0001.json` e atualiza um +índice de cobertura. Para documentos longos, um `segment` contém um localizador +estável (página, capítulo, intervalo de linhas ou timestamp) e o hash do trecho +extraído. O texto completo não é duplicado no ledger. + +### Módulo 4 — Retomada e consistência + +`import status --run` é somente leitura. `import resume --run` escolhe apenas +o próximo lote `queued` ou um lote abandonado com lease expirada. Cada lote +recebe lease, heartbeat e checkpoint operacional equivalentes aos do runtime, +mas não recebe autorização de escrita em artefatos canônicos. Uma mudança de +hash da fonte marca seus lotes e mapeamentos como `stale`; não apaga análise +anterior. + +### Módulo 5 — Materialização por seleção + +`import materialize` aceita somente mappings cujas fontes estejam revisadas ou +explicitamente `not_applicable` com justificativa. A operação é atômica por +seleção: valida todos os destinos antes de escrever, não sobrescreve existentes +e atualiza traceabilidade somente após todos os rascunhos serem publicados. +O formato atual de `mapping.json` continua aceito; v2 adiciona `source_ids` e +`source_segments` em paralelo a `source_documents` durante a migração. + +## CLI proposta + +```text +spec-framework import create --source ... --include ... + --exclude ... --max-files 500 --max-total-bytes 200MB + --max-file-bytes 10MB --chunk-size 25 --binary-policy inventory_only + +spec-framework import status --run IMPORT-001 [--json] +spec-framework import resume --run IMPORT-001 --agent [--chunk CHUNK-0003] +spec-framework import retry --run IMPORT-001 --chunk CHUNK-0003 --yes +spec-framework import materialize --run IMPORT-001 --approved-by --yes +``` + +`resume` não executa um modelo nem materializa rascunhos: prepara e reclama o +lote para o Artefact Importer. A análise continua sendo responsabilidade do +skill e do humano que revisa os resultados. + +## Dados, ativação e remoção + +| Item | Padrão | Ativação | Remoção segura | +| --- | --- | --- | --- | +| Inventário v2 | Novo run | `import create` | manter run como evidência; não apagar em upgrade | +| Filtros/orçamentos | Conservadores | flags gravadas no config | criar novo run, não reescrever config | +| Lotes/leases | Desligados para v1 | run v2 | expirar/reconciliar, nunca apagar análise | +| Extrator de PDF/DOCX | Ausente | extensão declarada | voltar a `inventory_only` | +| Materialização v2 | Igual ao atual | aprovação + `--yes` | não reversível automaticamente; drafts seguem fluxo normal | + +## Compatibilidade e upgrade + +- Nenhum `IMPORT-NNN` existente é reescrito. O leitor detecta v1 e apresenta + status legado; uma migração explícita cria um índice v2 derivado, preservando + JSONs originais e hashes. +- Produtos novos usam v2 quando a feature estiver estável. Produtos existentes + só adotam v2 em novos runs ou via `import migrate --run ... --yes`. +- `upgrade` nunca modifica fontes copiadas, traceabilidade, mapeamentos, + aprovações, rascunhos ou checkpoints do adotante. + +## Entregas incrementais + +| Fase | Entrega | Esforço | Critério de aceite | +| --- | --- | --- | --- | +| 0 | DEC de política de limites e binários | P | decisão aprovada e contratos atualizados | +| 1 | Filtros, orçamentos e inventário v2 paginado | M | falha previsível antes de copiar além do orçamento | +| 2 | Chunks, status e retomada sem análise automática | M | interromper e retomar não duplica nem perde fontes | +| 3 | Segmentos e traceabilidade por trecho | M | conclusão aponta localizador estável da fonte | +| 4 | Mapeamento/materialização v2 atômicos | M | preserva regra de não sobrescrever e aprovação explícita | +| 5 | Extratores opcionais e observabilidade | G | extensão desabilitada não altera o fluxo padrão | + +## Testes de combinação obrigatórios + +- 10 mil arquivos pequenos: memória limitada, páginas determinísticas e ordem + estável. +- Arquivo acima do limite; total acima do limite; exceção humana registrada. +- Glob include/exclude sobre caminhos Windows e Unix; `.git`, symlinks, binários + e permissões negadas. +- Interrupção durante cópia, hash, criação de lote e atualização de checkpoint. +- Retomada com hash de fonte alterado, lease expirada e lote bloqueado. +- Dois agentes tentando o mesmo lote; lotes distintos em paralelo. +- Mistura de run v1 e v2; upgrade e migração sem reescrita de conteúdo adotante. +- Materialização com uma fonte não revisada, destino duplicado e falha no meio: + nenhum artefato parcial pode permanecer. +- Confirmação de que nenhum caminho cria aprovação, commit, push, merge ou + resolve review remoto. + +## Superfícies afetadas + +`internal/sourceimport`, `internal/cli`, `internal/validator`, +`framework/skills/artifact-importer`, templates de importação, starter, +`FRAMEWORK.md`, `docs/starting-points.md`, testes de CLI e de upgrade. Avaliar +uma extensão de extratores somente depois da Fase 4; ela não entra no núcleo. + +## Decisões humanas pendentes + +1. Quais limites padrão são aceitáveis (arquivos, bytes totais e arquivo único)? +2. A cópia integral para `knowledge/imports/sources/` continua obrigatória para + todos os formatos, ou referências externas com hash são permitidas? +3. Quais formatos terão extrator oficial inicial (Markdown apenas, PDF, DOCX)? +4. Um lote pode ser marcado `not_applicable` por um humano sem leitura integral? +5. Qual política de retenção se aplica às fontes que contenham dados sensíveis? + +## Não recomendações + +Não fazer uma única chamada de agente para “entender toda a pasta”, não gravar +resumos sem fonte/trecho, não paralelizar dois agentes sobre o mesmo arquivo, +não transformar importação em aprovação e não introduzir banco de dados como +fonte canônica. JSON paginado e Markdown continuam auditáveis e preserváveis. diff --git a/docs/todo/subagent-orchestration-evolution-plan.md b/docs/todo/subagent-orchestration-evolution-plan.md new file mode 100644 index 0000000..6e29dd3 --- /dev/null +++ b/docs/todo/subagent-orchestration-evolution-plan.md @@ -0,0 +1,122 @@ +# Plano de evolução — Subagents governados + +## Objetivo + +Evoluir dos contratos atuais de tasks, chunks, leases, grafo e evidências para +uma orquestração assistida. O objetivo é paralelismo controlado sem delegar +decisões de produto, aprovação humana ou autoridade de entrega ao runtime. + +## Limites permanentes + +- Grafo e chunks são as unidades canônicas; o orquestrador não inventa trabalho. +- Cada subagent recebe escopo, leitura obrigatória, lease e evidência esperada. +- QA, Code Review e Security Review continuam independentes e read-only. +- Nenhum subagent aprova, commita, envia, faz merge, publica ou resolve review. +- Artefatos do produto permanecem auditáveis e pertencem ao adotante. + +## Fase 0 — Fundamentos + +Definir um `DispatchEnvelope` com unidade, agente, hash de entrada, leituras, +write scope, recursos compartilhados, evidências, status e operações proibidas. +Estados: `candidate`, `assigned`, `running`, `returned`, `reviewed`, `closed`, +`blocked`, `expired` e `cancelled`. + +Aceite: atribuição e retorno são reconstruíveis sem conversa anterior. + +## Fase 1 — Planejamento read-only + +Criar `dispatch plan --work WORK-NNN`. Mostra tasks/chunks elegíveis, +dependências, ondas, conflitos, risco e especialista recomendado. Não cria +agente, lease ou arquivo de produto. + +## Fase 2 — Atribuição explícita + +Criar `dispatch assign --unit --agent --yes`. Cria lease, envelope +imutável, handoff inicial e evento operacional. Falha para unidade não pronta, +dependência pendente, lease existente ou conflito de escopo. + +Padrão desligado por configuração do produto. Remoção segura: impedir novas +atribuições e preservar envelopes/evidências existentes. + +## Fase 3 — Retorno e observação + +Criar `dispatch return`, `dispatch observe` e `dispatch reconcile`. O retorno +exige resumo, hashes, evidências, bloqueios e rota sugerida. Observação mostra +leases, trabalho ativo, worktrees e gates pendentes. Reconciliação só reporta +estado órfão; não repara nada. + +## Fase 4 — Execução local supervisionada + +Iniciar somente um harness explicitamente configurado para um envelope já +atribuído. Código exige worktree; importação exige chunk. O processo recebe +envelope e diretórios permitidos, não uma instrução livre. Logs e transcrições +são evidência operacional. + +Guardrails: limite de concorrência, timeout, heartbeat, cancelamento explícito, +allowlist de harnesses e nenhuma credencial externa por padrão. + +## Fase 5 — Ondas supervisionadas + +Criar `dispatch wave --wave WAVE-003 --max-parallel 3 --yes`. Inicia apenas +unidades atribuídas, prontas e sem escopo/recurso compartilhado sobreposto. +Novas unidades param quando surge blocker; as já iniciadas não são apagadas. + +Primeiro rollout: chunks de importação. Segundo rollout: tasks em worktrees. + +## Fase 6 — Retorno para gates independentes + +Retornos viram candidatos a QA, Code Review e Security Review, nunca aprovação. +O orquestrador verifica hashes/evidências, mas não altera estados canônicos. + +## Fase 7 — Recomendações adaptativas + +Depois de uso comprovado, recomendar tamanho de lote, capacidade, gargalos e +prioridade por dependência. Recomendações exigem confirmação e não mudam escopo +ou critérios de aprovação. + +## Contrato mínimo + +```json +{ + "dispatch_id": "DISPATCH-001", + "unit": { "kind": "task", "id": "TK-014" }, + "agent": "code-runner-1", + "input_hash": "sha256:...", + "required_reading": ["specification.md", "tasks/TK-014.md"], + "write_scope": ["src/billing"], + "expected_evidence": ["test log", "diff hash"], + "status": "assigned", + "forbidden": ["approval", "commit", "push", "merge", "release"] +} +``` + +## Testes obrigatórios + +- Duas atribuições concorrentes para a mesma unidade. +- Dependência pendente, conflito de escopo e recurso compartilhado. +- Processo perdido, lease expirada, retorno duplicado e envelope órfão. +- Hash incompatível, evidência ausente e escrita fora do escopo. +- Wave parcialmente iniciada, cancelamento e retomada. +- Tentativas de aprovação, commit, push, merge, release e review externo. +- Upgrade e remoção sem apagar histórico operacional. + +## Métricas + +- Tempo entre unidade pronta e atribuída. +- Taxa de lease expirada, conflito e retorno bloqueado. +- Cobertura de evidência por retorno. +- Tempo até as revisões independentes. + +## Decisões humanas necessárias + +1. Harnesses locais suportados inicialmente. +2. Limite de concorrência por máquina e workspace. +3. Tipos de task permitidos após o piloto de importação. +4. Retenção e acesso a transcrições. +5. Política de cancelamento e recuperação de leases expiradas. + +## Não recomendações + +Não criar agentes ilimitados, não automatizar decisões de produto, não permitir +memória compartilhada sem proveniência e não automatizar merge/aprovação. Cada +fase só avança após demonstrar auditabilidade e critérios de aceite. diff --git a/framework/install.md b/framework/install.md index 02b2319..8a06907 100644 --- a/framework/install.md +++ b/framework/install.md @@ -57,9 +57,9 @@ spec-framework init ../my-product --agents codex,cursor,claude --yes | `existing-implementation` | `knowledge/assessments/implementation-assessment.md` plus full Foundation | | `audit-only` | Read-only bootstrap; mutating CLI commands are refused | -Use `--starting-point existing-documents` with `--source-dir` or `--sources` to bootstrap from existing product material. This creates an analysis-only import run with a per-source `traceability.json` ledger for an LLM or agent to fill with evidence, extracted claims, destinations, and unmapped gaps; it never treats source prose as approved product truth. +Use `--starting-point existing-documents` with `--source-dir` or `--sources` to bootstrap from existing product material. This creates a scalable analysis-only import run with paged inventory and review chunks; use `--import-max-files`, `--import-max-total-bytes`, `--import-max-file-bytes`, and `--import-chunk-size` to set the explicit bootstrap budget. It never treats source prose as approved product truth. -Draft materialization is a separate explicit command after the import mappings have been reviewed: `spec-framework import materialize --run IMPORT-001 --approved-by "Product Owner" --yes`. +For a large document set, create a bounded scalable run with `import create`, inspect it with `import status`, and resume one bounded chunk at a time. Draft materialization remains a separate explicit command after sources/chunks and mappings have been reviewed: `spec-framework import materialize --run IMPORT-001 --approved-by "Product Owner" --yes`. The generated `product/BOOTSTRAP.md` is rendered from the starting-point map in `framework/init/bootstrap.json`; each step names the user goal, agent reading set, writable draft paths, prompt, gate, and next handoff. For imports it also pins the active run id. `spec-framework work` remains blocked until that latest run records explicit materialization approval and at least one materialized draft path. This gate does not approve the resulting product artifacts. diff --git a/framework/skills/README.md b/framework/skills/README.md index aa42657..4cb0f99 100644 --- a/framework/skills/README.md +++ b/framework/skills/README.md @@ -26,8 +26,8 @@ When a skill mentions a product-relative path such as `knowledge/conventions/gat ## Expected Files - `/SKILL.md`: one skill per folder. -- Specialist skills: problem, vision, strategy, domain, goal, journey, feature, use case, Design System, UX/UI, specification, Engineering System, technical discovery, engineering proposal, engineering review, implementation planning, graph, task, code runner, bug fixer, QA, code review, security review, threat modeler, commit crafter, PR finalizer, audit, documentation, history, and artifact import. -- Orchestrator skills: product, domain evolution, existing product import, new feature, audit, evolution, documentation, release, delivery, execution scheduling, and integration. +- Specialist skills: problem, vision, strategy, domain, goal, journey, feature, use case, Design System, UX/UI, specification, Engineering System, technical discovery, engineering proposal, engineering review, implementation planning, graph, task, code runner, bug fixer, QA, code review, security review, threat modeler, commit crafter, PR finalizer, audit, documentation, history, artifact import, and subagent return review. +- Orchestrator skills: product, domain evolution, existing product import, new feature, audit, evolution, documentation, release, delivery, execution scheduling, integration, and dispatch. - Guidance skill: Framework Guide translates human goals into current CLI state, the smallest safe command, and the correct specialist or approval handoff without authoring artifacts. Runtime v2 also includes the `command-planner` and `command-executor` operational skills. The planner owns immutable argv-based plans; the executor is restricted to local R0/R1 plans. diff --git a/framework/skills/artifact-importer/SKILL.md b/framework/skills/artifact-importer/SKILL.md index da10fb1..419c71d 100644 --- a/framework/skills/artifact-importer/SKILL.md +++ b/framework/skills/artifact-importer/SKILL.md @@ -33,14 +33,14 @@ Source documents; product context; existing Domains, User Goals, Features, gloss ## Workflow 1. Copy or reference sources without modifying their content and compute a SHA-256 hash for each source. 2. Inventory source path, format, size, and hash. -3. Read each imported source and update `traceability.json` with review status, section-level evidence, extracted claims, candidate ids, mapped targets, and gaps. Every source must end as `reviewed`, `partially_mapped`, `mapped`, or `not_applicable`; never leave the reason implicit. +3. For a scalable run, claim only one `CHUNK-NNNN` with `import resume`, read its sources, then use `import record-review` with section-level evidence for every non-excluded source. For a legacy run, update `traceability.json`. Every source must end as reviewed, partially mapped, mapped, or not applicable; never leave the reason implicit. 4. Extract candidate Domains, User Goals, Features, rules, decisions, priorities, and dependencies with section-level evidence. 5. Compare candidates with existing artifacts and the glossary. 6. Record duplicates, contradictions, ambiguous parents, and open questions; never resolve them silently. 7. Propose source-to-artifact mappings in `draft` and leave `materialization_approved` false. 8. Stop for explicit human approval before creating canonical product artifacts. 9. When approved, materialize only selected mappings as `draft`, preserving `source_documents` traceability, recording `provenance.kind: import-draft` and `provenance.import_run`, and never creating approval records. -10. Use `spec-framework import materialize --run IMPORT-NNN --approved-by --yes` for mechanical materialization; do not edit the approval fields manually. +10. Use `spec-framework import materialize --run IMPORT-NNN --approved-by --yes` for mechanical materialization only after every scalable chunk is reviewed or excluded; do not edit the approval fields manually. ## Quality checklist - [ ] Preserves traceability to affected artifacts. diff --git a/framework/skills/dispatch-orchestrator/SKILL.md b/framework/skills/dispatch-orchestrator/SKILL.md new file mode 100644 index 0000000..336a460 --- /dev/null +++ b/framework/skills/dispatch-orchestrator/SKILL.md @@ -0,0 +1,52 @@ +--- +name: dispatch-orchestrator +description: "Dispatch Orchestrator Skill. Use when Codex needs to plan, assign, observe, reconcile, or supervise bounded subagent work in the Spec Framework workflow." +--- + +# Dispatch Orchestrator Skill + +## Layer +Execution + +## Responsibility +Owns dispatch envelopes, waves, capacity observation, and handoff sequencing. It does not author product artifacts, approve work, or deliver remotely. + +## Operating modes +- create: create an explicit assignment envelope. +- update: return or reconcile persisted dispatch state. +- audit: inspect leases, transcripts, scope conflicts, and stale returns. +- explain: summarize eligible work and blockers. + +## Inputs +Approved Execution Graph; ready task or import chunk; workspace state; dispatch configuration; current product decisions. + +## Outputs +Persisted envelopes; dispatch plan; wave observation; reconciliation findings; handoffs. + +## Required reading +- the framework root's `FRAMEWORK.md` +- Relevant parent context.md files. +- Relevant templates in framework/template/. +- Approved product decisions in the active product root's `knowledge/decisions/` and `.product/decisions.json`. + +## Workflow +1. Read current graph/chunk readiness and dispatch configuration. +2. Plan only canonical units with no dependency or scope conflict. +3. Require explicit human confirmation before assignment or execution. +4. Persist envelope, lease, required reading, hashes, scope, forbidden operations, and expected evidence. +5. Dispatch QA, Code Review, and Security Review only from the returned Code Runner diff hash. +6. Reconcile and route blockers; never repair approval or product state. + +## Quality checklist +- [ ] Preserves traceability to affected artifacts. +- [ ] Uses the correct template and naming conventions. +- [ ] Never assigns overlapping write scopes or resources. +- [ ] Execution uses an enabled harness and explicit confirmation. +- [ ] Detects gaps, conflicts, and dependencies. +- [ ] Records or requests decisions for meaningful changes. +- [ ] Leaves a clear handoff for the next skill or orchestrator. + +## Handoff +Next: code-runner, artifact-importer, qa, code-review, security-review, or product-historian. + +Pass forward envelope, hashes, evidence, blockers, risks, and required follow-up work. diff --git a/framework/skills/subagent-return-reviewer/SKILL.md b/framework/skills/subagent-return-reviewer/SKILL.md new file mode 100644 index 0000000..8c690d7 --- /dev/null +++ b/framework/skills/subagent-return-reviewer/SKILL.md @@ -0,0 +1,50 @@ +--- +name: subagent-return-reviewer +description: "Subagent Return Reviewer Skill. Use when Codex needs to validate a bounded subagent return against its envelope, hashes, evidence, and route in the Spec Framework workflow." +--- + +# Subagent Return Reviewer Skill + +## Layer +Validation + +## Responsibility +Owns `dispatch-return.md` validation and routing. It does not fix code, approve artifacts, accept residual risk, or close external reviews. + +## Operating modes +- create: record a validated return. +- update: refresh a return after new evidence. +- audit: find stale hashes, missing evidence, and scope violations. +- explain: summarize return readiness and blockers. + +## Inputs +Dispatch envelope; transcript; task/chunk context; hashes; evidence; findings. + +## Outputs +`dispatch-return.md`; route; blockers; stale-return verdict. + +## Required reading +- the framework root's `FRAMEWORK.md` +- Relevant parent context.md files. +- Relevant templates in framework/template/. +- Approved product decisions in the active product root's `knowledge/decisions/` and `.product/decisions.json`. + +## Workflow +1. Verify agent, unit, input hash, diff hash when applicable, scope, and evidence. +2. Reject returns with missing evidence, stale diff, or forbidden operation. +3. Preserve unresolved gaps and route to the independent owner. +4. Record the return without changing canonical task approval or lifecycle. + +## Quality checklist +- [ ] Preserves traceability to affected artifacts. +- [ ] Uses the correct template and naming conventions. +- [ ] Return matches its envelope and current diff. +- [ ] Findings have route and owner. +- [ ] Detects gaps, conflicts, and dependencies. +- [ ] Records or requests decisions for meaningful changes. +- [ ] Leaves a clear handoff for the next skill or orchestrator. + +## Handoff +Next: qa, code-review, security-review, artifact-importer, code-runner, or product-historian. + +Pass forward return, hashes, evidence, blockers, findings, risks, and required follow-up work. diff --git a/framework/template/dispatch-return-template.md b/framework/template/dispatch-return-template.md new file mode 100644 index 0000000..7f45fd6 --- /dev/null +++ b/framework/template/dispatch-return-template.md @@ -0,0 +1,26 @@ +# Dispatch Return + +| Field | Value | +| --- | --- | +| Dispatch ID | `[DISPATCH-NNN]` | +| Unit | `[task/chunk/research boundary]` | +| Agent | `[agent id]` | +| Status | `[returned/blocked]` | +| Input hash | `[sha256]` | +| Diff hash | `[sha256 or N/A]` | + +## Summary + +[Outcome and bounded scope.] + +## Evidence + +- [Links, hashes, logs, source locators, or findings.] + +## Blockers And Gaps + +- [Explicit blocker, ambiguity, or `None`.] + +## Recommended Route + +[QA, Code Review, Security Review, Artifact Importer, Product Historian, or human.] diff --git a/internal/cli/app.go b/internal/cli/app.go index 2691a6e..ae8e049 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -8,6 +8,7 @@ import ( "io" "os" "path/filepath" + "strconv" "strings" "github.com/JonatasFreireDev/spec-framework/internal/adapters" @@ -23,6 +24,39 @@ import ( type App struct { version string } +type multiFlag []string + +func (m *multiFlag) String() string { return strings.Join(*m, ",") } +func (m *multiFlag) Set(value string) error { *m = append(*m, value); return nil } + +func absoluteProductRoot(value string) (string, error) { + cwd, err := os.Getwd() + if err != nil { + return "", err + } + if filepath.IsAbs(value) { + return value, nil + } + return filepath.Join(cwd, value), nil +} +func parseByteLimit(value string) (int64, error) { + value = strings.ToUpper(strings.TrimSpace(value)) + multiplier := int64(1) + for _, unit := range []struct { + suffix string + size int64 + }{{"GB", 1 << 30}, {"MB", 1 << 20}, {"KB", 1 << 10}, {"B", 1}} { + if strings.HasSuffix(value, unit.suffix) { + value, multiplier = strings.TrimSpace(strings.TrimSuffix(value, unit.suffix)), unit.size + break + } + } + number, err := strconv.ParseInt(value, 10, 64) + if err != nil || number < 1 { + return 0, fmt.Errorf("invalid byte limit %q", value) + } + return number * multiplier, nil +} func New(version string) App { if version == "" { @@ -327,8 +361,126 @@ func runGraph(args []string, stdout, stderr io.Writer) int { } func runImport(args []string, stdout, stderr io.Writer) int { - if len(args) == 0 || args[0] != "materialize" { - fmt.Fprintln(stderr, "usage: spec-framework import materialize --run IMPORT-NNN --approved-by --yes") + if len(args) == 0 { + fmt.Fprintln(stderr, "usage: spec-framework import create|status|resume|materialize") + return 2 + } + if args[0] == "create" { + flags := flag.NewFlagSet("import create", flag.ContinueOnError) + flags.SetOutput(stderr) + productRoot := flags.String("product-root", "product", "product root") + include := multiFlag{} + exclude := multiFlag{} + flags.Var(&include, "include", "include glob (repeatable)") + flags.Var(&exclude, "exclude", "exclude glob (repeatable)") + maxFiles := flags.Int("max-files", 500, "maximum matched files") + maxTotal := flags.String("max-total-bytes", "200MB", "maximum copied bytes") + maxFile := flags.String("max-file-bytes", "10MB", "maximum bytes per file") + chunkSize := flags.Int("chunk-size", 25, "sources per analysis chunk") + binaryPolicy := flags.String("binary-policy", "inventory_only", "inventory_only or reject") + if err := flags.Parse(args[1:]); err != nil { + return 2 + } + if len(flags.Args()) == 0 { + fmt.Fprintln(stderr, "import create requires one or more source paths") + return 2 + } + total, err := parseByteLimit(*maxTotal) + if err != nil { + fmt.Fprintln(stderr, err) + return 2 + } + file, err := parseByteLimit(*maxFile) + if err != nil { + fmt.Fprintln(stderr, err) + return 2 + } + root, err := absoluteProductRoot(*productRoot) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + run, err := sourceimport.CreateScalableRun(root, flags.Args(), sourceimport.CreateOptions{Include: include, Exclude: exclude, MaxFiles: *maxFiles, MaxTotalBytes: total, MaxFileBytes: file, ChunkSize: *chunkSize, BinaryPolicy: *binaryPolicy}) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + fmt.Fprintln(stdout, "Created scalable import run", run) + return 0 + } + if args[0] == "status" || args[0] == "resume" || args[0] == "record-review" { + flags := flag.NewFlagSet("import "+args[0], flag.ContinueOnError) + flags.SetOutput(stderr) + productRoot := flags.String("product-root", "product", "product root") + runID := flags.String("run", "", "import run id") + agent := flags.String("agent", "", "importer identity") + chunk := flags.String("chunk", "", "chunk id") + input := flags.String("input", "", "review JSON input") + yes := flags.Bool("yes", false, "confirm review record") + jsonOutput := flags.Bool("json", false, "structured output") + if err := flags.Parse(args[1:]); err != nil { + return 2 + } + if strings.TrimSpace(*runID) == "" { + fmt.Fprintln(stderr, "import "+args[0]+" requires --run") + return 2 + } + root, err := absoluteProductRoot(*productRoot) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + if args[0] == "resume" { + claimed, err := sourceimport.Resume(root, *runID, *chunk, *agent) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + if *jsonOutput { + data, _ := json.Marshal(claimed) + fmt.Fprintln(stdout, string(data)) + } else { + fmt.Fprintln(stdout, "RESUMED", claimed.ID, "for", claimed.Agent) + } + return 0 + } + if args[0] == "record-review" { + if !*yes || *chunk == "" || *agent == "" || *input == "" { + fmt.Fprintln(stderr, "import record-review requires --chunk, --agent, --input, and --yes") + return 2 + } + data, err := os.ReadFile(*input) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + var review sourceimport.ChunkReview + if err := json.Unmarshal(data, &review); err != nil { + fmt.Fprintln(stderr, err) + return 2 + } + if err := sourceimport.RecordChunkReview(root, *runID, *chunk, *agent, review); err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + fmt.Fprintln(stdout, "REVIEWED", *chunk) + return 0 + } + status, err := sourceimport.ImportStatus(root, *runID) + if err != nil { + fmt.Fprintln(stderr, err) + return 1 + } + if *jsonOutput { + data, _ := json.Marshal(status) + fmt.Fprintln(stdout, string(data)) + } else { + fmt.Fprintf(stdout, "%s: %d source(s), %d chunk(s): %d queued, %d reviewing, %d reviewed, %d blocked, %d excluded\n", status.ImportID, status.Sources, status.Chunks, status.Queued, status.Reviewing, status.Reviewed, status.Blocked, status.Excluded) + } + return 0 + } + if args[0] != "materialize" { + fmt.Fprintln(stderr, "usage: spec-framework import create|status|resume|materialize") return 2 } flags := flag.NewFlagSet("import materialize", flag.ContinueOnError) @@ -561,6 +713,11 @@ func (app App) runInit(args []string, stdout, stderr io.Writer) int { startingPoint := flags.String("starting-point", "new-product", "new-product, existing-product, existing-documents, existing-feature, existing-implementation, or audit-only") sourcesValue := flags.String("sources", "", "comma-separated source files or directories for existing-documents") sourceDir := flags.String("source-dir", "", "source directory for existing-documents") + importMaxFiles := flags.Int("import-max-files", 500, "maximum files for existing-documents import") + importMaxTotal := flags.String("import-max-total-bytes", "200MB", "maximum copied bytes for existing-documents import") + importMaxFile := flags.String("import-max-file-bytes", "10MB", "maximum bytes per imported file") + importChunkSize := flags.Int("import-chunk-size", 25, "sources per import review chunk") + importBinaryPolicy := flags.String("import-binary-policy", "inventory_only", "inventory_only or reject") force := flags.Bool("force", false, "compatibility flag; never overwrites an existing product directory") installImpeccable := flags.Bool("install-impeccable", false, "install the optional Impeccable adapter after init") impeccableVersion := flags.String("impeccable-version", "", "explicit Impeccable CLI version") @@ -617,7 +774,17 @@ func (app App) runInit(args []string, stdout, stderr io.Writer) int { if strings.TrimSpace(*sourceDir) != "" { sources = append(sources, *sourceDir) } - result, err := install.Init(install.Options{Target: *target, Version: app.version, Agents: agents, StartingPoint: point, Sources: sources, Force: *force}) + importTotal, err := parseByteLimit(*importMaxTotal) + if err != nil { + fmt.Fprintln(stderr, err) + return 2 + } + importFile, err := parseByteLimit(*importMaxFile) + if err != nil { + fmt.Fprintln(stderr, err) + return 2 + } + result, err := install.Init(install.Options{Target: *target, Version: app.version, Agents: agents, StartingPoint: point, Sources: sources, ImportOptions: sourceimport.CreateOptions{MaxFiles: *importMaxFiles, MaxTotalBytes: importTotal, MaxFileBytes: importFile, ChunkSize: *importChunkSize, BinaryPolicy: *importBinaryPolicy}, Force: *force}) if err != nil { fmt.Fprintln(stderr, err) return 1 diff --git a/internal/cli/cobra.go b/internal/cli/cobra.go index 0f60b5f..acbec11 100644 --- a/internal/cli/cobra.go +++ b/internal/cli/cobra.go @@ -67,6 +67,7 @@ func (app App) NewCommand(stdout, stderr io.Writer) *cobra.Command { app.legacyCommand("impact", "Inspect a decision's validity and propagation.", func(args []string, out, errout io.Writer) int { return runImpact(args, out, errout) }, stdout, stderr), app.legacyCommand("dashboard", "Show a consolidated workflow dashboard.", func(args []string, out, errout io.Writer) int { return runDashboard(args, out, errout) }, stdout, stderr), app.legacyCommand("decisions", "Check or migrate product decisions.", func(args []string, out, errout io.Writer) int { return runDecisions(args, out, errout) }, stdout, stderr), + app.legacyCommand("dispatch", "Plan and supervise governed subagent assignments.", func(args []string, out, errout io.Writer) int { return runDispatch(args, out, errout) }, stdout, stderr), ) for _, name := range []string{"resume", "handoff", "checkpoint", "lease", "commands", "schedule", "integrate", "runtime", "reviews"} { command := name diff --git a/internal/cli/dispatch.go b/internal/cli/dispatch.go new file mode 100644 index 0000000..4adbe6e --- /dev/null +++ b/internal/cli/dispatch.go @@ -0,0 +1,254 @@ +package cli + +import ( + "encoding/json" + "flag" + "fmt" + "github.com/JonatasFreireDev/spec-framework/internal/dispatch" + "github.com/JonatasFreireDev/spec-framework/internal/sourceimport" + "io" + "os" + "path/filepath" + "strings" +) + +func mustRecommendations(items []dispatch.Recommendation, err error) []dispatch.Recommendation { + if err != nil { + return []dispatch.Recommendation{{Kind: "unavailable", Detail: err.Error(), RequiresConfirmation: true}} + } + return items +} + +func runDispatch(args []string, out, errout io.Writer) int { + if len(args) == 0 { + fmt.Fprintln(errout, "dispatch requires plan, assign, return, observe, or reconcile") + return 2 + } + fs := flag.NewFlagSet("dispatch", flag.ContinueOnError) + fs.SetOutput(errout) + root := fs.String("product-root", "product", "product root") + work := fs.String("work", "", "workspace") + graph := fs.String("graph", "", "execution graph") + task := fs.String("task", "", "task") + agent := fs.String("agent", "", "agent") + role := fs.String("role", "code-runner", "role") + id := fs.String("id", "", "dispatch id") + summary := fs.String("summary", "", "return summary") + evidence := fs.String("evidence", "", "comma-separated evidence") + command := fs.String("command", "", "supervised executable") + enable := fs.Bool("enable", false, "explicitly enable supervised execution") + wave := fs.String("wave", "", "persisted scheduler wave id") + max := fs.Int("max-parallel", 1, "maximum concurrent dispatches") + diffHash := fs.String("diff-hash", "", "immutable working-tree diff hash") + parent := fs.String("parent", "", "returned code-runner dispatch id") + runID := fs.String("run", "", "scalable import run") + chunk := fs.String("chunk", "", "scalable import chunk") + reviewInput := fs.String("review-input", "", "structured import review JSON") + harnesses := fs.String("harnesses", "", "comma-separated allowed harness basenames") + enabled := fs.Bool("enabled", false, "enable dispatch capability") + retention := fs.Int("transcript-retention", 100, "transcripts to retain per workspace") + yes := fs.Bool("yes", false, "confirm mutation") + if err := fs.Parse(args[1:]); err != nil { + return 2 + } + wd, _ := os.Getwd() + p := *root + if !filepath.IsAbs(p) { + p = filepath.Join(wd, p) + } + g := *graph + if g != "" && !filepath.IsAbs(g) { + g = filepath.Join(p, filepath.FromSlash(g)) + } + switch args[0] { + case "configure": + if !*yes { + fmt.Fprintln(errout, "dispatch configure requires --yes") + return 2 + } + if err := dispatch.SaveConfig(p, dispatch.Config{Version: 1, Enabled: *enabled, Harnesses: splitCSV(*harnesses), MaxParallel: *max, TranscriptRetention: *retention}); err != nil { + fmt.Fprintln(errout, err) + return 1 + } + fmt.Fprintln(out, "DISPATCH CONFIGURED") + return 0 + case "plan": + items, e := dispatch.Plan(p, g) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + for _, x := range items { + fmt.Fprintf(out, "%s %s ready=%t %s\n", x.TaskID, x.Role, x.Ready, strings.Join(x.Blockers, "; ")) + } + return 0 + case "assign": + if *role == "technical-discovery" { + if !*yes || *work == "" || *task == "" || *agent == "" { + fmt.Fprintln(errout, "technical-discovery assignment requires --work --task --agent --yes") + return 2 + } + path := *task + if !filepath.IsAbs(path) { + path = filepath.Join(p, filepath.FromSlash(path)) + } + x, e := dispatch.AssignResearch(p, *work, path, *agent) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "ASSIGNED", x.ID) + return 0 + } + if *role == "threat-modeler" { + if !*yes || *work == "" || *task == "" || *agent == "" { + fmt.Fprintln(errout, "threat-modeler assignment requires --work --task --agent --yes") + return 2 + } + path := *task + if !filepath.IsAbs(path) { + path = filepath.Join(p, filepath.FromSlash(path)) + } + x, e := dispatch.AssignThreatModel(p, *work, path, *agent) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "ASSIGNED", x.ID) + return 0 + } + if *role == "artifact-importer" { + if !*yes || *work == "" || *runID == "" || *agent == "" { + fmt.Fprintln(errout, "artifact-importer assignment requires --work --run --agent --yes") + return 2 + } + x, e := dispatch.AssignImportChunk(p, *work, *runID, *chunk, *agent) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "ASSIGNED", x.ID) + return 0 + } + if !*yes || *work == "" || *agent == "" || ((*role != "qa" && *role != "code-review" && *role != "security-review") && *task == "") || ((*role == "qa" || *role == "code-review" || *role == "security-review") && *parent == "") { + fmt.Fprintln(errout, "dispatch assign requires --work --task --agent --yes") + return 2 + } + if *role == "qa" || *role == "code-review" || *role == "security-review" { + x, e := dispatch.AssignReview(p, *work, *parent, *role, *agent) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "ASSIGNED", x.ID) + return 0 + } + x, e := dispatch.Assign(p, *work, g, *task, *role, *agent) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "ASSIGNED", x.ID) + return 0 + case "return": + if !*yes || *work == "" || *id == "" || *agent == "" { + fmt.Fprintln(errout, "dispatch return requires --work --id --agent --yes") + return 2 + } + if *reviewInput != "" { + data, e := os.ReadFile(*reviewInput) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + var review sourceimport.ChunkReview + if e = json.Unmarshal(data, &review); e != nil { + fmt.Fprintln(errout, e) + return 2 + } + x, e := dispatch.ReturnImport(p, *work, *id, *agent, *summary, review) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "RETURNED", x.ID) + return 0 + } + x, e := dispatch.Return(p, *work, *id, *agent, *summary, *diffHash, splitCSV(*evidence)) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "RETURNED", x.ID) + return 0 + case "observe": + if *work == "" { + fmt.Fprintln(errout, "dispatch observe requires --work") + return 2 + } + xs, e := dispatch.Observe(p, *work) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + for _, x := range xs { + fmt.Fprintf(out, "%s %s %s %s\n", x.ID, x.TaskID, x.Role, x.Status) + } + return 0 + case "reconcile": + if *work == "" { + fmt.Fprintln(errout, "dispatch reconcile requires --work") + return 2 + } + xs, e := dispatch.Reconcile(p, *work) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + for _, x := range xs { + fmt.Fprintf(out, "%s %s -> %s\n", x.Kind, x.DispatchID, x.Owner) + } + return 0 + case "run": + if !*yes || !*enable || *work == "" || *id == "" || *command == "" { + fmt.Fprintln(errout, "dispatch run requires --work --id --command --enable --yes") + return 2 + } + t, e := dispatch.Run(p, *work, *id, *enable, *command, fs.Args()) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + fmt.Fprintln(out, "RAN", t.DispatchID, t.OutputHash) + return 0 + case "wave": + if !*yes || !*enable || *work == "" || *wave == "" || *command == "" { + fmt.Fprintln(errout, "dispatch wave requires --work --wave --command --enable --yes") + return 2 + } + waveIDs, e := dispatch.WaveIDs(p, *work, *wave) + if e != nil { + fmt.Fprintln(errout, e) + return 1 + } + for _, r := range dispatch.RunWave(p, *work, waveIDs, *max, *enable, *command, fs.Args()) { + if r.Error != "" { + fmt.Fprintln(errout, r.ID, r.Error) + } else { + fmt.Fprintln(out, "RAN", r.ID, r.Transcript.OutputHash) + } + } + return 0 + case "recommend": + if *work == "" { + fmt.Fprintln(errout, "dispatch recommend requires --work") + return 2 + } + for _, r := range mustRecommendations(dispatch.Recommend(p, *work, *max)) { + fmt.Fprintln(out, r.Kind, r.Detail) + } + return 0 + } + fmt.Fprintln(errout, "unknown dispatch operation") + return 2 +} diff --git a/internal/cli/integration_test.go b/internal/cli/integration_test.go index 385e6a1..afb09df 100644 --- a/internal/cli/integration_test.go +++ b/internal/cli/integration_test.go @@ -8,8 +8,20 @@ import ( "testing" "github.com/JonatasFreireDev/spec-framework/internal/cli" + "github.com/JonatasFreireDev/spec-framework/internal/sourceimport" ) +func reviewInitialImportChunk(t *testing.T, productRoot string) { + t.Helper() + chunk, err := sourceimport.Resume(productRoot, "IMPORT-001", "CHUNK-0001", "test-importer") + if err != nil { + t.Fatal(err) + } + if err := sourceimport.RecordChunkReview(productRoot, "IMPORT-001", chunk.ID, "test-importer", sourceimport.ChunkReview{SourceEvidence: map[string][]sourceimport.Evidence{"SRC-000001": {{Locator: "line 1", Claim: "fixture reviewed"}}}}); err != nil { + t.Fatal(err) + } +} + func TestGoCLIInitValidateUpgradeAndMove(t *testing.T) { t.Setenv("SPEC_FRAMEWORK_CACHE", filepath.Join(t.TempDir(), "cache")) t.Setenv("SPEC_FRAMEWORK_AGENT_HOME", filepath.Join(t.TempDir(), "agents")) @@ -160,6 +172,7 @@ func TestCLIExistingDocumentsMaterialization(t *testing.T) { stdout.Reset() stderr.Reset() productRoot := filepath.Join(target, "product") + reviewInitialImportChunk(t, productRoot) if code := app.Run([]string{"import", "materialize", "--product-root", productRoot, "--run", "IMPORT-001", "--approved-by", "Product Owner", "--yes"}, &stdout, &stderr); code != 0 { t.Fatalf("materialize=%d stdout=%s stderr=%s", code, stdout.String(), stderr.String()) } @@ -193,6 +206,7 @@ func TestCLIExistingDocumentsNormalizeApproveAndWork(t *testing.T) { t.Fatal(err) } sourceRel := inv["sources"].([]any)[0].(map[string]any)["path"].(string) + reviewInitialImportChunk(t, productRoot) featurePath := "domains/imported/goals/imported/features/imported/context.md" draft := "---\nid: FT-IMPORT-001\ntype: feature\nname: Imported Feature\nstatus: draft\nowner_skill: feature\nslug: imported\nrigor_tier: S\nsource_documents:\n - " + sourceRel + "\n---\n\n# Imported Feature\n" mapping := map[string]any{"schema_version": 1, "import_id": "IMPORT-001", "mappings": []any{map[string]any{"id": "MAP-001", "target": featurePath, "artifact_type": "feature", "selected": true, "source_documents": []string{sourceRel}, "draft_content": draft}}} diff --git a/internal/dispatch/dispatch.go b/internal/dispatch/dispatch.go new file mode 100644 index 0000000..709ab96 --- /dev/null +++ b/internal/dispatch/dispatch.go @@ -0,0 +1,579 @@ +// Package dispatch persists supervised subagent assignments. It never starts a +// process, grants approval, or performs delivery operations. +package dispatch + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/JonatasFreireDev/spec-framework/internal/sourceimport" + "github.com/JonatasFreireDev/spec-framework/internal/workflow" +) + +type Envelope struct { + Version int `json:"version"` + ID string `json:"id"` + WorkspaceID string `json:"workspace_id"` + TaskID string `json:"task_id"` + UnitKind string `json:"unit_kind"` + UnitPath string `json:"unit_path,omitempty"` + ImportRun string `json:"import_run,omitempty"` + ImportChunk string `json:"import_chunk,omitempty"` + Role string `json:"role"` + Agent string `json:"agent"` + Graph string `json:"graph"` + TaskPath string `json:"task_path"` + InputHash string `json:"input_hash"` + DiffHash string `json:"diff_hash,omitempty"` + ParentID string `json:"parent_id,omitempty"` + RequiredReading []string `json:"required_reading"` + WriteScope []string `json:"write_scope"` + ExpectedEvidence []string `json:"expected_evidence"` + Status string `json:"status"` + CreatedAt string `json:"created_at"` + ReturnedAt string `json:"returned_at,omitempty"` + Summary string `json:"summary,omitempty"` + Evidence []string `json:"evidence,omitempty"` + Forbidden []string `json:"forbidden"` +} +type Candidate struct { + TaskID, Role, Path string + WriteScope []string + Ready bool + Blockers []string +} +type Finding struct { + Kind string `json:"kind"` + DispatchID string `json:"dispatch_id"` + Detail string `json:"detail"` + Owner string `json:"owner"` +} +type Transcript struct { + DispatchID string `json:"dispatch_id"` + StartedAt string `json:"started_at"` + FinishedAt string `json:"finished_at"` + ExitCode int `json:"exit_code"` + OutputHash string `json:"output_hash"` +} +type WaveResult struct { + ID string `json:"id"` + Transcript *Transcript `json:"transcript,omitempty"` + Error string `json:"error,omitempty"` +} +type Recommendation struct { + Kind string `json:"kind"` + Detail string `json:"detail"` + RequiresConfirmation bool `json:"requires_confirmation"` +} +type Config struct { + Version int `json:"version"` + Enabled bool `json:"enabled"` + Harnesses []string `json:"harnesses"` + MaxParallel int `json:"max_parallel"` + TranscriptRetention int `json:"transcript_retention"` +} + +func dir(root, work string) string { + return filepath.Join(root, ".product", "workspaces", work, "dispatches") +} +func configPath(root string) string { return filepath.Join(root, ".product", "dispatch.json") } +func LoadConfig(root string) (Config, error) { + data, err := os.ReadFile(configPath(root)) + if os.IsNotExist(err) { + return Config{Version: 1, Enabled: false, MaxParallel: 1, TranscriptRetention: 100}, nil + } + if err != nil { + return Config{}, err + } + var c Config + if err := json.Unmarshal(data, &c); err != nil { + return Config{}, err + } + if c.MaxParallel < 1 { + c.MaxParallel = 1 + } + return c, nil +} +func SaveConfig(root string, c Config) error { + if c.Version == 0 { + c.Version = 1 + } + if c.MaxParallel < 1 { + c.MaxParallel = 1 + } + data, err := json.MarshalIndent(c, "", " ") + if err != nil { + return err + } + if err = os.MkdirAll(filepath.Dir(configPath(root)), 0755); err != nil { + return err + } + return os.WriteFile(configPath(root), append(data, '\n'), 0644) +} +func Plan(root, graph string) ([]Candidate, error) { + nodes, err := workflow.ReadyUnclaimed(root, graph) + if err != nil { + return nil, err + } + items := make([]Candidate, 0, len(nodes)) + for _, node := range nodes { + readiness, err := workflow.CheckTaskReadiness(root, graph, node.ID) + item := Candidate{TaskID: node.ID, Role: "code-runner", Path: node.Path, WriteScope: node.WriteScope, Ready: err == nil && readiness.Ready} + if err != nil { + item.Blockers = []string{err.Error()} + } else { + for _, check := range readiness.Checks { + if check.Status == "block" { + item.Blockers = append(item.Blockers, check.Detail) + } + } + } + items = append(items, item) + } + sort.Slice(items, func(i, j int) bool { return items[i].TaskID < items[j].TaskID }) + return items, nil +} +func Assign(root, work, graph, task, role, agent string) (Envelope, error) { + if role == "" { + role = "code-runner" + } + if role != "code-runner" { + return Envelope{}, errors.New("only code-runner assignments are enabled; independent review dispatch remains read-only") + } + readiness, err := workflow.CheckTaskReadiness(root, graph, task) + if err != nil { + return Envelope{}, err + } + if !readiness.Ready { + return Envelope{}, errors.New("task is not ready for dispatch") + } + if _, err = workflow.ClaimLease(root, graph, task, agent, 30*time.Minute); err != nil { + return Envelope{}, err + } + taskPath := "" + for _, node := range mustNodes(graph) { + if node.ID == task { + taskPath = filepath.Join(filepath.Dir(graph), filepath.FromSlash(node.Path)) + break + } + } + data, err := os.ReadFile(taskPath) + if err != nil { + _ = workflow.ReleaseLease(root, task, agent) + return Envelope{}, err + } + sum := sha256.Sum256(data) + if err = os.MkdirAll(dir(root, work), 0755); err != nil { + return Envelope{}, err + } + id := fmt.Sprintf("DISPATCH-%d", time.Now().UTC().UnixNano()) + e := Envelope{Version: 1, ID: id, WorkspaceID: work, TaskID: task, UnitKind: "task", UnitPath: filepath.ToSlash(taskPath), Role: role, Agent: agent, Graph: filepath.ToSlash(graph), TaskPath: filepath.ToSlash(taskPath), InputHash: hex.EncodeToString(sum[:]), RequiredReading: []string{filepath.ToSlash(taskPath), filepath.ToSlash(graph)}, ExpectedEvidence: []string{"diff hash", "test log"}, Status: "assigned", CreatedAt: time.Now().UTC().Format(time.RFC3339), Forbidden: []string{"approval", "commit", "push", "merge", "release", "review-resolution"}} + for _, node := range mustNodes(graph) { + if node.ID == task { + e.WriteScope = node.WriteScope + } + } + return e, write(filepath.Join(dir(root, work), id+".json"), e) +} +func Return(root, work, id, agent, summary, diffHash string, evidence []string) (Envelope, error) { + var e Envelope + path := filepath.Join(dir(root, work), id+".json") + if err := read(path, &e); err != nil { + return e, err + } + if e.Agent != agent || e.Status != "assigned" { + return e, errors.New("dispatch is not assigned to this agent") + } + if strings.TrimSpace(summary) == "" || strings.TrimSpace(diffHash) == "" || len(evidence) == 0 { + return e, errors.New("return requires summary, diff hash, and evidence") + } + e.Status = "returned" + e.Summary = summary + e.Evidence = evidence + e.DiffHash = diffHash + e.ReturnedAt = time.Now().UTC().Format(time.RFC3339) + if err := write(path, e); err != nil { + return e, err + } + if e.TaskID == "" { + return e, nil + } + return e, workflow.ReleaseLease(root, e.TaskID, agent) +} +func ReturnImport(root, work, id, agent, summary string, review sourceimport.ChunkReview) (Envelope, error) { + var e Envelope + path := filepath.Join(dir(root, work), id+".json") + if err := read(path, &e); err != nil { + return e, err + } + if e.UnitKind != "import-chunk" || e.Agent != agent || e.Status != "assigned" { + return e, errors.New("dispatch is not an assigned import chunk for this agent") + } + if strings.TrimSpace(summary) == "" { + return e, errors.New("import return requires summary") + } + if err := sourceimport.RecordChunkReview(root, e.ImportRun, e.ImportChunk, agent, review); err != nil { + return e, err + } + e.Status = "returned" + e.Summary = summary + e.Evidence = []string{"structured source evidence"} + e.ReturnedAt = time.Now().UTC().Format(time.RFC3339) + return e, write(path, e) +} + +// AssignReview creates an independent read-only review envelope for the exact +// diff returned by a code-runner. It never claims write ownership. +func AssignReview(root, work, parentID, role, agent string) (Envelope, error) { + if role != "qa" && role != "code-review" && role != "security-review" { + return Envelope{}, errors.New("review role must be qa, code-review, or security-review") + } + var parent Envelope + if err := read(filepath.Join(dir(root, work), parentID+".json"), &parent); err != nil { + return Envelope{}, err + } + if parent.Role != "code-runner" || parent.Status != "returned" || parent.DiffHash == "" { + return Envelope{}, errors.New("review requires a returned code-runner dispatch with diff hash") + } + id := fmt.Sprintf("DISPATCH-%d", time.Now().UTC().UnixNano()) + e := parent + e.ID = id + e.Role = role + e.Agent = agent + e.ParentID = parent.ID + e.Status = "assigned" + e.CreatedAt = time.Now().UTC().Format(time.RFC3339) + e.ReturnedAt = "" + e.Summary = "" + e.Evidence = nil + e.WriteScope = nil + e.Forbidden = []string{"approval", "code-change", "commit", "push", "merge", "release", "review-resolution"} + e.ExpectedEvidence = []string{"review verdict", "findings", "diff hash: " + parent.DiffHash} + return e, write(filepath.Join(dir(root, work), id+".json"), e) +} + +// AssignResearch creates a read-only Technical Discovery assignment. Results +// are proposals with cited evidence, never decisions or Engineering Proposals. +func AssignResearch(root, work, unitPath, agent string) (Envelope, error) { + if strings.TrimSpace(unitPath) == "" || strings.TrimSpace(agent) == "" { + return Envelope{}, errors.New("research dispatch requires unit path and agent") + } + data, err := os.ReadFile(unitPath) + if err != nil { + return Envelope{}, err + } + sum := sha256.Sum256(data) + if err := os.MkdirAll(dir(root, work), 0755); err != nil { + return Envelope{}, err + } + id := fmt.Sprintf("DISPATCH-%d", time.Now().UTC().UnixNano()) + e := Envelope{Version: 1, ID: id, WorkspaceID: work, UnitKind: "technical-research", UnitPath: filepath.ToSlash(unitPath), Role: "technical-discovery", Agent: agent, InputHash: hex.EncodeToString(sum[:]), RequiredReading: []string{filepath.ToSlash(unitPath)}, ExpectedEvidence: []string{"sources", "options", "uncertainties"}, Status: "assigned", CreatedAt: time.Now().UTC().Format(time.RFC3339), Forbidden: []string{"decision", "approval", "engineering-proposal", "commit", "push", "merge", "release"}} + return e, write(filepath.Join(dir(root, work), id+".json"), e) +} + +// AssignThreatModel is a read-only assignment for one named trust boundary. +func AssignThreatModel(root, work, boundaryPath, agent string) (Envelope, error) { + e, err := AssignResearch(root, work, boundaryPath, agent) + if err != nil { + return e, err + } + e.UnitKind = "security-boundary" + e.Role = "threat-modeler" + e.ExpectedEvidence = []string{"threats", "controls", "residual risks"} + e.Forbidden = []string{"risk-acceptance", "approval", "release", "commit", "push", "merge"} + return e, write(filepath.Join(dir(root, work), e.ID+".json"), e) +} + +// AssignImportChunk claims one existing scalable-import chunk. It cannot record +// review evidence or materialize mappings; those remain explicit import steps. +func AssignImportChunk(root, work, run, chunkID, agent string) (Envelope, error) { + chunk, err := sourceimport.Resume(root, run, chunkID, agent) + if err != nil { + return Envelope{}, err + } + id := fmt.Sprintf("DISPATCH-%d", time.Now().UTC().UnixNano()) + unit := filepath.Join(root, "knowledge", "imports", "runs", run, "chunks", chunk.ID+".json") + data, err := os.ReadFile(unit) + if err != nil { + return Envelope{}, err + } + sum := sha256.Sum256(data) + if err := os.MkdirAll(dir(root, work), 0755); err != nil { + return Envelope{}, err + } + e := Envelope{Version: 1, ID: id, WorkspaceID: work, UnitKind: "import-chunk", UnitPath: filepath.ToSlash(unit), ImportRun: run, ImportChunk: chunk.ID, Role: "artifact-importer", Agent: agent, InputHash: hex.EncodeToString(sum[:]), RequiredReading: []string{filepath.ToSlash(unit)}, ExpectedEvidence: []string{"evidence per source", "gaps"}, Status: "assigned", CreatedAt: time.Now().UTC().Format(time.RFC3339), Forbidden: []string{"materialize", "approval", "mapping-selection", "commit", "push", "merge", "release"}} + return e, write(filepath.Join(dir(root, work), id+".json"), e) +} +func Observe(root, work string) ([]Envelope, error) { + entries, err := os.ReadDir(dir(root, work)) + if os.IsNotExist(err) { + return []Envelope{}, nil + } + if err != nil { + return nil, err + } + var out []Envelope + for _, entry := range entries { + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + var e Envelope + if err := read(filepath.Join(dir(root, work), entry.Name()), &e); err != nil { + return nil, err + } + out = append(out, e) + } + sort.Slice(out, func(i, j int) bool { return out[i].ID < out[j].ID }) + return out, nil +} + +// Reconcile is read-only and never changes an envelope, task, or lease. +func Reconcile(root, work string) ([]Finding, error) { + xs, err := Observe(root, work) + if err != nil { + return nil, err + } + byID := map[string]Envelope{} + for _, x := range xs { + byID[x.ID] = x + } + var out []Finding + for _, x := range xs { + if x.Role != "code-runner" && x.ParentID != "" { + p, ok := byID[x.ParentID] + if !ok { + out = append(out, Finding{"orphaned-review", x.ID, "parent dispatch missing", "delivery-orchestrator"}) + } else if p.DiffHash == "" || p.DiffHash != x.DiffHash { + out = append(out, Finding{"review-diff-mismatch", x.ID, "review does not match parent diff", "code-review"}) + } + } + if x.Status == "assigned" && x.Role != "code-runner" && x.DiffHash == "" { + out = append(out, Finding{"review-missing-diff", x.ID, "independent review lacks diff hash", "delivery-orchestrator"}) + } + } + sort.Slice(out, func(i, j int) bool { return out[i].DispatchID < out[j].DispatchID }) + return out, nil +} + +// Run executes only an assigned code-runner envelope after explicit enablement. +// It deliberately refuses Git delivery commands and records a transcript. +func Run(root, work, id string, enabled bool, command string, args []string) (Transcript, error) { + if !enabled { + return Transcript{}, errors.New("supervised dispatch execution is disabled") + } + cfg, err := LoadConfig(root) + if err != nil { + return Transcript{}, err + } + if !cfg.Enabled { + return Transcript{}, errors.New("dispatch capability is disabled for this product") + } + allowed := false + for _, h := range cfg.Harnesses { + if strings.EqualFold(filepath.Base(command), filepath.Base(h)) { + allowed = true + } + } + if !allowed { + return Transcript{}, errors.New("command is not an enabled dispatch harness") + } + var e Envelope + path := filepath.Join(dir(root, work), id+".json") + if err := read(path, &e); err != nil { + return Transcript{}, err + } + if e.Role != "code-runner" || e.Status != "assigned" { + return Transcript{}, errors.New("only assigned code-runner dispatches can run") + } + if strings.EqualFold(filepath.Base(command), "git") { + return Transcript{}, errors.New("dispatch runner cannot invoke git delivery commands") + } + started := time.Now().UTC() + cmd := exec.Command(command, args...) + cmd.Dir = filepath.Dir(root) + output, err := cmd.CombinedOutput() + if err == nil { + if scopeErr := validateWriteScope(filepath.Dir(root), e.WriteScope); scopeErr != nil { + err = scopeErr + } + } + exit := 0 + if x, ok := err.(*exec.ExitError); ok { + exit = x.ExitCode() + } + sum := sha256.Sum256(output) + t := Transcript{DispatchID: id, StartedAt: started.Format(time.RFC3339), FinishedAt: time.Now().UTC().Format(time.RFC3339), ExitCode: exit, OutputHash: hex.EncodeToString(sum[:])} + data, _ := json.MarshalIndent(t, "", " ") + tdir := filepath.Join(dir(root, work), "transcripts") + if mk := os.MkdirAll(tdir, 0755); mk != nil { + return t, mk + } + if writeErr := os.WriteFile(filepath.Join(tdir, id+".json"), append(data, '\n'), 0644); writeErr != nil { + return t, writeErr + } + if cfg.TranscriptRetention > 0 { + _ = retainTranscripts(tdir, cfg.TranscriptRetention) + } + return t, err +} +func validateWriteScope(repo string, scopes []string) error { + if len(scopes) == 0 { + return errors.New("dispatch task has no write scope") + } + out, err := exec.Command("git", "-C", repo, "diff", "--name-only").CombinedOutput() + if err != nil { + return err + } + for _, raw := range strings.Fields(string(out)) { + ok := false + for _, scope := range scopes { + scope = filepath.ToSlash(strings.TrimSuffix(scope, "/")) + if raw == scope || strings.HasPrefix(raw, scope+"/") { + ok = true + } + } + if !ok { + return fmt.Errorf("working-tree path %s escapes dispatch write scope", raw) + } + } + return nil +} +func retainTranscripts(path string, max int) error { + entries, err := os.ReadDir(path) + if err != nil { + return err + } + var names []string + for _, e := range entries { + if !e.IsDir() && filepath.Ext(e.Name()) == ".json" { + names = append(names, e.Name()) + } + } + sort.Strings(names) + for len(names) > max { + if err := os.Remove(filepath.Join(path, names[0])); err != nil { + return err + } + names = names[1:] + } + return nil +} + +// RunWave runs already-assigned envelopes with bounded local concurrency. +func RunWave(root, work string, ids []string, max int, enabled bool, command string, args []string) []WaveResult { + if max < 1 { + max = 1 + } + sem := make(chan struct{}, max) + out := make(chan WaveResult, len(ids)) + for _, id := range ids { + id := id + go func() { + sem <- struct{}{} + defer func() { <-sem }() + t, e := Run(root, work, id, enabled, command, args) + r := WaveResult{ID: id, Transcript: &t} + if e != nil { + r.Error = e.Error() + } + out <- r + }() + } + results := make([]WaveResult, 0, len(ids)) + for range ids { + results = append(results, <-out) + } + sort.Slice(results, func(i, j int) bool { return results[i].ID < results[j].ID }) + return results +} + +// WaveIDs derives assigned code-runner envelopes from the persisted scheduler wave. +func WaveIDs(root, work, wave string) ([]string, error) { + data, err := os.ReadFile(filepath.Join(root, ".product", "scheduler", "waves", work+".json")) + if err != nil { + return nil, err + } + var schedule workflow.Schedule + if err := json.Unmarshal(data, &schedule); err != nil { + return nil, err + } + var tasks map[string]bool = map[string]bool{} + for _, w := range schedule.Waves { + if w.ID == wave { + for _, t := range w.Tasks { + tasks[t] = true + } + } + } + if len(tasks) == 0 { + return nil, errors.New("scheduled wave not found") + } + xs, err := Observe(root, work) + if err != nil { + return nil, err + } + var ids []string + for _, e := range xs { + if e.Role == "code-runner" && e.Status == "assigned" && tasks[e.TaskID] { + ids = append(ids, e.ID) + } + } + sort.Strings(ids) + return ids, nil +} + +// Recommend is advisory only; it never assigns, reprioritizes, or executes. +func Recommend(root, work string, max int) ([]Recommendation, error) { + xs, err := Observe(root, work) + if err != nil { + return nil, err + } + active := 0 + var out []Recommendation + for _, x := range xs { + if x.Status == "assigned" { + active++ + } + if x.Status == "returned" && x.Role == "code-runner" { + out = append(out, Recommendation{"review-ready", "assign independent QA and Code Review for " + x.ID, true}) + } + } + if max > 0 && active >= max { + out = append(out, Recommendation{"capacity", "active dispatches reach configured capacity", true}) + } + if active == 0 { + out = append(out, Recommendation{"idle", "no active dispatches; inspect dispatch plan", true}) + } + return out, nil +} +func mustNodes(graph string) []workflow.Node { + var raw struct { + Nodes []workflow.Node `json:"nodes"` + } + data, _ := os.ReadFile(graph) + _ = json.Unmarshal(data, &raw) + return raw.Nodes +} +func write(path string, v any) error { + data, err := json.MarshalIndent(v, "", " ") + if err != nil { + return err + } + return os.WriteFile(path, append(data, '\n'), 0644) +} +func read(path string, v any) error { + data, err := os.ReadFile(path) + if err != nil { + return err + } + return json.Unmarshal(data, v) +} diff --git a/internal/dispatch/dispatch_test.go b/internal/dispatch/dispatch_test.go new file mode 100644 index 0000000..8c9a1c4 --- /dev/null +++ b/internal/dispatch/dispatch_test.go @@ -0,0 +1,44 @@ +package dispatch + +import ( + "os" + "path/filepath" + "testing" +) + +func TestReviewEnvelopePinsParentDiffAndCannotRun(t *testing.T) { + root := t.TempDir() + work := "WORK-001" + if err := os.MkdirAll(dir(root, work), 0755); err != nil { + t.Fatal(err) + } + parent := Envelope{Version: 1, ID: "DISPATCH-1", WorkspaceID: work, TaskID: "TK-1", Role: "code-runner", Agent: "runner", Status: "returned", DiffHash: "abc", InputHash: "input"} + if err := write(filepath.Join(dir(root, work), parent.ID+".json"), parent); err != nil { + t.Fatal(err) + } + review, err := AssignReview(root, work, parent.ID, "qa", "qa-1") + if err != nil { + t.Fatal(err) + } + if review.DiffHash != "abc" || review.ParentID != parent.ID || len(review.WriteScope) != 0 { + t.Fatalf("review=%+v", review) + } + if _, err := Run(root, work, review.ID, false, "echo", nil); err == nil { + t.Fatal("review run accepted") + } +} + +func TestReconcileReportsOrphanReview(t *testing.T) { + root := t.TempDir() + work := "WORK-001" + if err := os.MkdirAll(dir(root, work), 0755); err != nil { + t.Fatal(err) + } + if err := write(filepath.Join(dir(root, work), "DISPATCH-2.json"), Envelope{ID: "DISPATCH-2", WorkspaceID: work, Role: "code-review", ParentID: "missing", DiffHash: "abc", Status: "assigned"}); err != nil { + t.Fatal(err) + } + items, err := Reconcile(root, work) + if err != nil || len(items) != 1 || items[0].Kind != "orphaned-review" { + t.Fatalf("items=%+v err=%v", items, err) + } +} diff --git a/internal/install/install.go b/internal/install/install.go index 1c04622..79247c7 100644 --- a/internal/install/install.go +++ b/internal/install/install.go @@ -31,6 +31,7 @@ type Options struct { Agents []Agent StartingPoint string Sources []string + ImportOptions sourceimport.CreateOptions Force bool } type Result struct { @@ -141,7 +142,7 @@ func Init(opts Options) (Result, error) { for _, action := range plan.Actions { switch action { case "create-import-run": - runID, err := sourceimport.CreateRun(filepath.Join(stagingRoot, "product"), opts.Sources) + runID, err := sourceimport.CreateScalableRun(filepath.Join(stagingRoot, "product"), opts.Sources, opts.ImportOptions) if err != nil { return Result{}, err } diff --git a/internal/sourceimport/import.go b/internal/sourceimport/import.go index c92d586..4adcec5 100644 --- a/internal/sourceimport/import.go +++ b/internal/sourceimport/import.go @@ -178,17 +178,14 @@ func Materialize(productRoot, runID, approvedBy string) ([]string, error) { return nil, fmt.Errorf("traceability import_id %q does not match %q", traceability.ImportID, runID) } } - invData, err := os.ReadFile(filepath.Join(runRoot, "inventory.json")) + known, scalable, err := materializableSources(productRoot, runID) if err != nil { return nil, err } - var inventory Inventory - if err := json.Unmarshal(trimBOM(invData), &inventory); err != nil { - return nil, fmt.Errorf("parse inventory: %w", err) - } - knownSources := map[string]bool{} - for _, source := range inventory.Sources { - knownSources[source.Path] = true + if scalable { + if err := requireReviewedChunks(productRoot, runID); err != nil { + return nil, err + } } var selected []Mapping seen := map[string]bool{} @@ -200,7 +197,7 @@ func Materialize(productRoot, runID, approvedBy string) ([]string, error) { return nil, fmt.Errorf("mapping %s: %w", mapping.ID, err) } for _, source := range mapping.SourceDocuments { - if !knownSources[source] { + if !known[source] { return nil, fmt.Errorf("mapping %s references uninventoried source %s", mapping.ID, source) } } diff --git a/internal/sourceimport/materializable.go b/internal/sourceimport/materializable.go new file mode 100644 index 0000000..94905ed --- /dev/null +++ b/internal/sourceimport/materializable.go @@ -0,0 +1,56 @@ +package sourceimport + +import ( + "encoding/json" + "errors" + "os" + "path/filepath" +) + +func materializableSources(productRoot, runID string) (map[string]bool, bool, error) { + runRoot := filepath.Join(productRoot, "knowledge", "imports", "runs", runID) + if _, err := os.Stat(filepath.Join(runRoot, "inventory", "index.json")); err == nil { + sources, err := scalableSources(productRoot, runID) + if err != nil { + return nil, true, err + } + known := map[string]bool{} + for _, source := range sources { + known[source.Path] = true + } + return known, true, nil + } + data, err := os.ReadFile(filepath.Join(runRoot, "inventory.json")) + if err != nil { + return nil, false, err + } + var inventory Inventory + if err := json.Unmarshal(trimBOM(data), &inventory); err != nil { + return nil, false, err + } + known := map[string]bool{} + for _, source := range inventory.Sources { + known[source.Path] = true + } + return known, false, nil +} + +func requireReviewedChunks(productRoot, runID string) error { + entries, err := os.ReadDir(filepath.Join(productRoot, "knowledge", "imports", "runs", runID, "chunks")) + if err != nil { + return err + } + for _, entry := range entries { + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + var chunk Chunk + if err := readJSONFile(filepath.Join(productRoot, "knowledge", "imports", "runs", runID, "chunks", entry.Name()), &chunk); err != nil { + return err + } + if chunk.Status != "reviewed" && chunk.Status != "excluded" { + return errors.New("all scalable import chunks must be reviewed or excluded before materialization") + } + } + return nil +} diff --git a/internal/sourceimport/scalable.go b/internal/sourceimport/scalable.go new file mode 100644 index 0000000..5ea6f51 --- /dev/null +++ b/internal/sourceimport/scalable.go @@ -0,0 +1,465 @@ +package sourceimport + +import ( + "bufio" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "sort" + "strings" + "time" +) + +const scalableSchemaVersion = 2 + +type CreateOptions struct { + Include []string `json:"include,omitempty"` + Exclude []string `json:"exclude,omitempty"` + MaxFiles int `json:"max_files"` + MaxTotalBytes int64 `json:"max_total_bytes"` + MaxFileBytes int64 `json:"max_file_bytes"` + ChunkSize int `json:"chunk_size"` + BinaryPolicy string `json:"binary_policy"` +} + +type ScalableSource struct { + ID string `json:"id"` + OriginalPath string `json:"original_path"` + Path string `json:"path"` + Format string `json:"format"` + Size int64 `json:"size"` + SHA256 string `json:"sha256"` + Status string `json:"status"` + Reason string `json:"reason,omitempty"` +} +type InventoryIndex struct { + SchemaVersion int `json:"schema_version"` + ImportID string `json:"import_id"` + Pages int `json:"pages"` + Sources int `json:"sources"` + TotalBytes int64 `json:"total_bytes"` + ConfigHash string `json:"config_hash"` +} +type Chunk struct { + SchemaVersion int `json:"schema_version"` + ID string `json:"id"` + SourceIDs []string `json:"source_ids"` + Status string `json:"status"` + Agent string `json:"agent,omitempty"` + LeaseExpires string `json:"lease_expires,omitempty"` +} +type ScalableStatus struct { + ImportID string `json:"import_id"` + Sources, Chunks, Queued, Reviewing, Reviewed, Blocked, Excluded int +} +type ChunkReview struct { + SourceEvidence map[string][]Evidence `json:"source_evidence"` + Gaps map[string][]string `json:"gaps,omitempty"` +} + +func DefaultCreateOptions() CreateOptions { + return CreateOptions{MaxFiles: 500, MaxTotalBytes: 200 << 20, MaxFileBytes: 10 << 20, ChunkSize: 25, BinaryPolicy: "inventory_only", Exclude: []string{".git/**", "node_modules/**", "**/.git/**", "**/node_modules/**"}} +} +func (o CreateOptions) normalized() (CreateOptions, error) { + d := DefaultCreateOptions() + if o.MaxFiles == 0 { + o.MaxFiles = d.MaxFiles + } + if o.MaxTotalBytes == 0 { + o.MaxTotalBytes = d.MaxTotalBytes + } + if o.MaxFileBytes == 0 { + o.MaxFileBytes = d.MaxFileBytes + } + if o.ChunkSize == 0 { + o.ChunkSize = d.ChunkSize + } + if o.BinaryPolicy == "" { + o.BinaryPolicy = d.BinaryPolicy + } + if len(o.Exclude) == 0 { + o.Exclude = d.Exclude + } + if o.MaxFiles < 1 || o.MaxTotalBytes < 1 || o.MaxFileBytes < 1 || o.ChunkSize < 1 { + return o, errors.New("import limits and chunk size must be positive") + } + if o.BinaryPolicy != "inventory_only" && o.BinaryPolicy != "reject" { + return o, errors.New("binary policy must be inventory_only or reject") + } + return o, nil +} + +// CreateScalableRun creates a v2 analysis-only run. It checks budgets before +// copying, writes paged JSONL inventory, and never creates product artifacts. +func CreateScalableRun(productRoot string, inputs []string, options CreateOptions) (string, error) { + options, err := options.normalized() + if err != nil { + return "", err + } + files, err := expand(inputs) + if err != nil { + return "", err + } + if len(files) == 0 { + return "", errors.New("no regular source files found") + } + var chosen []string + var total int64 + for _, file := range files { + rel := filepath.ToSlash(file) + if matchesAny(rel, options.Exclude) || (len(options.Include) > 0 && !matchesAny(rel, options.Include)) { + continue + } + info, err := os.Stat(file) + if err != nil { + return "", err + } + if info.Size() > options.MaxFileBytes { + return "", fmt.Errorf("source exceeds max file bytes: %s", file) + } + if len(chosen)+1 > options.MaxFiles || total+info.Size() > options.MaxTotalBytes { + return "", fmt.Errorf("import budget exceeded before copying %s", file) + } + chosen, total = append(chosen, file), total+info.Size() + } + if len(chosen) == 0 { + return "", errors.New("no sources matched import filters") + } + base := filepath.Join(productRoot, "knowledge", "imports") + runID, err := nextRun(filepath.Join(base, "runs")) + if err != nil { + return "", err + } + runRoot := filepath.Join(base, "runs", runID) + sourceRoot := filepath.Join(base, "sources", runID) + published := false + defer func() { + if !published { + _ = os.RemoveAll(runRoot) + _ = os.RemoveAll(sourceRoot) + } + }() + for _, dir := range []string{runRoot, sourceRoot, filepath.Join(runRoot, "inventory", "pages"), filepath.Join(runRoot, "chunks")} { + if err := os.MkdirAll(dir, 0755); err != nil { + return "", err + } + } + if err := writeJSON(filepath.Join(runRoot, "import-config.json"), options); err != nil { + return "", err + } + pageSize := options.ChunkSize + var page []ScalableSource + var sources []ScalableSource + for index, input := range chosen { + if isBinary(input) && options.BinaryPolicy == "reject" { + return "", fmt.Errorf("binary source rejected: %s", input) + } + name := fmt.Sprintf("%06d-%s", index+1, filepath.Base(input)) + dest := filepath.Join(sourceRoot, name) + if err := copyFile(input, dest); err != nil { + return "", err + } + info, _ := os.Stat(dest) + hash, err := fileHash(dest) + if err != nil { + return "", err + } + rel, _ := filepath.Rel(productRoot, dest) + source := ScalableSource{ID: fmt.Sprintf("SRC-%06d", index+1), OriginalPath: input, Path: filepath.ToSlash(rel), Format: strings.TrimPrefix(strings.ToLower(filepath.Ext(input)), "."), Size: info.Size(), SHA256: hash, Status: "queued"} + if isBinary(input) { + source.Status, source.Reason = "excluded", "binary inventory only" + } + sources, page = append(sources, source), append(page, source) + if len(page) == pageSize || index == len(chosen)-1 { + if err := writeJSONL(filepath.Join(runRoot, "inventory", "pages", fmt.Sprintf("PAGE-%04d.jsonl", (index/pageSize)+1)), page); err != nil { + return "", err + } + page = nil + } + } + configData, _ := json.Marshal(options) + sum := sha256.Sum256(configData) + index := InventoryIndex{SchemaVersion: scalableSchemaVersion, ImportID: runID, Pages: (len(sources) + pageSize - 1) / pageSize, Sources: len(sources), TotalBytes: total, ConfigHash: hex.EncodeToString(sum[:])} + if err := writeJSON(filepath.Join(runRoot, "inventory", "index.json"), index); err != nil { + return "", err + } + // Compatibility summary for existing bootstrap/readers. The paged index is + // authoritative for scalable operations; this summary is never used to + // schedule or resume chunks. + legacy := Inventory{SchemaVersion: scalableSchemaVersion, ImportID: runID} + for _, source := range sources { + legacy.Sources = append(legacy.Sources, Source{Path: source.Path, Format: source.Format, Size: source.Size, SHA256: source.SHA256}) + } + if err := writeJSON(filepath.Join(runRoot, "inventory.json"), legacy); err != nil { + return "", err + } + if err := writeJSON(filepath.Join(runRoot, "traceability.json"), Traceability{SchemaVersion: scalableSchemaVersion, ImportID: runID, Status: "chunked"}); err != nil { + return "", err + } + if err := os.WriteFile(filepath.Join(runRoot, "conflicts.md"), []byte("# Import Conflicts\n\nNo conflicts have been classified yet.\n"), 0644); err != nil { + return "", err + } + if err := os.WriteFile(filepath.Join(runRoot, "import-report.md"), []byte("# Import Report\n\nScalable review is organized by chunks.\n"), 0644); err != nil { + return "", err + } + for offset := 0; offset < len(sources); offset += options.ChunkSize { + end := offset + options.ChunkSize + if end > len(sources) { + end = len(sources) + } + ids := make([]string, 0, end-offset) + status := "queued" + for _, source := range sources[offset:end] { + ids = append(ids, source.ID) + if source.Status == "excluded" { + status = "excluded" + } + } + if err := writeJSON(filepath.Join(runRoot, "chunks", fmt.Sprintf("CHUNK-%04d.json", offset/options.ChunkSize+1)), Chunk{SchemaVersion: scalableSchemaVersion, ID: fmt.Sprintf("CHUNK-%04d", offset/options.ChunkSize+1), SourceIDs: ids, Status: status}); err != nil { + return "", err + } + } + if err := writeJSON(filepath.Join(runRoot, "mapping.json"), MappingFile{SchemaVersion: scalableSchemaVersion, ImportID: runID}); err != nil { + return "", err + } + if err := writeJSON(filepath.Join(runRoot, "import-plan.json"), map[string]any{"schema_version": scalableSchemaVersion, "import_id": runID, "status": "draft", "materialization_approved": false}); err != nil { + return "", err + } + published = true + return runID, nil +} + +func ImportStatus(productRoot, runID string) (ScalableStatus, error) { + runRoot := filepath.Join(productRoot, "knowledge", "imports", "runs", runID) + indexData, err := os.ReadFile(filepath.Join(runRoot, "inventory", "index.json")) + if err != nil { + return ScalableStatus{}, err + } + var index InventoryIndex + if err := json.Unmarshal(indexData, &index); err != nil { + return ScalableStatus{}, err + } + status := ScalableStatus{ImportID: runID, Sources: index.Sources} + entries, err := os.ReadDir(filepath.Join(runRoot, "chunks")) + if err != nil { + return status, err + } + for _, entry := range entries { + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + var chunk Chunk + if readJSONFile(filepath.Join(runRoot, "chunks", entry.Name()), &chunk) != nil { + continue + } + status.Chunks++ + switch chunk.Status { + case "queued": + status.Queued++ + case "reviewing": + status.Reviewing++ + case "reviewed": + status.Reviewed++ + case "blocked": + status.Blocked++ + case "excluded": + status.Excluded++ + } + } + return status, nil +} + +// Resume claims one queued/expired chunk for the named importer. It only +// persists operational ownership; the skill still owns the review content. +func Resume(productRoot, runID, chunkID, agent string) (Chunk, error) { + if strings.TrimSpace(agent) == "" { + return Chunk{}, errors.New("import resume requires agent") + } + dir := filepath.Join(productRoot, "knowledge", "imports", "runs", runID, "chunks") + entries, err := os.ReadDir(dir) + if err != nil { + return Chunk{}, err + } + for _, entry := range entries { + if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" { + continue + } + var chunk Chunk + path := filepath.Join(dir, entry.Name()) + unlock, lockErr := lockChunk(dir, strings.TrimSuffix(entry.Name(), ".json")) + if lockErr != nil { + continue + } + if err := readJSONFile(path, &chunk); err != nil { + unlock() + return Chunk{}, err + } + if chunkID != "" && chunk.ID != chunkID { + unlock() + continue + } + expired := chunk.LeaseExpires != "" && mustParseTime(chunk.LeaseExpires).Before(time.Now().UTC()) + if chunk.Status != "queued" && !(chunk.Status == "reviewing" && expired) { + unlock() + continue + } + chunk.Status, chunk.Agent, chunk.LeaseExpires = "reviewing", agent, time.Now().UTC().Add(30*time.Minute).Format(time.RFC3339) + err := writeJSON(path, chunk) + unlock() + return chunk, err + } + return Chunk{}, errors.New("no resumable import chunk") +} + +func lockChunk(dir, id string) (func(), error) { + lockDir := filepath.Join(dir, ".locks") + if err := os.MkdirAll(lockDir, 0755); err != nil { + return nil, err + } + path := filepath.Join(lockDir, id+".lock") + file, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0644) + if err != nil { + return nil, err + } + _ = file.Close() + return func() { _ = os.Remove(path) }, nil +} + +// RecordChunkReview records evidence for every non-excluded source in a leased +// chunk. It is deliberately unable to select mappings or materialize drafts. +func RecordChunkReview(productRoot, runID, chunkID, agent string, review ChunkReview) error { + if strings.TrimSpace(agent) == "" { + return errors.New("chunk review requires agent") + } + path := filepath.Join(productRoot, "knowledge", "imports", "runs", runID, "chunks", chunkID+".json") + var chunk Chunk + if err := readJSONFile(path, &chunk); err != nil { + return err + } + if chunk.Status != "reviewing" || chunk.Agent != agent || mustParseTime(chunk.LeaseExpires).Before(time.Now().UTC()) { + return errors.New("chunk is not actively leased by this importer") + } + sources, err := scalableSources(productRoot, runID) + if err != nil { + return err + } + sourceByID := map[string]ScalableSource{} + for _, source := range sources { + sourceByID[source.ID] = source + } + for _, id := range chunk.SourceIDs { + source, ok := sourceByID[id] + if !ok { + return fmt.Errorf("chunk references unknown source %s", id) + } + if source.Status == "excluded" { + continue + } + if len(review.SourceEvidence[id]) == 0 { + return fmt.Errorf("review evidence is required for source %s", id) + } + } + traceDir := filepath.Join(productRoot, "knowledge", "imports", "runs", runID, "traceability") + if err := os.MkdirAll(traceDir, 0755); err != nil { + return err + } + if err := writeJSON(filepath.Join(traceDir, chunkID+".json"), review); err != nil { + return err + } + chunk.Status, chunk.Agent, chunk.LeaseExpires = "reviewed", "", "" + return writeJSON(path, chunk) +} + +func scalableSources(productRoot, runID string) ([]ScalableSource, error) { + runRoot := filepath.Join(productRoot, "knowledge", "imports", "runs", runID) + data, err := os.ReadFile(filepath.Join(runRoot, "inventory", "index.json")) + if err != nil { + return nil, err + } + var index InventoryIndex + if err := json.Unmarshal(data, &index); err != nil { + return nil, err + } + var all []ScalableSource + for page := 1; page <= index.Pages; page++ { + file, err := os.Open(filepath.Join(runRoot, "inventory", "pages", fmt.Sprintf("PAGE-%04d.jsonl", page))) + if err != nil { + return nil, err + } + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 1024), 4<<20) + for scanner.Scan() { + var source ScalableSource + if err := json.Unmarshal(scanner.Bytes(), &source); err != nil { + file.Close() + return nil, err + } + all = append(all, source) + } + if err := scanner.Err(); err != nil { + file.Close() + return nil, err + } + file.Close() + } + return all, nil +} +func matchesAny(path string, patterns []string) bool { + path = filepath.ToSlash(path) + for _, pattern := range patterns { + pattern = filepath.ToSlash(pattern) + if ok, _ := filepath.Match(pattern, path); ok { + return true + } + if strings.HasPrefix(pattern, "**/") { + if ok, _ := filepath.Match(strings.TrimPrefix(pattern, "**/"), filepath.Base(path)); ok { + return true + } + } + if strings.HasSuffix(pattern, "/**") && strings.Contains(path, strings.TrimPrefix(strings.TrimSuffix(pattern, "/**"), "**/")) { + return true + } + } + return false +} +func isBinary(path string) bool { + switch strings.ToLower(filepath.Ext(path)) { + case ".md", ".txt", ".json", ".yaml", ".yml", ".csv", ".html", ".xml": + return false + } + return true +} +func writeJSONL(path string, items []ScalableSource) error { + file, err := os.Create(path) + if err != nil { + return err + } + defer file.Close() + writer := bufio.NewWriter(file) + for _, item := range items { + data, _ := json.Marshal(item) + if _, err := writer.Write(append(data, '\n')); err != nil { + return err + } + } + return writer.Flush() +} +func readJSONFile(path string, value any) error { + data, err := os.ReadFile(path) + if err != nil { + return err + } + return json.Unmarshal(data, value) +} +func mustParseTime(value string) time.Time { + parsed, _ := time.Parse(time.RFC3339, value) + return parsed +} + +var _ = io.EOF +var _ = sort.Strings diff --git a/internal/sourceimport/scalable_test.go b/internal/sourceimport/scalable_test.go new file mode 100644 index 0000000..fbe0016 --- /dev/null +++ b/internal/sourceimport/scalable_test.go @@ -0,0 +1,115 @@ +package sourceimport + +import ( + "os" + "path/filepath" + "testing" +) + +func TestScalableRunFiltersBudgetsPagesAndResumes(t *testing.T) { + root, source := t.TempDir(), t.TempDir() + for _, name := range []string{"a.md", "b.md", "c.md", "skip.bin"} { + if err := os.WriteFile(filepath.Join(source, name), []byte("content"), 0644); err != nil { + t.Fatal(err) + } + } + run, err := CreateScalableRun(root, []string{source}, CreateOptions{Include: []string{"*.md", "**/*.md"}, MaxFiles: 3, MaxTotalBytes: 100, MaxFileBytes: 20, ChunkSize: 2}) + if err != nil { + t.Fatal(err) + } + status, err := ImportStatus(root, run) + if err != nil { + t.Fatal(err) + } + if status.Sources != 3 || status.Chunks != 2 || status.Queued != 2 { + t.Fatalf("status=%+v", status) + } + chunk, err := Resume(root, run, "", "importer") + if err != nil || chunk.Status != "reviewing" { + t.Fatalf("chunk=%+v err=%v", chunk, err) + } + if _, err := Resume(root, run, chunk.ID, "other"); err == nil { + t.Fatal("active chunk was claimed twice") + } +} + +func TestScalableRunRejectsBudgetBeforeCopy(t *testing.T) { + root, source := t.TempDir(), t.TempDir() + file := filepath.Join(source, "large.md") + if err := os.WriteFile(file, []byte("too large"), 0644); err != nil { + t.Fatal(err) + } + if _, err := CreateScalableRun(root, []string{file}, CreateOptions{MaxFiles: 1, MaxTotalBytes: 1, MaxFileBytes: 20, ChunkSize: 1}); err == nil { + t.Fatal("budget overflow accepted") + } + entries, _ := os.ReadDir(filepath.Join(root, "knowledge", "imports", "sources")) + if len(entries) != 0 { + t.Fatal("source copied after failed budget") + } +} + +func TestScalableRunRejectsBinaryBeforeCopyAndHonorsDefaultExcludes(t *testing.T) { + root, source := t.TempDir(), t.TempDir() + dependency := filepath.Join(source, "node_modules") + if err := os.MkdirAll(dependency, 0755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dependency, "ignored.md"), []byte("ignored"), 0644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(source, "bad.pdf"), []byte("binary"), 0644); err != nil { + t.Fatal(err) + } + if _, err := CreateScalableRun(root, []string{source}, CreateOptions{MaxFiles: 5, MaxTotalBytes: 100, MaxFileBytes: 100, ChunkSize: 1, BinaryPolicy: "reject"}); err == nil { + t.Fatal("binary accepted") + } + if _, err := os.Stat(filepath.Join(root, "knowledge", "imports", "sources", "IMPORT-001")); !os.IsNotExist(err) { + t.Fatal("partial sources preserved after rejected binary") + } +} + +func TestScalableReviewRequiresEvidenceAndGuardsMaterialization(t *testing.T) { + root, source := t.TempDir(), t.TempDir() + file := filepath.Join(source, "a.md") + if err := os.WriteFile(file, []byte("content"), 0644); err != nil { + t.Fatal(err) + } + run, err := CreateScalableRun(root, []string{file}, CreateOptions{MaxFiles: 1, MaxTotalBytes: 100, MaxFileBytes: 100, ChunkSize: 1}) + if err != nil { + t.Fatal(err) + } + chunk, err := Resume(root, run, "CHUNK-0001", "agent") + if err != nil { + t.Fatal(err) + } + if err := RecordChunkReview(root, run, chunk.ID, "agent", ChunkReview{}); err == nil { + t.Fatal("review without evidence accepted") + } + if err := RecordChunkReview(root, run, chunk.ID, "agent", ChunkReview{SourceEvidence: map[string][]Evidence{"SRC-000001": {{Locator: "line 1", Claim: "content"}}}}); err != nil { + t.Fatal(err) + } + if err := requireReviewedChunks(root, run); err != nil { + t.Fatal(err) + } +} + +func TestScalableResumeSkipsLockedChunk(t *testing.T) { + root, source := t.TempDir(), t.TempDir() + file := filepath.Join(source, "a.md") + if err := os.WriteFile(file, []byte("content"), 0644); err != nil { + t.Fatal(err) + } + run, err := CreateScalableRun(root, []string{file}, CreateOptions{MaxFiles: 1, MaxTotalBytes: 100, MaxFileBytes: 100, ChunkSize: 1}) + if err != nil { + t.Fatal(err) + } + dir := filepath.Join(root, "knowledge", "imports", "runs", run, "chunks") + unlock, err := lockChunk(dir, "CHUNK-0001") + if err != nil { + t.Fatal(err) + } + defer unlock() + if _, err := Resume(root, run, "CHUNK-0001", "agent"); err == nil { + t.Fatal("locked chunk claimed") + } +} diff --git a/starter/.spec-framework/README.md b/starter/.spec-framework/README.md index c903432..f4360a8 100644 --- a/starter/.spec-framework/README.md +++ b/starter/.spec-framework/README.md @@ -13,7 +13,6 @@ It explains how to do Specification Driven Development. It is not product scope. | `FRAMEWORK.md` | Method contract copied from the framework release. | | `AGENTS.framework.md` | Agent instructions for using the installed framework without mixing product scope into `.spec-framework/`. | | `manifest.json` | Installed framework version and asset map. | -| `decisions/` | Framework Decision Records copied from the framework release. | | `skills/` | Operational agent skills. | | `templates/` | Reusable artifact templates. | | `validators/` | Mechanical validation gates. | @@ -25,4 +24,4 @@ Product artifacts live in `../product/`. Product decisions live in `../product/knowledge/decisions/`. -Framework decisions live in `decisions/`. +Framework method evolution is incorporated directly into `FRAMEWORK.md`, the affected contracts, validators, and tests. Git history is the maintenance record. diff --git a/starter/.spec-framework/decisions/README.md b/starter/.spec-framework/decisions/README.md deleted file mode 100644 index ea18794..0000000 --- a/starter/.spec-framework/decisions/README.md +++ /dev/null @@ -1,15 +0,0 @@ -# Framework Decisions - -## Purpose - -Store Framework Decision Records installed from the framework release. - -FDRs explain method, gates, validators, skill contracts, and workflow policy. They are not product decisions. - -## Product Decision Boundary - -Product decisions live in: - -```text -product/knowledge/decisions/ -``` diff --git a/starter/AGENTS.md b/starter/AGENTS.md index 6eeee99..5553a62 100644 --- a/starter/AGENTS.md +++ b/starter/AGENTS.md @@ -30,7 +30,7 @@ Do not edit framework-owned assets to encode product scope. Put product-specific ## First Handoff -Start with Framework Guide unless a current CLI result or explicit human request names both the specialist and the concrete artifact or workspace scope. Revalidate persisted handoffs/checkpoints with `dashboard`, `status`, `next`, or `guide` before routing directly. A skill name without scope is only a hint. For a new product, continue through Product Orchestrator, Problem Discovery, Vision, and Strategy before creating domains or features. For `existing-documents`, let Artifact Importer read every inventoried source and complete `traceability.json` and `mapping.json` before requesting materialization. +Start with Framework Guide unless a current CLI result or explicit human request names both the specialist and the concrete artifact or workspace scope. Revalidate persisted handoffs/checkpoints with `dashboard`, `status`, `next`, or `guide` before routing directly. A skill name without scope is only a hint. For a new product, continue through Product Orchestrator, Problem Discovery, Vision, and Strategy before creating domains or features. For `existing-documents`, let Artifact Importer read every inventoried source and complete `traceability.json` and `mapping.json` before requesting materialization. For a scalable import, claim and review one persisted chunk at a time with source evidence; never mark a chunk reviewed without it. ## Canonical Delivery Flow diff --git a/starter/README.md b/starter/README.md index 25f4d28..740f02c 100644 --- a/starter/README.md +++ b/starter/README.md @@ -40,7 +40,7 @@ These are the default `new-product` steps. For every initialized repository, rea 8. Use `spec-framework work`, then `resume`, leases, checkpoints, and handoffs so execution can be continued without conversation history. 9. Use `spec-framework engineering-system inspect` for the shared technical baseline, `spec-framework dashboard --work WORK-001` for the consolidated flow, and `spec-framework decisions migrate` to preview legacy decision metadata upgrades. 11. Run `spec-framework decisions check --strict` in CI to validate decision paths, domains, approvals, references, and navigable links. -10. For `existing-documents`, ask the Artifact Importer agent to read each source and fill `traceability.json` plus proposed `mapping.json` entries before materialization; review unmapped gaps explicitly. +10. For `existing-documents`, ask the Artifact Importer agent to read each source and fill `traceability.json` plus proposed `mapping.json` entries before materialization; for a large source set, use bounded import chunks with evidence per source and review unmapped gaps explicitly. ## Boundary Rule