Skip to content

Phase 15 — KB index analysis caches#280

Open
leeovery wants to merge 11 commits into
idea/inception-pr-14-provenancefrom
idea/inception-pr-15-kb-index-analysis
Open

Phase 15 — KB index analysis caches#280
leeovery wants to merge 11 commits into
idea/inception-pr-14-provenancefrom
idea/inception-pr-15-kb-index-analysis

Conversation

@leeovery
Copy link
Copy Markdown
Owner

Summary

  • Index .workflows/{wu}/.state/{research-analysis,gap-analysis}.md cache files into the knowledge base under a new analysis phase value, so the full reasoning behind surfaced topics is searchable alongside research/discussion/spec content.
  • Cache files are indexed automatically by the two self-healing analyses immediately after each cache is rewritten; removal is already covered by the broad knowledge remove --work-unit path.
  • Filename whitelist (research-analysis, gap-analysis) prevents operational .state/ files (migrations, environment-setup) from ever entering the KB.

Changes

  • src/knowledge/index.js: extend INDEXED_PHASES, deriveIdentity (separate .state/ match with whitelist), and discoverArtifacts (file-existence scan inside the per-work-unit loop, after imports).
  • skills/workflow-knowledge/scripts/knowledge.cjs: rebuilt bundle.
  • skills/workflow-knowledge/chunking/analysis.json: mirrors research.json with confidence: "low".
  • skills/workflow-shared/references/{research-analysis,discussion-gap-analysis}.md: append knowledge index call after manifest cache-metadata writes in section E (surface-and-continue error pattern).
  • skills/workflow-{inception-process,bridge}/SKILL.md: declare Bash(node .claude/skills/workflow-knowledge/scripts/knowledge.cjs) so self-healing-driven indexing doesn't prompt.
  • Tests: 12 new CLI tests (tests/scripts/test-knowledge-cli.sh) and 2 new integration cases (tests/scripts/test-knowledge-integration.cjs) covering happy path, whitelist enforcement, idempotency, bulk discovery, cancelled-wu skip, and path-traversal guards.

Catch-up for existing work units

Existing in-progress epics with stale caches will re-index naturally on the next analysis re-run (Phase 7's stale-cache trigger). Users wanting immediate searchability of an unchanged cache can run knowledge index .workflows/{wu}/.state/{filename}.md manually.

Test plan

  • node --test tests/scripts/test-*.cjs — 641 tests pass
  • bash tests/scripts/test-knowledge-cli.sh — 215 pass (12 new)
  • bash tests/scripts/test-knowledge-build.sh — bundle freshness verified
  • Re-run of build (npm run build) produces a bundle identical to the committed one

🤖 Generated with Claude Code

leeovery and others added 11 commits May 17, 2026 17:22
Add 'analysis' to INDEXED_PHASES, extend deriveIdentity to parse
.workflows/{wu}/.state/{research-analysis,gap-analysis}.md paths with
filename whitelist enforcement, extend discoverArtifacts to scan caches
by file existence. Mirror research.json as analysis.json chunking config
(confidence: low). Rebuild bundle and cover with CLI + integration tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Append a `knowledge index` call to section E after the manifest writes
so the analysis content surfaces in future contextual queries. Mirrors
the surface-and-continue error pattern from import-files.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Append a `knowledge index` call to section E after the manifest writes
so the gap-analysis content surfaces in future contextual queries.
Mirrors the surface-and-continue error pattern from import-files.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ia self-healing

Both skills load self-healing references that now invoke `knowledge index`
on analysis cache writes. Declare the tool preemptively so the bridge and
refinement-session flows don't hit a permission prompt mid-stream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The for-INDEXED_PHASES loop in discoverArtifacts would no-op on the
new 'analysis' value (no manifest.phases.analysis exists), but the
silent fall-through is unclear. Make the skip explicit, matching the
imports special-case pattern. Behaviour is identical.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three review-driven fixes:
- SKILL.md "What is indexed" / "rebuild" prose was a hardcoded
  enumeration, not auto-derived — add the analysis row.
- reindex-work-unit.md only walked manifest-backed phases plus imports;
  reactivating a cancelled work unit left its analysis caches indexable
  on disk but absent from the KB. New section C probes the two cache
  paths and re-indexes any present.
- Surface the .state operational-file exclusion in "What is NOT indexed".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The analysis writes its cache to .state/discussion-gap-analysis.md
(prefixed), but the indexing whitelist accepted only gap-analysis.md
(no prefix). The reference's `knowledge index ... discussion-gap-analysis.md`
call would have thrown UserError at runtime, and bulk discovery would
have looked at the wrong path and never found the file.

Tests passed pre-fix because they exercised the whitelist with the
wrong filename — a self-confirming gap. Updated tests to use the
canonical filename so the bug can't reintroduce.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two further doc-accuracy gaps surfaced during review:
- Confidence-tier table listed only research + imports under 'low';
  analysis sits in the same tier and consumers need to distinguish via
  the provenance line's phase field.
- 'index' command's path-derivation note covered the standard and
  imports shapes but not '.state/' analysis paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sertions

- reindex-work-unit.md section B's 'no imports' branch returned to caller,
  bypassing my new section C. An epic with no imports but with on-disk
  analysis caches would not have them re-indexed on reactivation. Route
  both B branches to C instead.
- CLI tests 90 and 91 grep for 'analysis' in query output — but the
  Source: line contains the .state filename, which also matches. Switch
  to anchored '^[analysis |' assertions so the test actually exercises
  the provenance line.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The [ -f file ] && cmd shortcut exits non-zero when the file is
absent (the test command's exit code), which Claude's Bash tool
surfaces as an error and aborts the reference. A fresh epic with
no analysis caches yet would always trip this. Switch to
if/then/fi so the absent-file branch exits 0 cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant