feat(runtime): close local telemetry evidence (#685)

[kunaldhongade]

Summary

• Close feat(runtime): ingest production telemetry as read-only engineering evidence #685 with downstream topology + latency budget correlation, cited deploy/error tasks, and canProveCurrentTree: false
• Expose MCP runtime_evidence, persist redacted .codedecay/local/runtime-evidence.json, document can/cannot prove in docs/runtime.md
• Cover UAT-RUNTIME-1..6; local-artifact provider only (zero network by default)

Test plan

• vitest runtime-evidence + CLI runtime + MCP runtime_evidence
• tsc -p tsconfig.base.json --noEmit
• built CLI runtime --help
• CI green on this PR

Closes #685

Made with Cursor

[github-actions]

CodeDecay PR Check

Lead catch: No direct high-signal catch found

CodeDecay did not find a strong direct regression or weak-test signal in this run.

Risk: Medium · Merge 39/100 · Decay 54/100 · Security 0/100

This score is conservative pending stronger direct evidence. Treat structural-only signals as review guidance, not proof of a regression.

Full CodeDecay report

CodeDecay Report

Overall risk: Medium

Score	Value
Merge risk	39/100
Decay risk	54/100
Security risk	0/100

Findings	Count
High	3
Medium	13
Low	16

Changed Files

• docs/runtime.md added (+29/-0)
• packages/cli/src/docs/command-docs/analysis.ts modified (+4/-1)
• packages/knowledge/src/index.ts modified (+6/-2)
• packages/knowledge/src/runtime/ingest.ts modified (+364/-53)
• packages/knowledge/src/runtime/render.ts modified (+39/-4)
• packages/knowledge/src/runtime/types.ts modified (+42/-2)
• packages/knowledge/test/runtime-evidence.test.ts modified (+192/-9)
• packages/mcp/src/handlers/runtime-evidence.ts added (+39/-0)
• packages/mcp/src/index.ts modified (+3/-0)
• packages/mcp/src/tools/register-analysis.ts modified (+11/-2)
• packages/mcp/src/tools/registry.ts modified (+3/-1)
• packages/mcp/src/tools/schemas.ts modified (+10/-0)
• packages/mcp/src/tools/types.ts modified (+10/-0)
• packages/mcp/test/mcp-runtime-evidence.test.ts added (+55/-0)

Likely Impacted Areas

• Low Documentation (docs): docs/runtime.md, packages/cli/src/docs/command-docs/analysis.ts
• Low Source code (source): packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts, packages/knowledge/src/runtime/render.ts, packages/knowledge/src/runtime/types.ts, packages/mcp/src/handlers/runtime-evidence.ts, packages/mcp/src/index.ts, packages/mcp/src/tools/register-analysis.ts, packages/mcp/src/tools/registry.ts, packages/mcp/src/tools/schemas.ts, packages/mcp/src/tools/types.ts
• Low Tests (test): packages/knowledge/test/runtime-evidence.test.ts, packages/mcp/test/mcp-runtime-evidence.test.ts

Normalized Impact Graph

Graph artifact: .codedecay/local/impact-graph.json (4597 node(s), 9349 edge(s))

• Confidence: Direct: 9349, inferred: 0, heuristic: 0
• codedecay-js-babel-symbols via @babel/parser (available, adapter 1.0.0)
  • Limitation: Call expressions are not connected to target symbols in this adapter version.
  • Limitation: Only JavaScript and TypeScript files parsed by @babel/parser are represented.
  • Limitation: Static import resolution does not resolve runtime dependency injection or dynamic imports.
• codedecay-python-lezer via @lezer/python (available, adapter 1.0.0)
  • Limitation: Dynamic imports, dependency injection, decorators without literal routes, and framework route registries are not resolved.
  • Limitation: Python impact evidence uses the @lezer/python grammar and conservative module-to-file resolution.
  • Limitation: Static test imports do not prove execution or assertion quality.
• Graph limitation: A static test import does not prove the symbol executed or that assertions cover its behavior.

Symbol Impact Evidence

Graph artifact: .codedecay/local/symbol-impact-graph.json (958 file(s), 5790 edge(s))

• packages/cli/src/docs/command-docs/analysis.ts#ANALYSIS_COMMAND_DOCS -> packages/cli/src/docs/commands.ts
• packages/knowledge/src/index.ts#ingestRuntimeEvidence -> packages/cli/src/commands/runtime.ts, packages/knowledge/test/runtime-evidence.test.ts, packages/mcp/src/handlers/runtime-evidence.ts
  • Likely tests: packages/knowledge/test/runtime-evidence.test.ts
• packages/knowledge/src/runtime/ingest.ts#ingestRuntimeEvidence -> packages/cli/src/commands/runtime.ts, packages/knowledge/src/index.ts, packages/knowledge/test/runtime-evidence.test.ts, packages/mcp/src/handlers/runtime-evidence.ts
  • Likely tests: packages/knowledge/test/runtime-evidence.test.ts
• packages/knowledge/src/runtime/ingest.ts#IngestRuntimeEvidenceOptions -> packages/knowledge/src/index.ts
• packages/knowledge/src/runtime/ingest.ts#persistRuntimeEvidenceArtifact -> packages/knowledge/src/index.ts
• packages/knowledge/src/runtime/ingest.ts#RUNTIME_EVIDENCE_ARTIFACT_PATH -> packages/knowledge/src/index.ts
• packages/knowledge/src/runtime/render.ts#renderRuntimeEvidenceMarkdown -> packages/cli/src/commands/runtime.ts, packages/knowledge/src/index.ts, packages/mcp/src/handlers/runtime-evidence.ts
• packages/knowledge/src/runtime/types.ts#RuntimeDeploymentEvidence -> packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts
• packages/knowledge/src/runtime/types.ts#RuntimeErrorEvidence -> packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts
• packages/knowledge/src/runtime/types.ts#RuntimeEvidenceReport -> packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts, packages/knowledge/src/runtime/render.ts
• packages/knowledge/src/runtime/types.ts#RuntimeEvidenceSource -> packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts
• packages/knowledge/src/runtime/types.ts#RuntimeInvestigationTask -> packages/knowledge/src/index.ts, packages/knowledge/src/runtime/ingest.ts
• ...and 11 more symbol impact(s)

Language And Parser Coverage

• Source files classified: 13
• Fully supported parser files: 13
• Limited files: 0
• Unsupported files: 0

Merge Risk Breakdown

• Score: 39/100
• Raw score before dampeners: 71/100
• Adjusted score before severity cap: 55/100
• Highest contributing severity: Low
• Evidence mode: heuristic-only

Top contributors:

• +9 Change size (structural): Changed lines amplify review cost across 14 file(s).
• +6 File spread (structural): Change breadth spans 14 file(s).
• +4 Docs area changed (heuristic): docs/runtime.md touches a docs area and should be reviewed for regression impact.
• +4 Docs area changed (heuristic): packages/cli/src/docs/command-docs/analysis.ts touches a docs area and should be reviewed for regression impact.
• +4 Source area changed (heuristic): packages/knowledge/src/index.ts touches a source area and should be reviewed for regression impact.

Dampeners:

• -16 Heuristic-only dampener: Merge risk stays conservative until direct evidence exists.

Notes:

• Heuristic-only merge risk is capped at 54/100 until direct evidence exists.
• Untrusted memory context is visible but contributes 0 score until trusted evidence corroborates it.

Decay Risk Breakdown

• Score: 54/100
• Raw score before dampeners: 100/100
• Adjusted score before severity cap: 84/100
• Highest contributing severity: Medium
• Evidence mode: heuristic-only

Top contributors:

• +10 Broad unrelated change set (heuristic): This PR changes 12 files across 1 top-level areas and 2 risk categories.
• +10 Duplicated added logic (heuristic): A similar block of added logic appears 2 times across 2 file(s).
• +10 Duplicated added logic (heuristic): A similar block of added logic appears 2 times across 2 file(s).
• +10 Duplicated added logic (heuristic): A similar block of added logic appears 2 times across 2 file(s).
• +10 Duplicated added logic (heuristic): A similar block of added logic appears 2 times across 2 file(s).

Dampeners:

• -16 Heuristic-only dampener: Decay stays conservative until direct evidence exists.

Notes:

• Heuristic-only decay is capped at 54/100 until direct evidence exists.

Security Risk Breakdown

• Score: 0/100
• Raw score before dampeners: 0/100
• Adjusted score before severity cap: 0/100

Security Matcher Coverage

• Changed source files scanned: 11
• Security candidates found: 0
• Skipped files: 0

Test Evidence

• Mode: heuristic-only
• Sources: none
• Changed source coverage:
• packages/cli/src/docs/command-docs/analysis.ts: not measured (no measurable changed lines)
• packages/knowledge/src/index.ts: not measured (no measurable changed lines)
• packages/knowledge/src/runtime/ingest.ts: not measured (no measurable changed lines)
• packages/knowledge/src/runtime/render.ts: not measured (no measurable changed lines)
• packages/knowledge/src/runtime/types.ts: not measured (no measurable changed lines)
• packages/mcp/src/handlers/runtime-evidence.ts: not measured (no measurable changed lines)
• packages/mcp/src/index.ts: not measured (no measurable changed lines)
• packages/mcp/src/tools/register-analysis.ts: not measured (no measurable changed lines)
• Notes:
• No runtime coverage artifact was found. Test audit remains heuristic-only.

Changed Path Test Proof

Status	Count
Runtime-proven	0
Static-only	5
Weakened by mocks	0
Unproven	18

• Unproven packages/cli/src/docs/command-docs/analysis.ts#ANALYSIS_COMMAND_DOCS (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/cli/src/docs/command-docs/analysis.ts#ANALYSIS_COMMAND_DOCS without mocking the changed boundary.
• Static-only packages/knowledge/src/index.ts#ingestRuntimeEvidence (static-reference, deterministic)
  • Evidence: Referenced by packages/knowledge/test/runtime-evidence.test.ts, but no runtime coverage artifact proves changed lines executed.
  • Static references: packages/knowledge/test/runtime-evidence.test.ts
  • Repair task: Strengthen packages/knowledge/test/runtime-evidence.test.ts so it executes packages/knowledge/src/index.ts#ingestRuntimeEvidence with assertions; static import alone is not proof.
• Static-only packages/knowledge/src/runtime/ingest.ts#ingestRuntimeEvidence (static-reference, deterministic)
  • Evidence: Referenced by packages/knowledge/test/runtime-evidence.test.ts, but no runtime coverage artifact proves changed lines executed.
  • Static references: packages/knowledge/test/runtime-evidence.test.ts
  • Repair task: Strengthen packages/knowledge/test/runtime-evidence.test.ts so it executes packages/knowledge/src/runtime/ingest.ts#ingestRuntimeEvidence with assertions; static import alone is not proof.
• Unproven packages/knowledge/src/runtime/ingest.ts#IngestRuntimeEvidenceOptions (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#IngestRuntimeEvidenceOptions without mocking the changed boundary.
• Unproven packages/knowledge/src/runtime/ingest.ts#persistRuntimeEvidenceArtifact (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#persistRuntimeEvidenceArtifact without mocking the changed boundary.
• Unproven packages/knowledge/src/runtime/ingest.ts#RUNTIME_EVIDENCE_ARTIFACT_PATH (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#RUNTIME_EVIDENCE_ARTIFACT_PATH without mocking the changed boundary.
• Unproven packages/knowledge/src/runtime/render.ts#renderRuntimeEvidenceMarkdown (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/knowledge/src/runtime/render.ts#renderRuntimeEvidenceMarkdown without mocking the changed boundary.
• Unproven packages/knowledge/src/runtime/types.ts#RuntimeDeploymentEvidence (missing-proof, heuristic)
  • Evidence: No runtime coverage or static test reference was found for this changed path.
  • Repair task: Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeDeploymentEvidence without mocking the changed boundary.
• ...and 15 more changed path proof entries

Untrusted Memory Context

• Project invariant may be impacted (packages/knowledge/src/index.ts:70): Untrusted memory context: invariant "No hidden cloud or model call" applies to this change. The OSS CLI must remain useful without telemetry, API keys, hosted services, required LLM calls, or CodeDecayCloud.
• Project invariant may be impacted (packages/knowledge/src/index.ts:70): Untrusted memory context: invariant "Commands are explicit" applies to this change. CodeDecay must not run project commands unless they are configured and safety.allowCommands is true.
• Project invariant may be impacted (docs/runtime.md:1): Untrusted memory context: invariant "Tool evidence is separate from AI suggestions" applies to this change. Reports must not present agent/model suggestions as verified evidence unless backed by deterministic checks or command output.
• Project invariant may be impacted (docs/runtime.md:1): Untrusted memory context: invariant "Output must be actionable" applies to this change. Redteam reports and agent bundles should say what behavior to verify, which test proof is weak or missing, and what task a coding agent should perform.
• Architecture note applies (packages/cli/src/docs/command-docs/analysis.ts:39): Untrusted architecture context: CLI is the published surface: The public npm package is @submuxhq/codedecay and the binary is codedecay. Internal workspace packages are implementation details.
• Architecture note applies (packages/mcp/src/handlers/runtime-evidence.ts:1): Untrusted architecture context: Agent-owned workflow: CodeDecay should produce evidence and task bundles that a user's own Codex, Claude Code, Cursor, OpenCode, Pi, desktop app, or MCP client can use.

Medium Risk Findings

• Broad unrelated change set: This PR changes 12 files across 1 top-level areas and 2 risk categories.
• Duplicated added logic (packages/mcp/src/handlers/runtime-evidence.ts:9): A similar block of added logic appears 2 times across 2 file(s).
• Duplicated added logic (packages/mcp/src/handlers/runtime-evidence.ts:10): A similar block of added logic appears 2 times across 2 file(s).
• Duplicated added logic (packages/mcp/src/handlers/runtime-evidence.ts:11): A similar block of added logic appears 2 times across 2 file(s).
• Duplicated added logic (packages/mcp/src/handlers/runtime-evidence.ts:12): A similar block of added logic appears 2 times across 2 file(s).
• Duplicated added logic (packages/mcp/src/handlers/runtime-evidence.ts:13): A similar block of added logic appears 2 times across 2 file(s).
• High complexity in changed function (packages/knowledge/src/runtime/ingest.ts:154): loadOtlp has estimated cyclomatic complexity 19.
• High complexity in changed function (packages/knowledge/src/runtime/ingest.ts:235): loadErrorsAndDeployments has estimated cyclomatic complexity 14.
• High complexity in changed function (packages/knowledge/src/runtime/ingest.ts:381): correlateTopology has estimated cyclomatic complexity 16.
• High complexity in changed function (packages/knowledge/src/runtime/ingest.ts:431): createTasks has estimated cyclomatic complexity 12.
• High complexity in changed function (packages/knowledge/src/runtime/render.ts:3): renderRuntimeEvidenceMarkdown has estimated cyclomatic complexity 14.
• Large changed function (packages/mcp/src/tools/register-analysis.ts:37): registerAnalysisMcpTools spans 141 lines, which increases review and regression risk.

Low Risk Findings

• Docs area changed (docs/runtime.md:1): docs/runtime.md touches a docs area and should be reviewed for regression impact.
• Docs area changed (packages/cli/src/docs/command-docs/analysis.ts:39): packages/cli/src/docs/command-docs/analysis.ts touches a docs area and should be reviewed for regression impact.
• Source area changed (packages/knowledge/src/index.ts:70): packages/knowledge/src/index.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/knowledge/src/runtime/ingest.ts:2): packages/knowledge/src/runtime/ingest.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/knowledge/src/runtime/render.ts:8): packages/knowledge/src/runtime/render.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/knowledge/src/runtime/types.ts:4): packages/knowledge/src/runtime/types.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/handlers/runtime-evidence.ts:1): packages/mcp/src/handlers/runtime-evidence.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/index.ts:32): packages/mcp/src/index.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/tools/register-analysis.ts:16): packages/mcp/src/tools/register-analysis.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/tools/registry.ts:23): packages/mcp/src/tools/registry.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/tools/schemas.ts:134): packages/mcp/src/tools/schemas.ts touches a source area and should be reviewed for regression impact.
• Source area changed (packages/mcp/src/tools/types.ts:64): packages/mcp/src/tools/types.ts touches a source area and should be reviewed for regression impact.
• Test area changed (packages/knowledge/test/runtime-evidence.test.ts:1): packages/knowledge/test/runtime-evidence.test.ts touches a test area and should be reviewed for regression impact.
• Test area changed (packages/mcp/test/mcp-runtime-evidence.test.ts:1): packages/mcp/test/mcp-runtime-evidence.test.ts touches a test area and should be reviewed for regression impact.

Recommended Checks

• Add an integration test that reaches packages/cli/src/docs/command-docs/analysis.ts#ANALYSIS_COMMAND_DOCS without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#IngestRuntimeEvidenceOptions without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#persistRuntimeEvidenceArtifact without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/ingest.ts#RUNTIME_EVIDENCE_ARTIFACT_PATH without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/render.ts#renderRuntimeEvidenceMarkdown without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeDeploymentEvidence without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeErrorEvidence without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeEvidenceReport without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeEvidenceSource without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeInvestigationTask without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeOperationEvidence without mocking the changed boundary.
• Add an integration test that reaches packages/knowledge/src/runtime/types.ts#RuntimeProviderConfig without mocking the changed boundary.

Notes

CodeDecay is deterministic and local-first. This report was generated without telemetry, API keys, LLMs, or model calls.

---

Found by CodeDecay - deterministic, local-first, no telemetry.