Preserve leading declaration comments in bounded symbol search documents

[TheGreenCedar]

Outcome

A user investigating configuration values should be able to locate an indexed function whose declaration comment explicitly describes the value being searched. On pinned Terragrunt U04-C01, a public full-retrieval search for TF_VAR omits SetTerragruntInputsAsEnvVars and ToTerraformEnvVars, forcing ordinary source search to discover the configuration-to-Terraform handoff.

Reproduction and cause

Unpublished c3a5 full-retrieval public search completed with ten other rows and no gaps; internal/runner/run/run.go is indexed and complete. Its exported function comments contain TF_VAR at lines 491 and 714. Frozen core symbol_search_doc rows for the two functions omit it. Runtime semantic projection retains at most four nearest preceding comment lines and spends its 48-token source allocation on signature/body before comments. The lexical sidecar indexes the resulting document verbatim.

Retained development receipts:

• /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-public-search-devc3a5-03/run-receipt.json
• /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-current-public-route-devc3a5-02/packet-output.json
• /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-installed-adaptive-devc3a5-01/root-source-baseline-2026-09-23.md

Scope and acceptance

Repair the existing bounded symbol-document construction in codestory-runtime; do not add Terragrunt- or question-specific selectors, increase public confidence of diagnostic repo-text hits, or change the public schema. Cover a leading term in a multi-line declaration comment, bounded source/total tokens, retained identity/signature, deterministic document order and affected publication failure boundaries. Obtain independent exact-patch review and verify that an affected public search can locate useful evidence on a newly built candidate. A symbol-doc repair alone does not prove a complete adaptive answer or comparative gain.

Owner: root Sol 6 integrates and supervises; separate Sol 6 implements in an isolated worktree; independent Sol 6 challenger reviews. Base c3a5a7378ecdf57440312e433aaf2df8a8e21dc9. Refs #2173 and #2135.

[TheGreenCedar]

The first #2372 patch is rejected and preserved, with no source commit or candidate change. A base-head focused test reproduced the missing TF_VAR symbol-document text; the draft made its fixtures green, but exact independent review found two regressions: it discarded substantive opening/middle Godoc text (for example, Open ... STARTTLS) and could remove an existing DocumentedNontrivial dense anchor by shortening its body. The actual ToTerraformEnvVars comment places TF_VAR_xxx beyond the fixed 48-token source field even before signature/body; a safe generic loss policy and its dense effects remain to be designed. No repo-specific selector, larger global budget, V3 repo-text promotion or shared abstraction is admitted from one case. The failed diff is retained at /Users/albert/.codex/evidence/codestory-0176-codex-recovery/issue2372-rejected-first-patch-2026-09-23.diff, SHA-256 8e197bdaf29e4a195b44419e1124c03a72a5f8e4c13cd32b2873bed147dd48cf. Full development decision: /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-root-product-decision-2026-09-23.md, SHA-256 f5cd77be126fa123e861627c8341674b4e2955b011c32b33a14ffd28e2167041. Keep this issue open; the unified 0.17.6 goal continues through other outcomes.

[TheGreenCedar]

Three-repository transfer check for #2372

The leading-comment representation gap now has task-relevant examples in three unrelated repositories across infrastructure orchestration, build execution, and messaging. This satisfies the program's evidence threshold for considering a shared bounded symbol-document repair. It does not establish improved search ranking, complete adaptive answers, or comparative gain.

• Terragrunt U04-C01: pinned source comments for SetTerragruntInputsAsEnvVars and ToTerraformEnvVars describe TF_VAR; the c3a5 extracted symbol_search_doc rows at lines 495 and 720 omit that term, and the recorded public search missed those methods. The core rows retain signatures and body excerpts, so source was available.
• Bazel P04-Q12: leading JavaDoc for RemoteExecutionService.uploadInputsIfNotPresent, executeRemotely, and downloadOutputs describes input upload, remote execution, and output handling. The c3a5 extracted rows at lines 1981, 2040, and 1258 omit the substantive comments (the last retains only *). Method names still expose coarse upload/download terms; task-level search impact remains unverified.
• NATS P32-Q01: pinned server/client.go:5240-5242 says processMsgResults processes sublist results and reports delivery with queue filters. The b1ee extracted row 7536098353203745570, with source span 5242-5775, includes a signature and body excerpt but no comments:. The owning projection source is unchanged between b1ee and c3a5. This is representation evidence from an older accepted development candidate, not a c3a5 public-search result.

PocketBase U11-C02 did not show this specific loss: its relevant declarations have no task-bearing leading comments. Preserve that negative control.

Two rejected revisions remain rejected. A third positional reordering would be unsound because the fixed 48-unit source share can be saturated by signature and body; earlier attempts lost substantive STARTTLS documentation and changed DocumentedNontrivial dense eligibility. The next source slice must state its bounded-loss policy, preserve baseline dense-anchor decisions across source build and stored-core republish, pass hostile cases, and then demonstrate the affected public search on a new installed candidate. No adaptive-answer claim follows from a green unit test or retained document text.

[TheGreenCedar]

Rejected #2372 bounded-synopsis revision

The third uncommitted source revision is rejected. Its frozen two-file diff is issue2372-exact-review-patch-01.diff, SHA-256 030937ed0198de4db6e5f029dfcc3d18d080fb74449735541712d80c60460786, based on dev c3a5a7378ecdf57440312e433aaf2df8a8e21dc9. It was not committed, integrated, packaged, or exercised through a public search.

The implementer reproduced the original Terragrunt loss before editing: the v10 bounded row ended with comments: // Keys use and omitted TF_VAR_xxx. The draft passed 17 focused semantic-document tests, including representative Terragrunt, Bazel and NATS comment shapes, selected budget/dense-parity cases and STARTTLS opening/middle text, plus a stored-core runtime-cache fault test. Those tests do not establish usable public retrieval.

Independent Sol 6 high exact-patch review rejected the revision. At default source budget 48, a long signature and ordinary comment can leave only the body_summary: field label; the guard checks label presence while losing substantive body evidence (semantic_projection.rs:2075-2095). The comment synopsis selects code-shaped tokens and the first three prose lines, with a special second-line tail; a plain-language failure condition in a later line is predictably omitted (:2140-2225). This is another positional/example-sensitive tradeoff, not a sound general attached-comment representation. The v10 dense-selection parity check itself was conditionally sound, but it does not protect body content or uncaptured comment claims.

Root decision: preserve this failure, stop revising the same bounded dense text, and move the repair to the owning internal store/retrieval seam. Keep the existing dense symbol document, hash and anchor selection unchanged. Persist a separate bounded, source-attached comment field, version and hash; incorporate it into the existing symbol lexical row and sidecar input seal. Use source-ordered text with a simple 64-line/256-proxy-token cap and tail truncation. Require source repair for old/missing rows, stored-core consistency and prior-publication survival. The expanded boundary remains internal; no public API or result type is planned. The independent reviewer will challenge and review that exact replacement patch.

The implementer measured the pinned target comments with the same ceil(chars/3) proxy: Terragrunt 5/78 and 7/181 lines/units; Bazel 5/48, 6/72 and 13/211; NATS 2/46. All fit the proposed cap intact. This is a sizing check, not a new public-search result.

[TheGreenCedar]

PR #2373 merged into dev/codestory-next at 572e9364653aff9552bad26ba6b80c7c547bb4f3 (tree dc1b99c84951733fde705a143333bdaf1049d548). Independent exact-patch review accepted the source and two focused CI corrections; all applicable PR checks passed. The accepted source gives attached declaration comments a separate bounded lexical field while keeping dense symbol documents and anchors unchanged.

#2372 remains open for its public-search acceptance. The old c3a5 candidate and its failed TF_VAR search are preserved. Root is preparing a fresh unpublished candidate from the clean detached 572e dev merge under the 300-second/5-GiB preparation and 90-second installation controls, with a 50-GiB free-space reserve. Next proof: repeat the same public Terragrunt TF_VAR search and verify whether both previously omitted functions are reachable. No installed behavior, adaptive answer gain, formal acceptance, or release qualification is claimed yet.

[TheGreenCedar]

The merged 572e source fix now has a fresh unpublished package and an independently accepted isolated installation. The package archive is ddc51565…, runtime d1abc91d…, and installed plugin digest 62b4a1b4…; the 20-tool catalog and source pin match the merged dev head. This proves candidate/package identity, not the public TF_VAR search result.

The fresh Terragrunt search has not started. Exact-byte control reviews rejected five successive launcher revisions before admission. The latest review, controls-review-final-05.json (SHA-256 aacc5d7c…), found timeout cleanup could signal a process group the attempt no longer owns. Earlier rejected reviews and synthetic tests remain preserved at /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-public-search-dev572e-01/. No native/model attempt or budget was consumed, and no result is claimed from the new runtime. #2372 stays open for the same public-search acceptance; the wider 0.17.6 product program stays active while this route is reconsidered.

[TheGreenCedar]

The 572e candidate remains packaged and independently installed, but #2372's public TF_VAR search has still not run. I tested one changed admission approach: a single direct, candidate-pinned CLI supervisor instead of the rejected nested launchers. Its eight no-target synthetic checks passed. Independent Sol 6 review rejected target admission on the exact guard SHA-256 a07faa68879a074463eee3359b3b462df5be5d4c4b42ba9ca5654734b77a9f3b (test SHA-256 e0134b226b6a4196fee1af6d31084505c976ddf449abc69b7ac3b63cd8b5f524).

The runtime deliberately starts a detached embedding-server session (crates/codestory-cli/src/embedding_server_transport.rs:1536). In a native stall, its no-progress watchdog can outlast the proposed 120-second search and 120-second drain. The direct guard cannot reliably contain that server after CLI-group termination; its observed-PID cleanup also has an identity race. The exact rejected files are under /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-direct-guard-01/. No target CLI call, native slot claim, or attempt budget was consumed.

I am stopping this launcher route, not treating a control failure as a product-search result. The issue stays open for a public agent-profile search on the merged 572e runtime through an already trustworthy native lifecycle boundary. The wider 0.17.6 product and acceptance work remains active; no source, package, or release claim changes here.

[TheGreenCedar]

The source fix is merged through #2375 at dev b9122756950ff4f709cbee7b76f8842d6549c1b9. Its unpublished successor archive and isolated installation passed independent byte/control review (archive SHA-256 b9c8eead4d36dcb9f5239f83b3366cde2e90a44ce4e60f2006d41cad6365663f; installed runtime 340db0f5fc25321b3a5591bb0b3c726f3d2f8f50e7850d392237031aa1458a68). No release qualification or main promotion is claimed.

One fresh public TF_VAR search attempt ran and failed. The CLI returned retrieval_manifest_missing with exit 1: search on a fresh cache builds the core generation but requires a separate agent-profile retrieval index before full retrieval. The guard also rejected native closeout after a previously authenticated native process start identity became unreadable; the available receipt cannot distinguish a brief exit race from another unreadable state. The attempted native server was observed in the held invocation group, and the group was reaped. The native slot was kept unverified until a separate locked, repeated empty-process census reauthenticated its release. The failed attempt remains closed, with no accepted search result.

Receipts: /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-direct-guard-b912-01/run-receipt.json, retrieval-causal-decision.json, identity-causal-decision.json, and slot-reauthentication.json. The next work is a fresh, bounded agent-profile retrieval-index then public-search sequence with corrected native observation. This issue stays open until the public result is accepted; packet or source-document evidence alone does not establish a complete adaptive answer.

[TheGreenCedar]

The b912 public TF_VAR search now has an accepted installed-candidate execution receipt, but the user-visible omission remains. On pinned Terragrunt b07bca44, the agent-profile retrieval index, observational status, and public search all exited zero. Search returned a complete, available, full-retrieval response with ten evidence rows on the indexed generation and no reported gaps; neither SetTerragruntInputsAsEnvVars nor ToTerraformEnvVars appeared. The indexed core rows contain TF_VAR in both functions' attached declaration comments, so the next question is where the relevant candidates lose selection or presentation. This does not establish a complete adaptive answer or comparative gain.

The fresh attempt took 65.567 seconds, sampled at most 710,369,280 bytes added, kept the 50-GiB free reserve, authenticated separate index and search native lifetimes, closed both groups, preserved the source tree, and released the native slot. Evidence: /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-retrieval-sequence-b912-03/run-receipt.json (SHA-256 1d4cc990c5b5a460681f494066fca3f397e20364e05d8f162fa75a1ada20d712) and search.json (SHA-256 5f02c8e4e04e691c2b1e84699afc06d03307fa334b5f00cc0ae4be5ebaae8e4b). The earlier -01 and -02 failed attempts and their closed budgets remain unchanged.

Root owns integration and the next installed interaction. A separate Sol 6 implementer is tracing the exact loss stage on codex/2372-tfvar-selection from dev b9122756; an independent Sol 6 challenger is reviewing the contract and eventual patch. #2372 stays open until an affected public interaction actually exposes useful source evidence. No new general abstraction or repository-specific selector is authorized by this one case.

[TheGreenCedar]

The b912 Terragrunt follow-up demonstrates an adaptive locate step without a new source change. A fresh public full-retrieval search using the task-derived query Terragrunt inputs env TF_VAR returned SetTerragruntInputsAsEnvVars and ToTerraformEnvVars as evidence rows #1 and #2 on pinned source b07bca44. Independent review matched their published symbol IDs to internal/runner/run/run.go:495 and :720. All three CLI phases exited zero, index and search each authenticated their own native runtime and group closeout, source stayed clean, and the native slot was released. The run took 65.93 seconds with 710,979,584 bytes sampled peak growth, below its 3-GiB ceiling and above the 50-GiB free reserve.

This does not erase the broad TF_VAR top-ten omission from the earlier accepted -03 search. Both comments are in the core and lexical index, but the exact downstream fused rank is not exposed by public V3 output; a ranking-policy patch would be speculative. The refined query uses concepts present in the original investigation rather than function names learned from an answer key. CodeStory located the two handoff functions; ordinary pinned source reads support the larger dependency/remote-state explanation. No complete model-composed adaptive answer, comparative gain, live Terraform run, or release qualification is claimed. #2372 remains open as a documented broad-query relevance limit while complete workflow evidence continues under #2173.

Evidence: /Users/albert/.codex/evidence/codestory-0176-codex-recovery/terragrunt-u04-c01-focused-followup-b912-04/run-receipt.json (SHA-256 9266763e2ab7044b5480b1b732fb40b09c5fc57abd08393d280e7369f9c22a33), search.json (bfb651a7a1f1e90d027ad295a733a40e87a80fc3a80ebc4411284a7db94f0a7d), independent-result-review.json (e50f8f24f623196f9ad6b17d5cab33051fdd89fdd337d84248fe5754fff12e68), and root-product-decision.json. Failed -01/-02 and accepted broad -03 remain separate, unchanged attempts with their own budgets.