Skip to content

research: add System One code localization experiment - #16

Merged
BestNathan merged 46 commits into
mainfrom
research/system-one-code-localization
Sep 23, 2026
Merged

BestNathan merged 46 commits into
mainfrom
research/system-one-code-localization

Conversation

@BestNathan

Copy link
Copy Markdown
Owner

Summary

Adds a second System One research prototype under the existing progressive-action-space topic.

  • add a Python code-localization demo: repository -> directories -> files -> source lines -> grounded snippets
  • use independent Noul relevance judgments instead of Choice because multiple candidates may be relevant
  • preserve full candidate/decision provenance in JSONL traces
  • add deterministic fixture tests and a small websocket-oriented fixture repository
  • extend the System One topic with the shared StateSpaceGenerator -> CandidateSet -> DecisionPrimitive -> TransitionPolicy -> EvidenceRecorder model
  • add a GitHub Actions workflow that runs deterministic demos automatically and collects real System One evidence on manual dispatch

Research evidence

The workflow uploads 90-day artifacts containing:

  • run-manifest.json
  • Kubernetes result + human-readable execution log
  • code-locator result.json
  • code-locator trace.jsonl with candidate disclosure, model requests, scores, thresholds, latency, and usage
  • stdout.log
  • summary.md for real System One runs

The real-model job is manual, uses the existing ds environment, requires TYPESAFE_API_KEY, and accepts subject repository/ref, query, thresholds, and batch size as experiment inputs.

Validation

  • existing Kubernetes demo tests are executed by the new workflow
  • code-locator tests cover deterministic progressive localization and Noul request construction
  • the workflow's fixture job does not require external model credentials

Copy link
Copy Markdown
Owner Author

Research lineage update: this merged PR remains the historical initial Code Locator prototype in Narness. The later exploration line has moved to BestNathan/system-one-code-explore. Narness convergence and the distilled architectural conclusions are tracked in #26; the in-repo research/code-locator/ tree is now treated as an archived snapshot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant