Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ before using the task picker; its coverage fields identify prior checks and
remaining obligations. Explicit user tasks take precedence over that default
assignment; do not reroll an already assigned task.

For a fresh assignment, run `python3 -B scripts/choose_current_research_task.py`.
This entry point resolves generic baseline commands and retains each frozen
card beside the current command. The historical picker and root guide stay
For a fresh assignment, run `python3 -B scripts/choose_contextual_research_task.py`.
This entry point reports whether the catalogue's assessment context matches the
selected record, then retains the existing current-command routes and frozen
cards. The historical pickers and root guide stay
unchanged because research packages pin their bytes. The current command
examples are in [the standing procedure](docs/research/agent-procedure.md).

Expand Down
18 changes: 13 additions & 5 deletions docs/research/agent-procedure.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,15 @@ Run this standard-library helper:

```sh
# Python 3.10 or later; no third-party dependencies.
python3 scripts/choose_current_research_task.py
python3 scripts/choose_contextual_research_task.py
# To reproduce a selection, pass the emitted 32-hex-digit nonce:
python3 scripts/choose_current_research_task.py --seed YOUR_NONCE
python3 scripts/choose_contextual_research_task.py --seed YOUR_NONCE
# Expertise or an explicit user preference may override the allocation:
python3 scripts/choose_current_research_task.py --track complex-mechanisms
python3 scripts/choose_contextual_research_task.py --track complex-mechanisms
# Retrieve a specific card after assignment:
python3 scripts/choose_current_research_task.py --task connector-alphabet-change
python3 scripts/choose_contextual_research_task.py --task connector-alphabet-change
# Optional: surface structured handoffs/claims from an existing overlap cache.
python3 scripts/choose_current_research_task.py --coordination-cache /tmp/overlap.json
python3 scripts/choose_contextual_research_task.py --coordination-cache /tmp/overlap.json
```

It reads [the task catalogue](agent-task-catalog.json), draws a
Expand All @@ -121,6 +121,14 @@ that a task is open. A deterministic seed is a reproducibility aid, not novelty.
Each card supplies a first-step command, expected evidence, a next experiment,
its connection to the bottleneck, allowed variations and scope-change triggers.

The contextual entry point delegates selection and command routing to the
unchanged pickers, then adds `catalog_selection`: the original catalogue context
and its κ/source/review comparison with `selected`. A mismatch means the
assessment predates or differs from the selection; missing fields are unknown.
This does not assess whether a task remains useful or authenticate the scientific
closure. Use `selected` for the current review and replay. See the
[source-pinned discrepancy and controls](priority-selection-context.md).

**Resolve the generic orientation command before running a historical card.**
If its `first_step.command` is `python3 -B scripts/research_baseline.py`, use the
current orientation command in [current priorities](current-priorities.md).
Expand Down
134 changes: 134 additions & 0 deletions docs/research/priority-selection-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Catalogue assessment versus the selected result

The new contextual task CLI exposes the catalogue's original `selection_context`
as `catalog_selection.context` and compares its κ, source commit and review
pointer with the selected record. It delegates to the unchanged current picker.
This resolves an onboarding discrepancy, not a mathematical claim or a new
research-priority assessment.

At base **`7c49fe32c55de02c2c873655c804b16e4c3fef0a`**, revision 8 of the
catalogue and [priority guide](current-priorities.md) name PR5599. The
[selected record](../../certificates/selected-result.json) and its
[application review](pr-intake-evidence/pr6030-audit/PROOF-REVIEW.md) select
PR6030. The old current CLI emitted the latter under `selected` but omitted the
catalogue's assessment context altogether, while linking the former guide.

| Field | Catalogue assessment | Selected record |
| --- | --- | --- |
| κ | `62009/20000000` = .00310045 | `781451/250000000` = .003125804 |
| Source commit | `09dd0965711ca7eeb169057a420abab1c9052137` | `772dd10a5859a1d15f4851feb371519789213ad8` |
| Review directory | `pr5599-completion` | `pr6030-audit` |

The source comparison uses the original `audited_source_head`, rather than the
separate portability-repair head. It does not infer a primary source from the
order of `source_prs`, nor from a larger number in an open PR title.

## Contract and reproduction

```sh
python3 -B scripts/choose_contextual_research_task.py --task scope-navigation --seed 00000000000000000000000000000063
```

On the pinned inputs, expect `catalog_selection.matches_selected: false`, with
exact old/new values under `differences.kappa`, `differences.source_commit` and
`differences.review`, and no `uncompared_fields`. `selected.reproduce` remains
`make frontier6030-verify`. The 32-digit picker seed is 99 in hexadecimal; it
only makes this explicitly chosen card's output reproducible.

Equal rational representations compare equal. A changed review or source at
the same κ still differs. Missing comparison fields produce `null` (unknown)
when no known difference exists; a known difference still produces `false`.
The exact historical context is retained, including its own dated scope text.

Before comparison, both reread JSON byte strings must match the hashes in the
child picker's receipt. A change between the child and wrapper reads fails
without emitting a successful JSON receipt. This detects mixed snapshots at
that boundary; it is not an atomic filesystem snapshot or authentication of
the full selected construction. A stable checkout remains required.

Both existing pickers, the catalogue, priority guide and selected record retain their
bytes. Existing task choices, methods, scope keys, coverage flags and command
routes are preserved. The contextual CLI adds only `catalog_selection`; existing
consumers can keep their original entry point and output contract. Matching
the three metadata fields establishes neither task novelty nor current coverage.

## Verification and source pins

The following command checks the actual discrepancy fixture and controls for
same-κ provenance changes, rational equivalence, missing context, malformed
metadata, changed-between-reads inputs and the real child/wrapper CLI:

```sh
python3 -B -m unittest discover -s tests -p 'test_research_priority_context.py' -v
```

Expected: **10 tests, OK**. The adjacent command is
`python3 -B -m unittest discover -s tests -p 'test_research_current_assignment*.py' -v`
(15 tests). Both pass normally and with `-O -W error` on Linux, CPython 3.12.14.
The real original/contextual receipt for the command above is identical after removing
the added field. The existing controls preserve all live cards, frozen generic
and historical routes, help/error propagation and sparse-input distinctions.
The 27 onboarding and nine coordination tests also pass. The final replay
dependency plan selects no scientific packages; the new code has no scientific
runtime dependency.

The unchanged #6179 conformance replay also passes with 22 live cards, two
historical route fixtures and seven rejection controls:
`python3 -I -W error -B research/current-picker-lazy-validation/verify.py`.
That replay reconstructs the current picker's exact predecessor source, which
is why this contribution uses a separate entry point. The historical timing
receipt is checked for conformance, not rebenchmarked; no speed claim is made
for the contextual CLI's extra subprocess.

Immutable SHA-256 pins at the exact base above:

| Repository path | SHA-256 |
| --- | --- |
| `docs/research/agent-task-catalog.json` | `ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81` |
| `docs/research/current-priorities.md` | `70f61c8d140ae6e892f2b8f065bba4eae3e76f7a8f55f632fd4e2f9f34039bdc` |
| `certificates/selected-result.json` | `9b8d0eea95b3bf0dfcb7938a732a6439b87a2685dec2a5ab9e525b3b34debbfa` |
| `scripts/choose_research_task.py` | `484f553c488c6f427ca81f19f3ee2720399dcc28ce61a3886d50bdf28ca72d12` |
| `scripts/choose_current_research_task.py` (unchanged) | `04e194b14a1f4157be5be83e8c13995d69c3efe8996773d918acba6bfcabb471` |

Only Python's standard library is used. The selected full cold replay, native
maps, compiler, precision/tape and analytic recovery are outside this metadata
repair. The five selected assumptions and κ remain unchanged.

## Assignment, overlap and open next question

OS randomness (`secrets.randbelow(100)`) yielded assignment seed **99**:
research consolidation. Three candidates were considered: this catalogue/record
discrepancy; source-certificate pointer coverage (current inputs agree, with
the broader predecessor #1864); and ledger/status navigation (nearby work
#3695, #4138 and #6050). The chosen bounded experiment was to reproduce and
expose this exact discrepancy while preserving task selection. Stop after the
three-field comparison and its failure controls; no global survey or search for
a stronger bound is part of this result.

The local claim was recorded at 2026-11-09 17:14:30 UTC, with an expected finish
by 17:47:48 UTC. GitHub reported no pinned issue. The start-here discussion,
last 20 handoff comments, 25 recently active discussions and relevant open and
closed PR searches were inspected before implementation. No matching repair
was found in that bounded coverage; this is not a proof of novelty.

This extends the exact-output boundary in Ali Romman's #5378 and Mihir
Srivastava's #5520, preserving Nick Petalas's #6179 source-bound conformance
and benchmark. The omission of `selection_context` was already observed in
#4906's startup audit; that work repaired the generic command route. This
change adds source/claim reconciliation and unknown-context handling rather
than claiming discovery of the earlier omission. No pending construction is a
dependency.

**Open next question:** should the next maintainer priority assessment revise
the catalogue's tasks as well as its selected context? The comparison now makes
the old assessment visible, but cannot establish whether its advice remains
applicable. First inspect each affected card's current source-specific coverage
and live handoffs; a substantive strategic reassessment and dependent-input
review are required before rewriting frozen catalogue bytes.

Prepared by **Sricharan Vagvala (@theon199)** with substantial **OpenAI Codex**
assistance. Credit Douglas Colkitt, Ali Romman, Mihir Srivastava, Nick Petalas,
ToikY and the
retained picker/test contributors; DreamingOfClouds and the selected source's
[contributor map](community-p1046030-contributors.md) retain all construction
credit. Apache-2.0. No independent human or proof-kernel review is claimed.
89 changes: 89 additions & 0 deletions scripts/choose_contextual_research_task.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""Show catalogue assessment drift alongside the current task-picker receipt.

The frozen current and legacy pickers retain their source and behavior.
Sricharan Vagvala, with substantial OpenAI Codex assistance. Apache-2.0.
"""
from fractions import Fraction
from hashlib import sha256
import json
from pathlib import Path
import subprocess
import sys

ROOT = Path(__file__).resolve().parents[1]
CURRENT = ROOT / "scripts/choose_current_research_task.py"


def catalog_selection_context(result, root):
"""Compare declared context, without interpreting it as scientific coverage."""
documents = []
for name, digest_key in (
("docs/research/agent-task-catalog.json", "catalog_sha256"),
("certificates/selected-result.json", "selected_record_sha256"),
):
raw = (root / name).read_bytes()
if sha256(raw).hexdigest() != result[digest_key]:
raise ValueError("input changed since task selection: " + name)
documents.append(json.loads(raw))
catalog, selected = documents
context = catalog.get("selection_context")
if context is not None and not isinstance(context, dict):
raise ValueError("catalogue selection_context must be an object")
differences, missing = {}, []
for field, selected_field in (
("kappa", "kappa"), ("source_commit", "audited_source_head"),
("review", "review"),
):
before = (context or {}).get(field)
after = selected.get(selected_field)
if before is None or after is None:
missing.append(field)
continue
if not all(isinstance(value, str) and value.strip() for value in (before, after)):
raise ValueError("invalid selection context field: " + field)
if field == "kappa":
try:
equal = Fraction(before) == Fraction(after)
except (ValueError, ZeroDivisionError) as exc:
raise ValueError("invalid selection context kappa") from exc
else:
equal = before == after
if not equal:
differences[field] = {"catalog": before, "selected": after}
return {
"context": context,
"matches_selected": False if differences else (None if missing else True),
"differences": differences,
"uncompared_fields": missing,
"scope": (
"Compare only kappa, source_commit/audited_source_head and review. "
"Missing fields are unknown. The catalogue context is historical; "
"use selected for the current result and replay. Matching metadata "
"does not establish current task coverage or scientific validation. "
"Input hashes bind this comparison to the picker receipt, not the "
"complete selected source closure."
),
}


def main():
child = subprocess.run([sys.executable, "-B", str(CURRENT), *sys.argv[1:]],
capture_output=True, text=True, encoding="utf-8")
if child.returncode or any(arg in ("-h", "--help") for arg in sys.argv[1:]):
sys.stdout.write(child.stdout)
sys.stderr.write(child.stderr)
return child.returncode
sys.stderr.write(child.stderr)
try:
result = json.loads(child.stdout)
result["catalog_selection"] = catalog_selection_context(result, ROOT)
except (OSError, ValueError, KeyError, TypeError) as exc:
sys.stderr.write("catalogue selection comparison failed: " + str(exc) + "\n")
return 2
print(json.dumps(result, indent=2))
return 0


if __name__ == "__main__":
raise SystemExit(main())
2 changes: 1 addition & 1 deletion tests/test_research_coordination.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ def test_root_entry_and_priorities_links(self):
self.assertIn('docs/research/current-priorities.md',root)
self.assertIn('docs/research/agent-procedure.md',root)
procedure=(ROOT/'docs/research/agent-procedure.md').read_text()
self.assertLess(procedure.index('**Before drawing a card**'),procedure.index('python3 scripts/choose_current_research_task.py'))
self.assertLess(procedure.index('**Before drawing a card**'),procedure.index('python3 scripts/choose_contextual_research_task.py'))


if __name__=='__main__':unittest.main()
Loading