Goal
Close the defects found while landing #1416, #1421, #1423, #1424, #1426 and #1429, then re-measure the benchmarks on the surface that actually shipped.
Issues
|
area |
why it matters |
| #1414 |
project add / readiness |
Blocks the benchmark re-run. project add registers without indexing and status reports ready vacuously, so an eval runs against an empty project and returns a plausible zero instead of an error. Already invalidated one A/B run as 0/12. |
| #1432 |
MCP routing |
read_note and search resolve an unqualified name into another accessible workspace. The POSIX verbs were fixed in #1421; these two kept the bare-prefix fallback because they have no mount table to refuse against. |
| #1435 |
MCP paths |
find --meta returns search results that skip the re-qualification ls/find/cat now do, so its results don't round-trip. |
| #1431 |
search |
Vector/hybrid filter intersection caps the filter scan at 50,000 rows and treats everything beyond as disallowed. Deterministically biased against newer content on Postgres. |
| #1437 |
search index |
A relation permalink can collide with an observation's. Postgres raises IntegrityError; SQLite silently keeps both under one permalink. Pre-existing on main. |
| #1430 |
CI / SQLite on Windows |
Three symptoms — a hang, a database is locked, and a 45-minute job timeout — all consistent with SQLite operations serializing on Windows. Makes CI unreliable for every PR. |
Approach
Independent PRs off main, not a stack. The stack that produced these issues cost three separate cascade repairs: a rebase of a lower branch orphans everything above it, and GitHub's stacked-merge auto-rebases the child on every merge, doing the same thing again. These six touch largely separate areas, so only the second-merged in an overlapping pair pays a rebase.
Grouping by area rather than one PR each, so a shared rule gets stated once:
Then
Re-measure the agent-task A/B on the merged surface. The headline 12/12 at −36% tokens was recorded on an earlier build, before the routing rework, the MetadataPath funnel, and field-projection validation. Carrying that number forward would be reporting a result for code that no longer exists.
Also outstanding on #1398: curated-ingestion BEAM mode (only the raw baseline exists), the xAFS audit, and the Moby Dick qualitative run.
Goal
Close the defects found while landing #1416, #1421, #1423, #1424, #1426 and #1429, then re-measure the benchmarks on the surface that actually shipped.
Issues
project add/ readinessproject addregisters without indexing andstatusreports ready vacuously, so an eval runs against an empty project and returns a plausible zero instead of an error. Already invalidated one A/B run as 0/12.read_noteandsearchresolve an unqualified name into another accessible workspace. The POSIX verbs were fixed in #1421; these two kept the bare-prefix fallback because they have no mount table to refuse against.find --metareturns search results that skip the re-qualificationls/find/catnow do, so its results don't round-trip.IntegrityError; SQLite silently keeps both under one permalink. Pre-existing onmain.database is locked, and a 45-minute job timeout — all consistent with SQLite operations serializing on Windows. Makes CI unreliable for every PR.Approach
Independent PRs off
main, not a stack. The stack that produced these issues cost three separate cascade repairs: a rebase of a lower branch orphans everything above it, and GitHub's stacked-merge auto-rebases the child on every merge, doing the same thing again. These six touch largely separate areas, so only the second-merged in an overlapping pair pays a rebase.Grouping by area rather than one PR each, so a shared rule gets stated once:
Then
Re-measure the agent-task A/B on the merged surface. The headline 12/12 at −36% tokens was recorded on an earlier build, before the routing rework, the
MetadataPathfunnel, and field-projection validation. Carrying that number forward would be reporting a result for code that no longer exists.Also outstanding on #1398: curated-ingestion BEAM mode (only the raw baseline exists), the xAFS audit, and the Moby Dick qualitative run.