perf(store): read the operations journal from its end

[tom-sapletta-com]

operational_events parsed the whole append-only journal to return its newest rows. On the observed store that is 379 MB and ~170k json.loads calls for a limit=200 request — on the single-worker uvicorn event loop that also serves ticket writes, so every GET /operations stalled every concurrent ticket create and patch.

Change

The read walks backwards in chunks and stops once it has limit rows.

Measured on a 397 MB journal with the same line geometry (~2.2 KB/line):

limit=200, unfiltered:   1.163 s  ->  0.004 s   (268x)
output:                  byte-identical

A max_bytes budget (256 MiB, clamped to the chunk size) bounds the walk so a filter matching nothing recent degrades to a short read instead of a full scan. Such a query returns the matches inside that window rather than every match in history; an unfiltered read fills its limit from the last few kilobytes and never reaches the budget.

Blank-line, malformed-line, missing-file and no-trailing-newline handling are unchanged.

Validation

Full suite: 396 passed, 6 skipped, 0 failed. 23 new tests, including 12 randomized cases asserting row-for-row agreement with the forward reader this replaces, across chunk boundaries, rows longer than a chunk, and files with and without a trailing newline.

🤖 Generated with Claude Code

https://claude.ai/code/session_015QJeEkvVSGaDJyaxu4aoRL

Publication continuation: integrated current main without rewriting original history. The complete locked Python 3.13 suite passes: 463 passed, 6 skipped; all 23 journal regression cases pass. Docker Compose configuration passes. Current hosted Python 3.10/3.13 and existing CI gates must pass before independent exact-head Validator review.

Ticket: ticket-057. Closes #57.
Correlation ID: planfile-pr51-ticket057-publication-20260908

[ifuri-validator-agent]

Deterministic Validator approval for exact head 8cf64d57ac60498c73350ddd90bef2fa36c5a5b6.

Ticket: ticket-057
Correlation ID: local-semcod-planfile-pr-51-ticket-057
Model: zai/glm-5.3
Reviewed diff chunks: 2
Advisory LLM verdict: APPROVE
Advisory summary: Reviewed all 2 diff chunk(s). Replaces a full-file forward scan of the operations journal with a bounded reverse-walk (read_l_tail). Chunk-boundary, carry, budget-clamping, and no-trailing-newline logic are all correct; behavior parity with the forward reader is asserted by 23 tests including randomized oracle comparisons. All CI checks pass. | Chunk 2 contains only randomized test code comparing the backward tail reader against a forward reference implementation across blank lines, malformed JSON, chunk boundaries, and optional trailing newlines. No production code in this chunk. All CI checks pass (396 passed, 0 failed).
Advisory findings: none
The LLM output above is advisory and was not used as the approval trust root.

Actual PR impact radar

Exact range: 830d18f1bfff665229e7b624162ca00afd80d93c...8cf64d57ac60498c73350ddd90bef2fa36c5a5b6
Change digest: 78311d58a158185601af0b075f98f2090c3353f487da36fba2b0a73d50996813
Score: 60/100 (L), estimated 69 min, split recommended: true
Affected services/components: repository-wide/unclassified

Machine-readable radar JSONL and SVG

{"actual_change":{"additions":292,"base_sha":"830d18f1bfff665229e7b624162ca00afd80d93c","binary_files":0,"categories":{"code":2,"tests":1},"change_digest":"78311d58a158185601af0b075f98f2090c3353f487da36fba2b0a73d50996813","comparison":"830d18f1bfff665229e7b624162ca00afd80d93c...8cf64d57ac60498c73350ddd90bef2fa36c5a5b6","deletions":21,"file_count":3,"files":["planfile/core/jsonl_tail.py","planfile/core/store.py","tests/test_jsonl_tail.py"],"head_sha":"8cf64d57ac60498c73350ddd90bef2fa36c5a5b6","service_count":0,"services":[]},"assessment_mode":"observed-pr","axes":{"coupling":5,"delivery":2,"scope":4,"uncertainty":3,"validation":1},"complexity":"L","confidence":0.9,"diagnostics":["RADAR-ACCEPTANCE-MISSING","RADAR-BUDGET-EXCEEDED"],"estimate":{"budget_minutes":30,"minutes":69,"within_budget":false},"impact":{"components":["3.10","KB","claude.ai","claude.com","planfile","tests"],"files":["3.10/3.13","KB/line","claude.ai/code/session_015QJeEkvVSGaDJyaxu4aoRL","claude.com/claude-code","planfile/core/jsonl_tail.py","planfile/core/store.py","tests/test_jsonl_tail.py"],"public_interfaces":[],"runtime_dependencies":0},"schema":"subactor.ticket-radar/v1","score":60,"split":{"parts":[{"estimated_minutes":10,"name":"Implement 3.10","scope":["3.10"]},{"estimated_minutes":10,"name":"Implement KB","scope":["KB"]},{"estimated_minutes":10,"name":"Implement claude.ai","scope":["claude.ai"]},{"estimated_minutes":10,"name":"Implement claude.com","scope":["claude.com"]},{"estimated_minutes":10,"name":"Implement planfile","scope":["planfile"]},{"estimated_minutes":15,"name":"Validate and project to trackers","scope":["tests","planfile","github/gitlab/jira projections"]}],"reason":"estimated_minutes_exceed_budget","recommended":true},"standards":[{"id":"wellmanifest/dsl","revision":"6c60fc4e0dd1f1bb74f46a7745e28019908d1203","version":"0.1.0-dev"},{"id":"wellmanifest/ticket-lifecycle","revision":"5bf581907a87b46a13a73e6c033d3abe4d9a306f","version":"0.1.0-dev"},{"id":"wellmanifest/git-lifecycle","revision":"7d77d4b7af57e69bc75c3a0290b3a4805c5c4438","version":"0.2.0-dev"},{"id":"wellmanifest/logs","revision":"48c284ef7a069055c0bcb6b900147ce5e65f8b43","version":"0.3.0"}],"ticket_ref":"ticket-057"}

<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 128 128" role="img"><title>ticket-057: perf(store): read the operations journal from its end</title><rect width="128" height="128" rx="12" fill="#f8fafc"/><g stroke-width="1"><polygon points="64,55 72,61 69,71 59,71 56,61" fill="none" stroke="#d7dde5"/><polygon points="64,47 80,59 74,78 54,78 48,59" fill="none" stroke="#d7dde5"/><polygon points="64,38 89,56 79,85 49,85 39,56" fill="none" stroke="#d7dde5"/><polygon points="64,30 97,53 84,92 44,92 31,53" fill="none" stroke="#d7dde5"/><polygon points="64,21 105,51 89,99 39,99 23,51" fill="none" stroke="#d7dde5"/><line x1="64" y1="64" x2="64" y2="21" stroke="#aab4c0"/><line x1="64" y1="64" x2="105" y2="51" stroke="#aab4c0"/><line x1="64" y1="64" x2="89" y2="99" stroke="#aab4c0"/><line x1="64" y1="64" x2="39" y2="99" stroke="#aab4c0"/><line x1="64" y1="64" x2="23" y2="51" stroke="#aab4c0"/></g><polygon points="64,30 105,51 79,85 59,71 48,59" fill="#fb923c" fill-opacity="0.45" stroke="#c2410c" stroke-width="2"/><circle cx="64" cy="64" r="3" fill="#c2410c"/><g font-family="sans-serif" font-size="7" fill="#334155"><text x="64" y="11" text-anchor="middle">SCO</text><text x="114" y="48" text-anchor="middle">COU</text><text x="95" y="107" text-anchor="middle">UNC</text><text x="33" y="107" text-anchor="middle">VAL</text><text x="14" y="48" text-anchor="middle">DEL</text></g><text x="64" y="124" text-anchor="middle" font-family="sans-serif" font-size="8" fill="#0f172a">L · 69m</text></svg>

Merge will be attempted after this approval when explicitly authorized. ## Decision record (recomputable)

DECISION D-057-5276
TICKET ticket-057
HEAD_SHA 8cf64d57ac60498c73350ddd90bef2fa36c5a5b6
CORRELATION_ID local-semcod-planfile-pr-51-ticket-057
ACTOR agent:ifuri-validator-agent[bot]
APPLIED_RULE P-CORE-015
INPUT author_login = "tom-sapletta-com"
INPUT observed_checks = ["notify=PASS","test (3.13)=PASS","test (3.10)=PASS","ci-loop=PASS",".github/dependabot.yml=PASS"]
INPUT required_checks = ["ci-loop","notify"]
INPUT required_checks_source = "protected registry (env/request)"
INPUT reviewer_login = "ifuri-validator-agent[bot]"
INPUT superseded_checks = []
INPUT ticket_radar_receipt = {"schema":"subactor.ticket-radar/v1","base_sha":"830d18f1bfff665229e7b624162ca00afd80d93c","head_sha":"8cf64d57ac60498c73350ddd90bef2fa36c5a5b6","change_digest":"78311d58a158185601af0b075f98f2090c3353f487da36fba2b0a73d50996813","score":60,"complexity":"L","estimated_minutes":69,"split_recommended":true,"services":[],"authority":"ADVISORY","promotion":"FORBIDDEN"}
VERDICT APPROVE AUTHORITY DETERMINISTIC
REJECTED REQUEST_CHANGES BECAUSE NO_UNSAFE_CHANGE_REASON_FOUND
ADVISORY llm_verdict = "APPROVE" MODEL "zai/glm-5.3"
ASSERT VERDICT_AUTHORITY != "ADVISORY"