fix(dashboard): stream HTTP response bodies to EOF instead of one chunk (#4829)

[iamwhatever]

Problem / Motivation

Three dashboard HTTP readers read a response body with a single
aiohttp.StreamReader.read(cap + 1). read(n) returns up to n bytes,
resolving as soon as any data is buffered rather than waiting for EOF. On a
chunked response with no Content-Length it returns only the first buffered
chunk (~8–12 KiB), so the caller silently works on a truncated body:

• updates.py::_fetch_feed_bytes — the release feed is parsed from a partial
  document, so the update check can misread or fail on a chunked feed. The
  site also carried a comment claiming an oversized body "is DETECTED rather
  than silently truncated", which was false for a chunked feed.
• feedback.py::_read_capped_text — a truncated Aperture response is decoded
  and treated as the full reply.
• source_providers.py (Jira fetch) — json.loads on a partial Jira document
  surfaces as SourceProviderError, so the issue reads as unavailable rather
  than a parse bug.

All three fail silently, reading as "the remote had nothing". Same defect
class as #2222 and the official-registry site fixed in #4814.

Why it matters

Update checks, feedback submission acknowledgements, and Jira issue loading
each degrade nondeterministically depending on how the remote frames its
response — with no visible error pointing at the real cause. The failure only
appears with chunked transfer encoding, which makes it environment-dependent
and expensive to diagnose.

What changed (motivation → approach → change)

Symptom: bodies truncated to the first buffered chunk. Root cause: a single
read(n) cannot deliver a streamed body; the repo's established fix shape
(#4814, official.py::_fetch_json) drains iter_chunked to EOF with the cap
enforced against the accumulated total.

Because the single-read shape has now been reintroduced independently more
than once, the fix extracts one shared helper instead of three
near-copies: read_capped_response(resp, cap) in
dashboard/handlers/_shared.py, right next to the request-side
read_bounded_json that already consolidated the inbound twin of this
pattern. It streams to EOF, stops reading as soon as the accumulated total
exceeds the cap (an oversized body is refused mid-stream, never buffered
whole), and clamps the return to cap + 1 bytes so every caller's existing
over-cap sentinel (len(body) > cap) keeps working unchanged. All three
sites route through it; cap values and caller error semantics are untouched.
The stale updates.py comment is corrected to describe what the code now
actually guarantees.

Tests

test/test_read_capped_response.py (new), with a fake StreamReader whose
read(n) honors the real "up to n bytes" contract so a single-read
implementation is exercised and fails (all caller tests were proven red
against the pre-fix code):

• helper: a multi-chunk body is assembled whole to EOF
• helper: a body of exactly cap bytes arrives complete (sentinel stays off)
• helper: an over-cap body returns cap + 1 bytes and stops reading
  mid-stream (undelivered bytes are never consumed)
• feedback._read_capped_text: multi-chunk body decoded whole; over-cap body
  still raises ValueError
• updates._fetch_feed_bytes: chunked feed read whole; oversized feed still
  trips the caller's len(raw) > _FEED_MAX_BYTES check without buffering the
  whole body
• Jira fetch: a multi-chunk Jira document reaches json.loads complete and
  parses into the normalized issue payload

Manual verification

N/A — unit coverage sufficient: the fakes reproduce the exact chunked-stream
semantics of aiohttp.StreamReader, and the fix is confined to the read loop.

Related Issues

Closes #4829

Checklist

• Single commit with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
• Existing tests pass and new tests added for new functionality
• Self-review completed; code follows project style guidelines
• Documentation updated (if applicable) — N/A: caps and caller semantics unchanged; only the read mechanics moved to the documented fix(mcp): read official registry responses to EOF instead of one chunk #4814 shape
• No secrets, credentials, or internal references in the diff

[github-actions]

Design Review (Fable 5) — ✅ PASS

Design-level review of 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Real, recurring defect class; the fix reuses the repo's established streaming shape, consolidates three near-copies into one helper, and preserves every caller's cap sentinel.

[DESIGN-REVIEWED] 86a8d8f

[github-actions]

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 86a8d8f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7: <one-sentence reason>

[github-actions]

Opus 4.8 Review — ✅ no blocking findings

Reviewed 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 86a8d8f

Verdict parsed from the review's SHA-scoped output markers for commit 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7: <one-sentence reason>

[github-actions]

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 86a8d8ffb203dfd80b0d99161d4de479fec6dcc7 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All evidence gathered. The three buggy single-read(n) sites are all fixed, the helper has three real consumers, and I grepped for unfixed siblings of the root cause (.content.read( across src/): the remaining five hits are all bounded loops on the request side or correct manual drains (l0_probe.py, hooks.py, ops_mission_control, mochi, pptx_maker), so zero buggy siblings remain. The other cap-drain implementations (official.py, petdex_import.py, link_meta.py) predate this PR, carry different error semantics, and sit in layers that cannot import dashboard/handlers/_shared without inversion — not second spellings introduced here.

First-Principles-Verdict: PASS

Every item is the declared fix for a counted defect class, the helper has three real consumers, and zero buggy siblings remain unfixed.

What this change ships

Intent: stop three dashboard fetches from silently working on the first chunk of a chunked HTTP body — a FIX.

1. Update check now reads a chunked release feed whole — justified (declared defect, Three HTTP readers truncate chunked responses via a single StreamReader.read() #4829)
2. Feedback acknowledgement now decodes the full Aperture reply — justified
3. Jira issue fetch now hands json.loads a complete document — justified
4. New shared helper read_capped_response in _shared.py — justified; 3 counted consumers (feedback.py, source_providers.py, updates.py), grepped read_capped_response
5. Stale "is DETECTED" comment in updates.py corrected — part of the fix, declared
6. New test file pinning the stream-to-EOF contract — declared

Sibling count for the root cause: grepped .content.read( across src/ — 5 remaining hits (hooks.py:416, l0_probe.py:195, ops_mission_control/backend/routes.py:2195, mochi/backend/routes.py:1192, pptx_maker/backend/routes.py:479) are all already bounded drain-to-EOF loops, so the fix covers every instance of the defect. The helper sits at cause level within its layer: it removes the reintroduction path the description documents (the same single-read shape landed independently three times), placed beside read_bounded_json, the inbound twin that already consolidated this pattern. The extractions it does not attempt (official.py:_fetch_json, petdex_import._read_capped, link_meta._read_capped) carry different over-cap semantics (raise vs. sentinel vs. truncate-mode) and live in layers that cannot import dashboard handlers — not duplicates.

[FIRST-PRINCIPLES-REVIEWED] 86a8d8f

[iamwhatever]

🤖 Kiro Crew Auto-Pipeline [operator: iamwhatever]

Review-ready at 86a8d8f: all checks green on the first push, readiness: passed,
MERGEABLE. All four AI reviews are clean — GPT 5.6 no blocking findings, Opus 4.8
no findings, Design Review PASS, First Principles PASS — and there are no
unresolved threads. Awaiting human review + merge.

[bolichen97]

Picking this over #4836 — two things to port first

#4836 fixes the same issue (#4829) at the same three call sites (updates.py::_fetch_feed_bytes, feedback.py::_read_capped_text, source_providers.py::_fetch_jira_issue) with equivalent semantics: 64 KiB chunks read to EOF, cap enforced against the accumulated total, cap + 1 overflow sentinel preserved. Neither PR misses a site the other covers.

This PR is the one to land, because it extracts one shared read_capped_response(resp, cap) into handlers/_shared.py — next to the request-side read_bounded_json that already consolidated the inbound twin of this pattern — where #4836 repeats the same accumulate-and-check loop three times and defines _HTTP_READ_CHUNK_BYTES three times. That duplication is precisely what allowed this defect to be introduced independently three times now (#2222, #4814, #4829), so collapsing it to one entry point is what actually closes the recurrence path rather than just this instance.

Two things from #4836 are worth carrying over before this merges:

1. A Jira over-cap test. TestJiraFetchStreams here has only test_multi_chunk_jira_response_parses_whole. fix(dashboard): read chunked responses to EOF #4836 has test_accumulated_cap_stops_before_draining_tail, which proves the _MAX_PAYLOAD_BYTES path still raises SourceProviderError and stops mid-stream. This PR covers that case for feedback and updates but not Jira.
2. The system-spec paragraph. fix(dashboard): read chunked responses to EOF #4836 adds a "Bounded HTTP response reads" section to docs/system-specs/modules/learn-cron-dashboard.md. This PR declared docs N/A — but since this is the PR that creates the shared helper, that paragraph belongs here, reworded to name read_capped_response as the single entry point.

Credit for both to @leonlaiyc (#4836).

Approving now; holding auto-merge until those two land so they are not lost.

[bolichen97]

Approved after a description-vs-diff consistency review: every claim in the PR description is backed by the diff, and the diff carries no material change the description leaves unmentioned.