fix: don't fail responses whose body was capped by -response-size-to-read - #2642
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Essentials Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. Walkthrough
ChangesTruncated response handling
Estimated code review effort: 2 (Simple) | ~10 minutes Severity of issue fixed: Medium Suggested reviewers: Merge Risk: ⚪ Minimal · up to Capped responses now retain their successful HTTP status and metadata while keeping the configured body-size limit. The regression coverage shows no actionable merge-blocking risk remains. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
A rabbit trims the stream with care Comment |
Fixes #2641
What
A response whose body was capped by
-response-size-to-read(or by the default 50MB cap) was reported asstatus_code: 0, failed: truewithContentLength=100194 with Body length 10.The cap itself is intentional: since 78bf9c3 the body is wrapped in a
LimitReaderbeforepdhttputil.DumpResponseHeadersAndRaw, so buildingresp.Rawcannot load an unbounded body or an infinite stream in memory. The side effect was not: that dump callsresp.Write(), and net/http refuses to serialize a response whose body is shorter than the advertisedContent-Length, so httpx treated our own truncation as a transport error and dropped the whole result.When the truncation is ours (
ContentLength > MaxResponseBodySizeToRead), that serialization error is now tolerated. Headers and the truncated body are already dumped correctly at that point, soresp.Rawstays usable. The cap is untouched, memory stays bounded, and a server genuinely lying aboutContent-Lengthstill errors as before.Verification
TestDoTruncatedBodyIsNotAFailurefails without the fix (ContentLength=4096 with Body length 10) and passes with it. Full./common/httpx/...suite is green.Against the URL from the issue:
raw_headerkeepsContent-Length: 100194and the body holds the bytes actually read.Summary by CodeRabbit