Skip to content

Follow-ups from #805: the memory bound after the per-request cap, epoll's closing-conn reap by read time, no deterministic H2 refusal test, io_uring's dormant WRITEV path, body nits #818

Description

@FumingPower3925

Follow-ups from the round-1 review of #805 (celeris#761, celeris#802), none blocking. The blocking findings (pooled bodies reused before they were sent, the io_uring closing drain's 5 s cut, a streamed response cut at the cap, a backlog test that could not fail, CI time) are fixed in #805 itself; so are two minors: a failed read of a staged file now closes the conn (writeRefused) instead of leaving a header block with no body, and the gated TestWriteBufBackpressureClosesSlowConsumer asserts the server's close, not the client's view of the orphaned send queue.

  1. The memory bound moved; state or track it. An HTTP/2 connection may queue 64 MiB before a write is refused (16 times the H1 limit), and a peer that grants large windows and stops reading can pin that much per connection (maxPendingBytesH2, maxSendQueueBytesH2). A slow reader of a large HTTP/1 response holds a copy of the unsent tail (the flush copies the remainder into writeBuf). c.File responses pipelined behind one another are read whole into writeBuf by unstage, on the loop thread, with no cap check in makeSendFileFn (the case already existed through bufferedFileFallback). And a StreamWriter used without Detach on epoll or io_uring is buffered whole until its handler returns (Flush is a no-op on the H1 adapter): since fix(epoll, iouring): send a large response whole, keep pipelined responses in order, never send a body the handler has given back (celeris#761, celeris#802, celeris#817) #805 it is delivered however large it is, where it was cut at 4 MiB, but its memory is the whole response. The docs' streaming page says "Flush guarantees the bytes are on the wire", which is not true there.
  2. epoll reaps a conn left to closeWhenFlushed by ReadTimeout/IdleTimeout measured from the last read. lastActivity is stamped only by reads (engine/epoll/loop.go, drainRead), and checkTimeouts gives a non-dirty conn ReadTimeout, so a large response that a client reads steadily for longer than ReadTimeout (60 s by default) is cut. It predates fix(epoll, iouring): send a large response whole, keep pipelined responses in order, never send a body the handler has given back (celeris#761, celeris#802, celeris#817) #805 but is now reachable, since responses over 4 MiB are delivered. (fix(epoll, iouring): send a large response whole, keep pipelined responses in order, never send a body the handler has given back (celeris#761, celeris#802, celeris#817) #805 makes io_uring's closing drain restart its clock on send progress.)
  3. The HTTP/2 parts of fix(epoll, iouring): send a large response whole, keep pipelined responses in order, never send a body the handler has given back (celeris#761, celeris#802, celeris#817) #805's fix are caught only now and then. Neither the 64 MiB HTTP/2 cap nor the pendingBytes resync after the H2 write-queue flush has a deterministic test: in review, a mutant of each was missed in at least one of two TestLargeResponseIsDeliveredH2 runs, and the resync mutant's one failure was a 5 s stall with the connection open, so the H2 refused-write close path (closeWhenFlushed from the H2 queue loop) has no test. (On the real head the reviewer could not make an H2 refusal happen at all: a peer granting 2^31-1 windows and reading nothing for 2 s still got all 96 MiB.)
  4. io_uring's WRITEV scatter-gather path is unreachable now. fix(epoll, iouring): send a large response whole, keep pipelined responses in order, never send a body the handler has given back (celeris#761, celeris#802, celeris#817) #805 stops installing io_uring's zero-copy body writer (celeris#817): the kernel read the handler's buffer only at the next submit. bodyBuf, sendBody, iov and the WRITEV branches of flushSend and completeSend remain, dormant. Remove them, or bring the path back for bodies the engine may keep (a body backed by an immutable string, from c.String or c.HTML, or a buffer whose release the engine controls).
  5. Nit (body wording): the round-1 body's per-cell H2 failing-first numbers were one sample of a timing-dependent result on epoll and adaptive (io_uring's fail every time); and its cost line inferred "well under the end-to-end noise floor" from a micro-benchmark, with the benchmark-tier row queued, not run, and the per-turn csPendingBytes resync in the H2 queue loop not measured.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/engineEngine interface or implementationbugSomething isn't workingengine/epollEpoll engine specificsengine/iouringio_uring engine specifics

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions