[Bug] APIServer reuses released ByteBufs for validation error responses

[MrlixiangWE]

Describe the bug

PubHandler and KillHandler store several validation-error bodies in static ByteBuf fields and pass the same buffer to every DefaultFullHttpResponse. A full response owns its content, so Netty releases that buffer after the response is written. The next request that reaches the same validation path receives the already-released buffer and cannot return the documented error body.

On current main, the second invalid-QoS response fails with:

io.netty.util.IllegalReferenceCountException: refCnt: 0

Expected behavior: every request should receive an independently valid 400 or 403 response, regardless of previous requests.

Environment

• Version: current main at a92c5a51245596bf28b0b9241b85519e77dc4142
• JVM Version: Amazon Corretto 17.0.20
• Hardware Spec: Apple Silicon, arm64
• OS: macOS Darwin 25.5.0
• Testing Tools: focused TestNG regression test through Maven Wrapper 3.9.9

Reproducible Steps

Add this test to PubHandlerTest:

@Test
public void repeatedInvalidQoSResponseHasReadableContent() {
    DefaultFullHttpRequest req = buildRequest();
    req.headers().set(HEADER_TOPIC.header, "/greeting");
    req.headers().set(HEADER_CLIENT_TYPE.header, "admin_team");
    req.headers().set(HEADER_QOS.header, "3");

    PubHandler handler = new PubHandler(settingProvider, distClient);
    FullHttpResponse firstResponse = handler.handle(123, "bifromq_dev", req).join();
    firstResponse.release();

    FullHttpResponse secondResponse = handler.handle(124, "bifromq_dev", req).join();
    assertEquals(secondResponse.content().toString(StandardCharsets.UTF_8), "Invalid QoS");
    secondResponse.release();
}

Run:

./mvnw -q -pl bifromq-apiserver \
  -Dtest=PubHandlerTest#repeatedInvalidQoSResponseHasReadableContent test

The test consistently fails while reading the second response because its content has refCnt == 0.

The same ownership pattern is used by five validation responses:

• PubHandler: unacceptable topic, invalid QoS, and invalid expiry seconds
• KillHandler: invalid server redirect and overlong server reference

Proposed fix

Create independently owned content for each response and add regression coverage that releases the first response before exercising the same validation path again. I can prepare the patch if this approach looks right.

Publicly Accessible Diagnostic Data(If Reproducible Steps are not available)

Not required; the focused source-level test above reproduces the issue without deployment-specific data or custom plugins.

Additional context

This is separate from #264 / #266. That change removed an unmatched retain on inbound FullHttpRequest objects in TenantAwareHandler; this report concerns reuse of static outbound response-content buffers in PubHandler and KillHandler.

[popduke]

Thx for the report and the fix.