Unchecked varint64 reader is incompatible with Netty leak-aware composite buffers

[lhotari]

Enabling Netty's paranoid leak detection exposes a compatibility problem in LightProto's generated unchecked varint64 reader for composite buffers. Valid broker-entry metadata parses with leak detection disabled, but fails with IndexOutOfBoundsException: Truncated protobuf message when the same composite buffer is wrapped for leak detection.

Versions

• LightProto: 0.8.1
• Netty: 4.2.18.Final (standalone reproduction)
• The direct reader-index field access is also present in the current master template.

Reproduction

The following standalone probe uses Apache Pulsar's BrokerEntryMetadata, generated by LightProto 0.8.1. Its relevant protobuf fields are:

message BrokerEntryMetadata {
    optional uint64 broker_timestamp = 1;
    optional uint64 index = 2;
}

Put the generated classes plus netty-buffer and netty-common on the classpath, then run the probe in separate JVMs with DISABLED and PARANOID as the argument:

import io.netty.buffer.*;
import io.netty.util.ResourceLeakDetector;
import org.apache.pulsar.common.api.proto.BrokerEntryMetadata;
public class CompositeMetadataProbe {
 public static void main(String[] args) {
  ResourceLeakDetector.setLevel(ResourceLeakDetector.Level.valueOf(args[0]));
  BrokerEntryMetadata metadata = new BrokerEntryMetadata().setIndex(9);
  ByteBuf header = PooledByteBufAllocator.DEFAULT.buffer();
  header.writeShort(0x0e02).writeInt(metadata.getSerializedSize());
  metadata.writeTo(header);
  ByteBuf payload = PooledByteBufAllocator.DEFAULT.buffer().writeBytes(new byte[1025]);
  CompositeByteBuf composite = PooledByteBufAllocator.DEFAULT.compositeBuffer();
  composite.addComponents(true, header, payload);
  System.out.println("buffer=" + composite.getClass().getName());
  composite.skipBytes(2); int size = composite.readInt();
  try {
   BrokerEntryMetadata parsed = new BrokerEntryMetadata();
   parsed.parseFrom(composite, size);
   System.out.println("PASS index=" + parsed.getIndex() + " readerIndex=" + composite.readerIndex());
  } catch (Exception e) { e.printStackTrace(); }
  finally { composite.release(); }
 }
}

Observed with DISABLED:

buffer=io.netty.buffer.CompositeByteBuf
PASS index=9 readerIndex=8

Observed with PARANOID:

buffer=io.netty.buffer.AdvancedLeakAwareCompositeByteBuf
java.lang.IndexOutOfBoundsException: Truncated protobuf message
    at org.apache.pulsar.common.api.proto.BrokerEntryMetadata.parseFrom(BrokerEntryMetadata.java:187)
    at CompositeMetadataProbe.main(CompositeMetadataProbe.java:18)

Expected: both modes parse index 9 and leave the reader index at 8. Leak detection should not change protobuf parsing behavior.

Additional buffer cases tested

The following cases were tested with LightProto 0.8.1-generated classes and Netty 4.2.18.Final:

Buffer case	Result
Composite buffer + SIMPLE leak detection, forced sampling	Fails with Truncated protobuf message
Composite buffer + ADVANCED leak detection, forced sampling	Fails with Truncated protobuf message
Composite buffer + PARANOID leak detection	Fails with Truncated protobuf message
Direct buffer + PARANOID leak detection	Passes: index 9, reader index 8
Heap buffer + PARANOID leak detection	Passes: index 9, reader index 8
Slice of composite buffer + PARANOID leak detection	Passes: index 9, reader index 8
Duplicate of composite buffer + PARANOID leak detection	Passes: index 9, reader index 8
Read-only view of composite buffer + PARANOID leak detection	Passes: index 9, reader index 8

For SIMPLE and ADVANCED, sampling was forced with -Dio.netty.leakDetection.samplingInterval=1. With normal sampling, the failure can be intermittent because only tracked composite allocations receive the problematic wrapper. This is not specific to PARANOID detection or test code: production parsing can also be affected if an affected generated parser receives a leak-aware composite buffer.

Root cause

LightProtoByteBufAccess.java directly reads and writes the protected AbstractByteBuf.readerIndex field:

int i = buf.readerIndex;
byte b0 = buf._getByte(i);
// ...
buf.readerIndex = i + 1;

The codec selects this path for any AbstractByteBuf. However, Netty's leak-aware composite buffers inherit from WrappedCompositeByteBuf, which delegates readerIndex() and readerIndex(int) to its wrapped buffer. The wrapper's inherited field is not the effective reader index. _getByte also delegates to the wrapped buffer, so the unchecked reader accesses bytes at the wrong position and does not advance the effective reader index.

A possible fix is to use the reader-index accessors, or limit the direct-field optimization to buffer implementations where the field is authoritative and use a compatible fallback for delegating wrappers. A regression test should cover a leak-aware composite buffer with a nonzero starting reader index.

Apache Pulsar CI impact

This was exposed by apache/pulsar#26605, which restores paranoid leak detection for Gradle tests:

• Other job: CommandUtilsTest.testAddBrokerEntryMetadataLargePayload fails with the truncated-protobuf exception, including on retry.
• Pulsar Client job: MessageImplTest.testMessageBrokerAndEntryMetadataTimestampMissed also fails on retry and uses the same composite-buffer parsing path. Its catch block discards the original exception, so that CI report does not independently confirm the underlying exception.

CI used NETTY_LEAK_DETECTION=report; these are parser/test failures, not jobs deliberately failed by detected leak reports.