2024 edition with `nest_in_file_class = YES` throws `IllegalStateException` when used in JUnit test

[zbrtr]

What version of protobuf and what language are you using?

Version: 35.1, but can also reproduce on 36.0-rc1

Language: Java 17

What supported operating system version are you using (e.g. Linux, Windows) ?

Windows 11, but can also reproduce under Ubuntu 22.04

What supported runtime / compiler version are you using (e.g. python version, gcc version)

Java 17, JUnit 5, runtime 4.35.1

What did you do?

We have the following proto3 file:

// src/main/proto/item3.proto
syntax = "proto3";

package w;

option java_multiple_files = false;
option java_package = "com.company.x.proto";
option java_outer_classname = "ItemProto";

message Item {
  optional string name = 1;
}

We are trying to migrate to edition 2024, which, from my understanding, should be done as so:

// src/main/proto/item2024.proto
edition = "2024";

package w;

import "google/protobuf/java_features.proto";

option java_package = "com.company.x.proto";
option java_outer_classname = "ItemProto";

message Item {
  option features.(pb.java).nest_in_file_class = YES;

  string name = 1;
}

However, unlike the one generated by the proto3 file, the class generated by the 2024 edition causes the following JUnit 5 test to throw the following exception:

package com.company.x;

import org.junit.jupiter.api.Test;

import com.company.x.proto.ItemProto;

public class ItemTest {
    @Test
    public void testMyThing() {
        ItemProto.Item item = ItemProto.Item.newBuilder().setName("foobar").build();
        System.out.println("Hello world!" + item);
    }
}

com.company.x.ItemTest.testMyThing()  Time elapsed: 0.143 sec  <<< FAILURE!
java.lang.AssertionError: java.lang.IllegalStateException: getDescriptor() called before internalInit()
        at com.google.protobuf.Descriptors.getJavaEditionDefaults(Descriptors.java:106)
        at com.google.protobuf.Descriptors.getEditionDefaults(Descriptors.java:115)
        at com.google.protobuf.Descriptors$GenericDescriptor.resolveFeatures(Descriptors.java:3198)
        at com.google.protobuf.Descriptors$FileDescriptor.resolveAllFeaturesInternal(Descriptors.java:769)
        at com.google.protobuf.Descriptors$FileDescriptor.resolveAllFeaturesImmutable(Descriptors.java:750)
        at com.google.protobuf.DescriptorProtos.<clinit>(DescriptorProtos.java:49678)
        at com.google.protobuf.JavaFeaturesProto.<clinit>(JavaFeaturesProto.java:1577)
        at com.company.x.proto.ItemProto.<clinit>(ItemProto.java:599)
        at com.company.x.proto.ItemProto$Item.getDescriptorForType(ItemProto.java:82)
        at com.google.protobuf.TextFormat$Printer.print(TextFormat.java:384)
        at com.google.protobuf.TextFormat$Printer.printToString(TextFormat.java:684)
        at com.google.protobuf.AbstractMessage.toString(AbstractMessage.java:95)
        at java.base/java.lang.String.valueOf(String.java:4220)
        at com.company.x.ItemTest.testMyThing(ItemTest.java:11)
Caused by: java.lang.IllegalStateException: getDescriptor() called before internalInit()
        at com.google.protobuf.GeneratedMessage$GeneratedExtension.getDescriptor(GeneratedMessage.java:1980)
        at com.google.protobuf.ExtensionRegistry.newExtensionInfo(ExtensionRegistry.java:212)
        at com.google.protobuf.ExtensionRegistry.add(ExtensionRegistry.java:203)
        at com.google.protobuf.ExtensionRegistry.add(ExtensionRegistry.java:208)
        at com.google.protobuf.Descriptors.getJavaEditionDefaults(Descriptors.java:99)
        ... 31 more

Maven POM for reference:

<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.company.x.y.z</groupId>
    <artifactId>my-app</artifactId>
    <version>1</version>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>4.35.1</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.14.4</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

Steps to reproduce the behavior:

1. Download and extract proto-repro.zip or create the above files manually
2. protoc --java_out src/main/java src/main/proto/item3.proto
3. mvn test -> succeeds
4. protoc --java_out src/main/java src/main/proto/item2024.proto
5. mvn test -> produces above error

Notably, this issue only occurs when features.(pb.java).nest_in_file_class = YES is set; removing the option (and updating the test accordingly) works fine.

What did you expect to see

No exception being thrown.

[esrauchg]

The issue listed looks very surprising (and somewhat suspicious that something could be that broken somehow without us noticing until now).

Just to try to gather some more info so we may have something to go on here:

1. Can you run protoc --version. Note that it would absolutely not be working as intended if an old protoc and new runtime break the way you have listed, but it might help narrow down somewhere (including possibly a previous bug that was fixed)

2. Can you double check that there's not other lingering .java gencode around in those directories in between? One off the cuff theory to rule out is if you had protoc and made some files, then ran protoc again with a similar but not identical message and it only created some subset of those files, that the mixed combination when doing javac *.java could be arbitrarily bad combination

[zhangskz]

We mostly use import option to import java_features.proto in our tests which should avoid this problem (and is also recommended anyways for java_features.proto and all custom options).

We should validate that this reproduces (to rule out the types of problems @esrauchg pointed out), and if this also reproduces at older editions or with a different feature. This is a bit surprising, since we should've had coverage for direct imports of java_features.proto prior to import option even being added.

[zbrtr]

Can you run protoc --version. Note that it would absolutely not be working as intended if an old protoc and new runtime break the way you have listed, but it might help narrow down somewhere (including possibly a previous bug that was fixed)

libprotoc 35.1

Can you double check that there's not other lingering .java gencode around in those directories in between?

Yes, I'm positive there's no other code. I can even reproduce this with nothing but the above files (plus protoc curl'd from the GH Releases page) in a blank LXC container.

We mostly use import option to import java_features.proto in our tests which should avoid this problem (and is also recommended anyways for java_features.proto and all custom options).

I see; I cannot reproduce if I use import option. It would likely be a good idea to update https://protobuf.dev/editions/features/ to use import option in the example snippets, since that's what I was following.

[shaod2]

Thanks for the confirmation.

Yeah I'll go ahead to update the recommendation to using import option for now, but the real issue under the hood should be fixed and we should also have test coverage for that