Repository navigation
[SwiftExtract] Preserve source file path for deferred (out-of-order) extensions - #924
Conversation
|
Thanks for the contribution, looks fine I think. One thing I'd like to request for the future though is to cut down on the amount of wording in a PR description, it's a bit of a wall of text -- it is much easier to review if you summarize the problem using your own words in a few minimal sentences. |
0d2c6b6 to
b0bb56c
Compare
Sorry for the wall of text! It’s a habit of mine to write detailed PR descriptions. In my projects, I tend to include a lot of information to build solid documentation and provide context for future reference. I’ll keep it more concise in the future. Thanks for the tip! |
Problem
jextract's symbol table builder processes source files in a single pass, inwhatever order
FileManagerenumerates the input directory. If anextension Outer.Inner { ... }is visited before the file that declaresOuter(orInner), its extended type can't be resolved yet, so it's queuedin
unresolvedExtensionsand retried infinalize().When
finalize()resolves a deferred extension, it currently hardcodes thesource file path as
"FIXME_MISSING_FILEPATH.swift"(
SwiftParsedModuleSymbolTableBuilder.swift, with a// FIXMEcommentalready marking this).
Because the JNI thunk printer groups generated thunks by source file
(
writeSwiftThunkSources), everything with this fake path lands in a singleFIXME_MISSING_FILEPATH+SwiftJava.swift. The JExtractSwiftPlugin onlydeclares the real per-source-file
+SwiftJava.swiftoutputs to SwiftPM, sothis file is never compiled. The generated Java bindings are correct (the
Java class exists with all its methods, including
nativedeclarations), butthe corresponding Swift/JNI implementations are silently missing from the
built library — this fails at runtime with
UnsatisfiedLinkError/"No implementation found for ..." instead of at build time.
This is order-dependent, so it's easy to hit by accident: on a case-insensitive
APFS volume,
FileManager's directory enumeration order is filename-hashbased, not alphabetical, so whether an extension's file is read before or
after its extended type's declaring file depends on unrelated filenames
elsewhere in the module. For example,
extension Outer.Inner { struct X {} }in
Feature.swiftandenum Outer { enum Inner {} }inCore.swiftworkfine as long as
Core.swifthappens to be enumerated first — flip thatorder (as can happen simply by adding or renaming unrelated files in the
same directory) and
X's JNI thunks silently disappear from the build.Fix
SwiftParsedModuleSymbolTableBuilder.unresolvedExtensionsnow stores thesourceFilePathalongside each deferredExtensionDeclSyntax, andfinalize()passes that real path through totryHandle(extension:sourceFilePath:)instead of the placeholder string. Deferred extensions are now attributed
to their actual source file like any other declaration, so their JNI thunks
land in the correct, plugin-declared output file and actually get compiled.
collectAllFilesnow sorts the files gathered from each input path beforereturning them. Enumeration order was previously whatever the filesystem
handed back (non-deterministic across machines/filesystems for the reasons
above); sorting makes extraction output reproducible. This alone doesn't
fix the bug (many still-valid orderings would still defer extensions
declared in files before their type's declaring file), but it removes
filesystem/OS as a source of flakiness on top of fix (1).
Testing
Added
deferredExtensionKeepsSourceFilePathtoTests/SwiftExtractTests/SwiftSymbolTableTests.swift: it feeds the symboltable an extension file before the file declaring the extended type (mirroring
the real bug), then asserts the nested type's
sourceFilePathmatches theextension's actual file rather than the fixed fallback.
Ran the full
SwiftExtractTestssuite (swift test --filter SwiftExtractTests):91 tests, 11 suites, all passing.
Verified end-to-end against a real multi-target Swift package with a
JExtractSwiftPlugin-driven Android build: before the fix,HomeFeed'sJNI thunks (and everything else in the affected files) were generated into
FIXME_MISSING_FILEPATH+SwiftJava.swiftand silently dropped from the build;after the fix they're written to their real per-file output and the resulting
.soexports the expected JNI symbols.