Marshal const void* buffers, pass JNI includes, cross-compile ABI builds

CLAUDE.md

@@ -61,6 +61,18 @@ Bindings are `internal` by default and each carries **`@JvmName`**. Without it K
 internal functions (`kniBridge0$module`) while JNI resolves the C symbol from the unmangled method
 name: it compiles, links, and dies on the first call.
 
+**The bridges are renamed to the C function they call.** cinterop numbers them; `parseBridgeNames`
+recovers the name from the wrapper cinterop generated beside each one — the same place the doc comment
+comes from — and `stripCinterop`/`marshalStub` rename both sides together. The rename is skipped, and
+the number kept, when two bridges would share a name, when one bridge is reached from wrappers that
+disagree, or when the name collides with `kniCString`/`nativeLibrary`.
+
+**JNI escapes `_` in a method name as `_1`.** The symbol is `Java_<package>_<class>_<method>` with
+`_` as the separator, so `FPDFText_CountChars` has to emit `Java_pdfium_pdfium_FPDFText_1CountChars`.
+Get this wrong and the symbol reads as method `CountChars` on class `FPDFText`: it compiles, links,
+and dies on the first call, exactly like a missing `@JvmName`. Almost every C API worth binding has
+underscores in it, so this is the failure mode to check first after touching the naming.
+
 ## Kotlin DSL gotchas
 
 - The `kotlin-dsl` plugin adds a `T.() -> Unit` overload of `whenObjectAdded`, making the SAM form

README.md

@@ -145,7 +145,10 @@ externalNativeBuild {
 ```
 
 Each ABI gets its own configure/build pair, the libraries land in `jniLibs/<abi>/`, and the plugin
-wires that directory into the Android variants so they are packaged into the AAR. The plugin sets
+wires that directory into the Android variants so they are packaged into the AAR. The Android
+toolchain comes from the NDK the plugin finds — `ANDROID_NDK_HOME`, then `ndk.dir` or `sdk.dir` in
+`local.properties`, then `ANDROID_HOME` — unless the build passes `-DCMAKE_TOOLCHAIN_FILE` itself.
+`platform.set(26)` inside an `abi` block chooses the minimum API; it is `android-21` otherwise. The plugin sets
 `CMAKE_LIBRARY_OUTPUT_DIRECTORY`, so the CMakeLists needs no knowledge of the layout.
 
 The plugin supplies what only it knows, as cache entries your `CMakeLists.txt` reads:
@@ -154,6 +157,7 @@ The plugin supplies what only it knows, as cache entries your `CMakeLists.txt` r
 |---|---|
 | `KONAN_JNI_STUB_DIR` | directory holding the generated `.c` stub |
 | `KONAN_JNI_LIB_NAME` | the name the generated bindings will `System.loadLibrary` |
+| `KONAN_JNI_INCLUDE_DIRS` | include roots for `jni.h` and its platform header, `;`-separated |
 
 It also points `JAVA_HOME` at a JDK that ships `include/jni.h` — the one running Gradle often does
 not, since IDE-bundled JBRs strip the headers — and locates `cmake` itself, because the Gradle daemon
@@ -162,16 +166,33 @@ does not inherit a login shell's PATH.
 ```cmake
 set(KONAN_JNI_STUB_DIR "" CACHE PATH "")
 set(KONAN_JNI_LIB_NAME "" CACHE STRING "")
+set(KONAN_JNI_INCLUDE_DIRS "" CACHE STRING "")
 
 file(GLOB JNI_SOURCES "${KONAN_JNI_STUB_DIR}/*.c")
 add_library(mylib-jni SHARED ${JNI_SOURCES})
 set_target_properties(mylib-jni PROPERTIES OUTPUT_NAME "${KONAN_JNI_LIB_NAME}")
+target_include_directories(mylib-jni PRIVATE ${KONAN_JNI_INCLUDE_DIRS})
 target_link_libraries(mylib-jni PRIVATE mylib)   # frameworks, libc++ etc. arrive transitively
 ```
 
+`find_package(JNI)` would work too, but it is free to pick a different JDK from the one the plugin
+chose for the bindings; the cache variable is the JDK that generated them.
+
 ### Generated bindings
-One `external fun kniBridgeN(...)` per C function, in header order, each carrying its C-derived
-signature as a doc comment. Parameters are marshalled by shape:
+One `external fun` per C function, in header order, named for the function it calls and carrying its
+C-derived signature as a doc comment:
+
+```kotlin
+/** C: FPDFText_CountChars(text_page: FPDF_TEXTPAGE?): Int */
+@JvmName("FPDFText_CountChars")
+internal external fun FPDFText_CountChars(p0: Long): Int
+```
+
+cinterop numbers the bridges `kniBridge0…N`; the plugin renames them, so a binding reads as the C API
+does and a linker error or stack frame names something searchable. A bridge keeps its number when the
+rename would be ambiguous — two bridges sharing a name, or a name the generated file already uses.
+
+Parameters are marshalled by shape:
 
 | C parameter                        | Kotlin parameter | Crosses as                                    |
 |------------------------------------|------------------|------------------------------------------------|
@@ -181,7 +202,7 @@ signature as a doc comment. Parameters are marshalled by shape:
 | opaque handle, pointer to struct   | `Long`           | raw address                                    |
 
 The bridges are `internal` by default — they are an implementation detail of the module that writes
-the idiomatic API over them, and this keeps `kniBridge0…N` out of its published surface. Each carries
+the idiomatic API over them, and this keeps them out of its published surface. Each carries
 `@JvmName`, because Kotlin mangles internal functions on the JVM and JNI resolves the symbol from the
 unmangled method name. Set `visibility.set(BindingVisibility.PUBLIC)` if the wrapper is in a
 different Gradle module.

build.gradle.kts

@@ -8,7 +8,7 @@ plugins {
     `kotlin-dsl`
 }
 
-version = "1.2.0-alpha05"
+version = "1.2.0-alpha07"
 group = "io.github.lemcoder"
 
 java {

examples/README.md

@@ -42,14 +42,15 @@ expect fun add(a: Int, b: Int): Int
 // nativeMain  — over the cinterop binding
 actual fun add(a: Int, b: Int): Int = mymath.my_add(a, b)
 
-// jvmMain / androidMain  — over the generated JNI bridge
-actual fun add(a: Int, b: Int): Int = example.kniBridge0(a, b)
+// jvmMain / androidMain  — over the generated JNI bridge, which carries the same name
+actual fun add(a: Int, b: Int): Int = example.my_add(a, b)
 ```
 
 Pointer/string params arrive at the JNI bridge as a raw address (`Long`); convert in your `actual`.
 
 ## What's verified here vs. needs a device
 
 - `jvm` and `native` run to completion and print `5` / `40.0`.
-- `android` produces and verifies the `.so` (ELF + exported `Java_..._kniBridgeN` symbols). Running it
+- `android` produces and verifies the `.so` (ELF + exported `Java_..._my_1add` symbols, JNI escaping
+  each `_` in the name as `_1`). Running it
   on Android needs an emulator/device and an AGP module — see [`android/README.md`](android/README.md).

examples/android/src/main/kotlin/example/Api.kt

@@ -1,6 +1,6 @@
 package example
 
-// Idiomatic API the user writes on top of the generated low-level JNI bridges
-// (build/generated/jvmInterop/kotlin/example/example.kt).
-fun add(a: Int, b: Int): Int = kniBridge0(a, b)        // C: my_add(a: Int, b: Int): Int
-fun scale(x: Double): Double = kniBridge1(x)           // C: my_scale(x: Double): Double
+// Idiomatic API the user writes on top of the generated low-level JNI bridges, which carry the names
+// of the C functions they call (build/generated/jvmInterop/kotlin/example/example.kt).
+fun add(a: Int, b: Int): Int = my_add(a, b)
+fun scale(x: Double): Double = my_scale(x)

examples/jvm/build.gradle.kts

@@ -37,9 +37,10 @@ application { mainClass.set("example.MainKt") }
 
 tasks.named<JavaExec>("run") {
     dependsOn("linkJvmInteropMymath")
-    // loadLibrary() resolves the stub from java.library.path.
+    // loadLibrary() resolves the stub from java.library.path. Host libraries live under lib/, not
+    // jniLibs/ — only ABI-named directories may sit under the latter, which is AGP's contract.
     jvmArgs(
         "-Djava.library.path=" +
-            layout.buildDirectory.dir("jvmInterop/mymath/jniLibs/${host.abiDir}").get().asFile.absolutePath
+            layout.buildDirectory.dir("jvmInterop/mymath/lib/${host.abiDir}").get().asFile.absolutePath
     )
 }

examples/jvm/src/main/kotlin/Main.kt

@@ -1,9 +1,9 @@
 package example
 
-// Idiomatic API the *user* writes on top of the generated low-level JNI bridges.
-// (See build/generated/jvmInterop/kotlin/.../example.kt for the generated `kniBridgeN` + their C signatures.)
-fun add(a: Int, b: Int): Int = kniBridge0(a, b)        // C: my_add(a: Int, b: Int): Int
-fun scale(x: Double): Double = kniBridge1(x)           // C: my_scale(x: Double): Double
+// Idiomatic API the *user* writes on top of the generated low-level JNI bridges, which carry the
+// names of the C functions they call. (See build/generated/jvmInterop/kotlin/.../example.kt.)
+fun add(a: Int, b: Int): Int = my_add(a, b)
+fun scale(x: Double): Double = my_scale(x)
 
 fun main() {
     println("add(2, 3)    = ${add(2, 3)}")   // expect 5

src/main/kotlin/io/github/lemcoder/interop/ExternalNativeBuild.kt

@@ -89,6 +89,9 @@ abstract class CMakeAbiSettings @Inject constructor(private val abiName: String)
 
     override fun getName(): String = abiName
 
+    /** Minimum Android API the stub is built against; `android-21` when unset. */
+    abstract val platform: Property<Int>
+
     /** Configure preset for this ABI. */
     abstract val preset: Property<String>
 

src/main/kotlin/io/github/lemcoder/interop/JvmInteropRegistry.kt

@@ -45,6 +45,7 @@ abstract class JvmInteropRegistry @Inject constructor(
          */
         const val STUB_DIR_VARIABLE = "KONAN_JNI_STUB_DIR"
         const val LIB_NAME_VARIABLE = "KONAN_JNI_LIB_NAME"
+        const val JNI_INCLUDE_VARIABLE = "KONAN_JNI_INCLUDE_DIRS"
     }
 
     /** How a declaration site takes the generated Kotlin; keeps KGP types out of this class. */
@@ -283,6 +284,39 @@ abstract class JvmInteropRegistry @Inject constructor(
                     generate.flatMap { it.stubSourceDirectory }.map { it.asFile.absolutePath },
                 )
                 cacheEntries.put(LIB_NAME_VARIABLE, generate.flatMap { it.stubLibraryBaseName })
+                // The stub includes jni.h, and the JDK running Gradle often has no headers — an
+                // IDE-bundled JBR strips them. JAVA_HOME alone does not reach the compiler, so the
+                // include roots are passed too; find_package(JNI) would be free to pick another JDK.
+                cacheEntries.put(
+                    JNI_INCLUDE_VARIABLE,
+                    project.provider {
+                        val home = settings.jniHome.orNull?.let { java.io.File(it) } ?: JvmInteropSupport.detectJniHome()
+                        val include = home.resolve("include")
+                        val platform = include.listFiles()?.firstOrNull { it.isDirectory }
+                        listOfNotNull(include, platform).joinToString(";") { it.absolutePath }
+                    },
+                )
+                // An ABI is an Android ABI, and building for one means cross-compiling: without the
+                // NDK toolchain CMake uses the host compiler, and the failure is a linker complaining
+                // about an unknown file type rather than anything mentioning Android. Skipped when the
+                // build names a toolchain itself.
+                if (abi != null) {
+                    val named = cmake.arguments.get() + abi.arguments.get()
+                    if (named.none { it.startsWith("-DCMAKE_TOOLCHAIN_FILE") }) {
+                        val ndk = JvmInteropSupport.findNdk(project.projectDir)
+                        checkNotNull(ndk) {
+                            "No Android NDK found for ABI '${abi.name}'. Set ANDROID_NDK_HOME, or ndk.dir " +
+                                "in local.properties, or pass -DCMAKE_TOOLCHAIN_FILE yourself."
+                        }
+                        cacheEntries.put(
+                            "CMAKE_TOOLCHAIN_FILE",
+                            ndk.resolve("build/cmake/android.toolchain.cmake").absolutePath,
+                        )
+                        cacheEntries.put("ANDROID_ABI", abi.name)
+                        cacheEntries.put("ANDROID_PLATFORM", abi.platform.map { "android-$it" }.orElse("android-21"))
+                    }
+                }
+
                 // Land the library where the plugin says, so a consumer never guesses the build
                 // layout and Android gets the jniLibs/<abi>/ shape AGP packages.
                 cacheEntries.put(

src/main/kotlin/io/github/lemcoder/jvm/GenerateJvmInteropTask.kt

@@ -6,6 +6,7 @@ import io.github.lemcoder.util.ParamKind
 import io.github.lemcoder.util.execCapture
 import io.github.lemcoder.util.marshalStub
 import io.github.lemcoder.util.parseBridgeKinds
+import io.github.lemcoder.util.parseBridgeNames
 import io.github.lemcoder.util.stripCinterop
 import org.gradle.api.DefaultTask
 import org.gradle.api.file.ConfigurableFileCollection
@@ -138,10 +139,12 @@ abstract class GenerateJvmInteropTask @Inject constructor(
         val kinds = kotlinFiles.fold(emptyMap<Int, List<ParamKind>>()) { acc, kt ->
             acc + parseBridgeKinds(kt.readText())
         }
+        // cinterop numbers the bridges; the wrappers beside them name the C function each one calls.
+        val names = kotlinFiles.fold(emptyMap<Int, String>()) { acc, kt -> acc + parseBridgeNames(kt.readText()) }
         val internal = internalBindings.getOrElse(true)
-        kotlinFiles.forEach { kt -> kt.writeText(stripCinterop(kt.readText(), kinds, internal)) }
+        kotlinFiles.forEach { kt -> kt.writeText(stripCinterop(kt.readText(), kinds, internal, names)) }
         out.resolve(C_DIR).walkTopDown().filter { it.extension == "c" }
-            .forEach { c -> c.writeText(marshalStub(c.readText(), kinds)) }
+            .forEach { c -> c.writeText(marshalStub(c.readText(), kinds, names)) }
     }
 
     private companion object {

src/main/kotlin/io/github/lemcoder/jvm/JvmInteropSupport.kt

@@ -60,6 +60,30 @@ internal object JvmInteropSupport {
         ?: error("No JDK with include/jni.h found. Set the interop's jniHome.")
 
     /** The home of the JDK [detectJniIncludeDirs] would use. */
+    /**
+     * The Android NDK, for cross-compiling a stub per ABI.
+     *
+     * Checked in the order a developer would expect to win: an explicit environment variable, then
+     * `ndk.dir` or an SDK in `local.properties`, then `ANDROID_HOME`. The newest installed NDK is
+     * taken when a directory holds several.
+     */
+    fun findNdk(projectDir: File): File? {
+        System.getenv("ANDROID_NDK_HOME")?.let { return File(it).takeIf { home -> home.isDirectory } }
+        System.getenv("ANDROID_NDK_ROOT")?.let { return File(it).takeIf { root -> root.isDirectory } }
+
+        val properties = generateSequence(projectDir) { it.parentFile }
+            .map { it.resolve("local.properties") }
+            .firstOrNull { it.isFile }
+            ?.let { file -> file.readLines().mapNotNull { line -> line.split("=", limit = 2).takeIf { it.size == 2 } } }
+            ?.associate { (key, value) -> key.trim() to value.trim() }
+            .orEmpty()
+
+        properties["ndk.dir"]?.let { return File(it).takeIf { dir -> dir.isDirectory } }
+
+        val sdk = properties["sdk.dir"]?.let(::File) ?: System.getenv("ANDROID_HOME")?.let(::File)
+        return sdk?.resolve("ndk")?.listFiles()?.filter { it.isDirectory }?.maxByOrNull { it.name }
+    }
+
     fun detectJniHome(): File = jdkCandidates()
         .flatMap { listOf(it, it.resolve("Contents/Home")) }
         .firstOrNull { it.resolve("include/jni.h").isFile }

src/main/kotlin/io/github/lemcoder/util/CInteropFormatter.kt

@@ -6,6 +6,40 @@ private val LOAD_LIB = Regex("""loadKonanLibrary\("([^"]+)"\)""")
 // Captures each friendly wrapper's name/params/return and the kniBridge it delegates to, so we can keep
 // the original C-derived signature as documentation next to the otherwise opaque bridge.
 private val WRAPPER = Regex("""(?s)fun\s+(\w+)\(([^)]*)\)\s*:\s*([^{]+?)\s*\{[^}]*?kniBridge(\d+)""")
+private val IDENTIFIER = Regex("""[A-Za-z_]\w*""")
+
+/** Names the generated file already uses, which a bridge must not take. */
+private val RESERVED = setOf(C_STRING_HELPER, "nativeLibrary")
+
+/**
+ * Bridge index -> the name of the C function it calls, taken from the wrapper cinterop generated
+ * beside it.
+ *
+ * cinterop numbers the bridges, so a binding reads `kniBridge54(handle)` where the C API says
+ * `FPDFText_CountChars(text_page)`. The number is an implementation detail of the generator and
+ * carries nothing for a reader: renaming the bridge to the function it calls is what makes a
+ * hand-written `expect`/`actual` on top legible, and it keeps stack traces and linker errors
+ * pointing at something searchable.
+ *
+ * A bridge is left numbered whenever the rename would be unsafe: two bridges sharing a name, one
+ * bridge reached from wrappers that disagree, a name the generated file already uses, or anything
+ * that is not a plain identifier.
+ */
+fun parseBridgeNames(rawKotlin: String): Map<Int, String> {
+    val pairs =
+        WRAPPER.findAll(rawKotlin).mapNotNull { m -> m.groupValues[4].toIntOrNull()?.to(m.groupValues[1]) }.toList()
+    val timesUsed = pairs.groupingBy { it.second }.eachCount()
+
+    return pairs
+        .groupBy({ it.first }, { it.second })
+        .mapNotNull { (index, names) ->
+            val name = names.distinct().singleOrNull() ?: return@mapNotNull null
+            if (timesUsed.getValue(name) != names.size) return@mapNotNull null
+            if (name in RESERVED || !IDENTIFIER.matches(name)) return@mapNotNull null
+            index to name
+        }
+        .toMap()
+}
 
 /**
  * Rewrites a cinterop-generated JVM `.kt` into a runtime-free form: keeps `@file:JvmName` + package +
@@ -21,6 +55,7 @@ fun stripCinterop(
     src: String,
     kinds: Map<Int, List<ParamKind>> = emptyMap(),
     internalBindings: Boolean = false,
+    names: Map<Int, String> = emptyMap(),
 ): String {
     val jvmName = JVM_NAME.find(src)?.groupValues?.get(1)
     val pkg = src.lineSequence().firstOrNull { it.trimStart().startsWith("package ") }?.trim()
@@ -47,10 +82,11 @@ fun stripCinterop(
         }.joinToString(", ")
         val ret = m.groupValues[3].replace("NativePtr", "Long")
         val doc = origByBridge[idx]?.let { "/** C: $it */\n" } ?: ""
+        val name = names[idx.toIntOrNull()] ?: bridge
         // @JvmName because Kotlin mangles internal functions on the JVM ("kniBridge0${'$'}module"),
         // and JNI resolves the C symbol from the unmangled method name.
-        val jvmNameAnnotation = if (internalBindings) "@JvmName(\"$bridge\")\n" else ""
-        "$doc$jvmNameAnnotation${modifier}external fun $bridge($params): $ret"
+        val jvmNameAnnotation = if (internalBindings) "@JvmName(\"$name\")\n" else ""
+        "$doc$jvmNameAnnotation${modifier}external fun $name($params): $ret"
     }.toList()
 
     return buildString {