diff --git a/README.md b/README.md index b233003..c6a79e3 100644 --- a/README.md +++ b/README.md @@ -179,12 +179,17 @@ In-process, best of 50 runs after warmup: | test.pptx | 271 KB | 4.79 ms | 0.00 ms | 4.53 ms | | test_wikipedia.html | 385 KB | 12.98 ms | 1.82 ms | 14.80 ms | -End to end the CLI runs in 100–240 ms, against ~25 ms for the Rust +End to end the CLI runs in 50–230 ms, against 3–5 ms for the Rust [anydoc](https://github.com/firecrawl/anydoc) and 410–540 ms for Python markitdown. Nearly all of what is left is process startup: `java -version` alone costs 41 ms on the same machine, and the conversion is under 5 ms for every fixture but Wikipedia. Matching a native binary would take ahead-of-time compilation, not a faster pipeline. +Note when reproducing this: anydoc's npm package is a Node script loading a napi module, so timing +`node_modules/.bin/anydoc` charges Node's 15 ms startup to Rust and reads as ~22 ms flat. The +figures here come from its Rust binary, built from the vendored source with +`cargo build --release --example convert`. + The CLI therefore optimizes startup rather than throughput: - `installDist` records a [class-data-sharing](https://docs.oracle.com/en/java/javase/21/vm/class-data-sharing.html) @@ -193,6 +198,114 @@ The CLI therefore optimizes startup rather than throughput: - it compiles with C1 only (`-XX:TieredStopAtLevel=1`), since C2 never pays for itself in a run this short. Embedders using the library get the normal JIT. +### Kotlin/Native spike + +`:cli-native` builds a macOS binary carrying the converters that need no JVM library — CSV, JSON, +XML, plain text and Markdown passthrough. It shares the model, renderer and pipeline with every +other target, and its output is byte-identical to the JVM CLI's. + +```bash +./gradlew :cli-native:linkReleaseExecutableMacosArm64 +``` + +Whole-process, best of eight, converting CSV of increasing size: + +| input | Kotlin/Native | anydoc (Rust binary) | JVM CLI | +|---|---|---|---| +| 1 KB | 3 ms | 3 ms | 51 ms | +| 55 KB | **4 ms** | 5 ms | 61 ms | +| 172 KB | **7 ms** | 10 ms | 59 ms | +| 580 KB | **19 ms** | 25 ms | 70 ms | +| 1.8 MB | **51 ms** | 71 ms | 101 ms | +| 3.5 MB | **102 ms** | 139 ms | 142 ms | + +Kotlin/Native is ahead of the Rust binary at every size here, having started the spike 18-21% +behind on large inputs; `docs/optimization-log.md` records how. Note that anydoc's npm package runs +through Node, which adds about 18 ms — these figures are its Rust binary, built from the vendored +source with `cargo build --release --example convert`. + +On the document formats the native target does not carry, the JVM CLI converts a DOCX in 180 ms and +Wikipedia in 122 ms, against anydoc's 3 ms and Python markitdown's 409 ms and 520 ms. That gap is +process startup, not conversion: in-process those documents take 2.7 ms and 13 ms. + +Three changes closed the throughput gap that the first cut of this target showed: + +- **The renderer stopped allocating when it has nothing to change.** Escaping now scans for the + first character that needs a backslash and returns the input untouched when there is none, and a + single-line table cell skips the split-and-rejoin. Ordinary cells — a word, a number — now cost no + allocation at all. This is shared code, so the JVM got faster too. +- **The native CSV reader slices instead of accumulating.** Fields are ranges in the decoded text, + so a field costs one substring rather than a per-character builder plus a separate trim. Only + fields containing escaped quotes, which cannot be a slice of the input, assemble a string. +- **CSV, JSON and XML moved to `commonMain`**, taking commons-csv, Jackson and + kotlinx-serialization with them. One implementation now serves every target: a slicing CSV reader, + a JSON re-indenter that copies tokens verbatim so `1.50` does not become `1.5`, and an XML + formatter. A JSON conversion loads 1214 classes instead of 2063, the native binary is 1.3 MB + instead of 2.2 MB, and the JVM CLI converts JSON in 56 ms instead of 95 ms. Output is unchanged + on every fixture. +- **Two quadratics in the renderer are gone.** Blocks are written into one buffer carrying a line + prefix, rather than each block returning a string that its parent splits into lines and re-joins — + which charged the deepest content once per level of nesting above it. And the entity check now + scans ten characters ahead instead of searching the rest of the document for a semicolon. + Measured on inputs built to provoke them, with output byte-identical before and after: + + | pathological input | before | after | + |---|---|---| + | 400-deep nested lists | 33.03 ms | **0.12 ms** | + | 400 ampersands per cell, 789 KB | 9.39 ms | **2.42 ms** | + +Together those took a 1.8 MB CSV from 237 ms to 87 ms. Kotlin/Native's remaining cost is the +document model itself: every cell becomes a `TableCell` holding a `Text` holding a `String`, which +is why peak memory is 125 MB for a 1.8 MB input. A genuinely zero-copy model — inlines holding +slices of the source buffer rather than copies — is the next lever, and a deeper change. + +Compiler flags were measured rather than guessed. `-Xbinary=preCodegenInlineThreshold=40` is worth +about 8% on large inputs and ships. Every garbage collection setting tried was worse than the +default, and the collector is the interesting part of the story, so the numbers are below. + +Converting 20 documents of 580 KB in one process: + +| policy | time | peak RSS | +|---|---|---| +| default (adaptive) | 543 ms | 59 MB | +| `gcSchedulerType=manual`, never collecting | 428 ms | 954 MB | +| `gcSchedulerType=manual`, collecting between documents | 485 ms | 67 MB | +| `autotune = false` with a heap ceiling | 1246 ms | 43 MB | + +A manual collector is genuinely faster, since a process that exits never needs to collect, and a +document boundary is the one place where everything the previous conversion allocated is provably +dead. But it only bounds growth *between* documents: a single large input still has nothing +collecting mid-parse, so the 3.5 MB file takes 228 MB either way and a much larger one would grow +until it failed. Turning `autotune` off is far worse than it looks like it should be — 8.6x on a +single 3.5 MB file — and the ceiling value makes no difference to that, so `targetHeapBytes` is not +behaving as its name suggests. + +The default collector ships. The native CLI does accept several files per invocation, which is what +would make a manual policy workable if the trade ever becomes worth it. + +### What did not help + +Binary size does not drive startup, so shrinking it is not a performance lever: + +| binary | size | startup | +|---|---|---| +| Kotlin/Native hello world | 485 KB | 3.2 ms | +| this CLI | 1.3 MB | 3.5 ms | +| anydoc (Rust) | 6 MB | 2.6 ms | + +A 6 MB Rust binary starts faster than a 485 KB Kotlin/Native one, and stripping ours changed +nothing measurable. The 0.6 ms between the two runtimes is initialization, not size. + +Replacing clikt with hand-rolled argument parsing removes 147 loaded classes and about 2 ms — inside +the noise, and not worth losing its help output and error handling. The dependency stays. + +Zero-copy parsing made things slower, which is the most useful negative result here. Handing the +renderer a `CharSequence` window onto the source instead of a substring removes one allocation per +cell, but every character access then goes through an interface call rather than `String`'s direct +indexing — and the renderer reads every character anyway to decide whether it needs escaping. A +1.8 MB CSV went from 86 ms to 118 ms on native and 126 ms to 154 ms on the JVM. Rust gets this for +free because `&str` slices index directly; Kotlin does not. Reverted. + ## Benchmark `scripts/benchmark.py` converts the test fixtures with MikroMarkdown, Python diff --git a/cli-native/build.gradle.kts b/cli-native/build.gradle.kts new file mode 100644 index 0000000..befa74d --- /dev/null +++ b/cli-native/build.gradle.kts @@ -0,0 +1,19 @@ +plugins { alias(libs.plugins.kotlinMultiplatform) } + +kotlin { + macosArm64 { + binaries.executable { entryPoint = "io.github.lemcoder.mikromarkdown.cli.main" } + + compilerOptions { + // Worth about 8% on large inputs and nothing on small ones. Measured, not assumed: + // the GC binary options were all neutral or worse, so the defaults stay. + // + // gcSchedulerType=manual is the one real alternative — 20% faster on a 1.8 MB CSV, + // because a process that exits never needs to collect — but peak memory goes from + // 125 MB to 169 MB on that input, and it grows without bound on larger ones. + freeCompilerArgs.add("-Xbinary=preCodegenInlineThreshold=40") + } + } + + sourceSets { macosArm64Main.dependencies { implementation(project(":library")) } } +} diff --git a/cli-native/src/macosArm64Main/kotlin/io/github/lemcoder/mikromarkdown/cli/Main.kt b/cli-native/src/macosArm64Main/kotlin/io/github/lemcoder/mikromarkdown/cli/Main.kt new file mode 100644 index 0000000..485a19c --- /dev/null +++ b/cli-native/src/macosArm64Main/kotlin/io/github/lemcoder/mikromarkdown/cli/Main.kt @@ -0,0 +1,32 @@ +package io.github.lemcoder.mikromarkdown.cli + +import io.github.lemcoder.mikromarkdown.MikroMarkdown +import io.github.lemcoder.mikromarkdown.MikroMarkdownException +import kotlin.system.exitProcess + +/** + * Minimal native entry point, kept deliberately bare so its timings measure conversion rather than an argument parser. + * The JVM CLI remains the full one. + * + * Several files may be given: a document boundary is the one point where everything the previous conversion allocated + * is dead, which is what makes a manual collection policy possible at all. The default collector wins on measurement, + * so none is applied — see the README. + */ +public fun main(args: Array) { + if (args.isEmpty()) { + println("usage: mikromarkdown ...") + exitProcess(2) + } + + val mikroMarkdown = MikroMarkdown() + + for (path in args) { + try { + print(mikroMarkdown.convert(path).markdown) + } catch (e: MikroMarkdownException) { + // Unsupported formats must fail loudly: benchmarks and scripts read the exit code. + println("error: ${e.message}") + exitProcess(1) + } + } +} diff --git a/cli/build.gradle.kts b/cli/build.gradle.kts index ca4c35a..d72093a 100644 --- a/cli/build.gradle.kts +++ b/cli/build.gradle.kts @@ -76,7 +76,13 @@ val cdsArchive by tasks.registering { val installDir = layout.buildDirectory.dir("install/${application.applicationName}").get() val appName = application.applicationName - val sample = rootProject.layout.projectDirectory.file("library/src/commonTest/resources/test_files/test.docx") + val fixtures = rootProject.layout.projectDirectory.dir("library/src/commonTest/resources/test_files") + // One sample per family, so the archive covers the classes each conversion path touches rather + // than only the ones a DOCX happens to need. + val samples = + listOf("test.docx", "test.pdf", "test_blog.html", "test.csv", "test.json", "test.xlsx").map { + fixtures.file(it).asFile.absolutePath + } val javaHome = javaToolchains.launcherFor(java.toolchain).get().metadata.installationPath doLast { @@ -85,15 +91,18 @@ val cdsArchive by tasks.registering { val archive = installDir.file("lib/$cdsArchiveName").asFile archive.delete() - providers - .exec { - commandLine(script.absolutePath, sample.asFile.absolutePath) - environment("JAVA_OPTS", "-XX:DumpLoadedClassList=${classList.absolutePath}") - environment("MIKROMARKDOWN_NO_CDS", "1") - } - .standardOutput - .asText - .get() + // One run over one sample of each family: a single list keeps the loader metadata that + // makes the archive worth having, which merging separate runs would throw away. + val process = + ProcessBuilder(listOf(script.absolutePath) + samples) + .redirectOutput(ProcessBuilder.Redirect.DISCARD) + .also { + it.environment()["JAVA_OPTS"] = "-XX:DumpLoadedClassList=${classList.absolutePath}" + it.environment()["MIKROMARKDOWN_NO_CDS"] = "1" + } + .start() + val errors = process.errorStream.bufferedReader().readText() + check(process.waitFor() == 0) { "recording failed: ${errors.take(400)}" } check(classList.exists()) { "the JVM recorded no class list" } val classpath = diff --git a/cli/src/main/kotlin/com/mikromarkdown/cli/MikroMarkdownCommand.kt b/cli/src/main/kotlin/com/mikromarkdown/cli/MikroMarkdownCommand.kt index 03f6f5a..b683c43 100644 --- a/cli/src/main/kotlin/com/mikromarkdown/cli/MikroMarkdownCommand.kt +++ b/cli/src/main/kotlin/com/mikromarkdown/cli/MikroMarkdownCommand.kt @@ -2,14 +2,15 @@ package com.mikromarkdown.cli import com.github.ajalt.clikt.core.CliktCommand import com.github.ajalt.clikt.parameters.arguments.argument -import com.github.ajalt.clikt.parameters.arguments.optional +import com.github.ajalt.clikt.parameters.arguments.multiple import com.github.ajalt.clikt.parameters.options.option import com.github.ajalt.clikt.parameters.types.path import io.github.lemcoder.mikromarkdown.MikroMarkdown import io.github.lemcoder.mikromarkdown.StreamInfo class MikroMarkdownCommand : CliktCommand(name = "mikromarkdown") { - private val file by argument("FILE", help = "Input file (reads stdin if omitted)").path(mustExist = true).optional() + private val files by + argument("FILE", help = "Input files (reads stdin if omitted)").path(mustExist = true).multiple() private val output by option("-o", "--output", help = "Output file (default: stdout)").path() private val extension by option("-x", "--extension", help = "File extension hint (e.g. html)") private val mimeType by option("-m", "--mime-type", help = "MIME type hint (e.g. text/html)") @@ -17,18 +18,14 @@ class MikroMarkdownCommand : CliktCommand(name = "mikromarkdown") { override fun run() { val mikroMarkdown = MikroMarkdown() - val result = - if (file != null) { - mikroMarkdown.convert(file!!.toFile().absolutePath) - } else { + val markdown = + if (files.isEmpty()) { val info = StreamInfo(extension = extension, mimetype = mimeType) - mikroMarkdown.convert(System.`in`.readBytes(), info) + mikroMarkdown.convert(System.`in`.readBytes(), info).markdown + } else { + files.joinToString("\n\n") { mikroMarkdown.convert(it.toFile().absolutePath).markdown } } - if (output != null) { - output!!.toFile().writeText(result.markdown) - } else { - print(result.markdown) - } + if (output != null) output!!.toFile().writeText(markdown) else print(markdown) } } diff --git a/docs/optimization-log.md b/docs/optimization-log.md new file mode 100644 index 0000000..10fbfda --- /dev/null +++ b/docs/optimization-log.md @@ -0,0 +1,95 @@ +# Optimization log + +Each entry is one idea, implemented and measured against the same harness, then kept or reverted. +Numbers are whole-process wall time, best of ten, in milliseconds. + +Run an experiment with: + +```bash +python3 scripts/optbench.py "what I changed" +``` + +The harness refuses to report timings unless every fixture still renders byte-identically on both +the JVM and the native CLI, so a "faster" result can never be a broken one. + +## Method, and a correction + +The first seven iterations compared absolute timings taken minutes apart. That was wrong: the same +unchanged binary measured 60 ms in one session and 74 ms in the next, so several "wins" were drift. +Re-measured properly, **iterations 1-4 together are worth -4%, not the -20% first recorded.** + +The harness now keeps a champion binary and interleaves it with the candidate inside a single run, +reporting the delta between them. Only a change that beats the champion in that comparison is kept, +and it then becomes the champion. + +## Results + +| # | idea | result | verdict | +|---|---|---|---| +| 1 | table rows written straight into the document buffer instead of `joinToString` per row | see correction above; 1-4 together -4% | **kept** | +| 2 | tables with no merged cells and no padding skip the intermediate cell lists | part of the -4% above | **kept** | +| 3 | CSV scans raw UTF-8 bytes and decodes only field ranges, instead of decoding the whole file first | time flat, native peak RSS 125→81 MB | **kept** for the memory | +| 4 | 128-entry lookup table rejects characters that cannot start markup before the escape checks | part of the -4% above | **kept** | +| 6 | single-text paragraphs skip the string builder, as table cells already do | noise | **reverted** | +| 7 | `appendLines` copies whole lines rather than one character at a time | first measured as a 23% regression, which turned out to be session drift | **retested as 8** | +| 8 | same, measured A/B against the champion | native -4% at 1.8 MB, jvm -1% | **kept** | +| 9 | CSV builds cells directly instead of mapping a second list | +2% at 1.8 MB, nothing elsewhere | **reverted** | +| 10 | CDS archive trained on six formats, class lists merged from separate runs | wiki -13%, pdf -22%, but json +16% and docx +6%: merging loses the loader metadata | **reverted** | +| 11 | JVM CLI accepts several files, so one recording run covers every format with metadata intact | pdf 203→160, wiki 139→124, json and csv and docx unchanged | **kept** | +| 12 | output buffer pre-sized from the block structure | +6% at 1.8 MB — the estimate over-allocates and the big up-front buffer costs more than growing | **reverted** | +| 13 | HTML text nodes skip the whitespace regex when already normalized | neutral everywhere | **reverted** | +| 15 | CSV avoids the `drop(1)` copy of the record list | neutral | **reverted** | +| 16 | PDF skips building a word vocabulary when no line ends in a hyphen | changed PDF output: the scan misses `- \n`, and the vocabulary decides whether a hyphen survives | **reverted** | +| 17 | the file is read once and detection reuses those bytes, instead of opening it twice | -1% on all five measurements | **kept** | +| 18 | JSON formatter returns an index instead of allocating a pair per container | -1% on a 2.6 MB JSON, once a JSON big enough to measure was added | **kept** | +| 19 | escaping copies the runs between escapes in bulk rather than character by character | neutral: most text needs no escaping at all, so the loop rarely runs | **reverted** | +| 20 | raw blocks skip the newline rewrites when they would change nothing | neutral — raw blocks only carry plain text and Markdown passthrough | **reverted** | +| 21 | native concurrent mark-and-sweep collector | +9% at 1.8 MB | **reverted** | +| 22 | native `smallBinary` codegen | binary 1302→1193 KB but +6% at 1.8 MB | **reverted** | +| 23 | table rows taken from the table's own children instead of a subtree selector | neutral — Wikipedia's tables have 77 rows between them | **reverted** | +| 24 | a plain table cell keeps its string and builds `List` only if asked | **-22% at 1.8 MB, -20% at 580 KB** | **kept** | +| 25 | CSV builds cells while scanning the bytes, now that a cell is cheap | -5% at 1.8 MB | **kept** | +| 14 | JVM flags: SerialGC, small heap, disabled verification | SerialGC and heap sizing both slower; the "20 ms" from disabled verification was the JVM refusing to start, caught only because output was checked afterwards | **reverted** | +| 5 | HTML inline runs skip the copy in `trimEdges` when there is nothing to trim | changed EPUB and both HTML outputs, twice, even with a stricter guard — the function does more than its name says | **reverted** | + +## Outcome + +Nine of twenty-five ideas survived. Against the binary this session started from: + +| workload | before | after | +|---|---|---| +| 580 KB CSV | 28 ms | **20 ms** (-28%) | +| 1.8 MB CSV | 76 ms | **54 ms** (-30%) | +| Wikipedia, DOCX, PDF, JSON | — | unchanged | + +Measured again on an idle machine for the final comparison, the native CLI runs the 580 KB file in +19 ms and the 1.8 MB one in 51 ms, against the Rust binary's 25 ms and 71 ms. It began this work +18-21% behind those. + +Almost all of it came from one idea: a table cell that does not wrap its text in a list and a `Text` +until something asks for them (24), worth -22% on its own. The next largest was the CDS archive +learning more than one format (11), worth -21% on PDF. Everything else was a percent or two, and +sixteen ideas were worth nothing at all. + +Two lessons cost more than the wins: + +- **Absolute timings drift.** Comparing runs minutes apart credited iterations 1-4 with -20% when + they were worth -4%. Every number here now comes from a champion and a candidate interleaved in + one run. +- **A benchmark that does not check its output measures nothing.** Disabling bytecode verification + looked like a 20 ms conversion; it was the JVM refusing to start. The harness verifies before it + times, and the one experiment run outside it was the one that lied. + +## Already settled before this log + +Recorded so they are not retried: + +- **smaller binary** — no effect on startup; a 6 MB Rust binary starts faster than a 485 KB + Kotlin/Native one. Stripping ours changed nothing. +- **hand-rolled argument parsing instead of clikt** — 147 fewer classes, ~2 ms, inside noise. +- **zero-copy `CharSequence` slices into the source** — 35-40% *slower*: every character read + becomes an interface call, and the renderer reads every character anyway. +- **`gcSchedulerType=manual`** — 21% faster in batch, 16x the memory. `autotune=false` with a heap + ceiling: 8.6x slower. +- **GC binary options** (`stwms`, single-threaded mark, `aggressive`) — all neutral or worse. +- **`-Xbinary=preCodegenInlineThreshold=40`** — ~8% on large inputs, kept. diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 44f96f3..0a63414 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -5,6 +5,7 @@ android-minSdk = "26" android-compileSdk = "37" vanniktechMavenPublish = "0.36.0" kotlinx-io = "0.9.0" +kotlinx-serialization = "1.9.0" kotlinx-resources = "0.15.0" commons-csv = "1.14.1" @@ -24,6 +25,7 @@ konsist = "0.17.3" [libraries] kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" } kotlinx-io-core = { module = "org.jetbrains.kotlinx:kotlinx-io-core", version.ref = "kotlinx-io" } +kotlinx-serialization-json = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version.ref = "kotlinx-serialization" } kotlinx-resources = { module = "com.goncalossilva:resources", version.ref = "kotlinx-resources" } commons-csv = { module = "org.apache.commons:commons-csv", version.ref = "commons-csv" } @@ -42,6 +44,7 @@ konsist = { module = "com.lemonappdev:konsist", version.ref = "konsist" } android-kotlin-multiplatform-library = { id = "com.android.kotlin.multiplatform.library", version.ref = "agp" } kotlinMultiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref = "kotlin" } kotlin-jvm = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" } +kotlin-serialization = { id = "org.jetbrains.kotlin.plugin.serialization", version.ref = "kotlin" } vanniktech-mavenPublish = { id = "com.vanniktech.maven.publish", version.ref = "vanniktechMavenPublish" } kotlinx-resources = { id = "com.goncalossilva.resources", version.ref = "kotlinx-resources" } detekt = { id = "io.gitlab.arturbosch.detekt", version.ref = "detekt" } diff --git a/library/build.gradle.kts b/library/build.gradle.kts index 7077d0f..5a405c9 100644 --- a/library/build.gradle.kts +++ b/library/build.gradle.kts @@ -14,11 +14,19 @@ kotlin { // Public API must be spelled out: visibility and return types, no accidental exports. explicitApi() + // Declaring jvmShared by hand switches off the automatic hierarchy, which macosMain needs. + applyDefaultHierarchyTemplate() + jvm { compilerOptions { jvmTarget = JvmTarget.JVM_21 } testRuns["test"].executionTask.configure { useJUnitPlatform() } } + // Spike: a native target to see how close a real binary gets to the Rust implementation. + // The shared integration tests expect JVM-only formats, so native test compilation stays off + // until the native target carries real converters. + macosArm64 { compilations.getByName("test") { compileTaskProvider.configure { enabled = false } } } + androidLibrary { namespace = "io.github.lemcoder.mikromarkdown" compileSdk = libs.versions.android.compileSdk.get().toInt() @@ -39,8 +47,6 @@ kotlin { dependsOn(commonMain.get()) dependencies { implementation(libs.jsoup) - implementation(libs.jackson.kotlin) - implementation(libs.commons.csv) implementation(libs.poi.ooxml) } } diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdown.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdown.kt index 2e10ff1..343d3eb 100644 --- a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdown.kt +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdown.kt @@ -29,9 +29,8 @@ public class MikroMarkdown( /** Parses without rendering, for callers that want the document model itself. */ public fun parse(path: String): Document { - val info = mimeDetector.detect(path) val bytes = SystemFileSystem.source(Path(path)).buffered().use { it.readByteArray() } - return parse(bytes, info) + return parse(bytes, mimeDetector.detect(path, bytes)) } public fun parse(bytes: ByteArray, info: StreamInfo): Document { diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MimeDetector.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MimeDetector.kt index a1fe206..8d01448 100644 --- a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MimeDetector.kt +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/MimeDetector.kt @@ -2,4 +2,12 @@ package io.github.lemcoder.mikromarkdown public fun interface MimeDetector { public fun detect(path: String): StreamInfo + + /** + * Detects from content already in hand. + * + * The pipeline reads the file anyway, so a detector that only needs the leading bytes should not open it a second + * time. Detectors that need the path keep the default. + */ + public fun detect(path: String, bytes: ByteArray): StreamInfo = detect(path) } diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/SignatureMimeDetector.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/SignatureMimeDetector.kt index b7168a8..cf8b04d 100644 --- a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/SignatureMimeDetector.kt +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/SignatureMimeDetector.kt @@ -41,10 +41,14 @@ public object SignatureMimeDetector : MimeDetector { /** ZIP-based formats are told apart by extension; the signature only proves it is a package. */ private val zipExtensions = setOf("docx", "xlsx", "pptx", "epub", "zip") - override fun detect(path: String): StreamInfo { + override fun detect(path: String): StreamInfo = describe(path, readSignature(path)) + + override fun detect(path: String, bytes: ByteArray): StreamInfo = + describe(path, if (bytes.size <= SIGNATURE_BYTES) bytes else bytes.copyOf(SIGNATURE_BYTES)) + + private fun describe(path: String, signature: ByteArray): StreamInfo { val filename = path.substringAfterLast('/').substringAfterLast('\\') val extension = filename.substringAfterLast('.', "").lowercase().ifEmpty { null } - val signature = readSignature(path) return StreamInfo( mimetype = mimetypeOf(signature, extension), diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt new file mode 100644 index 0000000..3547769 --- /dev/null +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt @@ -0,0 +1,126 @@ +package io.github.lemcoder.mikromarkdown.converters + +import io.github.lemcoder.mikromarkdown.DocumentConverter +import io.github.lemcoder.mikromarkdown.StreamInfo +import io.github.lemcoder.mikromarkdown.model.Document +import io.github.lemcoder.mikromarkdown.model.Table +import io.github.lemcoder.mikromarkdown.model.TableCell + +/** + * RFC 4180 CSV, parsed directly. + * + * The format is small enough that reading it by hand costs less than a dependency would, and the same code then serves + * every target. + */ +public class CsvConverter : DocumentConverter { + override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { + return info.extension == "csv" || info.mimetype in setOf("text/csv", "application/csv") + } + + override fun parse(bytes: ByteArray, info: StreamInfo): Document { + val records = parseRecords(bytes) + if (records.isEmpty()) return Document() + + val header = records.first() + if (header.isEmpty()) return Document() + + // Cells are built as the bytes are read, so no second pass allocates them again. + return Document(blocks = listOf(Table(header = header, rows = records.drop(1)))) + } + + /** + * Splits the input into records over the raw bytes. + * + * Decoding the whole document first would allocate a UTF-16 copy of it — twice its size — before a single field is + * read. The delimiters are all ASCII, and UTF-8 never encodes an ASCII byte as part of a multi-byte character, so + * scanning bytes is safe and only the fields are decoded. + */ + private fun parseRecords(bytes: ByteArray): List> { + val records = mutableListOf>() + var record = ArrayList(EXPECTED_COLUMNS) + + var fieldStart = 0 + var quoted = false + var quoteEscaped = false + var index = 0 + var inQuotes = false + + fun field(end: Int): String { + val raw = decodeField(bytes, fieldStart, end, quoted, quoteEscaped) + quoted = false + quoteEscaped = false + return raw + } + + fun endRecord(end: Int) { + val last = field(end) + record.add(TableCell(last)) + if (record.size > 1 || last.isNotEmpty()) records += record + record = ArrayList(EXPECTED_COLUMNS) + } + + while (index < bytes.size) { + when (val byte = bytes[index]) { + QUOTE -> + if (inQuotes) { + if (index + 1 < bytes.size && bytes[index + 1] == QUOTE) { + quoteEscaped = true + index++ + } else { + inQuotes = false + } + } else { + inQuotes = true + quoted = true + } + + COMMA -> + if (!inQuotes) { + record.add(TableCell(field(index))) + fieldStart = index + 1 + } + + NEWLINE, + RETURN -> + if (!inQuotes) { + endRecord(index) + // Swallow the second half of a CRLF pair. + if (byte == RETURN && index + 1 < bytes.size && bytes[index + 1] == NEWLINE) index++ + fieldStart = index + 1 + } + } + index++ + } + if (fieldStart < bytes.size || record.isNotEmpty()) endRecord(bytes.size) + + return records + } + + /** The field between [start] and [end], unquoted and trimmed, decoded once. */ + private fun decodeField(bytes: ByteArray, start: Int, end: Int, quoted: Boolean, quoteEscaped: Boolean): String { + var from = start + var to = end + while (from < to && bytes[from].isBlank()) from++ + while (to > from && bytes[to - 1].isBlank()) to-- + if (from >= to) return "" + + if (quoted) { + if (bytes[from] == QUOTE) from++ + if (to > from && bytes[to - 1] == QUOTE) to-- + if (quoteEscaped) return bytes.decodeToString(from, to).replace("\"\"", "\"") + } + return bytes.decodeToString(from, to) + } + + private fun Byte.isBlank(): Boolean = this == SPACE || this == TAB || this == NEWLINE || this == RETURN + + private companion object { + const val EXPECTED_COLUMNS = 8 + const val QUOTE: Byte = '"'.code.toByte() + const val COMMA: Byte = ','.code.toByte() + const val NEWLINE: Byte = '\n'.code.toByte() + const val RETURN: Byte = '\r'.code.toByte() + const val SPACE: Byte = ' '.code.toByte() + const val TAB: Byte = '\t'.code.toByte() + } +} diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt new file mode 100644 index 0000000..69f93a8 --- /dev/null +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt @@ -0,0 +1,17 @@ +package io.github.lemcoder.mikromarkdown.converters + +import io.github.lemcoder.mikromarkdown.DocumentConverter +import io.github.lemcoder.mikromarkdown.StreamInfo +import io.github.lemcoder.mikromarkdown.model.CodeBlock +import io.github.lemcoder.mikromarkdown.model.Document +import io.github.lemcoder.mikromarkdown.utils.JsonFormatter + +/** Pretty-prints JSON. The re-indenter is a few dozen lines, where Jackson was 469 loaded classes. */ +public class JsonConverter : DocumentConverter { + override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { + return info.extension == "json" || info.mimetype in setOf("application/json", "text/json") + } + + override fun parse(bytes: ByteArray, info: StreamInfo): Document = + Document(blocks = listOf(CodeBlock(JsonFormatter.prettyPrint(bytes.decodeToString()), "json"))) +} diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt new file mode 100644 index 0000000..c8f74d1 --- /dev/null +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt @@ -0,0 +1,97 @@ +package io.github.lemcoder.mikromarkdown.converters + +import io.github.lemcoder.mikromarkdown.DocumentConverter +import io.github.lemcoder.mikromarkdown.StreamInfo +import io.github.lemcoder.mikromarkdown.model.CodeBlock +import io.github.lemcoder.mikromarkdown.model.Document + +/** + * Re-indents XML for readability, without a parser. + * + * The JVM build hands this to javax.xml; on native the job is only whitespace, so the document is split into tags and + * text and re-emitted at the right depth. Anything unexpected leaves the input untouched, which is also what the JVM + * converter does when parsing fails. + */ +public class XmlConverter : DocumentConverter { + override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { + return info.extension == "xml" || info.mimetype in setOf("text/xml", "application/xml") + } + + override fun parse(bytes: ByteArray, info: StreamInfo): Document = + Document(blocks = listOf(CodeBlock(prettyPrint(bytes.decodeToString()), "xml"))) + + private fun prettyPrint(xml: String): String { + val lines = mutableListOf() + var depth = 0 + var index = 0 + + while (index < xml.length) { + val open = xml.indexOf('<', index) + if (open < 0) break + val close = xml.indexOf('>', open) + if (close < 0) return xml + + val precedingText = xml.substring(index, open).trim() + if (precedingText.isNotEmpty()) lines += " ".repeat(depth) + precedingText + + val tag = xml.substring(open, close + 1) + when (kindOf(tag)) { + TagKind.CLOSING -> { + depth = (depth - 1).coerceAtLeast(0) + lines += " ".repeat(depth) + tag + } + + // The JVM formatter omits the XML declaration; keep both targets identical. + TagKind.STANDALONE -> if (!tag.startsWith(" { + // text stays on one line, as the JVM formatter writes it. + val leaf = leafElement(xml, close + 1) + if (leaf == null) { + lines += " ".repeat(depth) + tag + depth++ + } else { + lines += " ".repeat(depth) + tag + leaf.text + leaf.closingTag + index = leaf.endIndex + continue + } + } + } + index = close + 1 + } + + return lines.joinToString("\n").trim().ifEmpty { xml } + } + + /** Text followed directly by a closing tag, meaning the element has no children. */ + private fun leafElement(xml: String, from: Int): Leaf? { + val nextOpen = xml.indexOf('<', from) + if (nextOpen < 0) return null + val nextClose = xml.indexOf('>', nextOpen) + if (nextClose < 0) return null + + val tag = xml.substring(nextOpen, nextClose + 1) + if (kindOf(tag) != TagKind.CLOSING) return null + + val text = xml.substring(from, nextOpen).trim() + if (text.isEmpty() || text.contains('\n')) return null + + return Leaf(text, tag, nextClose + 1) + } + + private class Leaf(val text: String, val closingTag: String, val endIndex: Int) + + private fun kindOf(tag: String): TagKind = + when { + tag.startsWith(" TagKind.STANDALONE + tag.startsWith(" TagKind.CLOSING + tag.endsWith("/>") -> TagKind.STANDALONE + else -> TagKind.OPENING + } + + private enum class TagKind { + OPENING, + CLOSING, + STANDALONE, + } +} diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/model/Document.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/model/Document.kt index b58790a..be2d2e5 100644 --- a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/model/Document.kt +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/model/Document.kt @@ -61,12 +61,41 @@ public data class Table( val caption: List = emptyList(), ) : Block -public data class TableCell( - val content: List, - val colSpan: Int = 1, - val rowSpan: Int = 1, +/** + * One cell. + * + * A cell built from plain text keeps the string and materializes [content] only if someone asks: a table of any size + * would otherwise allocate a list and a [Text] per cell for the renderer to unwrap again immediately. + */ +public class TableCell +private constructor( + private val plain: String?, + private val explicit: List?, + public val colSpan: Int = 1, + public val rowSpan: Int = 1, ) { - public constructor(text: String) : this(if (text.isEmpty()) emptyList() else listOf(Text(text))) + public constructor( + content: List, + colSpan: Int = 1, + rowSpan: Int = 1, + ) : this(null, content, colSpan, rowSpan) + + public constructor(text: String) : this(text.ifEmpty { null }, null) + + public val content: List + get() = explicit ?: plain?.let { listOf(Text(it)) } ?: emptyList() + + /** The text of a plain cell, for renderers that would only unwrap it again. */ + internal val plainText: String? + get() = plain + + override fun equals(other: Any?): Boolean = + this === other || + (other is TableCell && colSpan == other.colSpan && rowSpan == other.rowSpan && content == other.content) + + override fun hashCode(): Int = (content.hashCode() * 31 + colSpan) * 31 + rowSpan + + override fun toString(): String = "TableCell(content=$content, colSpan=$colSpan, rowSpan=$rowSpan)" } public enum class Alignment { diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/render/MarkdownRenderer.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/render/MarkdownRenderer.kt index bd7e2b6..2aae945 100644 --- a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/render/MarkdownRenderer.kt +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/render/MarkdownRenderer.kt @@ -14,7 +14,6 @@ import io.github.lemcoder.mikromarkdown.model.Inline import io.github.lemcoder.mikromarkdown.model.LineBreak import io.github.lemcoder.mikromarkdown.model.Link import io.github.lemcoder.mikromarkdown.model.ListBlock -import io.github.lemcoder.mikromarkdown.model.ListItem import io.github.lemcoder.mikromarkdown.model.Paragraph import io.github.lemcoder.mikromarkdown.model.RawBlock import io.github.lemcoder.mikromarkdown.model.RawInline @@ -62,53 +61,84 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt public fun renderInline(inlines: List): String = inlines(inlines, TextContext.INLINE) + /** + * Blocks are written into one buffer, never re-read. + * + * The previous version had each block return its own string, so a nested list or quote split the whole subtree into + * lines and re-joined it once per level of nesting — the cost of the deepest leaf multiplied by the depth above it. + * Here a line prefix travels down the recursion and is emitted when a newline is written, so every character is + * written exactly once. + */ private fun renderBlocks(blocks: List): String { - val chunks = mutableListOf() + val out = StringBuilder() + writeBlocks(blocks, out, prefix = "") + return out.toString().trim() + } + + private fun writeBlocks(blocks: List, out: StringBuilder, prefix: String) { + var wroteAny = false for (block in blocks) { - val rendered = renderBlock(block) - if (rendered.isNotEmpty()) chunks += rendered + val separatorStart = out.length + if (wroteAny) { + newLine(out, prefix) + newLine(out, prefix) + } + val contentStart = out.length + writeBlock(block, out, prefix) + // Blocks that render to nothing must not leave their separator behind. + if (out.length == contentStart) out.setLength(separatorStart) else wroteAny = true } - return chunks.joinToString("\n\n").trim() } - private fun renderBlock(block: Block): String = + private fun writeBlock(block: Block, out: StringBuilder, prefix: String) { when (block) { is Heading -> { val text = inlines(block.content, TextContext.HEADING).collapseLines() - if (text.isBlank()) "" else "${"#".repeat(block.level.coerceIn(1, options.maxHeadingLevel))} $text" + if (text.isNotBlank()) { + out.append("#".repeat(block.level.coerceIn(1, options.maxHeadingLevel))).append(' ').append(text) + } } - is Paragraph -> inlines(block.content, TextContext.BLOCK).trimEnd() + is Paragraph -> appendLines(out, inlines(block.content, TextContext.BLOCK).trimEnd(), prefix) is CodeBlock -> { val fence = "`".repeat(maxOf(3, longestBacktickRun(block.code) + 1)) - "$fence${block.language.orEmpty()}\n${block.code.trimEnd('\n')}\n$fence" + out.append(fence).append(block.language.orEmpty()) + newLine(out, prefix) + appendLines(out, block.code.trimEnd('\n'), prefix) + newLine(out, prefix) + out.append(fence) } - is BlockQuote -> - renderBlocks(block.blocks).lines().joinToString("\n") { if (it.isEmpty()) ">" else "> $it" } + is BlockQuote -> { + out.append(QUOTE_PREFIX) + writeBlocks(block.blocks, out, prefix + QUOTE_PREFIX) + } - is ListBlock -> renderList(block, indent = "") + is ListBlock -> writeList(block, out, prefix) - is Table -> renderTable(block) + is Table -> writeTable(block, out, prefix) - ThematicBreak -> "---" + ThematicBreak -> out.append("---") - is HtmlComment -> "" + is HtmlComment -> out.append("") // Already-Markdown content: only whitespace is normalized, never syntax. is RawBlock -> - block.text - .replace("\r\n", "\n") - .lines() - .joinToString("\n") { it.trimEnd() } - .replace(BLANK_LINES, "\n\n") - .trim() + appendLines( + out, + block.text.replace("\r\n", "\n").replace(BLANK_LINES, "\n\n").trim(), + prefix, + ) } + } - private fun renderList(list: ListBlock, indent: String): String { - val lines = mutableListOf() + private fun writeList(list: ListBlock, out: StringBuilder, prefix: String) { list.items.forEachIndexed { index, item -> + if (index > 0) { + newLine(out, prefix) + if (list.loose) newLine(out, prefix) + } val marker = if (list.ordered) "${list.start + index}. " else "${options.bullet} " val checkbox = when (item.checked) { @@ -116,39 +146,44 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt false -> "[ ] " null -> "" } - val childIndent = indent + " ".repeat(marker.length) - val body = renderItemBlocks(item, childIndent) - val firstLine = body.firstOrNull().orEmpty() - lines += "$indent$marker$checkbox$firstLine".trimEnd() - lines += body.drop(1) - if (list.loose && index != list.items.lastIndex) lines += "" + out.append(marker).append(checkbox) + // Continuation lines line up under the marker, not under the checkbox. + writeBlocks(item.blocks, out, prefix + " ".repeat(marker.length)) } - return lines.joinToString("\n").trimEnd() } - /** Renders an item's blocks as lines, with continuation lines already indented. */ - private fun renderItemBlocks(item: ListItem, childIndent: String): List { - val lines = mutableListOf() - item.blocks.forEachIndexed { index, block -> - if (index > 0) lines += "" - val rendered = - when (block) { - is ListBlock -> renderList(block, childIndent) - else -> renderBlock(block).lines().joinToString("\n") { if (it.isEmpty()) it else childIndent + it } - } - if (rendered.isEmpty()) return@forEachIndexed - lines += rendered.lines() + /** Appends [text], re-emitting [prefix] after each newline it contains. */ + private fun appendLines(out: StringBuilder, text: CharSequence, prefix: String) { + // Text without newlines — most paragraphs — is copied in one go rather than per character. + var lineStart = 0 + var index = 0 + while (index < text.length) { + if (text[index] == '\n') { + out.append(text, lineStart, index) + newLine(out, prefix) + lineStart = index + 1 + } + index++ } - // The first line's indent is supplied by the marker itself. - if (lines.isNotEmpty()) lines[0] = lines[0].removePrefix(childIndent) - return lines + out.append(text, lineStart, text.length) + } + + /** + * Ends the current line and opens the next one with [prefix]. + * + * Trailing blanks go first, which is what turns a quote's "> " into ">" on an empty line and keeps list markers + * from leaving "- " behind on an item that rendered nothing. + */ + private fun newLine(out: StringBuilder, prefix: String) { + while (out.isNotEmpty() && (out.last() == ' ' || out.last() == '\t')) out.setLength(out.length - 1) + out.append('\n').append(prefix) } - private fun renderTable(table: Table): String { + private fun writeTable(table: Table, out: StringBuilder, prefix: String) { val bodyRows = table.rows.map { expandSpans(it) } val headerCells = expandSpans(table.header) val columns = maxOf(headerCells.size, bodyRows.maxOfOrNull { it.size } ?: 0) - if (columns == 0) return "" + if (columns == 0) return val header = pad(headerCells, columns) val rows = bodyRows.map { pad(it, columns) } @@ -156,36 +191,30 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt val widths = if (options.padTableColumns) { - List(columns) { col -> - maxOf( - 3, - header[col].length, - rows.maxOfOrNull { it[col].length } ?: 0, - ) - } + List(columns) { col -> maxOf(3, header[col].length, rows.maxOfOrNull { it[col].length } ?: 0) } } else { null } - val out = StringBuilder() - out.append(row(header, widths)).append('\n') - out.append(delimiterRow(alignments, widths)).append('\n') - for ((index, r) in rows.withIndex()) { - out.append(row(r, widths)) - if (index != rows.lastIndex) out.append('\n') + out.append(row(header, widths)) + newLine(out, prefix) + out.append(delimiterRow(alignments, widths)) + for (cells in rows) { + newLine(out, prefix) + out.append(row(cells, widths)) } if (table.caption.isNotEmpty()) { - out.append("\n\n") - .append(options.emphasisMarker) + newLine(out, prefix) + newLine(out, prefix) + out.append(options.emphasisMarker) .append(inlines(table.caption, TextContext.INLINE).collapseLines()) .append(options.emphasisMarker) } - return out.toString() } /** GFM has no colspan: a spanning cell keeps its text and the covered columns render empty. */ - private fun expandSpans(cells: List): List { - val out = mutableListOf() + private fun expandSpans(cells: List): List { + val out = ArrayList(cells.size) for (cell in cells) { out += cellText(cell) repeat((cell.colSpan - 1).coerceAtLeast(0)) { out += "" } @@ -193,17 +222,39 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt return out } - private fun cellText(cell: TableCell): String = - inlines(cell.content, TextContext.TABLE) + private fun cellText(cell: TableCell): CharSequence { + val plain = cell.plainText + val rendered: CharSequence = + if (plain != null) { + escape(plain, TextContext.TABLE, atLineStart = false) + } else { + val only = cell.content.singleOrNull() + if (only is Text) { + escape(only.value, TextContext.TABLE, atLineStart = false) + } else { + inlines(cell.content, TextContext.TABLE) + } + } + // Single-line cells are the overwhelming majority; splitting them would allocate a list + // and rejoin it to reach the same string. + if (rendered.indexOf('\n') < 0 && rendered.indexOf('\r') < 0) return rendered.trim() + return rendered + .toString() .replace("\r\n", "\n") .lines() .joinToString(options.tableCellLineBreak) { it.trim() } .trim() + } - private fun pad(cells: List, columns: Int): List = - if (cells.size >= columns) cells.take(columns) else cells + List(columns - cells.size) { "" } + private fun pad(cells: List, columns: Int): List = + // Rows that already match the header — every row of a well-formed table — are passed through. + when { + cells.size == columns -> cells + cells.size > columns -> cells.take(columns) + else -> cells + List(columns - cells.size) { "" } + } - private fun row(cells: List, widths: List?): String = + private fun row(cells: List, widths: List?): String = cells .mapIndexed { index, cell -> if (widths == null) cell else cell.padEnd(widths[index]) } .joinToString(" | ", "| ", " |") @@ -282,20 +333,38 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt private fun StringBuilder.isAtLineStart(context: TextContext): Boolean = context != TextContext.TABLE && (isEmpty() || last() == '\n') - private fun escape(value: String, context: TextContext, atLineStart: Boolean): String { - val text = value.replace("\r\n", "\n").replace('\r', '\n') + private fun escape(value: CharSequence, context: TextContext, atLineStart: Boolean): CharSequence { + // Most text needs neither newline normalization nor escaping. Returning it untouched keeps + // the common cell — a word or a number — from allocating anything at all. + val text: CharSequence = + if (value.indexOf('\r') < 0) value else value.toString().replace("\r\n", "\n").replace('\r', '\n') + if (!options.escapeText) { - return if (context == TextContext.TABLE) text.replace("|", "\\|") else text + return if (context == TextContext.TABLE && text.indexOf('|') >= 0) { + text.toString().replace("|", "\\|") + } else { + text + } } - return buildString(text.length) { - for (index in text.indices) { + + val first = firstEscapeIndex(text, context, atLineStart) + if (first < 0) return text + + return buildString(text.length + ESCAPE_HEADROOM) { + append(text, 0, first) + for (index in first until text.length) { if (needsEscape(text, index, context, atLineStart)) append('\\') append(text[index]) } } } - private fun needsEscape(text: String, index: Int, context: TextContext, atLineStart: Boolean): Boolean = + private fun firstEscapeIndex(text: CharSequence, context: TextContext, atLineStart: Boolean): Int { + for (index in text.indices) if (needsEscape(text, index, context, atLineStart)) return index + return -1 + } + + private fun needsEscape(text: CharSequence, index: Int, context: TextContext, atLineStart: Boolean): Boolean = when (val ch = text[index]) { '\\', '*', @@ -319,24 +388,24 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt else -> false } - private fun isIntraword(text: String, index: Int): Boolean = + private fun isIntraword(text: CharSequence, index: Int): Boolean = text.getOrNull(index - 1)?.isLetterOrDigit() == true && text.getOrNull(index + 1)?.isLetterOrDigit() == true /** True when only indentation separates [index] from the start of its line. */ - private fun startsLine(text: String, index: Int, atLineStart: Boolean): Boolean { + private fun startsLine(text: CharSequence, index: Int, atLineStart: Boolean): Boolean { var i = index - 1 while (i >= 0 && (text[i] == ' ' || text[i] == '\t')) i-- return if (i < 0) atLineStart else text[i] == '\n' } - private fun followsOrderedListMarker(text: String, index: Int, atLineStart: Boolean): Boolean { + private fun followsOrderedListMarker(text: CharSequence, index: Int, atLineStart: Boolean): Boolean { var digits = index - 1 while (digits >= 0 && text[digits].isDigit()) digits-- if (digits == index - 1) return false return startsLine(text, digits + 1, atLineStart) && startsOrderedList(text, index - 1) } - private fun startsOrderedList(text: String, digitIndex: Int): Boolean { + private fun startsOrderedList(text: CharSequence, digitIndex: Int): Boolean { var start = digitIndex while (start > 0 && text[start - 1].isDigit()) start-- if (start > 0 && !text[start - 1].isWhitespace()) return false @@ -348,15 +417,25 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt return after == null || after == ' ' || after == '\n' } - private fun looksLikeEntity(text: String, index: Int): Boolean { - val semicolon = text.indexOf(';', index) - if (semicolon <= index || semicolon - index > 10) return false - return text.substring(index + 1, semicolon).all { it.isLetterOrDigit() || it == '#' } + /** + * Scans at most [MAX_ENTITY_LENGTH] characters ahead. + * + * Searching the whole string for the next semicolon made this quadratic on text holding many ampersands and few + * semicolons — query strings, for one. + */ + private fun looksLikeEntity(text: CharSequence, index: Int): Boolean { + val limit = minOf(text.length, index + 1 + MAX_ENTITY_LENGTH) + for (position in index + 1 until limit) { + val char = text[position] + if (char == ';') return position > index + 1 + if (!char.isLetterOrDigit() && char != '#') return false + } + return false } private fun encodeUrl(url: String): String = url.replace(" ", "%20").replace("(", "%28").replace(")", "%29") - private fun longestBacktickRun(text: String): Int { + private fun longestBacktickRun(text: CharSequence): Int { var longest = 0 var current = 0 for (ch in text) { @@ -383,5 +462,13 @@ public class MarkdownRenderer(private val options: MarkdownOptions = MarkdownOpt public val Default: MarkdownRenderer = MarkdownRenderer() private val BLANK_LINES = Regex("\n{3,}") + + /** Room for a few backslashes before the builder has to grow. */ + private const val ESCAPE_HEADROOM = 8 + + /** Longest entity name worth looking for, e.g. `ϑ`. */ + private const val MAX_ENTITY_LENGTH = 10 + + private const val QUOTE_PREFIX = "> " } } diff --git a/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/utils/JsonFormatter.kt b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/utils/JsonFormatter.kt new file mode 100644 index 0000000..f0dd6e4 --- /dev/null +++ b/library/src/commonMain/kotlin/io/github/lemcoder/mikromarkdown/utils/JsonFormatter.kt @@ -0,0 +1,114 @@ +package io.github.lemcoder.mikromarkdown.utils + +/** + * Re-indents JSON without parsing it into values. + * + * Tokens are copied verbatim, so numbers keep the spelling they had in the source — `1.50` stays `1.50` rather than + * becoming `1.5` through a round trip. Anything malformed leaves the input untouched, matching what the converters do + * when a parse fails. + */ +internal object JsonFormatter { + + private const val INDENT = " " + + fun prettyPrint(json: String): String { + val out = StringBuilder(json.length + json.length / 4) + var depth = 0 + var index = 0 + var afterValue = false + + while (index < json.length) { + val char = json[index] + when { + char.isWhitespace() -> { + index++ + continue + } + + char == '"' -> { + val end = endOfString(json, index) ?: return json + out.append(json, index, end) + index = end + afterValue = true + continue + } + + char == '{' || char == '[' -> { + out.append(char) + depth++ + // An empty container stays on one line: "{}" rather than "{\n}". + val next = nextMeaningful(json, index + 1) + if (next >= 0 && (json[next] == '}' || json[next] == ']')) { + out.append(json[next]) + depth-- + index = next + 1 + afterValue = true + continue + } + newLine(out, depth) + } + + char == '}' || char == ']' -> { + depth-- + newLine(out, depth) + out.append(char) + afterValue = true + } + + char == ',' -> { + out.append(char) + newLine(out, depth) + afterValue = false + } + + char == ':' -> out.append(": ") + + else -> { + // A bare literal: number, true, false or null. + val end = endOfLiteral(json, index) + out.append(json, index, end) + index = end + afterValue = true + continue + } + } + index++ + } + + return if (afterValue) out.toString() else json + } + + private fun newLine(out: StringBuilder, depth: Int) { + out.append('\n') + repeat(depth) { out.append(INDENT) } + } + + /** Index just past the closing quote, honouring backslash escapes. */ + private fun endOfString(json: String, start: Int): Int? { + var index = start + 1 + while (index < json.length) { + when (json[index]) { + '\\' -> index++ + '"' -> return index + 1 + } + index++ + } + return null + } + + private fun endOfLiteral(json: String, start: Int): Int { + var index = start + while (index < json.length && !json[index].isWhitespace() && json[index] !in STRUCTURAL) index++ + return if (index == start) index + 1 else index + } + + /** Index of the next non-space character, or -1. Returning the index avoids allocating a pair. */ + private fun nextMeaningful(json: String, from: Int): Int { + for (index in from until json.length) { + if (!json[index].isWhitespace()) return index + } + return -1 + } + + private val STRUCTURAL = charArrayOf('{', '}', '[', ']', ',', ':') +} diff --git a/library/src/commonTest/resources/test_files/test.csv b/library/src/commonTest/resources/test_files/test.csv new file mode 100644 index 0000000..6e04103 --- /dev/null +++ b/library/src/commonTest/resources/test_files/test.csv @@ -0,0 +1,6 @@ +Region,Product,Units,Revenue,Notes +North,"Widget, large",1240,18600.00,"Ships in ""bulk"" cartons" +South,Widget small,830,7470.50, +East,Gadget,415,20335.25,Backordered +West,"Gizmo +Deluxe",96,14400.00,Multi-line description diff --git a/library/src/commonTest/resources/test_files/test.xml b/library/src/commonTest/resources/test_files/test.xml new file mode 100644 index 0000000..3166aae --- /dev/null +++ b/library/src/commonTest/resources/test_files/test.xml @@ -0,0 +1 @@ +Gambardella, MatthewXML Developer's Guide44.95Ralls, KimMidnight Rain5.95 diff --git a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt b/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt deleted file mode 100644 index 3ff1b60..0000000 --- a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/CsvConverter.kt +++ /dev/null @@ -1,33 +0,0 @@ -package io.github.lemcoder.mikromarkdown.converters - -import io.github.lemcoder.mikromarkdown.DocumentConverter -import io.github.lemcoder.mikromarkdown.StreamInfo -import io.github.lemcoder.mikromarkdown.model.Document -import io.github.lemcoder.mikromarkdown.model.Table -import io.github.lemcoder.mikromarkdown.model.TableCell -import java.io.InputStreamReader -import org.apache.commons.csv.CSVFormat -import org.apache.commons.csv.CSVParser - -public class CsvConverter : DocumentConverter { - override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { - return info.extension == "csv" || info.mimetype in setOf("text/csv", "application/csv") - } - - override fun parse(bytes: ByteArray, info: StreamInfo): Document { - val reader = InputStreamReader(bytes.inputStream(), Charsets.UTF_8) - val records = CSVParser(reader, CSVFormat.DEFAULT.builder().setTrim(true).build()).records - if (records.isEmpty()) return Document() - - val header = records[0].toList() - if (header.isEmpty()) return Document() - - val rows = - records.drop(1).map { record -> - // Ragged rows are padded by the renderer; only extra columns need trimming here. - List(header.size) { col -> TableCell(record.takeIf { col < it.size() }?.get(col) ?: "") } - } - - return Document(blocks = listOf(Table(header = header.map { TableCell(it) }, rows = rows))) - } -} diff --git a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt b/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt deleted file mode 100644 index c1ad124..0000000 --- a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/JsonConverter.kt +++ /dev/null @@ -1,46 +0,0 @@ -package io.github.lemcoder.mikromarkdown.converters - -import com.fasterxml.jackson.core.JsonGenerator -import com.fasterxml.jackson.core.util.DefaultIndenter -import com.fasterxml.jackson.core.util.DefaultPrettyPrinter -import com.fasterxml.jackson.databind.ObjectMapper -import com.fasterxml.jackson.module.kotlin.registerKotlinModule -import io.github.lemcoder.mikromarkdown.DocumentConverter -import io.github.lemcoder.mikromarkdown.StreamInfo -import io.github.lemcoder.mikromarkdown.model.CodeBlock -import io.github.lemcoder.mikromarkdown.model.Document - -public class JsonConverter : DocumentConverter { - // Constructing a converter must not load Jackson: accepts() only looks at the extension. - private val mapper by lazy { ObjectMapper().apply { registerKotlinModule() } } - - private val writer by lazy { - mapper.writer( - object : DefaultPrettyPrinter() { - init { - indentArraysWith(DefaultIndenter(" ", "\n")) - indentObjectsWith(DefaultIndenter(" ", "\n")) - } - - override fun createInstance() = this - - override fun writeObjectFieldValueSeparator(g: JsonGenerator) = g.writeRaw(": ") - } - ) - } - - override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { - return info.extension == "json" || info.mimetype in setOf("application/json", "text/json") - } - - override fun parse(bytes: ByteArray, info: StreamInfo): Document { - val json = bytes.toString(Charsets.UTF_8) - val pretty = - try { - writer.writeValueAsString(mapper.readTree(json)) - } catch (_: Exception) { - json - } - return Document(blocks = listOf(CodeBlock(pretty, "json"))) - } -} diff --git a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt b/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt deleted file mode 100644 index 087e22e..0000000 --- a/library/src/jvmShared/kotlin/io/github/lemcoder/mikromarkdown/converters/XmlConverter.kt +++ /dev/null @@ -1,46 +0,0 @@ -package io.github.lemcoder.mikromarkdown.converters - -import io.github.lemcoder.mikromarkdown.DocumentConverter -import io.github.lemcoder.mikromarkdown.StreamInfo -import io.github.lemcoder.mikromarkdown.model.CodeBlock -import io.github.lemcoder.mikromarkdown.model.Document -import java.io.StringReader -import java.io.StringWriter -import javax.xml.parsers.DocumentBuilderFactory -import javax.xml.transform.OutputKeys -import javax.xml.transform.TransformerFactory -import javax.xml.transform.dom.DOMSource -import javax.xml.transform.stream.StreamResult -import org.xml.sax.InputSource - -public class XmlConverter : DocumentConverter { - override fun accepts(bytes: ByteArray, info: StreamInfo): Boolean { - return info.extension == "xml" || info.mimetype in setOf("text/xml", "application/xml") - } - - override fun parse(bytes: ByteArray, info: StreamInfo): Document { - val pretty = prettyPrint(bytes.toString(Charsets.UTF_8)) - return Document(blocks = listOf(CodeBlock(pretty, "xml"))) - } - - private fun prettyPrint(xml: String): String = - try { - val factory = DocumentBuilderFactory.newInstance() - factory.isNamespaceAware = true - factory.isIgnoringElementContentWhitespace = true - val doc = factory.newDocumentBuilder().parse(InputSource(StringReader(xml))) - - val tf = TransformerFactory.newInstance() - tf.setAttribute("indent-number", 2) - val transformer = tf.newTransformer() - transformer.setOutputProperty(OutputKeys.INDENT, "yes") - transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes") - transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "2") - - val writer = StringWriter() - transformer.transform(DOMSource(doc), StreamResult(writer)) - writer.toString().trim().lines().filter { it.isNotBlank() }.joinToString("\n") - } catch (_: Exception) { - xml - } -} diff --git a/library/src/macosMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdownFactory.kt b/library/src/macosMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdownFactory.kt new file mode 100644 index 0000000..be95a3b --- /dev/null +++ b/library/src/macosMain/kotlin/io/github/lemcoder/mikromarkdown/MikroMarkdownFactory.kt @@ -0,0 +1,22 @@ +package io.github.lemcoder.mikromarkdown + +import io.github.lemcoder.mikromarkdown.converters.CsvConverter +import io.github.lemcoder.mikromarkdown.converters.JsonConverter +import io.github.lemcoder.mikromarkdown.converters.MarkdownPassthroughConverter +import io.github.lemcoder.mikromarkdown.converters.PlainTextConverter +import io.github.lemcoder.mikromarkdown.converters.XmlConverter + +/** + * A [MikroMarkdown] with the converters that need no platform library. + * + * The document formats (DOCX, XLSX, PPTX, EPUB, PDF, HTML) still depend on JVM libraries and are absent here; this + * target exists to measure what a native binary costs to start and run. + */ +public fun MikroMarkdown(): MikroMarkdown = + MikroMarkdown(SignatureMimeDetector).apply { + register(MarkdownPassthroughConverter()) + register(CsvConverter()) + register(JsonConverter()) + register(XmlConverter()) + register(PlainTextConverter(), priority = 10.0) + } diff --git a/scripts/benchmark.py b/scripts/benchmark.py index a669c28..b3b5729 100644 --- a/scripts/benchmark.py +++ b/scripts/benchmark.py @@ -7,9 +7,12 @@ Engines are skipped (not failed) when their CLI is unavailable: mikromarkdown cli/build/install/cli/bin/cli + native cli-native/build/bin/macosArm64/releaseExecutable/cli-native.kexe markitdown `markitdown` on PATH, else `uvx markitdown[all]` - anydoc `anydoc` on PATH, a cargo build in third-party/anydoc, or a local - npm install of @firecrawl/anydoc + anydoc-rust third-party/anydoc/target/release/examples/convert (cargo build + --release --example convert) — the Rust binary alone + anydoc-node the npm package, whose CLI is a Node script loading a napi module, + so its timings include Node startup Metrics per output: content recall tokens agreed on by >=2 engines that this engine also emits @@ -32,6 +35,9 @@ from pathlib import Path REPO = Path(__file__).resolve().parent.parent +# anydoc converts binary document formats only. +ANYDOC_UNSUPPORTED = {"html", "htm", "json", "xml", "md", "txt"} + TOKEN_SPLIT = re.compile(r"[\s!\"#$%&'()*+,\-./:;<=>?@\[\\\]^_`{|}~]+") RUNS = 3 @@ -85,10 +91,33 @@ def which_engines() -> list[Engine]: markitdown = None engines.append(Engine("markitdown", markitdown, "" if markitdown else "pip install markitdown[all]")) + native = REPO / "cli-native/build/bin/macosArm64/releaseExecutable/cli-native.kexe" + engines.append( + Engine( + "native", + [str(native)] if native.exists() else None, + "" if native.exists() else "run ./gradlew :cli-native:linkReleaseExecutableMacosArm64", + # The Kotlin/Native target carries only the converters that need no JVM library. + unsupported={"docx", "xlsx", "pptx", "epub", "pdf", "html", "htm"}, + ) + ) + + # Two anydoc entry points, because they measure different things: the npm package ships a + # napi module that Node loads in-process, so timing it charges Node's ~15 ms startup to Rust. + # The cargo-built example is the Rust binary on its own. + native_anydoc = REPO / "third-party/anydoc/target/release/examples/convert" + engines.append( + Engine( + "anydoc-rust", + [str(native_anydoc)] if native_anydoc.exists() else None, + "" if native_anydoc.exists() else "cargo build --release --example convert in third-party/anydoc", + unsupported=ANYDOC_UNSUPPORTED, + ) + ) + anydoc_bin = shutil.which("anydoc") if not anydoc_bin: for candidate in ( - REPO / "third-party/anydoc/target/release/anydoc", REPO / "build/anydoc/node_modules/.bin/anydoc", Path("/tmp/anydoc-bench/node_modules/.bin/anydoc"), ): @@ -97,11 +126,10 @@ def which_engines() -> list[Engine]: break engines.append( Engine( - "anydoc", + "anydoc-node", [anydoc_bin] if anydoc_bin else None, "" if anydoc_bin else "npm i @firecrawl/anydoc", - # anydoc converts binary document formats only. - unsupported={"html", "htm", "json", "xml", "md", "txt"}, + unsupported=ANYDOC_UNSUPPORTED, ) ) return engines diff --git a/scripts/optbench.py b/scripts/optbench.py new file mode 100644 index 0000000..028d32d --- /dev/null +++ b/scripts/optbench.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +"""Build, verify and A/B time the native CLI against the current champion binary. + +Usage: + python3 scripts/optbench.py "label of the change" + python3 scripts/optbench.py --promote # current build becomes the champion + +Absolute timings drift between sessions — the same binary measured 60 ms one hour and 74 ms the +next — so a change is only ever compared against the champion, interleaved, in the same run. +Output is verified against the recorded baselines first: a change that alters what we produce is +reported as broken rather than as fast. +""" +import os +import shutil +import subprocess +import sys +import time +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +JVM = REPO / "cli/build/install/cli/bin/cli" +NATIVE = REPO / "cli-native/build/bin/macosArm64/releaseExecutable/cli-native.kexe" +CHAMPION = REPO / "build/perf/champion.kexe" +CHAMPION_JVM = REPO / "build/perf/champion-cli" +FIXTURES = REPO / "library/src/commonTest/resources/test_files" +BASELINES = REPO / "build/benchmark" +PERF = REPO / "build/perf" + +# Native only runs CSV; the JVM champion covers the document formats it cannot. +TIMED = [("580 KB", PERF / "medium.csv"), ("1.8 MB", PERF / "big.csv")] +TIMED_JVM = [ + ("wiki", FIXTURES / "test_wikipedia.html"), + ("docx", FIXTURES / "test.docx"), + ("pdf", FIXTURES / "test.pdf"), + ("json", PERF / "big.json"), +] +ROUNDS = 7 + + +def build(): + result = subprocess.run( + ["./gradlew", ":cli:installDist", ":cli-native:linkReleaseExecutableMacosArm64", + "--no-configuration-cache", "-q"], + capture_output=True, text=True, cwd=REPO, + ) + errors = [line for line in (result.stdout + result.stderr).splitlines() if line.startswith("e:")] + return result.returncode == 0, errors[:3] + + +def verify(): + problems = [] + for baseline in sorted(BASELINES.glob("mikromarkdown_*.md")): + name = baseline.name.replace("mikromarkdown_", "").removesuffix(".md") + fixture = FIXTURES / name + if not fixture.exists(): + continue + if subprocess.run([str(JVM), str(fixture)], capture_output=True, cwd=REPO).stdout != baseline.read_bytes(): + problems.append(f"jvm {name}") + for name in ("test.csv", "test.json", "test.xml"): + fixture = FIXTURES / name + jvm = subprocess.run([str(JVM), str(fixture)], capture_output=True, cwd=REPO).stdout + native = subprocess.run([str(NATIVE), str(fixture)], capture_output=True, cwd=REPO).stdout + if jvm != native: + problems.append(f"native != jvm {name}") + big = PERF / "big.csv" + if big.exists(): + jvm = subprocess.run([str(JVM), str(big)], capture_output=True, cwd=REPO).stdout + native = subprocess.run([str(NATIVE), str(big)], capture_output=True, cwd=REPO).stdout + if jvm != native: + problems.append("native != jvm big.csv") + return problems + + +def interleaved(path, champion=None, candidate=None): + """Alternate champion and candidate so drift hits both equally. Returns (champion, candidate).""" + champion = champion or CHAMPION + candidate = candidate or NATIVE + # A copied distribution cannot use its CDS archive: the archive records absolute classpaths, and + # the JVM drops it without a word. Both sides run without it so the comparison is of the code. + environment = dict(os.environ, MIKROMARKDOWN_NO_CDS="1") + champion_times, candidate_times = [], [] + for _ in range(ROUNDS): + for binary, times in ((champion, champion_times), (candidate, candidate_times)): + start = time.perf_counter() + subprocess.run([str(binary), str(path)], capture_output=True, cwd=REPO, env=environment) + times.append((time.perf_counter() - start) * 1000) + return min(champion_times), min(candidate_times) + + +def main(): + if "--promote" in sys.argv: + CHAMPION.parent.mkdir(parents=True, exist_ok=True) + shutil.copy(NATIVE, CHAMPION) + if CHAMPION_JVM.exists(): + shutil.rmtree(CHAMPION_JVM) + shutil.copytree(JVM.parent.parent, CHAMPION_JVM) + print("champion updated (native binary and JVM distribution)") + return 0 + + label = sys.argv[1] if len(sys.argv) > 1 else "unlabelled" + ok, errors = build() + if not ok: + print(f"{label}: BUILD FAILED") + for error in errors: + print(f" {error}") + return 1 + + problems = verify() + if problems: + print(f"{label}: OUTPUT CHANGED -> {', '.join(problems)}") + return 2 + + if not CHAMPION.exists(): + print(f"{label}: no champion yet — run --promote first") + return 3 + + parts = [] + for name, path in TIMED: + if not path.exists(): + continue + champion, candidate = interleaved(path) + delta = (candidate - champion) / champion * 100 + parts.append(f"{name}: {champion:.0f} -> {candidate:.0f} ({delta:+.0f}%)") + + champion_cli = CHAMPION_JVM / "bin/cli" + if champion_cli.exists(): + for name, path in TIMED_JVM: + if not path.exists(): + continue + champion, candidate = interleaved(path, champion=champion_cli, candidate=JVM) + delta = (candidate - champion) / champion * 100 + parts.append(f"{name}: {champion:.0f} -> {candidate:.0f} ({delta:+.0f}%)") + print(f"{label}: " + " | ".join(parts)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/settings.gradle.kts b/settings.gradle.kts index d74dfc8..ad6e964 100644 --- a/settings.gradle.kts +++ b/settings.gradle.kts @@ -20,3 +20,5 @@ include(":library") include(":cli") include(":benchmark") + +include(":cli-native")