Kotlin/Native target, and the performance work it started - #6
Merged
Conversation
Measures whether a native binary can compete with anydoc before committing to
porting the parsers that actually need work (OOXML, PDF).
A macosArm64 target carries CSV, JSON and XML — the three formats that need no
JVM library — plus plain text and Markdown passthrough. The model, renderer and
pipeline come along unchanged from commonMain, which is the point: only the
parsers had to be written.
- CSV is parsed by hand (RFC 4180, ~60 lines) instead of commons-csv
- JSON goes through kotlinx-serialization instead of Jackson
- XML is re-indented by a small formatter instead of javax.xml
- :cli-native links a 2.2 MB binary from those
Output is byte-identical to the JVM CLI for all three formats.
Whole-process, best of six, CSV of increasing size:
input native anydoc jvm-cli
1 KB 3 ms 21 ms 61 ms
55 KB 10 ms 25 ms 73 ms
172 KB 25 ms 30 ms 82 ms
580 KB 75 ms 47 ms 114 ms
1.8 MB 237 ms 93 ms 193 ms
Startup is a rout in native's favour and throughput is not: Kotlin/Native runs
the per-cell table work at about half the JVM's speed, so anydoc leads from
~250 KB and even the JVM CLI overtakes native around 1.5 MB. A plain-text
conversion of the same 3.5 MB file takes native 71 ms against the JVM's 108 ms,
so the deficit is allocation-heavy work rather than IO.
The native CLI lives in its own module rather than the library, so the library
keeps its no-printing rule. Native test compilation is off: the shared
integration tests expect formats this target does not have yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ulating
The native spike matched Rust on startup but ran the per-cell work of a large
table at about half the JVM's speed. Two changes close most of that, and the
first one speeds up every target.
Renderer, shared by all targets:
- escaping scans for the first character that needs a backslash and returns the
input string untouched when there is none, instead of rebuilding every string
through a StringBuilder
- newline normalization only runs when the text actually contains a carriage return
- a single-line table cell skips the split-and-rejoin that produced the same string
- rows that already match the header width are passed through rather than copied
An ordinary table cell — a word, a number — now allocates nothing.
Native CSV reader:
- fields are ranges in the decoded text, so a field costs one substring rather
than a per-character builder plus a separate trim
- only fields with escaped quotes, which cannot be a slice of the input, assemble
a string
Whole process, best of eight, CSV by size:
input native anydoc jvm-cli (native before)
1 KB 3 ms 21 ms 63 ms 3 ms
55 KB 6 ms 25 ms 65 ms 10 ms
172 KB 10 ms 30 ms 77 ms 25 ms
580 KB 29 ms 47 ms 99 ms 75 ms
1.8 MB 87 ms 94 ms 152 ms 237 ms
3.5 MB 176 ms 161 ms 223 ms 498 ms
Native now leads anydoc up to 1.8 MB and trails it by under 10% at 3.5 MB. The
JVM CLI gained from the shared renderer work too, 193 -> 152 ms at 1.8 MB.
Compiler flags were measured, not assumed: -Xbinary=preCodegenInlineThreshold=40
is worth about 8% on large inputs and ships. Every GC binary option tried was
neutral or worse except gcSchedulerType=manual, which is 20% faster on 1.8 MB but
takes peak memory from 125 MB to 169 MB and grows unbounded on larger inputs, so
it is documented rather than enabled.
Output is byte-identical: native matches the JVM CLI on all three native formats,
and the JVM output is unchanged across all eight fixtures.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gcSchedulerType=manual means the collector runs only when code calls it, so a
document boundary — where everything the previous conversion allocated is dead —
becomes a place to schedule one. Converting 20 documents of 580 KB in one process:
default (adaptive) 543 ms 59 MB
manual, never collecting 428 ms 954 MB
manual, collecting between documents 485 ms 67 MB
autotune off with a heap ceiling 1246 ms 43 MB
Collecting per document does recover the memory and keeps some of the speed, but
it only bounds growth between documents: nothing collects mid-parse, so a single
large input grows unchecked either way. Turning autotune off is 8.6x slower on one
3.5 MB file, and the ceiling value changes nothing, so targetHeapBytes does not
behave as its name suggests.
The default collector stays. The native CLI now takes several files per invocation,
which is what would make a manual policy workable if that trade ever pays.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both found by scaling pathological inputs rather than by reading the code, and
both verified byte-identical against the previous implementation.
Nested blocks were charged once per level above them. Each block returned its own
string, so a list or quote split its whole subtree into lines and re-joined it to
add one level of indentation — the deepest leaf paid for every ancestor. Blocks
now write into one buffer, carrying a line prefix that is emitted when a newline
is written, so every character is written exactly once. Trailing blanks are
trimmed at each newline, which is what keeps a quote's "> " as ">" on empty lines
and stops an empty list item leaving "- " behind.
The entity check searched the rest of the string for a semicolon on every
ampersand it met, so text with many ampersands and few semicolons — query
strings, for instance — cost O(n^2). It now scans ten characters, the longest
entity name worth looking for.
Render time on inputs built to provoke them:
400-deep nested lists 33.03 ms -> 0.12 ms
400 ampersands per cell, 789 KB 9.39 ms -> 2.42 ms
Both scale linearly now. Real documents gain less, since neither pattern is
common: Wikipedia renders in 1.51 ms against 1.82 ms.
A third suspicion did not survive measurement: the backward whitespace scan in
the line-start check never showed superlinear growth, because the character
before a marker is almost never a space. It is unchanged.
Output is byte-identical across all ten fixtures and all three pathological inputs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every anydoc figure quoted so far was measured through node_modules/.bin/anydoc,
which is a Node script loading a napi module. Node's startup is 15 ms on this
machine, so anydoc read as a flat ~22 ms regardless of input, and the comparison
flattered us on small documents.
The npm package ships no standalone executable, so the honest binary comes from
building the vendored source: cargo build --release --example convert. The
benchmark harness now runs both and names them apart.
Corrected, whole process, best of eight:
input kotlin-native anydoc-rust anydoc-node jvm-cli
1 KB 3 ms 3 ms 21 ms 63 ms
55 KB 5 ms 5 ms 24 ms 65 ms
172 KB 10 ms 10 ms 30 ms 73 ms
580 KB 29 ms 27 ms 45 ms 95 ms
1.8 MB 86 ms 73 ms 93 ms 151 ms
3.5 MB 171 ms 141 ms 163 ms 222 ms
So Kotlin/Native matches Rust up to ~172 KB, where both are just process startup,
and trails by 7% at 580 KB growing to 21% at 3.5 MB. The earlier claim that it beat
anydoc at every size up to 1.8 MB was an artifact of timing Node.
The document fixtures were also overstated: anydoc converts docx in 4 ms and pdf
in 6 ms, not the 22-25 ms the wrapper suggested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
Correction: every anydoc number above was measured through Node
Rebuilt the real binary from the vendored source (
Kotlin/Native matches Rust up to ~172 KB — at that size both are almost entirely process startup — and trails by 7% at 580 KB, growing to 21% at 3.5 MB. My earlier claim that it beat anydoc at every size through 1.8 MB was an artifact of timing Node. The document fixtures were overstated the same way: anydoc does docx in 4 ms and pdf in 6 ms, not the 22–25 ms the wrapper suggested. The JVM CLI's real gap on those is 187 ms vs 4 ms, not 190 vs 22. What does not change: the conclusion. Native is still the right shape for a CLI and for iOS, the throughput deficit is still real and still concentrated in allocation-heavy work, and the renderer fixes still stand on their own. The comparison is just no longer flattering us by 18 ms.
|
The native spike wrote these three formats without a platform library, and its output was byte-identical to the JVM's. That made the JVM versions redundant, so the native ones move to commonMain and commons-csv, Jackson and kotlinx-serialization all go. JSON now goes through a re-indenter that copies tokens verbatim, which also means numbers keep their source spelling rather than surviving a parse and re-print; 1.50 stays 1.50. Checked against escapes, empty containers, deep nesting, big integers and top-level arrays, comparing parsed data rather than text. - a JSON conversion loads 1214 classes, down from 2063 - the native binary is 1.3 MB, down from 2.2 MB - the JVM CLI converts JSON in 56 ms, down from 95 ms, and a 1.8 MB CSV in 126 ms, down from 151 ms, since the slicing reader beats commons-csv - the Konsist duplicate-file rule is strict again: nothing is copied per target except the factory and the PDF converter Two things measured and rejected. Binary size does not drive startup — a hello-world Kotlin/Native binary of 485 KB starts in 3.2 ms, ours at 1.3 MB in 3.5 ms, and anydoc's 6 MB Rust binary in 2.6 ms — so stripping and shrinking buy nothing. Replacing clikt with hand-rolled parsing removes 147 classes and about 2 ms, inside the noise and not worth the help text it would cost. Output is byte-identical across all ten fixtures on both targets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Trying zero-copy parsing produced a useful negative result and one keeper.
The idea was to hand the renderer a CharSequence window onto the source rather
than a substring per cell. It is slower: every character access becomes an
interface call instead of String's direct indexing, and the renderer reads every
character anyway to decide whether it needs escaping. A 1.8 MB CSV went from
86 ms to 118 ms on native, 126 ms to 154 ms on the JVM. Rust gets this for free
because &str slices index directly. Reverted, along with the CharSequence in the
public model that it needed.
What survived is the fast path found along the way: a table cell holding a single
run of plain text is escaped directly, instead of being pushed through a
StringBuilder to reach the same characters. Combined with escaping that already
returned its input untouched when nothing needed a backslash, an ordinary cell now
reaches the output without being copied at all.
input kotlin-native anydoc-rust jvm-cli (native before)
55 KB 5 ms 6 ms 62 ms 5 ms
580 KB 27 ms 26 ms 77 ms 29 ms
1.8 MB 76 ms 72 ms 115 ms 86 ms
3.5 MB 145 ms 141 ms 167 ms 171 ms
Native is now within 3-6% of the Rust binary at every size, where it trailed by
18-21% before. The JVM CLI gained too, 126 ms to 115 ms at 1.8 MB.
Output is byte-identical across all ten fixtures on both targets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Table rows and simple tables write straight into the document buffer, CSV scans raw bytes instead of decoding the whole file, an ASCII table rejects characters that cannot start markup, and appendLines copies whole lines. Together -4% on a 1.8 MB CSV against the pre-session binary, and native peak RSS on that input drops from 125 MB to 81 MB. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The archive was trained on a DOCX alone, so a PDF or HTML conversion loaded most of its classes the slow way. Training over one sample of each family cuts PDF from 203 ms to 160 ms and Wikipedia from 139 ms to 124 ms. Recording has to happen in a single run: merging class lists from separate runs drops the loader metadata and made JSON 16% slower. That needs the CLI to accept several files, which it now does, matching the native one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A plain table cell now keeps its string and builds List<Inline> only when asked, which is worth -22% on a 1.8 MB CSV by itself: a table of that size was allocating a list and a Text per cell for the renderer to unwrap again immediately. CSV then builds cells while scanning the bytes, for another -5%. Together with the earlier iterations, a 1.8 MB CSV goes from 76 ms to 54 ms and a 580 KB one from 28 ms to 20 ms. Document formats are unchanged. Sixteen ideas were measured and discarded, including pre-sizing the output buffer, a concurrent collector, smallBinary codegen, and four separate attempts at skipping work that turned out not to be on any hot path. docs/optimization-log.md records each one so they are not tried again. Two methodology fixes matter more than most of the wins. Timings drift between sessions, so every measurement is now a champion and a candidate interleaved in one run — that correction alone revised iterations 1-4 from -20% to -4%. And the harness verifies output before reporting time: the one experiment I ran outside it reported a 20 ms conversion that was really the JVM failing to start. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The README and the optimization log carried numbers from before the last two iterations. Re-measured on an idle machine: the native CLI converts a 580 KB CSV in 19 ms and a 1.8 MB one in 51 ms, against the Rust binary's 25 ms and 71 ms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lemcoder
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5 — merge that first, or GitHub retargets this at
mainand the diff swallows #5's commits.Started as a spike to answer one question — can Kotlin/Native compete with Rust — and turned into the performance work that followed from the answer.
Where it ends up
CSV, whole process, best of 10:
Document fixtures, where the native target has no converters yet:
And output quality, which speed alone does not capture:
We have the widest coverage at full recall; anydoc is fastest per format but drops the most content (74% on EPUB) and handles the fewest.
What is in the branch
A
macosArm64target carrying the formats that need no platform library — CSV, JSON, XML, plain text, Markdown passthrough — plus:cli-native. The model, renderer and pipeline came fromcommonMainunchanged; only parsers had to be written. Output is byte-identical to the JVM CLI.Three dependencies gone. Since the native implementations matched the JVM's byte for byte, the JVM versions were redundant: commons-csv, Jackson and kotlinx-serialization all left, and CSV/JSON/XML moved to
commonMain. A JSON conversion loads 1214 classes instead of 2063.Two quadratics removed from the renderer, both found by scaling pathological inputs rather than by reading code: nested blocks were charged once per level above them (400-deep lists: 33 ms → 0.12 ms), and the entity check scanned the rest of the document on every
&(789 KB of ampersands: 9.4 ms → 2.4 ms).Twenty-five measured experiments, nine kept, recorded individually in
docs/optimization-log.md. Almost all the gain came from one: a plain table cell now keeps its string and buildsList<Inline>only if something asks, worth -22% on a 1.8 MB CSV. A 60k-row table had been allocating a list and aTextper cell for the renderer to unwrap again.What was tried and rejected
Recorded so nobody spends the time again:
CharSequenceslices — 35-40% slower. Every character read becomes an interface call, and the renderer reads every character anyway to decide about escaping. Rust gets this free because&strindexes directly.gcSchedulerType=manual— 21% faster in batch, 16x the memory, and it bounds nothing within a single document.smallBinary(+6%), pre-sized output buffer (+6%), hand-rolled argument parsing (~2 ms, inside noise), and eleven others worth nothing at all.Two methodology corrections
Both cost more than most of the wins, and both are in the log:
Absolute timings drift. The same unchanged binary measured 60 ms in one session and 74 ms in the next, which credited four iterations with -20% when they were worth -4%. Every number here now comes from a champion and a candidate interleaved in a single run.
A benchmark that does not verify its output measures nothing. Disabling bytecode verification looked like a 20 ms conversion; it was the JVM refusing to start.
scripts/optbench.pyverifies every fixture on both targets before it reports a timing, and the one experiment I ran outside it was the one that lied.An earlier correction stands too: every anydoc figure before that point was measured through its Node wrapper, which added ~18 ms and flattered us. These come from the Rust binary.
Verification
./gradlew checkpasses. Output is byte-identical across all ten fixtures on both targets, and the harness refuses to report a timing otherwise.🤖 Generated with Claude Code