From 99ba83c0db2b633503f9498ad2adc9746a5ce716 Mon Sep 17 00:00:00 2001 From: mikolaj Date: Mon, 10 Aug 2026 13:34:56 +0200 Subject: [PATCH] Parse documents into a shared model, render Markdown once MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every converter used to build Markdown strings directly, so escaping, table shaping, list indentation and blank-line handling were reimplemented (and drifted) across nine converters in two source sets. Converters now parse into a format-independent Document model — blocks, inlines, tables with spans, and embedded assets — and a single MarkdownRenderer serializes it to GFM. Fixes to output quirks now land in one place. - DocumentConverter.convert -> parse(bytes, info): Document; MikroMarkdown renders - MarkdownRenderer owns escaping, heading clamping, nested lists, GFM tables (ragged rows padded, colspan expanded, pipes escaped, newlines ->
) and fence sizing; MarkdownNormalizer's post-hoc regex pass is gone - HTML/EPUB go through a Jsoup DOM -> model walker instead of flexmark, which is dropped as a dependency - Document and DocumentBuilder are public: MikroMarkdown.parse() returns the model and ConversionResult carries it, so callers can render with their own options Also adds scripts/benchmark.py, comparing output against Python markitdown and Rust anydoc. It found three defects, fixed here: - the Log4j banner PDFBox triggers was printed to stdout, corrupting CLI Markdown - PDF text became a single paragraph; PDFBox paragraph formatting is now enabled - de-hyphenation merged real compounds ("conversation-centric"); it now keeps the hyphen when both fragments appear as standalone words in the document PythonComparisonTest normalizes Unicode spacing (Java's \s ignores hair/thin spaces, Python's does not) and ignores