Palimpsest (named after the ancient manuscript pages scraped clean and rewritten over) is a high-performance, cross-platform C++17 CLI tool designed to reconstruct clean, formatted digital documents (.txt or .docx) from screen recordings of someone scrolling through text, PDFs, presentations, or documentation.
When people record presentations, documentation, code, or manuals by scrolling through them on screen, the resulting video contains thousands of redundant frames, motion blur, overlapping text segments, and visual window chrome (toolbars, ribbons, taskbars).
Palimpsest automates the entire document recovery workflow:
- Intelligent Dwell & Frame Decimation: Discards 99%+ of identical video frames while guaranteeing crystal-clear OCR by capturing frames that settle for a configurable dwell period.
- ROI Cropping: Strips UI chrome and window borders via bounding boxes, corner coordinates, or CSV files before text processing.
- Hardware-Accelerated Preprocessing: Employs Gaussian filtering and Hough line deskewing across CPU SIMD cores or iGPU OpenCL compute units.
- Multithreaded OCR: High-throughput parallel OCR via multithreaded Tesseract LSTM worker pool with protected digit confidence.
- Garbage & Noise Filtering: Discards low-confidence words and heuristically detected noise lines (symbol runs, fragmented characters) before stitching.
- Fuzzy Suffix-Prefix Stitching: Automatically aligns and deduplicates overlapping lines across scrolled frames without duplicating content.
- Cross-GAP Deduplication: Maintains a rolling seen-lines history across content gaps so previously seen text is never repeated even after scroll jumps.
- Ignore Phrase Stripping: Filters watermarks, headers, and recurring boilerplate via exact substring or fuzzy phrase matching (
ignore_phrases.txt). - Config File Support: All parameters readable from a
key=valueconfig file (config.csv) with CLI flags as overrides. - Native Document Generation: Directly outputs clean plain text or fully valid Office Open XML (
.docx) archives compatible with Microsoft Word and LibreOffice Writer.
Note on Accuracy & Human Review: Palimpsest achieves >90% accuracy (94.56% in benchmarks) on reconstructed text. Because screen recordings inherently contain video compression artifacts, motion blur, and watermarks, the generated output is not 100% perfect and may occasionally contain minor OCR artifacts or missing edge words. The final document may require quick human review and light manual editing to achieve maximum accuracy.
Note on Output Files: When Palimpsest starts, it automatically truncates / overwrites the destination output file (e.g.
output.txtor.docx) with a fresh document. To preserve existing output, specify a different destination with-o / --outputoroutput = <path>inconfig.csv.
Important
Palimpsest works best on videos where the presenter pauses on content for a few seconds at a time — lecture recordings, documentation walkthroughs, slide-by-slide presentations, terminal sessions, and similar material. Scrolling is handled fine, but if the screen is in continuous motion every couple of seconds without settling, the dwell gating will forward very few or no frames, and results may be sparse or empty. For fast, non-stop scrolling recordings, consider lowering stable-dwell-ms or switching to motion-threshold mode (stable-dwell-ms = 0).
All benchmarks run on a 1200p (1920×1200), H.264 @ 6 Mbps, 60 FPS screen recording (21,000 frames / 5m 50s).
Test Machine & Build:
- Build Type: Release build (
-DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3 -march=native") - CPU: AMD Ryzen AI 7 350 (8 Cores, 16 Threads)
- RAM: 16 GB DDR5 @ 5600 MT/s (Single-Channel, ~44 GB/s)
- GPU: AMD Radeon 860M (8 CUs, 8056 MB, rusticl / ACO, DRM 3.64, Linux 7.1.8-arch1-3)
Typical real-world use — 21,000 frames decoded, but only a handful forwarded to OCR after dwell gating. Bottleneck: video decoder. Avg FPS = decoded frames (21,000) ÷ elapsed time.
| Backend | OCR Threads | Forwarded to OCR | Time | Avg FPS | Real-Time Factor |
|---|---|---|---|---|---|
| CPU (SIMD) | 14 (auto) | 5 | 1m 06s | 318 FPS | ~5.3× |
| OpenCL (iGPU) | 14 (auto) | 5 | 1m 05s | 323 FPS | ~5.4× |
| CPU (SIMD) | 16 (full) | 5 | 1m 05s | 323 FPS | ~5.4× |
In dwell mode almost no frames reach preprocessing, so CPU and OpenCL backends are decode-bound and perform identically.
Forces ~20,000 uniformly sampled frames through the full preprocessing + Tesseract OCR pipeline, bypassing dwell gating. Bottleneck: Tesseract OCR (always CPU). Avg FPS = forwarded frames ÷ elapsed time.
| Backend | Forwarded Frames¹ | OCR'd | Time | Avg FPS² | Per-Forwarded Frame |
|---|---|---|---|---|---|
| CPU (SIMD) | 20,000 | 10,710 | 2m 47s | ~120 FPS | ~8.3 ms/frame |
| OpenCL (iGPU) | 17,157 | 8,594 | 2m 41s | ~107 FPS | ~9.4 ms/frame |
¹ The OpenCL run forwarded fewer frames (17,157) than the CPU run (20,000) because it reached the physical end of the 21,000-frame video earlier after stream/empty drops (a known bench-mode artifact when
stride ≈ 1). Both runs processed substantial OCR work.² Average FPS = forwarded frames ÷ total elapsed time, calculated independently of the in-progress terminal display.
Real-world note: Most recordings do not contain 20,000 unique, non-overlapping content frames. A typical 1-hour lecture with 5-second pauses produces ~700 unique dwell frames — not 20,000. This stress test represents a pathological maximum load scenario.
Both backends produce nearly identical throughput in real-world usage. The true bottleneck is always Tesseract OCR, which runs on CPU regardless of backend selection. The preprocessing ops (grayscale, Gaussian blur, deskew) account for a small fraction of per-frame time.
Use OpenCL if: You want to offload the image preprocessing work from your CPU cores to the iGPU so other applications stay more responsive during a long batch run.
Use CPU if: You want maximum stability and predictability, or your system's OpenCL driver (e.g. Mesa Rusticl) adds overhead that negates the benefit.
./build/bin/palimpsest <input_video> [OPTIONS]# Simplest run — auto output path, CPU backend, English OCR
./build/bin/palimpsest input_video.mkv
# Load all settings from config file (recommended)
./build/bin/palimpsest input_video.mkv -c config.csv
# With OpenCL GPU backend + crop region + DOCX output
./build/bin/palimpsest input_video.mkv -b opencl --crop crop_coords.txt -f docx -o result.docx
# With custom ignore phrases file
./build/bin/palimpsest input_video.mkv --ignore ignore_phrases.txt -o clean.txt
# Inspect GPU / OpenCL hardware capabilities and exit
./build/bin/palimpsest input_video.mkv -b opencl --noshit| Short | Long | Default | Description |
|---|---|---|---|
-o |
--output |
<input>.txt |
Output file path |
-f |
--output-format |
txt |
txt or docx |
-b |
--backend |
cpu |
cpu or opencl |
-c |
--config |
— | Path to config file (see config.csv) |
-l |
--lang |
eng |
Tesseract language code (e.g. eng+fra) |
--crop |
— | x,y,w,h · 4-corner coords · or CSV file path |
|
--ignore |
— | Phrase string or path to .txt file (one per line) |
|
--threads |
hw_concurrency - 2 |
OCR worker thread count | |
--filter-threshold |
2.5 |
Skip frames with pixel diff below this | |
--jump-threshold |
40.0 |
Flag hard boundary when pixel diff exceeds this | |
--min-interval-ms |
150 |
Min ms between forwarded frames | |
--stable-dwell-ms |
5000 |
Ms frame must stay stable to force OCR | |
--stable-threshold |
5.0 |
Pixel diff threshold for "same stable frame" | |
--overlap-threshold |
75.0 |
Fuzzy score to trigger line stitching | |
--overlap-window |
300 |
Chars used for overlap search | |
--seen-threshold |
75.0 |
Fuzzy score for cross-GAP duplicate detection | |
--seen-window |
50 |
Rolling seen-lines history size | |
--min-alpha-ratio |
0.4 |
Min fraction of alpha chars to keep a line | |
--min-line-length |
3 |
Min chars — shorter lines are discarded | |
--min-word-confidence |
40.0 |
Min Tesseract word confidence % | |
-d |
--dump-images |
— | Dump clean color cropped OCR frames as 1.png, 2.png into images/ |
--bench-preprocess |
0 |
Benchmark mode: forward N uniformly sampled frames through preprocess + OCR, implies --dry-run |
|
--verbose |
— | Print per-frame telemetry | |
--dry-run |
— | Run pipeline without writing output | |
--noshit |
— | Print hardware stats and exit |
All parameters above (except
--verbose,--dry-run,--noshit,--dump-images,--bench-preprocess) can be set inconfig.csv. The includedconfig.csvat the project root is a fully documented reference — every key, its default, its range, and what it does.
Load any config file with -c / --config. Both key=value and key,value formats are supported. Lines starting with # are comments. CLI flags always override config file values.
./build/bin/palimpsest input.mkv -c config.csv
./build/bin/palimpsest input.mkv -c config.csv --backend opencl # CLI winsSee config.csv for the fully annotated reference with every key, default, range, and description.
| Problem | Fix |
|---|---|
| Missed stationary slides / pause too short | Lower stable-dwell-ms (e.g. 2000 or 3000 ms) in config.csv |
| Low confidence or missing words | Lower min-word-confidence (e.g. 35 or 40) in config.csv |
| Garbage / symbol lines in output | Raise min-alpha-ratio (e.g. 0.50) and min-line-length (e.g. 4) |
| Duplicate content across slide transitions | Raise seen-threshold (e.g. 85 or 90) or increase seen-window |
| Watermarks or recurring header noise | Add phrases to ignore_phrases.txt (and set ignore = ignore_phrases.txt) |
| Edge text or window ribbons included | Adjust coordinates in crop_coords.txt (and set crop = crop_coords.txt) |
| Need visual verification of OCR frames | Enable image dumping with -d / --dump-images or dump-images = true |
Palimpsest must be compiled from source. All C++ dependencies (OpenCV, Tesseract, Leptonica, rapidfuzz, miniz, fmt, CLI11) are automatically fetched and built via CMake's FetchContent — no external package managers (like vcpkg) are required.
Platform Compatibility: Palimpsest is developed and actively tested on Linux (Arch Linux / Ubuntu 22.04+ with GCC 9+ or Clang 11+). Windows (MSVC 2019/2022) should work with standard CMake build workflows, but is currently untested.
# Clone the repository
git clone https://github.com/plexescor/Palimpsest.git
cd Palimpsest
# Build with provided compile script (runs cmake -B build && cmake --build build -j$(nproc))
./compile.sh- Operating Systems: Linux (GCC 9+, Clang 11+, tested on Arch Linux / Ubuntu 22.04+). Windows (MSVC VS 2019/2022) is untested.
- C++ Standard: C++17 compliant compiler (
-std=c++17or/std:c++17). - Build System: CMake 3.21 or newer.
- Dependencies (All automatically fetched via CMake
FetchContent):OpenCV 4.11.0(modules:core,imgproc,imgcodecs,videoio)Tesseract 5.3.4(libtesseractLSTM engine)Leptonica 1.84.1(libleptimage library)rapidfuzz-cpp v3.0.4(header-only fuzzy string alignment)miniz 3.0.2(ZIP compression engine for DOCX){fmt} 10.2.1(console output formatting)CLI11 v2.4.1(command line argument parser)
- Video Playback / Decoding Backends: GStreamer 1.0 (
gstreamer,gst-plugins-base,gst-plugins-good) or native V4L/FFmpeg runtime codecs. - Tessdata:
eng.traineddataplaced in./tessdata/or located viaTESSDATA_PREFIX/ system paths. - OpenCL Runtime (Optional for
--backend opencl):- Linux: Mesa OpenCL / Rusticl / ROCm runtime (
opencl-icd-loader,mesa-opencl). - Windows: AMD / Intel / NVIDIA OpenCL 2.0+ display drivers (untested).
- Linux: Mesa OpenCL / Rusticl / ROCm runtime (
Palimpsest is built as a strictly staged, lock-free producer-consumer pipeline where bounded SPSC (Single-Producer Single-Consumer) circular ring buffers connect all execution stages:
[ Video Source File ]
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 1: Video Decoder Thread (cv::VideoCapture) │
│ - Decodes frames sequentially │
│ - Attaches monotonic frame_index and video timestamp_ms │
└──────────────────────────────┬──────────────────────────────┘
│ SPSC RingBuffer<RawFrame, 128>
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 2: Frame Filter & Dwell Gatekeeper │
│ - Tracks frame stability over time (--stable-dwell-ms) │
│ - Unconditionally forwards settled frames to OCR │
│ - Suppresses redundant frames during scrolling/transitions │
└──────────────────────────────┬──────────────────────────────┘
│ SPSC RingBuffer<FilteredFrame, 64>
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 3: Preprocessor Thread │
│ - Applies ROI cropping (--crop coordinates / CSV) │
│ - Optional dump of clean color frames (--dump-images) │
│ - Grayscale conversion, Gaussian denoising & deskewing │
│ - OpenCL iGPU hardware acceleration support │
└──────────────────────────────┬──────────────────────────────┘
│ Bounded WorkQueue<PreprocessedFrame>
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 4: OCR Worker Pool (N Worker Threads) │
│ - Parallel Tesseract LSTM instances (PageIterator API) │
│ - Native line preservation and protected digit recognition │
└──────────────────────────────┬──────────────────────────────┘
│ Unordered ResultQueue<OcrResult>
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 5: Reorder Buffer (Consumer Thread) │
│ - Keyed by monotonic frame_index in std::map │
│ - Re-sequences out-of-order worker outputs into order │
└──────────────────────────────┬──────────────────────────────┘
│ Ordered Stream
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 6: Deduplication & Stitching │
│ - Token set & substring fuzzy deduplication (rapidfuzz) │
│ - Cross-GAP seen-lines history and ignore phrase filtering │
│ - Appends non-duplicate paragraphs into document state │
└──────────────────────────────┬──────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 7: Structure Recovery │
│ - Heuristic heading detection (H1 / H2 height ratios) │
│ - Bullet list and table column alignment detection │
└──────────────────────────────┬──────────────────────────────┘
│ Structured Blocks
▼
┌─────────────────────────────────────────────────────────────┐
│ Stage 8: Output Writer │
│ - Clean TXT Writer without gap noise │
│ - Native DOCX Writer via Office Open XML & miniz │
└─────────────────────────────────────────────────────────────┘
All inter-stage ring buffers and worker queues are bounded. If downstream OCR workers experience load, the work queue fills up, causing the Preprocessor to pause, which in turn blocks the Frame Filter and Decoder. This automatic upstream backpressure guarantees that process memory usage remains strictly bounded, regardless of video length or resolution.