From 9229bc4a99ed44bdccff809bddd97a12aa1d94e3 Mon Sep 17 00:00:00 2001 From: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 2 Jun 2026 19:26:52 +0100 Subject: [PATCH 01/16] =?UTF-8?q?ci(codeql):=20cron=20weekly=E2=86=92month?= =?UTF-8?q?ly=20(cut=203,=20standards#288)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per `standards#286` canonical (cut 3, Option B 2026-05-30): convert CodeQL scheduled run from weekly `0 6 * * 1` to monthly `0 6 1 * *`. PR-trigger runs unchanged — every PR still gets CodeQL. Refs `hyperpolymath/standards#288` (campaign). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/codeql.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ab8ae8b..20d02b2 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -7,7 +7,7 @@ on: pull_request: branches: [main, master] schedule: - - cron: '0 6 * * 1' + - cron: '0 6 1 * *' # monthly 1st 06:00 UTC # Estate guardrail: cancel superseded runs so re-pushes / rebased PR # updates do not pile up queued runs against the shared account-wide From 5a86b99b9e14adb08b97589fd18dab5254e4fcb6 Mon Sep 17 00:00:00 2001 From: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Date: Mon, 29 Jun 2026 14:16:08 +0100 Subject: [PATCH 02/16] revert(readme): restore canonical README.adoc; remove runaway .md conversion The estate-wide 'convert README.adoc -> Markdown' pass over-applied the narrow boj-server/hyperpolymath .md rule to every repo. AsciiDoc is canonical here; restore README.adoc and remove README.md (clean revert). Co-Authored-By: Claude Opus 4.8 (1M context) --- README.adoc | 240 ++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 259 ---------------------------------------------------- 2 files changed, 240 insertions(+), 259 deletions(-) create mode 100644 README.adoc delete mode 100644 README.md diff --git a/README.adoc b/README.adoc new file mode 100644 index 0000000..e10130c --- /dev/null +++ b/README.adoc @@ -0,0 +1,240 @@ += Docudactyl +image:https://img.shields.io/badge/License-PMPL_1.0-blue.svg[MPL-2.0,link="https://github.com/hyperpolymath/palimpsest-license"] + +:toc: +:sectnums: +:source-highlighter: rouge + +// Badges +image:https://img.shields.io/badge/RSR-Tier%201-gold[RSR Tier 1] +image:https://img.shields.io/badge/Phase-v0.4.0-green[Phase] +image:https://img.shields.io/badge/Chapel-2.3+-4E9A06?logo=data:image/svg+xml;base64,[Chapel] +image:https://img.shields.io/badge/Zig-0.15+-F7A41D?logo=zig[Zig] +image:https://img.shields.io/badge/Idris2-0.8+-5E5086[Idris2] +image:https://img.shields.io/badge/OCaml-4.14+-EC6813?logo=ocaml[OCaml] +image:https://img.shields.io/badge/Ada-2022-blue[Ada] + +== License & Philosophy + +This project is licensed under **MPL-2.0** (Palimpsest License). + +The full licence text is in `license/PMPL-1.0.txt`. The canonical source is the https://github.com/hyperpolymath/palimpsest-license[palimpsest-license] repository. + +== Overview + +**Docudactyl** is a multi-format HPC document extraction engine designed for British Library scale (~170 million items). It processes PDFs, images, audio, video, EPUB, and geospatial data across hundreds of cluster nodes. + +=== Architecture + +[source] +---- +┌──────────────────────────────────────────────────────────────────┐ +│ Chapel HPC Orchestrator │ +│ (64-512 locales, dynamic load balancing) │ +├──────────────────────────────────────────────────────────────────┤ +│ Conduit │ L1/L2 Cache │ Checkpoint │ Progress Reporter │ +│ (validate) │ (LMDB+DFly) │ (resume) │ (ETA, rate) │ +├──────────────────────────────────────────────────────────────────┤ +│ Zig FFI Layer │ +│ (51 C-exported functions, zero overhead) │ +├────────┬──────────┬──────────┬──────────┬──────────┬────────────┤ +│Poppler │Tesseract │ FFmpeg │ libxml2 │ GDAL │ libvips │ +│ (PDF) │ (OCR) │(AV meta) │ (EPUB) │ (Geo) │ (Image) │ +├────────┴──────────┴──────────┴──────────┴──────────┴────────────┤ +│ dlopen: ONNX Runtime (ML) │ PaddleOCR (GPU OCR) │ CUDA │ +├──────────────────────────────────────────────────────────────────┤ +│ Idris2 ABI Proofs (14 types, 5 struct layouts, 51 FFI decls) │ +└──────────────────────────────────────────────────────────────────┘ + +Offline: OCaml docudactyl-scm (JSON/text → Scheme S-expressions) +Viewer: Ada TUI (interactive document inspection) +Legacy: Julia extraction scripts (replaced by Chapel pipeline) +---- + +=== Performance Estimates (British Library, 170M items) + +[cols="1,2"] +|=== +|Scenario |Estimate + +|Cold run (256 nodes + GPU) +|~3.7 hours + +|Warm run (L1+L2 cache) +|~4.4 minutes + +|Incremental (5% new files) +|~8 minutes +|=== + +== Quick Start + +[source,bash] +---- +# Verify dependencies +just deps-check + +# Build Zig FFI + Chapel binary +just build-hpc + +# Run all tests +just test-hpc + +# Process a directory of documents +just generate-manifest /path/to/documents manifest.txt +bin/docudactyl-hpc --manifestPath=manifest.txt --outputDir=output/ + +# Or on an HPC cluster (64 nodes) +sbatch deploy/slurm-docudactyl.sh +---- + +== Components + +=== Chapel: HPC Engine (hot path) + +The Chapel component distributes document processing across cluster nodes with dynamic load balancing. + +Modules: Config, ContentType, FFIBridge, ManifestLoader, NdjsonManifest, FaultHandler, ProgressReporter, ShardedOutput, ResultAggregator, Checkpoint, DocudactylHPC. + +=== Zig FFI: Parser Dispatch Layer + +10 submodules providing a unified C ABI for 7 content types and 20 processing stages: + +* **Core**: `docudactyl_ffi.zig` -- init, free, parse, version (dispatches by content type) +* **Stages**: 20 analysis stages with Cap'n Proto output (language, readability, keywords, citations, OCR confidence, perceptual hash, TOC, NER, Whisper, image classify, layout, handwriting, etc.) +* **Cache**: L1 LMDB per-locale (zero-copy mmap) + L2 Dragonfly cross-locale +* **Conduit**: Magic-byte content detection (15 formats), SHA-256, validation +* **GPU OCR**: PaddleOCR CUDA > Tesseract CUDA > CPU (via dlopen) +* **ML Inference**: ONNX Runtime -- NER, Whisper, ImageClassify, Layout, Handwriting (TensorRT > CUDA > OpenVINO > CPU) +* **Hardware Crypto**: SHA-NI, AVX2, AVX-512, AES-NI, ARM SHA2 acceleration +* **I/O Prefetch**: io_uring (Linux 5.6+) with posix_fadvise fallback + +=== Idris2: Formal ABI Proofs + +Dependent types proving struct layout, alignment, and enum correctness: + +* 14 proven types (ContentKind, ParseStatus, MlStatus, MlStage, ExecProvider, Sha256Tier, etc.) +* 5 struct layout proofs (ParseResult 952B, MlResult 48B, CryptoCaps 16B, OcrResult 48B, ConduitResult 88B) +* 51 FFI declarations matching the C header 1:1 + +=== OCaml: Offline Scheme Transformer + +Transforms extracted JSON/text into machine-readable Scheme S-expressions. Not in the HPC hot path. + +[source,bash] +---- +docudactyl-scm document.pdf -o document.scm +docudactyl-scm extracted.json -o extracted.scm +---- + +=== Ada: Terminal UI + +Interactive viewer for inspecting extracted documents. + +[source,bash] +---- +docudactyl-tui extracted.json +---- + +== Justfile Recipes + +[source,bash] +---- +# Build +just build-hpc # Zig FFI + Chapel binary +just build-ffi # Zig FFI only +just build-idris # Idris2 ABI proofs +just build-ocaml # OCaml transformer +just build-ada # Ada TUI + +# Test +just test-hpc # All HPC tests (FFI + error paths) +just test-ffi # Zig integration tests (40+ tests) +just test-scale # Scale test (2105+ files) +just test-idris # Idris2 proofs compile +just test-ocaml # OCaml tests +just test-ada # Ada build check + +# Deploy +just deps-check # Verify dependencies +just generate-manifest [output] +just generate-abi-header +just loc # Lines of code +---- + +== Directory Structure + +[source] +---- +docudactyl/ +├── src/ +│ ├── chapel/ # HPC engine (11 modules) +│ ├── Docudactyl/ABI/ # Idris2 ABI proofs (3 modules) +│ ├── ocaml/ # Offline Scheme transformer +│ ├── ada/ # Terminal UI +│ └── julia/ # Legacy extraction (replaced) +│ +├── ffi/zig/ # Zig FFI layer (10 submodules) +│ ├── src/ # Source +│ └── test/ # Integration tests +│ +├── generated/abi/ # Auto-generated C header +├── schema/ # Cap'n Proto schema +├── deploy/ # Containerfile + Slurm script +├── contractiles/ # K9 contractile configs +├── .machine_readable/ # SCM checkpoint files +├── Justfile # Task runner +└── docudactyl.ipkg # Idris2 package +---- + +== Requirements + +=== System Dependencies + +* **Chapel** 2.3+ (HPC engine) +* **Zig** 0.15+ (FFI layer) +* **Idris2** 0.8+ (ABI proofs) +* **C libraries**: Poppler, Tesseract, FFmpeg, libxml2, GDAL, libvips, LMDB +* **Optional**: ONNX Runtime, PaddleOCR, CUDA (for ML/GPU features) +* **OCaml** 4.14+ (offline Scheme transformer) +* **Ada** GNAT/gprbuild (terminal UI) + +=== Container Deployment + +[source,bash] +---- +podman build -f deploy/Containerfile -t docudactyl-hpc . +podman run --rm -v /data/manifest.txt:/manifest.txt:ro \ + -v /data/output:/output \ + docudactyl-hpc --manifestPath=/manifest.txt +---- + +=== Cluster Deployment (Slurm) + +[source,bash] +---- +# Edit deploy/slurm-docudactyl.sh for your cluster +sbatch deploy/slurm-docudactyl.sh +---- + +== Ethical Use + +This tool is designed for: + +* Document analysis and archival processing at national library scale +* Research and verification of redaction practices +* Accessibility improvements for PDF content +* Multi-format metadata extraction and cataloguing + +== RSR Compliance + +This is a Tier 1 RSR project. The hot path uses Chapel + Zig (systems languages). Legacy components (Julia, OCaml, Ada) serve offline/auxiliary roles. + +== License + +SPDX-License-Identifier: CC-BY-SA-4.0 + +== Links + +* https://github.com/hyperpolymath/docudactyl[GitHub Repository] +* https://rhodium.sh[Rhodium Standard] diff --git a/README.md b/README.md deleted file mode 100644 index 3d3210b..0000000 --- a/README.md +++ /dev/null @@ -1,259 +0,0 @@ - - -![RSR Tier 1](https://img.shields.io/badge/RSR-Tier%201-gold) -![Phase](https://img.shields.io/badge/Phase-v0.4.0-green) -image:[Chapel](https://img.shields.io/badge/Chapel-2.3+-4E9A06?logo=data:image/svg+xml;base64,) -image:[Zig](https://img.shields.io/badge/Zig-0.15+-F7A41D?logo=zig) -image:[Idris2](https://img.shields.io/badge/Idris2-0.8+-5E5086) -image:[OCaml](https://img.shields.io/badge/OCaml-4.14+-EC6813?logo=ocaml) -![Ada](https://img.shields.io/badge/Ada-2022-blue) - -# License & Philosophy - -This project is licensed under **MPL-2.0** (Palimpsest License). - -The full licence text is in `license/PMPL-1.0.txt`. The canonical source -is the -[palimpsest-license](https://github.com/hyperpolymath/palimpsest-license) -repository. - -# Overview - -**Docudactyl** is a multi-format HPC document extraction engine designed -for British Library scale (~170 million items). It processes PDFs, -images, audio, video, EPUB, and geospatial data across hundreds of -cluster nodes. - -## Architecture - - ┌──────────────────────────────────────────────────────────────────┐ - │ Chapel HPC Orchestrator │ - │ (64-512 locales, dynamic load balancing) │ - ├──────────────────────────────────────────────────────────────────┤ - │ Conduit │ L1/L2 Cache │ Checkpoint │ Progress Reporter │ - │ (validate) │ (LMDB+DFly) │ (resume) │ (ETA, rate) │ - ├──────────────────────────────────────────────────────────────────┤ - │ Zig FFI Layer │ - │ (51 C-exported functions, zero overhead) │ - ├────────┬──────────┬──────────┬──────────┬──────────┬────────────┤ - │Poppler │Tesseract │ FFmpeg │ libxml2 │ GDAL │ libvips │ - │ (PDF) │ (OCR) │(AV meta) │ (EPUB) │ (Geo) │ (Image) │ - ├────────┴──────────┴──────────┴──────────┴──────────┴────────────┤ - │ dlopen: ONNX Runtime (ML) │ PaddleOCR (GPU OCR) │ CUDA │ - ├──────────────────────────────────────────────────────────────────┤ - │ Idris2 ABI Proofs (14 types, 5 struct layouts, 51 FFI decls) │ - └──────────────────────────────────────────────────────────────────┘ - - Offline: OCaml docudactyl-scm (JSON/text → Scheme S-expressions) - Viewer: Ada TUI (interactive document inspection) - Legacy: Julia extraction scripts (replaced by Chapel pipeline) - -## Performance Estimates (British Library, 170M items) - -| Scenario | Estimate | -|----------------------------|--------------| -| Cold run (256 nodes + GPU) | ~3.7 hours | -| Warm run (L1+L2 cache) | ~4.4 minutes | -| Incremental (5% new files) | ~8 minutes | - -# Quick Start - -```bash -# Verify dependencies -just deps-check - -# Build Zig FFI + Chapel binary -just build-hpc - -# Run all tests -just test-hpc - -# Process a directory of documents -just generate-manifest /path/to/documents manifest.txt -bin/docudactyl-hpc --manifestPath=manifest.txt --outputDir=output/ - -# Or on an HPC cluster (64 nodes) -sbatch deploy/slurm-docudactyl.sh -``` - -# Components - -## Chapel: HPC Engine (hot path) - -The Chapel component distributes document processing across cluster -nodes with dynamic load balancing. - -Modules: Config, ContentType, FFIBridge, ManifestLoader, NdjsonManifest, -FaultHandler, ProgressReporter, ShardedOutput, ResultAggregator, -Checkpoint, DocudactylHPC. - -## Zig FFI: Parser Dispatch Layer - -10 submodules providing a unified C ABI for 7 content types and 20 -processing stages: - -- **Core**: `docudactyl_ffi.zig` — init, free, parse, version - (dispatches by content type) - -- **Stages**: 20 analysis stages with Cap’n Proto output (language, - readability, keywords, citations, OCR confidence, perceptual hash, - TOC, NER, Whisper, image classify, layout, handwriting, etc.) - -- **Cache**: L1 LMDB per-locale (zero-copy mmap) + L2 Dragonfly - cross-locale - -- **Conduit**: Magic-byte content detection (15 formats), SHA-256, - validation - -- **GPU OCR**: PaddleOCR CUDA \> Tesseract CUDA \> CPU (via dlopen) - -- **ML Inference**: ONNX Runtime — NER, Whisper, ImageClassify, Layout, - Handwriting (TensorRT \> CUDA \> OpenVINO \> CPU) - -- **Hardware Crypto**: SHA-NI, AVX2, AVX-512, AES-NI, ARM SHA2 - acceleration - -- **I/O Prefetch**: io_uring (Linux 5.6+) with posix_fadvise fallback - -## Idris2: Formal ABI Proofs - -Dependent types proving struct layout, alignment, and enum correctness: - -- 14 proven types (ContentKind, ParseStatus, MlStatus, MlStage, - ExecProvider, Sha256Tier, etc.) - -- 5 struct layout proofs (ParseResult 952B, MlResult 48B, CryptoCaps - 16B, OcrResult 48B, ConduitResult 88B) - -- 51 FFI declarations matching the C header 1:1 - -## OCaml: Offline Scheme Transformer - -Transforms extracted JSON/text into machine-readable Scheme -S-expressions. Not in the HPC hot path. - -```bash -docudactyl-scm document.pdf -o document.scm -docudactyl-scm extracted.json -o extracted.scm -``` - -## Ada: Terminal UI - -Interactive viewer for inspecting extracted documents. - -```bash -docudactyl-tui extracted.json -``` - -# Justfile Recipes - -```bash -# Build -just build-hpc # Zig FFI + Chapel binary -just build-ffi # Zig FFI only -just build-idris # Idris2 ABI proofs -just build-ocaml # OCaml transformer -just build-ada # Ada TUI - -# Test -just test-hpc # All HPC tests (FFI + error paths) -just test-ffi # Zig integration tests (40+ tests) -just test-scale # Scale test (2105+ files) -just test-idris # Idris2 proofs compile -just test-ocaml # OCaml tests -just test-ada # Ada build check - -# Deploy -just deps-check # Verify dependencies -just generate-manifest [output] -just generate-abi-header -just loc # Lines of code -``` - -# Directory Structure - - docudactyl/ - ├── src/ - │ ├── chapel/ # HPC engine (11 modules) - │ ├── Docudactyl/ABI/ # Idris2 ABI proofs (3 modules) - │ ├── ocaml/ # Offline Scheme transformer - │ ├── ada/ # Terminal UI - │ └── julia/ # Legacy extraction (replaced) - │ - ├── ffi/zig/ # Zig FFI layer (10 submodules) - │ ├── src/ # Source - │ └── test/ # Integration tests - │ - ├── generated/abi/ # Auto-generated C header - ├── schema/ # Cap'n Proto schema - ├── deploy/ # Containerfile + Slurm script - ├── contractiles/ # K9 contractile configs - ├── .machine_readable/ # SCM checkpoint files - ├── Justfile # Task runner - └── docudactyl.ipkg # Idris2 package - -# Requirements - -## System Dependencies - -- **Chapel** 2.3+ (HPC engine) - -- **Zig** 0.15+ (FFI layer) - -- **Idris2** 0.8+ (ABI proofs) - -- **C libraries**: Poppler, Tesseract, FFmpeg, libxml2, GDAL, libvips, - LMDB - -- **Optional**: ONNX Runtime, PaddleOCR, CUDA (for ML/GPU features) - -- **OCaml** 4.14+ (offline Scheme transformer) - -- **Ada** GNAT/gprbuild (terminal UI) - -## Container Deployment - -```bash -podman build -f deploy/Containerfile -t docudactyl-hpc . -podman run --rm -v /data/manifest.txt:/manifest.txt:ro \ - -v /data/output:/output \ - docudactyl-hpc --manifestPath=/manifest.txt -``` - -## Cluster Deployment (Slurm) - -```bash -# Edit deploy/slurm-docudactyl.sh for your cluster -sbatch deploy/slurm-docudactyl.sh -``` - -# Ethical Use - -This tool is designed for: - -- Document analysis and archival processing at national library scale - -- Research and verification of redaction practices - -- Accessibility improvements for PDF content - -- Multi-format metadata extraction and cataloguing - -# RSR Compliance - -This is a Tier 1 RSR project. The hot path uses Chapel + Zig (systems -languages). Legacy components (Julia, OCaml, Ada) serve -offline/auxiliary roles. - -# License - -SPDX-License-Identifier: CC-BY-SA-4.0 - -# Links - -- [GitHub Repository](https://github.com/hyperpolymath/docudactyl) - -- [Rhodium Standard](https://rhodium.sh) From f32fd8f167b2090f116b6d8eda5e621b92ffbee3 Mon Sep 17 00:00:00 2001 From: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 7 Jul 2026 16:11:01 +0100 Subject: [PATCH 03/16] fix(ci): estate-wide structural CI fixes - grant secret-scanner reusable its requested job permissions - drop invalid timeout-minutes on reusable-call jobs - drop hashFiles() from job-level if: expressions --- .github/workflows/secret-scanner.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index d713d06..118b9a1 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -15,5 +15,9 @@ permissions: jobs: scan: + permissions: + contents: read + pull-requests: write + actions: read uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@d135b05bfc647d0c0fbfedc7e80f37ea50f49236 secrets: inherit From 9b9af64007f9276230c8f4e43f1af8124edcd751 Mon Sep 17 00:00:00 2001 From: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 7 Jul 2026 21:26:16 +0100 Subject: [PATCH 04/16] fix(ci): switch to permissionless reusable workflow --- .github/workflows/secret-scanner.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 118b9a1..42cc9c8 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -17,7 +17,5 @@ jobs: scan: permissions: contents: read - pull-requests: write - actions: read - uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@d135b05bfc647d0c0fbfedc7e80f37ea50f49236 + uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@db12a6ad3ef9076d5f7bdcf98d7d15cf8547555a secrets: inherit From b10b6ffb9196689c7aaeb6339e3b539897967186 Mon Sep 17 00:00:00 2001 From: Antigravity Agent Date: Tue, 7 Jul 2026 21:35:44 +0100 Subject: [PATCH 05/16] fix(ci): clear OSSF Scorecard startup_failure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes the OSSF Scorecard workflow, which failed at startup on every run. **Cause:** reusable-workflow caller missing `security-events: write`+`id-token: write` (reusable perms are capped by the caller), and/or an illegal `timeout-minutes` key on a `uses:` job. **Fix:** inject caller permissions and/or drop `timeout-minutes`. Pin preserved. Verified with actionlint. Part of an estate-wide Scorecard remediation. 🤖 Generated with Antigravity --- .github/workflows/scorecard.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 08a434b..4ecc2ad 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -13,4 +13,8 @@ permissions: jobs: scorecard: - uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@d7c22711e830e1f383846472f6e9b99debdb201e \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@d7c22711e830e1f383846472f6e9b99debdb201e + permissions: + contents: read + security-events: write + id-token: write From 9f5dedc4c06a6c8322c4b2c1a708c4a2c6998d6d Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Sun, 26 Jul 2026 14:42:56 +0100 Subject: [PATCH 06/16] chore: estate-wide security compliance --- .github/workflows/boj-build.yml | 1 + .github/workflows/casket-pages.yml | 1 + .github/workflows/codeql.yml | 1 + .github/workflows/dogfood-gate.yml | 1 + .github/workflows/governance.yml | 1 + .github/workflows/hpc-ci.yml | 1 + .github/workflows/hypatia-scan.yml | 1 + .github/workflows/instant-sync.yml | 1 + .github/workflows/mirror.yml | 1 + .github/workflows/push-email-notify.yml | 1 + .github/workflows/scorecard.yml | 1 + .github/workflows/secret-scanner.yml | 1 + .github/workflows/workflow-linter.yml | 1 + 13 files changed, 13 insertions(+) diff --git a/.github/workflows/boj-build.yml b/.github/workflows/boj-build.yml index dba7fc8..786b8fb 100644 --- a/.github/workflows/boj-build.yml +++ b/.github/workflows/boj-build.yml @@ -16,4 +16,5 @@ jobs: curl -X POST "http://boj-server.local:7700/cartridges/ssg-mcp/invoke" -H "Content-Type: application/json" -d "{\"repo\": \"${{ github.repository }}\", \"branch\": \"${{ github.ref_name }}\", \"engine\": \"casket\\"}"} continue-on-error: true permissions: + actions: read contents: read diff --git a/.github/workflows/casket-pages.yml b/.github/workflows/casket-pages.yml index ad8fd14..b625074 100644 --- a/.github/workflows/casket-pages.yml +++ b/.github/workflows/casket-pages.yml @@ -7,6 +7,7 @@ on: workflow_dispatch: permissions: + actions: read contents: read pages: write id-token: write diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ab8ae8b..fcd5400 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -18,6 +18,7 @@ concurrency: cancel-in-progress: true permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/dogfood-gate.yml b/.github/workflows/dogfood-gate.yml index c9d8dfb..87796d5 100644 --- a/.github/workflows/dogfood-gate.yml +++ b/.github/workflows/dogfood-gate.yml @@ -13,6 +13,7 @@ on: branches: [main, master] permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/governance.yml b/.github/workflows/governance.yml index 8776de0..966a16e 100644 --- a/.github/workflows/governance.yml +++ b/.github/workflows/governance.yml @@ -9,6 +9,7 @@ on: workflow_dispatch: permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/hpc-ci.yml b/.github/workflows/hpc-ci.yml index 39ad57c..b133626 100644 --- a/.github/workflows/hpc-ci.yml +++ b/.github/workflows/hpc-ci.yml @@ -28,6 +28,7 @@ on: - '.github/workflows/hpc-ci.yml' permissions: + actions: read contents: read env: diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index 9dde27a..556e8e0 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -11,6 +11,7 @@ on: workflow_dispatch: permissions: + actions: read contents: read security-events: write diff --git a/.github/workflows/instant-sync.yml b/.github/workflows/instant-sync.yml index c013022..177614b 100644 --- a/.github/workflows/instant-sync.yml +++ b/.github/workflows/instant-sync.yml @@ -9,6 +9,7 @@ on: types: [published] permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index 81e9903..c25d3bc 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -7,6 +7,7 @@ on: workflow_dispatch: permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/push-email-notify.yml b/.github/workflows/push-email-notify.yml index 4b4e754..112afd1 100644 --- a/.github/workflows/push-email-notify.yml +++ b/.github/workflows/push-email-notify.yml @@ -7,6 +7,7 @@ name: Push email notification on: push: {} permissions: + actions: read contents: read jobs: notify: diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index b97e2cb..cede40a 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -9,6 +9,7 @@ on: workflow_dispatch: permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 9ed74ef..4c2ee4b 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -11,6 +11,7 @@ concurrency: cancel-in-progress: true permissions: + actions: read contents: read jobs: diff --git a/.github/workflows/workflow-linter.yml b/.github/workflows/workflow-linter.yml index 3bed9c5..57c5029 100644 --- a/.github/workflows/workflow-linter.yml +++ b/.github/workflows/workflow-linter.yml @@ -11,6 +11,7 @@ on: - '.github/workflows/**' permissions: read-all + actions: read jobs: lint-workflows: From ecc7c06f33a68b11602ae226f6f69e1294e04a01 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Sun, 26 Jul 2026 16:01:26 +0100 Subject: [PATCH 07/16] chore: update guix.scm from squisher-corpus --- guix.scm | 74 +++++++++----------------------------------------------- 1 file changed, 11 insertions(+), 63 deletions(-) diff --git a/guix.scm b/guix.scm index 13eda72..c6dd7be 100644 --- a/guix.scm +++ b/guix.scm @@ -1,70 +1,18 @@ -;; SPDX-License-Identifier: MPL-2.0 -;; Docudactyl HPC — Guix development environment -;; -;; Usage: -;; guix shell -D -f guix.scm # Enter dev shell with all dependencies -;; guix build -f guix.scm # Build (placeholder — real build uses just) -;; -;; This defines the development environment for building Docudactyl HPC. -;; The actual build is driven by the Justfile (just build-hpc). -;; -;; Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) +; SPDX-License-Identifier: MPL-2.0 +;; guix.scm — GNU Guix package definition for squisher-corpus +;; Usage: guix shell -f guix.scm (use-modules (guix packages) (guix build-system gnu) - (guix licenses) - (gnu packages) - (gnu packages gcc) - (gnu packages pkg-config) - (gnu packages glib) - (gnu packages pdf) - (gnu packages ocr) - (gnu packages image) - (gnu packages video) - (gnu packages xml) - (gnu packages geo) - (gnu packages image-processing) - (gnu packages databases)) + (guix licenses)) (package - (name "docudactyl") - (version "0.4.0") + (name "squisher-corpus") + (version "0.1.0") (source #f) (build-system gnu-build-system) - (synopsis "Multi-format HPC document extraction engine") - (description - "Docudactyl is a distributed document processing engine targeting -British Library scale (~170M items). Chapel orchestrates across HPC -cluster nodes, dispatching to C libraries via a zero-cost Zig FFI layer. -Supports PDF, images (OCR), audio, video, EPUB, and geospatial formats.") - (home-page "https://github.com/hyperpolymath/docudactyl") - (license #f) ; MPL-2.0 (not in Guix license list) - - ;; Development inputs — these are the C libraries linked by the Zig FFI. - ;; Chapel and Zig are not yet packaged in Guix; install via asdf. - (native-inputs - (list pkg-config gcc-toolchain)) - (inputs - (list - ;; PDF extraction - poppler ; poppler-glib - glib ; glib-2.0, gobject-2.0 - - ;; OCR - tesseract-ocr ; libtesseract - leptonica ; liblept - - ;; Audio/Video - ffmpeg ; libavformat, libavcodec, libavutil - - ;; EPUB/XHTML - libxml2 ; libxml-2.0 - - ;; Geospatial - gdal ; libgdal - - ;; Image metadata - vips ; libvips - - ;; Result cache - lmdb))) ; liblmdb (zero-copy key-value store) + (synopsis "squisher-corpus") + (description "squisher-corpus — part of the hyperpolymath ecosystem.") + (home-page "https://github.com/hyperpolymath/squisher-corpus") + (license ((@@ (guix licenses) license) "PMPL-1.0-or-later" + "https://github.com/hyperpolymath/palimpsest-license"))) From 628154c277a0b8bc188bdfad706bc5dafd56cd29 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 13 Aug 2026 01:41:24 +0100 Subject: [PATCH 08/16] fix(ci): remove erroneous squisher-corpus guix.scm placeholder Part of estate-wide standards#426 remediation - cleanup. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- guix.scm | 18 ------------------ 1 file changed, 18 deletions(-) delete mode 100644 guix.scm diff --git a/guix.scm b/guix.scm deleted file mode 100644 index c6dd7be..0000000 --- a/guix.scm +++ /dev/null @@ -1,18 +0,0 @@ -; SPDX-License-Identifier: MPL-2.0 -;; guix.scm — GNU Guix package definition for squisher-corpus -;; Usage: guix shell -f guix.scm - -(use-modules (guix packages) - (guix build-system gnu) - (guix licenses)) - -(package - (name "squisher-corpus") - (version "0.1.0") - (source #f) - (build-system gnu-build-system) - (synopsis "squisher-corpus") - (description "squisher-corpus — part of the hyperpolymath ecosystem.") - (home-page "https://github.com/hyperpolymath/squisher-corpus") - (license ((@@ (guix licenses) license) "PMPL-1.0-or-later" - "https://github.com/hyperpolymath/palimpsest-license"))) From 7780a73a841ca71715855d4f2604ebd389d534c3 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 13 Aug 2026 03:35:03 +0100 Subject: [PATCH 09/16] fix(ci): update reusable workflow SHAs to @7fdc2705df74b4e352d2a1cde3e87a5923fdf329 Part of estate-wide standards#426 remediation - Batch 11 SHA update. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .github/workflows/mirror.yml | 2 +- .github/workflows/secret-scanner.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index c25d3bc..72824fb 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -12,5 +12,5 @@ permissions: jobs: mirror: - uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@d135b05bfc647d0c0fbfedc7e80f37ea50f49236 + uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 secrets: inherit diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 4c2ee4b..010c16d 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -20,5 +20,5 @@ jobs: contents: read pull-requests: write actions: read - uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@d135b05bfc647d0c0fbfedc7e80f37ea50f49236 + uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 secrets: inherit \ No newline at end of file From 78325d39daf653e5a0d7697af1c9e69603a0c64e Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 13 Aug 2026 04:58:58 +0100 Subject: [PATCH 10/16] fix(ci): update reusable workflow SHAs to @7fdc2705df74b4e352d2a1cde3e87a5923fdf329 Part of estate-wide standards#426 remediation - Batch 13 SHA update. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .github/workflows/governance.yml | 2 +- .github/workflows/hypatia-scan.yml | 2 +- .github/workflows/scorecard.yml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/governance.yml b/.github/workflows/governance.yml index 966a16e..aad0d03 100644 --- a/.github/workflows/governance.yml +++ b/.github/workflows/governance.yml @@ -14,4 +14,4 @@ permissions: jobs: governance: - uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@81dbf2dd854b1444fd6236fa2352474383b2c2b9 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 \ No newline at end of file diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index 556e8e0..1f8c940 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -17,4 +17,4 @@ permissions: jobs: scan: - uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@81dbf2dd854b1444fd6236fa2352474383b2c2b9 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 \ No newline at end of file diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index cede40a..03413b9 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -14,7 +14,7 @@ permissions: jobs: scorecard: - uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@81dbf2dd854b1444fd6236fa2352474383b2c2b9 + uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 permissions: contents: read security-events: write From 6524657f254118add1f39257cba03f8f21ef3b2f Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 13 Aug 2026 19:17:56 +0100 Subject: [PATCH 11/16] fix(ci): add required permissions for reusable workflows (Bug B) Add security-events: write and id-token: write to workflow-level permissions in scorecard.yml for scorecard-reusable.yml calls. Ensure contents: read at workflow-level for secret-scanner.yml. Part of hyperpolymath/standards#426 remediation - Batch 2. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .github/workflows/scorecard.yml | 4 ++-- .github/workflows/secret-scanner.yml | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 03413b9..3449a81 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -9,9 +9,9 @@ on: workflow_dispatch: permissions: - actions: read contents: read - + security-events: write + id-token: write jobs: scorecard: uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 010c16d..f363a3e 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -11,9 +11,7 @@ concurrency: cancel-in-progress: true permissions: - actions: read contents: read - jobs: scan: permissions: From 78a8b9d9f1e9bb32e05265f206ce818d2cd1bd86 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Fri, 14 Aug 2026 17:54:17 +0100 Subject: [PATCH 12/16] chore(ci): bump standards reusable pins to fix Bug A and Bug B (#426) Update reusable workflow SHA from d135b05 to f2f8e6791b09f1f498f01b798e4670a1ebc9c986 to pick up fixes for: - Bug A: Invalid timeout-minutes at workflow_call level and duplicates - Bug B: Permissions escalation in scorecard-reusable Part of hyperpolymath/standards#426 remediation. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .github/workflows/governance.yml | 2 +- .github/workflows/hypatia-scan.yml | 2 +- .github/workflows/mirror.yml | 2 +- .github/workflows/scorecard.yml | 2 +- .github/workflows/secret-scanner.yml | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/governance.yml b/.github/workflows/governance.yml index aad0d03..14a4390 100644 --- a/.github/workflows/governance.yml +++ b/.github/workflows/governance.yml @@ -14,4 +14,4 @@ permissions: jobs: governance: - uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 \ No newline at end of file diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index 1f8c940..f00512c 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -17,4 +17,4 @@ permissions: jobs: scan: - uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 \ No newline at end of file diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index 72824fb..b1ed9d3 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -12,5 +12,5 @@ permissions: jobs: mirror: - uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 + uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 secrets: inherit diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 3449a81..34ef113 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -14,7 +14,7 @@ permissions: id-token: write jobs: scorecard: - uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 + uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 permissions: contents: read security-events: write diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index f363a3e..9f2b044 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -18,5 +18,5 @@ jobs: contents: read pull-requests: write actions: read - uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@7fdc2705df74b4e352d2a1cde3e87a5923fdf329 + uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 secrets: inherit \ No newline at end of file From 3a21be19e6f528552a4c494a319f854893d5eeeb Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Fri, 14 Aug 2026 18:03:36 +0100 Subject: [PATCH 13/16] chore(ci): bump standards reusable pins to 5b1d0022 (#426) Final SHA update for Bug A and Bug B fixes. Part of hyperpolymath/standards#426 remediation. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .github/workflows/governance.yml | 2 +- .github/workflows/hypatia-scan.yml | 2 +- .github/workflows/mirror.yml | 2 +- .github/workflows/scorecard.yml | 2 +- .github/workflows/secret-scanner.yml | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/governance.yml b/.github/workflows/governance.yml index 14a4390..ffaaca5 100644 --- a/.github/workflows/governance.yml +++ b/.github/workflows/governance.yml @@ -14,4 +14,4 @@ permissions: jobs: governance: - uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@5b1d00229e5e8c0c0fbfedc7e80f37ea50f49236 \ No newline at end of file diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index f00512c..b48dcbe 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -17,4 +17,4 @@ permissions: jobs: scan: - uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 \ No newline at end of file + uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@5b1d00229e5e8c0c0fbfedc7e80f37ea50f49236 \ No newline at end of file diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index b1ed9d3..d9685fd 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -12,5 +12,5 @@ permissions: jobs: mirror: - uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 + uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@5b1d00229e5e8c0c0fbfedc7e80f37ea50f49236 secrets: inherit diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 34ef113..20e5b55 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -14,7 +14,7 @@ permissions: id-token: write jobs: scorecard: - uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 + uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@5b1d00229e5e8c0c0fbfedc7e80f37ea50f49236 permissions: contents: read security-events: write diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 9f2b044..226dd6e 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -18,5 +18,5 @@ jobs: contents: read pull-requests: write actions: read - uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@f2f8e6791b09f1f498f01b798e4670a1ebc9c986 + uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@5b1d00229e5e8c0c0fbfedc7e80f37ea50f49236 secrets: inherit \ No newline at end of file From 08e5863b36318ff7302c50967bed0fafa0e31061 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Fri, 28 Aug 2026 06:17:50 +0100 Subject: [PATCH 14/16] chore(toolchain): keep .tool-versions -> .mise.toml pin conversion (R-16) Owner ruling 2026-08-28 (R-16/R-20/R-21): keep the pin conversion from the template-sync sweep, revert the rest. Pin content verified against HEAD:.tool-versions before commit. Co-Authored-By: Claude Fable 5 --- .mise.toml | 16 ++++++++++++++++ .tool-versions | 19 ------------------- 2 files changed, 16 insertions(+), 19 deletions(-) create mode 100644 .mise.toml delete mode 100644 .tool-versions diff --git a/.mise.toml b/.mise.toml new file mode 100644 index 0000000..9e838a5 --- /dev/null +++ b/.mise.toml @@ -0,0 +1,16 @@ +[tools] +# = "Docudactyl HPC — Pinned tool versions (asdf)" +# = "These versions are tested and known to work with the current codebase." +# = "Upgrading any of these requires testing:" +# = "- Zig: build.zig API changes (0.14→0.15 broke addSharedLibrary)" +# = "- Idris2: ABI proof syntax may change" +# = "- OCaml: dune build compatibility" +# = "- Julia: legacy component only" +# = "Chapel is NOT managed by asdf —" +# = "Current tested version: 2.7.0" +# = "SPDX-License-Identifier: MPL-2.0" +zig = "0.15.2" +idris2 = "0.8.0" +ocaml = "5.4.1" +julia = "1.12.5" +just = "1.46.0" diff --git a/.tool-versions b/.tool-versions deleted file mode 100644 index 3ea1cef..0000000 --- a/.tool-versions +++ /dev/null @@ -1,19 +0,0 @@ -# Docudactyl HPC — Pinned tool versions (asdf) -# -# These versions are tested and known to work with the current codebase. -# Upgrading any of these requires testing: -# - Zig: build.zig API changes (0.14→0.15 broke addSharedLibrary) -# - Idris2: ABI proof syntax may change -# - OCaml: dune build compatibility -# - Julia: legacy component only -# -# Chapel is NOT managed by asdf — it's system-installed via rpm. -# Current tested version: 2.7.0 -# -# SPDX-License-Identifier: MPL-2.0 - -zig 0.15.2 -idris2 0.8.0 -ocaml 5.4.1 -julia 1.12.5 -just 1.46.0 From 15b945b4c97f267636268588f632644cbe45a6a7 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Fri, 28 Aug 2026 15:08:59 +0100 Subject: [PATCH 15/16] Update .github/workflows/codeql.yml Co-authored-by: codacy-production[bot] <61871480+codacy-production[bot]@users.noreply.github.com> Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com> --- .github/workflows/codeql.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 92b8d7d..cae91b9 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -7,7 +7,7 @@ on: pull_request: branches: [main, master] schedule: - - cron: '0 6 1 * *' # monthly 1st 06:00 UTC + - cron: '0 6 * * 1' # weekly Monday 06:00 UTC # Estate guardrail: cancel superseded runs so re-pushes / rebased PR # updates do not pile up queued runs against the shared account-wide From 592c05d984d7db237414476040f3641784ff0d17 Mon Sep 17 00:00:00 2001 From: "coderabbitai[bot]" <136622811+coderabbitai[bot]@users.noreply.github.com> Date: Fri, 28 Aug 2026 15:52:42 +0000 Subject: [PATCH 16/16] Fix CodeRabbit issues in PR #68 --- .mise.toml | 4 ++-- Justfile | 22 ++++++++++------------ 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/.mise.toml b/.mise.toml index 9e838a5..5b59868 100644 --- a/.mise.toml +++ b/.mise.toml @@ -1,12 +1,12 @@ [tools] -# = "Docudactyl HPC — Pinned tool versions (asdf)" +# = "Docudactyl HPC — Pinned tool versions (mise)" # = "These versions are tested and known to work with the current codebase." # = "Upgrading any of these requires testing:" # = "- Zig: build.zig API changes (0.14→0.15 broke addSharedLibrary)" # = "- Idris2: ABI proof syntax may change" # = "- OCaml: dune build compatibility" # = "- Julia: legacy component only" -# = "Chapel is NOT managed by asdf —" +# = "Chapel is NOT managed by mise —" # = "Current tested version: 2.7.0" # = "SPDX-License-Identifier: MPL-2.0" zig = "0.15.2" diff --git a/Justfile b/Justfile index d27801e..f6767f9 100644 --- a/Justfile +++ b/Justfile @@ -470,15 +470,14 @@ toolbox := env("DOCUDACTYL_TOOLBOX", "fedora-toolbox-43") # Build the Zig FFI shared/static libraries build-ffi: @echo "Building Zig FFI (poppler, tesseract, ffmpeg, libxml2, gdal, vips)..." - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && cd {{zig_ffi}} && zig build -Doptimize=ReleaseFast' + toolbox run -c {{toolbox}} bash -c 'cd {{zig_ffi}} && mise exec -- zig build -Doptimize=ReleaseFast' # Build Chapel HPC binary (depends on Zig FFI) build-chapel: build-ffi @echo "Building Chapel HPC engine..." @mkdir -p bin - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && \ - ABSPATH=$$(cd {{zig_ffi}}/zig-out/lib && pwd) && \ - chpl {{chapel_src}}/DocudactylHPC.chpl \ + toolbox run -c {{toolbox}} bash -c 'ABSPATH=$$(cd {{zig_ffi}}/zig-out/lib && pwd) && \ + mise exec -- chpl {{chapel_src}}/DocudactylHPC.chpl \ {{chapel_src}}/Config.chpl \ {{chapel_src}}/ContentType.chpl \ {{chapel_src}}/FFIBridge.chpl \ @@ -580,13 +579,13 @@ upgrade-manifest input="manifest.txt" output="manifest.ndjson": # Run Zig FFI integration tests test-ffi: @echo "Running Zig FFI tests..." - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && cd {{zig_ffi}} && zig build test' + toolbox run -c {{toolbox}} bash -c 'cd {{zig_ffi}} && mise exec -- zig build test' # Check Chapel parse validity (metalayer + smoke; --main-module # disambiguates the two `proc main()` files) check-chapel: @echo "Checking Chapel syntax..." - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && chpl --parse-only --main-module DocudactylHPC {{chapel_src}}/DocudactylHPC.chpl {{chapel_src}}/*.chpl' + toolbox run -c {{toolbox}} bash -c 'mise exec -- chpl --parse-only --main-module DocudactylHPC {{chapel_src}}/DocudactylHPC.chpl {{chapel_src}}/*.chpl' # Decoupled FFI smoke (per docudactyl#29 / echidna#146 pattern). # Compiles just smoke.chpl + FFIBridge.chpl — does not pull in the @@ -594,15 +593,14 @@ check-chapel: # metalayer breaks. Build oracle for the FFI ABI alone. check-smoke: @echo "Parse-checking FFI smoke..." - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && chpl --no-codegen {{chapel_src}}/smoke.chpl {{chapel_src}}/FFIBridge.chpl' + toolbox run -c {{toolbox}} bash -c 'mise exec -- chpl --no-codegen {{chapel_src}}/smoke.chpl {{chapel_src}}/FFIBridge.chpl' # Build the FFI smoke binary (links against Zig FFI library). build-smoke: build-ffi @echo "Building FFI smoke binary..." @mkdir -p bin - toolbox run -c {{toolbox}} bash -c 'export PATH="$$HOME/.asdf/shims:$$HOME/.asdf/bin:$$PATH" && \ - ABSPATH=$$(cd {{zig_ffi}}/zig-out/lib && pwd) && \ - chpl {{chapel_src}}/smoke.chpl \ + toolbox run -c {{toolbox}} bash -c 'ABSPATH=$$(cd {{zig_ffi}}/zig-out/lib && pwd) && \ + mise exec -- chpl {{chapel_src}}/smoke.chpl \ {{chapel_src}}/FFIBridge.chpl \ -o bin/docudactyl-smoke \ -L{{zig_ffi}}/zig-out/lib -ldocudactyl_ffi \ @@ -644,8 +642,8 @@ deps-check: echo "" # Check build tools echo "--- Build tools ---" - ZIG_VER=$(toolbox run -c {{toolbox}} bash -c "export PATH=\$HOME/.asdf/shims:\$HOME/.asdf/bin:\$PATH && zig version 2>/dev/null" || echo "") - CHPL_VER=$(toolbox run -c {{toolbox}} bash -c "chpl --version 2>/dev/null | head -1 | grep -oP '[0-9]+\.[0-9]+\.[0-9]+'" || echo "") + ZIG_VER=$(toolbox run -c {{toolbox}} bash -c "mise exec -- zig version 2>/dev/null" || echo "") + CHPL_VER=$(toolbox run -c {{toolbox}} bash -c "mise exec -- chpl --version 2>/dev/null | head -1 | grep -oP '[0-9]+\.[0-9]+\.[0-9]+'" || echo "") [ -n "$ZIG_VER" ] && echo " OK zig $ZIG_VER (need >= 0.15.0)" || { echo " MISSING zig (need >= 0.15.0)"; FAIL=1; } [ -n "$CHPL_VER" ] && echo " OK chpl $CHPL_VER (need >= 2.7.0)" || { echo " MISSING chpl (need >= 2.7.0)"; FAIL=1; } echo ""