Store JATS full text for publications, drop redundant abstracts from pubs.yaml - #206
Open
aarmey wants to merge 3 commits into
Open
Store JATS full text for publications, drop redundant abstracts from pubs.yaml#206aarmey wants to merge 3 commits into
aarmey wants to merge 3 commits into
Conversation
added 3 commits
August 27, 2026 07:56
…pubs.yaml Downloads and commits JATS XML from PMC for 49 publications, then removes the now-redundant `abstract` field from data/pubs.yaml wherever a JATS file covers it (48 entries), cutting the file from 2,849 to 1,983 lines. Hugo's pubentry partial now extracts the abstract from JATS at build time via regex tag-stripping, falling back to the YAML field where no JATS file exists. CV.typ and other pubs.yaml fields are untouched, so the Typst CV build is unaffected. scripts/fetch_jats.py is reusable to refresh JATS as new PMID'd publications are added.
Extracts plain text from JATS XML via pandoc's native JATS reader (scripts/extract_fulltext.py, gitignored fulltext/ output — regenerable from committed JATS files), generates one page per publication via a Hugo content adapter (content/publications/_content.gotmpl + layouts/pub/single.html) so Pagefind has something to index, and adds a lazy-loaded search box to the publications list page. Pagefind's JS/WASM/index (~150KB gzipped for the initial load) only fetches on first interaction with the search input, so browsing the site normally adds zero bytes. CI now runs pandoc extraction and `pagefind --site _site` after the Hugo build.
Silences Hugo's deprecation warning (languageCode was deprecated in v0.158.0 in favor of locale); output is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
data/JATS/PMC<id>.xml) for the 49 publications with a resolvable PMC recordabstractfield fromdata/pubs.yamlfor the 48 entries a JATS file covers (2,849 → 1,983 lines); all other fields verified byte-identical, and entries with no PMC record keep their YAML abstract as a fallbackpubentrypartial now extracts the abstract from JATS at build time (layouts/partials/jats-abstract.html) via regex tag-stripping — deliberately not Hugo's built-in XML unmarshaler, which was confirmed to silently drop text around inline<italic>/<sup>tagsscripts/fetch_jats.pyis a reusable script (uv run --with ruamel.yaml scripts/fetch_jats.py) to refresh JATS as new PMID'd publications are added_CV/CV.typand all otherpubs.yamlfields are untouched — the Typst CV build reads locator fields directly from YAML and doesn't need JATSTest plan
hugo --minifybuilds cleantypst compile --root . _CV/CV.typbuilds clean; publication list in the rendered PDF unaffectedpubs.yamlareabstractremoval andpmcidaddition — no other field values changed🤖 Generated with Claude Code
https://claude.ai/code/session_01Uc6rqkTyBH75kNfrirbxZE