dev: replace gapmacro2gapdoc.g with a Python rewrite - #6496
Open
fingolfin wants to merge 10 commits into
Open
Conversation
Tooling for converting the manuals still in the old plain-TeX format, see gap-system#639. Eight packages converted with it so far; 18 remain. The old script ran ~30 global find/replace passes over the whole file, so it had no way to tell a backquote inside \beginexample from one in running text, or a \> inside \beginitems (an item mark) from one in a chapter body (a declaration). \> alone occurs 1732 times in the remaining sources, and the old header lists it as not handled correctly. This one carves out verbatim regions, then parses structure line by line, then scans inline markup with an explicit mode stack. Output is validated as XML before writing; anything not understood becomes a TODO(g2g) marker rather than vanishing. It also consults sources the old script did not: * The package's own Declare... calls, which decide an element's type. 677 \> lines carry no type letter, and where letter and source disagree the source wins -- 44 corrections. * manual.six, in both formats, for cross-references. These are emitted with the target's own spelling, since the old format matched case-insensitively and manuals drifted. * .bib files, whose TeX accents become Unicode. Constructs GAPDoc cannot parse are reported: a TeX escape in an author name aborts the build from inside NormalizedNameAndKey. Examples get particular care. The old build extracted them to a gitignored file nothing ran, so they had gone stale everywhere. A blank line is layout in the old manuals but expected output in a .tst, so <Example> is split only where a new gap> command follows. All 18 remaining packages produce well-formed XML; all 715 verbatim blocks survive byte-identical. gapmacro2gapdoc.g also rewrote documentation embedded in GAP source files, which this does not. No package still using gapmacro.tex has any: none uses \Declaration or \FileHeader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GAPDoc derives a label from each declaration's name, book-wide, and otherwise reports "Label multiply defined" and resolves references to it arbitrarily. merge_overloads only sees one ManSection at a time, so it misses a name documented in two: liepring documents LiePRingsByLibrary once with a prime and once without. Report those rather than guess a label, since the right one says why the forms differ. 11 across the packages still using gapmacro.tex. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three fixes, all from SymbCompCC:
* \eqalign becomes an array environment and \cr becomes \\. GAPDoc loads
amssymb but not amsmath, so there is no `aligned`. 16 \eqalign and 75
\cr remain, in SymbCompCC, rds and sonata. \matrix, \cases and \over are
reported instead: they need a human without amsmath.
* In math, \{ and \} are literal braces. Unescaping them let a brace open
a group and swallow the rest of the formula.
* \cdots is no longer flattened to \ldots.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gapmacro kept section and declaration labels apart; GAPDoc has a single
namespace, so \Section{CHR} wrapping \>CHR(...) becomes "Label multiply
defined" and a reference resolves to whichever came last. grape does this
73 times, cohomolo 9.
Where the section wraps exactly that one declaration the section label is
redundant, so drop it and point any reference at the declaration instead:
same destination, one label. Sections holding more than the declaration
they are named after are left alone and reported -- one in grape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AutoDoc writes doc/main.xml, doc/title.xml and doc/_*.xml itself. A chapter source named main.tex therefore converts to a file AutoDoc then overwrites, losing the chapter with no warning. fwtree has one; rename such a chapter after the book. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ement GAP would prompt "> " and read the next line as input, so the extracted test consumes the manual's output as part of the command. modisom writes a rec() across six lines with no "> " prompts, which made Test read the first line as a whole statement, fail on the syntax, and treat the rest as expected output. Report rather than insert the prompts: the other case this finds is rds's "gap> d1=d2", where the semicolon is simply missing, and adding a prompt there would turn the recorded output into input and hide the real bug. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gapmacro.tex spells the apostrophe as \pif and "!" as \excl to dodge its own catcodes; \hyphenation is a TeX hint with no GAPDoc counterpart. All three used to leave a TODO marker. \pif alone occurs 42 more times in the manuals still to convert.
fingolfin
added
kind: enhancement
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6496 +/- ##
=======================================
Coverage 79.03% 79.04%
=======================================
Files 685 685
Lines 293805 293836 +31
Branches 8664 8664
=======================================
+ Hits 232220 232250 +30
+ Misses 59786 59783 -3
- Partials 1799 1803 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ThomasBreuer
approved these changes
Aug 14, 2026
ThomasBreuer
left a comment
ThomasBreuer
left a comment
Contributor
There was a problem hiding this comment.
Interesting.
Long time ago I had decided that it is hopeless to convince people not to use the old gapmacro.tex stuff for their packages.
If this new attempt helps to get rid of some gapmacro.tex dependencies, I am happy.
… not GAP
Three defects the sonata, cryst and unipot conversions turned up.
AutoDoc extracts <Example> verbatim and GAP's test format only recognises a
prompt at the start of a line, so an indented block is read as one lump of
expected output and fails against START_TEST. sonata indented every one of
its examples. <Example> is now dedented by its common prefix; <Log> keeps
its indentation, which cryst uses to centre two matrices. A prompt still
indented afterwards means the block indents some lines and not others, and
is reported rather than guessed at.
In a code span \pif{1}\pif came out as '{1}' where GAP writes '1', and
\^{} as ^{}. GAP only writes "{" after an identifier, ")" or "]", so a
brace following anything else is TeX grouping and goes.
\* reached the LaTeX unchanged and stopped pdflatex with "Missing {";
gapmacro.tex made * active, which is why the sources spell it that way.
Splitting a \beginlist on \item alone left the second "item" at the head of the body, so the mark was lost and the entry came out as literal "item1.". With the mark visible, is_enumeration sees 1..6 and the list becomes <Enum>, which is what unipot's info levels wanted. rds is the only other package that uses \itemitem.
Pinning them to a record fixes the package at what it needs today. Left at true, AutoDoc comments in the source are picked up without another edit to makedoc.g, and AutoDoc scaffolds the document as _main, so its LaTeX by-products all share one prefix and /doc/_main.* covers them in .gitignore. A bibliography called manual.bib says nothing about which manual, so it is renamed after the package, as the hand-converted packages already do.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tooling for converting the manuals still in the old plain-TeX format, see #639. Eight packages converted with it so far (radiroot, sglppow, nilmat, CaratInterface, cubefree, grpconst, crystcat, aclib); 18 remain.
The old script ran ~30 global find/replace passes over the whole file, so it had no way to tell a backquote inside
\beginexamplefrom one in running text, or a\>inside\beginitems(an item mark) from one in a chapter body (a declaration).\>alone occurs 1732 times in the remaining sources, and the old header lists it as not handled correctly.This one carves out verbatim regions, then parses structure line by line, then scans inline markup with an explicit mode stack. Output is validated as XML before writing; anything not understood becomes a
TODO(g2g)marker rather than vanishing.It also consults sources the old script did not:
Declare...calls, which decide an element's type. 677\>lines carry no type letter, and where letter and source disagree the source wins — 44 corrections.manual.six, in both formats, for cross-references. These are emitted with the target's own spelling, since the old format matched case-insensitively and manuals drifted..bibfiles, whose TeX accents become Unicode. Constructs GAPDoc cannot parse are reported: a TeX escape in an author name aborts the build from insideNormalizedNameAndKey.Examples get particular care. The old build extracted them to a gitignored file nothing ran, so they had gone stale everywhere. A blank line is layout in the old manuals but expected output in a
.tst, so<Example>is split only where a newgap>command follows.All 18 remaining packages produce well-formed XML; all 715 verbatim blocks survive byte-identical.
gapmacro2gapdoc.galso rewrote documentation embedded in GAP source files, which this does not. No package still usinggapmacro.texhas any: none uses\Declarationor\FileHeader.