fix: primitive space results; stdlib alpha/digit/word/any_character - #15
Merged
Merged
Conversation
moct-kor pads its working string with 'sub line_start space' and 'sub line_end space' before the liaison rules, and strips the pads at the end. The subst renderer had no branch for primitive results, so with on_unsupported=skip the pads were silently dropped — the before-space guards on initial consonants never fired and 불국사 came out ᄇulguksa. A primitive space in a result now renders as " ". alpha, digit, word and any_character are Ruby Stdlib built-in aliases, resolved before doc-local aliases (the interpreter's lookup order). The Python engine now knows them as expression keywords with Onigmo's ASCII-exact definitions — Python's \w and \d are unicode- wide where Onigmo's are not, so word is [a-zA-Z0-9_] and digit is [0-9] rather than \w/\d. Direct corpus sweep: 137 failures / 30 maps -> 98 / 27 (moct-kor, bgnpcgn-kor rok-2011 x2 fully healed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two conformance gaps, both measured against the Ruby interpreter:
Primitive
spacein results. moct-kor pads its working string withsub line_start space/sub line_end spacebefore the liaison rules and strips the pads at the end. The subst renderer had no branch for primitive results, so underon_unsupported=skipthe pads were silently dropped — thebefore spaceguards on initial consonants never fired and 불국사 came out ᄇulguksa (all 13 of the map's examples). A primitive space now renders as " ".Stdlib aliases.
alpha,digit,word,any_characterare RubyStdlib::ALIASESbuilt-ins, resolved before doc-local aliases (the interpreter's lookup order). The Python engine now knows them as expression keywords with Onigmo's ASCII-exact definitions — Python's\w/\dare unicode-wide where Onigmo's are not, soword=[a-zA-Z0-9_]anddigit=[0-9]. These are the only stdlib names the corpus uses in guards (19/20/21/1 occurrences).Direct corpus sweep: 137 failures / 30 maps → 98 / 27 — moct-kor, bgnpcgn-kor-Hang-Latn-rok-2011 and bgnpcgn-kor-Kore-Latn-rok-2011 fully healed. Suite 43 passed / 1 xpassed. Bridge sweep (Compiler::Python over the full corpus): 425 → 201.