Fix a dropped morpheme ID when a zero-width rule wraps one ending at the same node - #500
Open
johnml1135 wants to merge 40 commits into
Open
johnml1135 wants to merge 40 commits into
johnml1135 wants to merge 40 commits into
Conversation
Fixtures with per-fixture hashes, a published manifest, and a documented contract for running an external engine against them. This is what a consumer receives: the fixtures, the manifest, and PROTOCOL.md, and nothing else in this branch. Also a census of the compilations the pinned compiler produced, a denominator for rule interactions, and per-phase parse observation. The interaction ledger is stated as a corpus statistic rather than a bound. It counts what these grammars happen to contain, which is not a claim about what a grammar could contain. Derive coverage denominators from the DTD and the engine A denominator read off the corpus measures the corpus. So the interface inventory comes from the DTD, and RequiredToLoad splits into RequiredByDtd and RequiredByLoader, because "the DTD demands it" and "the loader demands it" are different facts about a grammar. The surfaces no consumer reads are named rather than quietly counted, and the host configuration the fixtures assume becomes part of the contract. Real grammars supply the SHAPE of a feature and never its scale. Establish presence-is-not-coverage and the layers enforcing it A construct can be present, referenced correctly, load cleanly, and change no parse. Only severance that changes a parse counts as evidence. Adds the interaction-chain layer, per-grammar witness traceability that holds presence and witness apart, exercises: corroboration, and fold-in candidates. The witness ledger records every fixture that witnesses a surface, not just the first. The earlier strictly-better tie-break silently preferred a fabricated edge case over a language grammar on every tie, which hid how much coverage the real grammars already carried. Add the authoring harness The doer skill, the review skill, the measurement, and the revision loop, with the suite documentation shipped alongside the fixtures. Measurement and revision stay separate deliberately. Doing both in one pass makes a change impossible to attribute to either. Make the mutant sweep reproducible, and add conformance CI Morpher.Synthesize sizes its Parallel.ForEach off ProcessorCount, so a mutant could return a different verdict per run. Pinning DOTNET_PROCESSOR_COUNT=1 in the child makes two independent sweeps byte-identical, which is the property the drift gates already assumed and did not have. Test child processes are bounded to the test's own lifetime. Adds the conformance workflow, the fixture index, and the RuleName/FailureAllomorph threading that lets an infinite-loop crash name the rule responsible. The expensive sweep runs on a schedule, not on every pull request. The dead-rule step is advisory: --coverage-report reports dead rules and still exits 0, and making that a hard failure while 14 exist would fail CI on its first run. Key coverage to the engine's own gates and three constraint layers An obligation is worth covering only where the engine can act on it, the DTD can declare it, and a real FieldWorks project can produce it. Failing any one of the three is an exclusion, not a gap. Layer one is FailureReason, HermitCrab's own 23-member enumeration of the decisions it makes when declining to apply something. Witness status comes from a real traced run, and six of those gates are reachable through no XML attribute at all, so an attribute-keyed denominator cannot see them. Layer three is HCLoader, the component that turns a LibLCM model into an HC grammar and therefore integrates both the model and what is reachable from it. Of 83 subjects, 61 are producible and 22 are not. Keying MC/DC to the gates also fixes a miscount: the attribute-keyed ledger emits four arms per chain but can certify at most one, because it reads the arm off the reader attribute's spelling. The older ledger is kept as a cross-check on writer-reader chains, which the gate ledger does not measure, and the docs say which number is the claim. Ledger assertions read the checked-in file; only a freshness check recomputes, behind [Explicit]. An earlier version reswept four times in the default suite. Triage every obligation, and record what authors keep relearning Fifty-one unmet obligations, each assigned a bucket from the blocker its own ledger records, with the construct stated in plain language so a reader can judge whether covering it is worth anything. The split that matters: thirteen are a HARNESS gap, not a coverage gap, and no amount of authoring touches them. Seven control arms already have their word in the corpus and fail only because GrammarRuleIndex cannot resolve Allomorph, MorphologicalInput, AffixTemplate or PhonologicalSubrule to a fired-rule id. Six more are reachable only through element content, for which no severance primitive exists. Also writes down the engine and tooling facts each author had been rediscovering at real cost: severance is fixture-wide and can unblock as easily as block; a DTD #REQUIRED attribute can never be severed, which makes both co-occurrence gates unwitnessable by any word; outputPartOfSpeech overrides rather than sets; stem names compare by object identity; and a Timeout is a statement about the machine, not the grammar. check-obligation-feasibility.ps1 answers the three mechanical questions before a budget is committed, and claims only that no known mechanism forbids a witness. Witness lexically-conditioned phonology and affix-conferred gates No grammar in the corpus declared an MPR gate on a phonological subrule, so exception features and minor rules -- one of the most ordinary things a real grammar does -- had no coverage at all. HCLoader emits both at lines 2057 and 2058, from FLEx's own Required and Excluded rule features. Satisfied cells go from four to nine, and the recurring shape is worth knowing: a feature the ROOT presets cannot witness a required-gate chain, because severing it can only turn a passing word into a failing one. What works is a feature an affix confers or destroys before the reader checks it. Two items are foreclosed by the engine rather than unattempted. Stem names compare by bare object identity, and a root-preset feature meets a plain set-membership test, so in both cases removing the payload can only break a match and never repair one. Raises the mutant budget from 45s to 180s and drops the confirmation retry. A Timeout roved to a new fixture that proved Unobservable in isolation while needing about 128s uncontended, and genuine non-termination has never once been observed here. Record what this measures, and where it deliberately stops Every figure was read from a checked-in ledger at the time of writing. Interface edges: 60 declared, 44 present, 19 witnessed. Obligation cells: 346 enumerated, 18 worth covering, 8 defensible. Gate arms: 46, of which 42 are worth covering and 14 are evidenced, giving 5 of 23 gates both MC/DC arms. Those are findings, not a backlog, and the docs now say so. The apparatus is frozen here on purpose: it established what the fixtures witness and produced the map of what remains, and its marginal return then fell to roughly nothing. obligation-triage.tsv classifies every unmet obligation by the blocker its own ledger records, and most of the remainder is impossible or strained rather than merely undone. One row changed what it MEASURES, and the docs say so rather than swapping a number quietly. The interaction-chain row reported chains whose writer and reader are each evidenced somewhere, which two separate words satisfy; it now reports chains with a same-word paired witness. Resolving a control arm to its nearest rule ancestor makes three more attributable. Four others never will be: Allomorph is always a child of LexicalEntry and never of a rule, and the DTD gives AffixTemplate no id at all, so neither has any identity to attribute a control to. An adversarial review then broke one claim. The affix-conferred blocking cell cited HCLoader.cs:1717 as its read, but that line populates ExcludedMprFeatures only from slot.ReferringObjects.OfType<ILexEntryInflType>(), FieldWorks' own irregular-form blocking. The witness is behaviourally genuine and the fixture stays as an engine test, but the claim is withdrawn. The general fact matters more than the cell. Producibility is keyed per attribute and does not compose along a chain, and no gate in this repository can catch that, because it requires reading another one. The branch's own working documents come out with it. Twenty plans, censuses and rationale files are deleted and their reasoning moves to the pull request, which is the one place a reader always looks and which survives being read a year later. Two were genuinely durable and move into conformance/docs instead: the HermitCrab XML semantic catalogue, and the pipeline design. Nothing shipped now points at a document a consumer does not receive. That eviction was not cosmetic. docs/conformance-migration-ledger.md was still load-bearing in code: it sat in ConformanceManifestGenerator's AdditionalSourceFiles, which is hashed into every manifest's SourceHash, so deleting the file would have permanently recorded an absent contributor to that hash, and a parametrized test would have thrown on a missing file. Make CI able to build, and stop gating on a deliberate backlog Two independent reasons every job failed. global.json pinned SDK 10.0.303 with rollForward disabled. Hosted runners carry 10.0.110, 10.0.204, 10.0.302 and 10.0.400 -- never 303 -- so the pin was unsatisfiable by construction and all three jobs died on their first dotnet invocation. That version was one developer's local SDK. The pin now states a floor with latestFeature instead of an exact patch. The part that matters is unchanged: the census still analyses this repository with the same Roslyn that builds it, taken from $(MSBuildToolsPath)/Roslyn/bincore rather than a NuGet package. An exact patch bought nothing beyond that, because SdkVersion feeds a graph hash that is recomputed every run and compared against nothing checked in, so any .NET 10 SDK is self-consistent. Pinning one that is not installable everywhere only made the repository unbuildable. Second, ci.yml gated on the semantic-coverage CLI, which exits non-zero whenever the catalog is incomplete. The catalog is INTENTIONALLY an incomplete proposal backlog, and unclassified-mapping is its expected state: the CLI reported zero new gaps, zero stale lines and zero unbacked quotients while still failing. The test named TheCheckedInCatalogMapsEveryRealSurfaceExactlyOnce already runs that identical audit under dotnet test, and asserts both that the catalog is incomplete and that unclassified-mapping is the only diagnostic class allowed, which is a stronger gate than an exit code and can express the intent. The redundant step is gone and the comment names its real gate. The CLI's exit code conflating "not complete" with "regressed" is left as it is, and recorded in the pull request rather than changed under time pressure. Apply CSharpier, and brace the bodies it wraps CI runs dotnet csharpier check, and 200 files in this branch had never been run through the formatter. Formatting them exposed a genuine disagreement between two pre-existing tools: .editorconfig sets csharp_prefer_braces to when_multiline, so a brace-less if is legal with a one-line body and an IDE0011 error once the body wraps. CSharpier wrapped 83 such bodies, all of them long argument lists, which turned legal code into build errors under TreatWarningsAsErrors. Master never hits this because its brace-less bodies are short enough not to wrap. Adding braces satisfies both tools, so neither .editorconfig nor the CSharpier configuration is touched -- the repository's style is the repository's own call. Convergence took two passes. The second was not redundant: while the Conformance project failed to build, the Tests project could not compile at all, so six of the 83 sites stayed invisible until the first pass fixed its project reference. Verified that no logic moved, rather than assuming it: comparing token streams with whitespace and braces stripped, every difference across all 108 changed files is either a trailing comma CSharpier adds to a multi-line initialiser or a using-directive reorder. Both are inert in C#. Handle both path shapes on either platform, and name mismatched words The compilation-graph hash is relocation-invariant so a graph captured on one machine can be verified on another, which means the path vocabulary has to understand a Windows-shaped path and a Unix-shaped one regardless of the host. LogicalPathTokens carries its own IsAbsolute and NormalizeAbsolute for exactly that reason. Three places reached for the platform's own path handling instead, and all three failed only on Linux. A csc switch is syntactically an absolute path on Unix, so switch shape must be decided before any path handling: /noconfig carries no value, /doc:/tmp/x.xml carries one, and /home/me/src/X.cs is a real path. The discriminator is what follows the switch name -- nothing, a colon, or a further separator. The ancestor .editorconfig probe used Path.GetFileName, whose notion of a separator is the host's, so a Windows-shaped path there returned the whole string as its filename on Linux and the probe silently declined a file it should have admitted. It now segments with this class's own rules. Finally, a mismatch reason gave only a count. A count cannot be diagnosed from a CI log on a machine you cannot reach, so it now names the words and their expected and actual analyses, truncated and capped at five. Read census specimens from code this branch owns Master replaced the SINGLE_THREADED compile symbol with a runtime MaxDegreeOfParallelism, so the engine no longer contains any code that exists in only some configurations. Two SemanticCoverage tests were reading engine code as their live specimen and lost it. Both point the census at real repository source rather than at a string literal, which is the whole reason they exist -- the synthetic-source tests beside each already cover the same logic on fixtures. So they still read real source, but source whose shape this branch is answerable for. The local-function keying test reads FailureRuleAttributor's Walk in place of Morpher's deleted GenerateSynthesis. The configuration-only census test reads Morpher's surviving #if OUTPUT_ANALYSES region, scoped to the type rather than to one method signature, so that an upstream rename cannot present itself here as a census defect. The second test's claim is narrower than it was, and the name now says so: it witnessed a method that existed only under a symbol, and no such method remains anywhere in the engine. What it still witnesses is a configuration-gated region censused under exactly the configurations that contain it. Its expected configuration string was confirmed by running the test, not derived. Stop naming the rule that hit the epenthesis loop cap Threading rule.Name down to the 256-node throw site cost a required parameter on two public constructors, SynthesisRewriteRuleSpec and EpenthesisSynthesisRewriteSubruleSpec. Both ship in the HermitCrab NuGet package, so that is a source and binary break for anyone constructing them. Nothing read what it bought. InfiniteLoopException.RuleName has no reader anywhere in the repository: the conformance runner mentions the exception only in a comment, the fixture that provokes it asserts expect_crash and not a message, and the engine test asserts only Throws.TypeOf<InfiniteLoopException>(). A capability with no consumer does not justify breaking a published constructor, so the four files go back to what master has. The 256-node cap itself is untouched, and edge-cases/simultaneous- epenthesis-cascade still pins it. Split the grammar health checker out to its own change GrammarHealthChecker and GrammarHealthFinding are a diagnostic feature, not part of the conformance suite: nothing in conformance/ or in the Conformance project calls them, and their only caller here was their own test file. They are wanted in FieldWorks on a schedule of their own, so they ship in #475 instead, which now carries the newer copies this branch had developed. Two tests go with them that cannot follow them there. Both load real conformance fixtures, and one uses the Conformance project's Fixture.DiscoverAll, so neither compiles on a branch that has no fixture tree. They belong here rather than in #475, and should come back once that lands; until then they are recoverable from 9dfb2f6. What remains of this branch's engine footprint is Trace.FailureAllomorph, its assignment in TraceManager from an argument that method already received and discarded, and the inert SemanticBranch capture point. Remove the semantic branch marker channel SemanticBranch was introduced by this branch's first commit as a way for engine code to declare a semantic path that the census would then count. It was never wired up. No engine or tool code has called SemanticBranch.Hit in any commit here, no ledger, catalog or coverage file has ever carried a branch: id or a branch-marker surface, and no document names it as a mechanism anyone intends to use. Every coverage number this branch reports comes from severance sweeps and trace evidence instead. So it was a public type in the shipped netstandard2.0 package with no consumer, and the census carried a fail-closed resolution path that could never fire. It could not move to the Conformance project either: that project references the engine and not the reverse, so a marker channel living there could never be called from the engine that is its only intended caller. Removed with it: CollectMarkers and its two resolution helpers in CSharpInventoryReader, the branch-marker catalog family, and three tests that existed only to exercise markers. Three further tests used a marker as an incidental probe and keep their subject. Two lose nothing -- the if/else-if arms keep an inert body, and the reachable-versus-dead claim was already carried by the xml-read assertions beside it. The third is narrower than it was and its name now says so: it asserted that an unresolved marker in dead code does not trip the fail-closed check, and with no such check left it now asserts only that an unresolved call reached from dead code raises no diagnostic. What remains of this branch's engine footprint is Trace.FailureAllomorph and its assignment in TraceManager: two files, thirteen lines, no new public type, and both read by FailureRuleAttributor.
The inventory reader compiles one engine file at a time, so members of other types resolve only from metadata stubs. Missing-member errors on an instance receiver (CS1061) were already on the approximation allowlist; the static-receiver form (CS0117) was not, so a static call such as Word.SynthesisAlternatives(...) in Morpher.cs made the reader treat the whole file as uncompilable and census no decisions at all. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two synthetic edge-case fixtures, each with a positive witness and negative controls, for behaviours a HermitCrab port got wrong and that no fixture exercised: - rewrite-analysis-feature-neutralization: a SegmentNaturalClass-based rewrite whose analysis-side reconstruction erases the changed feature, so GetMatchingStrReps unifies the pre-image with a segment from another table's stratum and the word gains a second analysis; a control word whose orthogonal feature blocks the collision keeps exactly one. - synthesis-stratum-render-stale-table: SynthesisStratumRule.Apply never reassigns Word.Stratum, so a root crossing strata renders its signature surface against its entry table (empty when that table lacks the segment); a root entered on the surface stratum renders normally. Expectations come from the self-check harness built here. Generated ledgers regenerated through the CLI; three hard-coded row counts updated for the new rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A segment is its feature bundle, and a CharacterDefinitionTable only spells
that bundle for one stratum (CharacterDefinitionTable.GetMatchingStrReps
matches by FeatureStruct.IsUnifiable alone). So a root entered on an inner
stratum surfaces on the final stratum spelled by whichever final-table
segment carries the same bundle, with no phonological rule involved.
Two fixtures already exhibited this as a side effect and both blamed the
wrong mechanism. rewrite-analysis-feature-neutralization attributed the
second analysis of "d" to FeatureAnalysisRewriteRuleSpec's analysis-side
feature erasure; deleting the rule from that grammar leaves hc.dll analysing
"d" as ROOT1|b, so erasure is not load-bearing there. The grammar and word
notes now say so, and the directory name is kept as a record of the earlier
attribution.
New fixture edge-cases/cross-table-root-respelling isolates the mechanism:
two strata with disjoint alphabets, no phonological rule, an inner root
("m", bundle X) that surfaces as the final table's "t" (also bundle X), a
native final-stratum root at the same spelling so "t" is ambiguous, a
suffix so respelling is exercised under affixation, an inner root whose
bundle no final-table segment carries, and negative controls. Every
signature is transcribed from hc-conformance's self-check.
Word "tu" pins a second finding worth the team's eye: HermitCrabExtensions.
ToRegexString renders ROOT1+SUF against the Inner table, where SUF's bundle
has no segment, so the signature is "ROOT1+SUF|m" with the suffix's own
segment silently absent. Same GetMatchingStrReps lookup, zero-match branch.
constructs.txt gains "CharacterDefinitionTable: cross-table respelling (an
inner-stratum root surfaces spelled by the final stratum's table)", claimed
by the new fixture, by rewrite-analysis-feature-neutralization "d", and by
synthesis-stratum-render-stale-table "j", so a consumer mapping the construct
cannot inherit coverage from the broader multi-table row.
Ledgers regenerated (coverage-report, manifest, interface inventory,
interaction chains, dataflow obligations, rule-interaction pairs, coverage
traceability, evidence cards, engine-gate inventory, gate obligations); the
engine-gate inventory had also drifted behind Morpher.cs line numbers and
the synthesis-stratum-render-stale-table fixture, and is now current.
Hard-coded ledger counts in four test classes updated to the measured
values; OrderingGeneratorTests had been asserting 33 fixtures since before
the two fixtures pinned in 25ddf91.
Question for review: is an inner-stratum root surfacing through the final
table's same-bundle segment, with no rule, the intended behaviour? The
conformance suite now treats it as ground truth.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…aml front matter Every fixture under conformance/languages/** and conformance/edge-cases/** now carries a fieldworks_producible: true|false field in its words.yaml front matter, directly after requires. The verdict names whether a real FieldWorks project (via HCLoader, Src/LexText/ParserCore/HCLoader.cs in the separate FieldWorks repo) could ever produce every construct the fixture's grammar.xml exercises -- a fixture no FieldWorks user could produce is HC-engine-only, and passing it is not evidence of FieldWorks-facing coverage. 17 of 36 fixtures land at false, well beyond the six already suspected (prefixal-discontinuous-slot-dependency, suffixing-evidential-adjacency-chain, fusional-realizational-morphology, morphotactic-attribute-breadth, loader-isactive-breadth, feature-gating-breadth) plus the two the prior triage flagged as likely (suffixing-extension-slot-ordering, loader-default-symbol). Checking every fixture rather than assuming found four more classes of non-producible construct beyond the two already-known cross-cutting findings (RealizationalRule never built by HCLoader; MprFeatureGroup fixed at three hardcoded groups): - Two custom-named MorphologicalPhonologicalRuleFeatureGroup fixtures (mpr-group-overwrite-without-realizational, mpr-overwrite-order-dependence) whose entire premise is a construct HCLoader can never emit. - Four fixtures (bistratal-overlapping-segment-representation, cross-table-root-respelling, rewrite-analysis-feature-neutralization, synthesis-stratum-render-stale-table) whose core premise is two distinct CharacterDefinitionTables across strata; HCLoader loads exactly one table per grammar and reuses it for every stratum it builds (HCLoader.cs:204, 227-233, 374, 2669-2742). - loader-isactive, which tests a second, inactive PhonologicalFeatureSystem block; HCLoader loads exactly one feature system per grammar directly from LibLCM (HCLoader.cs:198). - compounding-breadth's isolated CompoundingSubrule@isActive="no" decoy; HCLoader always synthesizes exactly one CompoundingSubrule per CompoundingRule with no per-subrule Disabled/isActive filter at all (HCLoader.cs:1842-2001). - feature-system-breadth, whose isActive decoys sit mostly on collection kinds (SymbolicFeature, ComplexFeature, FeatureValue, SegmentNaturalClass, FeatureNaturalClass, LexicalEntry) HCLoader has no Disabled/isActive filter for at all. Adds the two known cross-cutting findings to fieldworks-producibility.tsv as a new, hand-curated loader-gap subject kind (RealizationalRule, MorphologicalPhonologicalRuleFeatureGroup) alongside the two mechanically-enumerated kinds (failure-reason, interface-attribute) -- neither finding is a FailureReason or a single (element, attribute) pair, so the generator script, its own header comment, and FieldworksProducibilityLedgerTests.cs all gained a third, literal (not mechanically re-derived) subject list for this kind. Extends words.schema.json and WordsYamlLoader.cs/WordsYaml.cs to recognize the two new front-matter keys (a false verdict requires non-empty notes, enforced at parse time). Adds a new PROTOCOL.md section documenting the field's meaning and its three usage-level checks that a per-row TSV entry alone cannot express (a template-slot rule may not set MPR features; requiredMPRFeatures must be identical across a rule's subrules; co-occurrence rules must be type="exclude"), and updates how-it-is-computed.md's counts and prose to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nse so FieldWorks can produce it rrPast was a RealizationalRule, which FieldWorks' HCLoader never constructs (every Load*AffixProcessRule method builds AffixProcessRule only; LoadInflAffixProcessRule's own comment at HCLoader.cs:977 even says "TODO: use realizational affix process rules"). Converts it to an ordinary MorphologicalRule carrying the same OutputHeadFeatures (past tense) and no RealizationalFeatures, keeping every id, gloss, and every other rule untouched. Validated against HermitCrabInput.dtd. Re-verified against the C# founding oracle in self-check mode (hc-conformance.exe --fixtures conformance --propose --include-pathological, the same in-process Morpher self-check always uses): all 36 fixtures still pass, and -- stronger than expected -- kalid/kalmuid/kalidmu's signatures and traced rules come back byte-identical to the committed words.yaml; the RealizationalRule-to-MorphologicalRule swap is fully transparent to both. Only the RealizationalAffixProcessRule exercises claim on those three words is dropped (nothing producible can back it any more) and fieldworks_producible flips to true. Regenerates the mechanical ledgers this edit genuinely moves (engine-gate-inventory, gate-obligations, interface-witness, construct-claim-corroboration, grammar-coverage-ledger, the conformance manifest) -- each diff is scoped to exactly what changed for this one fixture. A handful of unrelated fixtures (strrep-identity, suffixing-extension-slot-ordering) show pre-existing, environment-driven drift in interface-witness.tsv on a fresh --coverage-traceability recompute (a null-symbol rendering difference and a differing stack-trace class name, neither caused by this change); that drift is deliberately left uncommitted rather than laundered in here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ersion grammar-coverage-ledger.tsv and construct-claim-corroboration.tsv both moved when rrPast stopped being a RealizationalRule (previous commit): kalid/kalmuid/kalidmu's RealizationalAffixProcessRule claims (prose, unmapped) became "Syntactic feature agreement (...)" claims that corroborate on OutputHeadFeatures instead. Rows: 677 -> 676 (construct 96 -> 95, one claimed-unmapped row removed rather than replaced) and confirmed/unmapped 210/250 -> 213/247 (rows unchanged at 475, three rows change status). Found by a full-suite run after the previous commit's ledger regeneration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…laim change Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ifest for deep-optional-affix-nesting fieldworks_producible: true claims a grammar.xml shape a real FieldWorks project could produce, but nothing previously backed that claim with an actual project. Add the real LibLCM project (authored via PanGloss's xample-projector from this fixture's own grammar.xml, verified to reproduce byte-for-byte under the existing determinism normalization) plus a small versioned manifest of phoneme-removal counterfactuals against it, document the new fieldworks/ directory shape in PROTOCOL.md, and gate that shape mechanically (this repo never opens the .fwdata itself). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ader-scope check disproved feature-gating-breadth was marked fieldworks_producible: true, but HCLoader hardcodes MorphologicalRuleOrder.Unordered at every Stratum it builds (HCLoader.cs:227,230,374) and can never emit linear -- and this fixture's committed expectations for kalidka/kalnoka/kalidmu genuinely depend on linear ordering, measured by flipping to unordered and re-running the C# founding oracle (each gains a spurious extra parse). Flip to false, independent of the fixture's earlier rrPast RealizationalRule->MorphologicalRule conversion, which stays and fixed a different gap. Record the rule-order gap as its own loader-gap subject (morphologicalRuleOrder) in fieldworks-producibility.tsv, generated via generate-fieldworks-producibility.ps1 rather than hand-edited, and state its real condition: not "the fixture declares linear" (the DTD defaults to linear, so that test would condemn most fixtures) but "the fixture's ground truth changes under the unordered flip". PROTOCOL.md section 9 gains this as a fourth usage-level check alongside the three already there, with the same measurement instruction. Narrow the MorphologicalInput.excludedMPRFeatures row's notes to its real scope: producible only on a rule reached through an AffixTemplate Slot (HCLoader.cs:1717's sole assignment site), never on a stratum-level rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…Loader.cs An adversarial review claimed four upstream `true` marks could not survive a FieldWorks project, and each is independently confirmed here by reading HCLoader.cs: - edge-cases/mpr-gated-exception: mrSuf is a plain stratum-level MorphologicalRule (no AffixTemplate/Slot in this grammar at all), and HCLoader.cs:1717 -- the sole site setting AffixProcessAllomorph.ExcludedMprFeatures -- fires only inside LoadAffixTemplate's own slot-blocking loop. The fixture's OTHER excludedMPRFeatures, on a PhonologicalSubrule, is unaffected (HCLoader.cs:2058, unconditional) and stays producible. - languages/polysynthetic-stratal-derivation-chain: both CompoundingRules declare blockable="false"; HCLoader never touches CompoundingRule.Blockable (zero hits for "lockable"), and the engine's own constructor defaults it to true. - languages/suffixing-vowel-harmony: ruleCollective's RequiredEnvironments sits on a genuinely discontinuous (Insert+Copy+Insert) subrule output. HCLoader's generic multi-action path for such a rule, LoadAffixProcessAllomorph, never touches .Environments -- only LoadRootAllomorph, LoadCircumfixAffixProcessAllomorph (built from two SEPARATE literal-form allomorphs), and LoadFormAffixProcessAllomorph (single-piece) do. strrep-identity's msubI is the boundary case that stays producible: same RequiredEnvironments shape, but a single contiguous Insert+Copy piece. - edge-cases/metathesis-comparison-crash: HCLoader always binds HC's LeftSwitchName to the pattern group at LibLCM's own RIGHT-switch index (HCLoader.cs:2122,2134-2135), and FieldWorks' own metathesis-rule editor forces LibLCM's LeftSwitch cell to be structurally earlier than its RightSwitch cell -- so a FieldWorks project can never produce the leftSwitch-names-the-earlier-group shape this fixture's crash regression depends on. Each flip carries a fieldworks_producible_notes entry naming the construct and citing the HCLoader evidence, in the voice the existing false fixtures use. Manifest regenerated to match the four updated words.yaml hashes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e grammar grammar.xml:104's PhonologicalRule (prule4) carries no multipleApplicationOrder attribute at all, so it takes the DTD default, leftToRightIterative (HermitCrabInput.dtd:179) -- confirmed by reading both the fixture and the DTD directly. The header comment claiming it is "tagged simultaneous" describes the source C# unit test this rule was ported from (RewriteRuleTests.EpenthesisRules sub-case (1), which really does run under RewriteApplicationMode.Simultaneous and completes cleanly), not what the port itself declares. The port drops the tag deliberately, not by oversight: words.yaml's own header already documents this fixture as "the OTHER direction of the Simultaneous-vs-Iterative divergence," and the Iterative default is exactly what produces the self-feeding epenthesis cascade that hits the engine's 256-node cap and crashes -- the fixture's whole reason to exist. Adding multipleApplicationOrder="simultaneous" would remove the crash this fixture pins (per the source unit test, Simultaneous mode completes without incident), so the grammar and its oracle-verified expect_crash pin are correct as committed; only the comment was wrong. Corrected the comment to describe what the XML actually declares and why. Manifest regenerated for this fixture's grammar hash. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…urement fieldworks-producibility.tsv's morphologicalRuleOrder loader-gap row has said "six fixtures were measured this way and found genuinely dependent" with nothing recording which six, so the claim could not be checked. This adds that ledger. Reproduced this session by copying each fixture into a scratch directory (never the checked-in copy), flipping every <Stratum> from morphologicalRuleOrder="linear" to "unordered", and re-running the C# founding oracle (hc-conformance.exe --propose --include-pathological) against the unchanged checked-in words.yaml: - languages/fusional-realizational-morphology: 4/63 words gain a spurious parse (ygofz, yxpedz, gofwz, gofhw) -- matches exactly. - languages/suffixing-extension-slot-ordering: 4/53 (yxkibz, sekwz, sekhw, dalehiz) -- matches. - edge-cases/mpr-group-overwrite-without-realizational: 1/5 (wudofq) -- matches. - edge-cases/feature-gating-breadth: 3/15 (kalidka, kalnoka, kalidmu) -- matches. The remaining two live in the PanGloss repository's own conformance-staging/filter-passes/ (exact-span, structural-transition) -- exactly why the count looked wrong when searching only this repo. Reproduced those too (read-only copy, same oracle build): both already-correct signatures are produced TWICE under "unordered" rather than a spurious new one, which their own words.yaml headers already documented informally. The other three upstream fixtures in this ledger are already false for independent, separately-cited reasons that never mention rule order; the doc says so explicitly so their notes are not misread as incomplete. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1. Adds a loader-gap subject for the metathesis switch-name/position inversion (the mechanism behind the metathesis-comparison-crash correction): HCLoader always binds HC's LeftSwitchName to the pattern group at LibLCM's own RIGHT-switch index (HCLoader.cs:2122,2134-2135), and FieldWorks' own metathesis-rule editor forces LibLCM's LeftSwitch cell to be structurally earlier than its RightSwitch cell -- so a FieldWorks project can never produce a MetathesisRule whose leftSwitch names the structurally earlier of its two pattern groups. 2. Narrows the generic Environments: Yes verdict with a fifth PROTOCOL.md section 9 usage-level semantic check (alongside the existing four): environments are producible only when a MorphologicalSubrule's output is expressible as a single literal-form affix (RootAllomorph, a paired Circumfix built from two separate literal-form allomorphs, or a single-piece Form) -- never a genuinely discontinuous multi-piece process-rule output. HCLoader's generic multi-action path for such a rule, LoadAffixProcessAllomorph, never touches .Environments at all. edge-cases/strrep-identity's msubI is named as the boundary case that keeps this check from being read over-broadly (same RequiredEnvironments shape, but a single contiguous piece, so it stays producible). 3. Adds a loader-gap row for lexical-pattern root-allomorph shapes (b[Vowel]t, [Any]*), which are producible (HCLoader.cs:2532-2571 IsLexicalPattern; 2731-2740 registers every natural class for lexical patterns) and previously had no row at all. Regenerated conformance/fieldworks-producibility.tsv through generate-fieldworks-producibility.ps1 (88 rows: 23 failure-reason + 60 interface-attribute + 5 loader-gap; Yes=62, No=26) rather than hand-editing it, and updated docs/how-it-is-computed.md's counts and prose to match. FieldworksProducibilityLedgerTests' pinned counts and loader-gap subject list updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Xplat Code Coverage attaches a CLR profiler to the test host via CORECLR_*/COR_* environment variables, and ProcessStartInfo inherits them into any child process by default. CounterfactualGate's --evaluate-mutant child (which intentionally crashes on a decoy grammar) and RepositoryCompilationGraphLoader's MSBuild capture children both spawned that way, uninstrumented. Reproduced the test host crash twice under `--collect:"Xplat Code Coverage"` (positions varied: mid TheCheckedInCatalogMapsEveryRealSurfaceExactlyOnce, then early in CounterfactualGateTests) and confirmed 0/3 crashes across the same invocations once the profiler variables are stripped before Process.Start.
…rocess AdapterEngine's external-adapter spawn was a third unguarded ProcessStartInfo site the first fix missed; traced its one construction path (Program.cs's --adapter flag) and confirmed no test reaches it (the only in-process Program.Main test invocation, NoMemoizationOptionRejectsAdapterMode, is rejected by --no-memoization validation before AdapterEngine is ever built). Fixed it anyway: it exercises the identical kill-on-timeout hazard, and `dotnet test`-reachability today doesn't guarantee it stays that way. A grep for the literal call sites also missed a fourth and fifth: the test project has its own child-process spawns (ChildProcessHarness and three tests that build a ProcessStartInfo directly to re-run --evaluate-mutant or exercise MsBuildProcessRunner), all equally reachable from `dotnet test` and none stripping the profiler either. Rather than keep patching call sites, made ChildProcessEnvironment.CreateStartInfo the only way to construct a child ProcessStartInfo in this assembly (and the test assembly it grants internals to): every site now gets one pre-stripped from the factory instead of building its own. ProcessStartInfoConstructionGateTests scans both assemblies' sources and fails, naming the file, if a raw `new ProcessStartInfo` (qualified or not) appears anywhere else. Falsified before trusting it: a first version matched only the unqualified spelling and missed a namespace-qualified `new System.Diagnostics.ProcessStartInfo` probe outright (silent false pass); the corrected regex-based version named the probe's file precisely, and cleanly passed once the probe was removed. Re-verified the original repro post-fix: isolated HermitCrab.Tests under `--collect:"Xplat Code Coverage" --blame-crash` (578 passed, 1 skipped, 579 total) and the full-solution CI-shaped run (4 + 83 + 800 + 578 passed, 4 skipped, 1469 total) both completed with exit code 0, no test host crash.
Re-authors the fixture's five FieldWorks-non-producible constructs (four the
prior notes named, plus one this conversion discovered was never named --
LexicalEntry.family + MorphologicalRule/RealizationalRule@blockable, neither of
which HCLoader ever writes) using mechanisms HCLoader actually emits:
- A template-slot rule writing MorphologicalOutput.MPRFeatures moves to a
stratum-level rule instead (mrGrpSetA/mrGrpSetB/mrGrpReq replace
mrMB/mrMA/mrMReq's MPR-group role); mrMB itself stays template-slotted for
its Slot-ordering role, minus the write.
- The MorphologicalPhonologicalRuleFeatureGroup gets a FIXED HCLoader-real name
("exceptionFeatures", matchType "all", default Overwrite) instead of a custom
name with outputType="append" -- new coverage, since no fixture previously
exercised any of HCLoader's three real group names.
- MorphemeCoOccurrenceRule/AllomorphCoOccurrenceRule move from type="require"
(HCLoader never builds one) to type="exclude" on a fresh isolated pair
(mrExA/mrExB).
- RealizationalRule (mrReal/mrRealDecoy) and the family+blockable/CheckBlocking
material (bak/dom/sim/rog and their words) are removed outright: neither has
any FieldWorks-producible substitute.
- Main's morphologicalRuleOrder flips from "linear" to "unordered" (HCLoader
hardcodes Unordered and can never emit "linear"), re-verified word-by-word.
- mrExclReader's excludedMPRFeatures moves into an AffixTemplate Slot: it and
MorphologicalOutput.MPRFeatures (mrConferExcl, kept stratum-level) have
OPPOSITE placement requirements, a fact the pre-conversion (both-stratum)
shape never had checked against it -- see mpr-gated-exception's own mrSuf for
the same gap in the fixture this pattern is modeled on.
Every signature and expect_fail verdict re-derived against the C# founding
oracle (self-check mode); all matched the predicted design on the first run.
Full conversion history, and what did not survive, recorded in words.yaml.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ible pattern Per the triage, converts ONLY the family-blocking third of this fixture's three named blockers (family, RealizationalRule, per-subrule compound MPR). RealizationalRule and per-subrule compound MPR stay untouched -- converting either would gut what "fusional-realizational-morphology" is actually testing -- so the fixture stays fieldworks_producible: false overall, with notes updated to say so precisely. Ordinary-rule family-blocking (duc/tul/ducit/ducunt, Family "famCarry") is converted to the edge-cases/mpr-gated-exception pattern: eDuc now carries ruleFeatures="mprSuppletivePast" directly (mirroring mpr-gated-exception's own eVokad) and mrPast2's own MorphologicalInput excludes that feature. Getting this genuinely producible (not just relocated) required moving mrPast2 into a new, minimal AffixTemplate: MorphologicalInput.excludedMPRFeatures is producible ONLY on a rule reached through a Slot (the opposite requirement from MorphologicalOutput.MPRFeatures), the same fact edge-cases/ morphotactic-attribute-breadth's own conversion found -- and this grammar had no AffixTemplate at all before now. Added "ferit" (FER+PAST2) to prove the exclusion is specific to DUC, not a blanket refusal. Compound-side family-blocking (the former genlav/corriv words, famCompBlock/ famHNBlock) is REMOVED outright, not converted: CompoundingSubrule's own requiredMPRFeatures/excludedMPRFeatures -- the mechanism that would otherwise stand in -- is unconditionally absent from both compounding-rule loaders, with no template-placement carve-out the way the ordinary-rule version has. No FieldWorks-producible substitute exists. mrCompoundHN/mrCompoundNH's own blockable="false" (never producible either, and for mrCompoundNH never even exercised) is dropped with it, reverting to the engine's own default. Every remaining word re-verified against the C# founding oracle (self-check mode); zero mismatches on the first run, including the two new/changed words. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
For every fixture still fieldworks_producible: false after this branch's two conversions (20 of 36), records the blocking construct(s) with an HCLoader.cs citation, what it covers, whether a producible fixture already covers the same thing, and a recommendation of CONVERT / DELETE / KEEP-ENGINE-ONLY. Analysis only -- no fixture is touched or deleted. Highlights: cross-table-root-respelling (the fixture PanGloss's newest FST backend capability was named for) stays KEEP-ENGINE-ONLY, with the consequence stated plainly -- that capability's conformance evidence is a construct no FieldWorks project can ever produce. Two "census" fixtures (feature-system-breadth, loader-isactive-breadth) are KEEP-ENGINE-ONLY rather than narrowed, since narrowing would duplicate other fixtures and destroy the one thing each uniquely provides (every isActive-bearing kind checked together). One DELETE candidate (mpr-group-overwrite-without-realizational) is flagged pending a check this analysis could not perform itself: whether PanGloss's own PlanComposed backend already treats morphotactic-attribute-breadth's converted MPR-group test as an equivalent witness. Six fixtures get a concrete CONVERT mechanism, three of them following this task's own two conversions almost verbatim (including one, polysynthetic-stratal-derivation-chain, where the fix is PROVABLY behavior- preserving from Word.CheckBlocking's own family==null short-circuit, not just expected to be). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Preserved verbatim so it is not lost. Contains coverage.csv/evidence-card/tsv regeneration plus a further morphotactic-attribute-breadth expansion and two census-pin test edits. NOT yet justified: InterfaceInventoryLedgerTests' typedEdges 53 -> 51 carries no explanation, and the CheckBlocking family+stratum evidence card now reports no exercising fixture at all.
…ge.csv rows The fieldworks_producible conversion (30216cd) and its later WIP expansion re-authored this fixture's words.yaml without ever re-adding per-word/per-parse "exercises:" tags, so the whole MorphotacticAttributeBreadth block (28 rows) silently vanished from conformance/coverage.csv with no test catching it: parity-check.py and CheckedInCoverageTablesAreUpToDate only check that coverage.csv matches what WordsYamlLoader finds, and a fixture with zero "exercises:" tags matches that trivially. Every word below now carries an "exercises:" tag naming what it ACTUALLY tests post-conversion, not a blind copy of the pre-conversion label -- topsakatana in particular moves from the removed co-occurrence construct to "Affix template slots", since mrCoA-D no longer carry any co-occurrence rule. Regenerated every downstream derived artifact this affects (coverage.csv, construct-claim-corroboration.tsv, engine-gate-inventory.tsv, the manifest, grammar-coverage-ledger.tsv, interface-witness.tsv, four evidence cards) via hc-conformance --write-coverage-traceability / --coverage-report, and fixed two more stale entries the same conversion left behind: semantic-coverage-presence-waivers.txt claimed a presence-only waiver for the removed custom MPR-group isActive decoy (now genuinely uncovered, not presence-only), and semantic-coverage-evidence.tsv still named four AffixTemplate Slot pairs whose Slots the conversion deleted outright (a full re-sweep is hours long over this corpus; the four rows are pruned with a note, not regenerated). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Seven SemanticCoverage tests pin exact or floor counts derived from the corpus; the two fieldworks_producible conversions in this branch (30216cd morphotactic-attribute-breadth, baa401f fusional-realizational-morphology) shifted several of them, and this branch's own restoration of morphotactic-attribute-breadth's "exercises:" tags shifted a few more. Each updated assertion carries a comment naming the exact mechanism (not just the new number): - ConstructClaimCorroborationTests: 475/213/15/247 -> 473/211/12/250 (removed family+blockable/ RealizationalRule/append-mode-group words took claims with them). - CoverageGapRatchetTests: the orphaned-evidence-row check (see the prior commit's semantic-coverage-evidence.tsv fix) had been failing on every run since 30216cd landed, so the gap-count assertion below it never actually executed post-conversion until now. Raised 18 -> 21; 6 of those are positively attributable to this branch (new adjacent-Slot pairs created by removing the Slots that used to separate them), named explicitly; the other 15 are unrelated (git-log confirmed those fixtures untouched since before this branch) and left for separate follow-up. - FoldInCandidateLedgerTests: MorphologicalInput.excludedMPRFeatures is no longer edge-case-only now that fusional-realizational-morphology's mrPast2 reaches it through a real AffixTemplate slot. - GrammarCoverageGateTests: TraceFloor 71 -> 69. dtd:enum/MorphologicalRule/blockable/false and dtd:enum/RealizationalRule/blockable/false lose their only Trace witness (mrUnblockable/kulgi and mrReal/kulru, both removed); confirmed by temporarily dumping every Trace item and checking no other fixture's blockable="false" fires in a verified parse. - GrammarCoverageLedgerTests / InterfaceWitnessLedgerTests: net row-count shifts from LexicalEntry.family losing two present fixtures and three of morphotactic-attribute-breadth's claimed-construct categories disappearing entirely. - InterfaceInventoryLedgerTests: typedEdges 53 -> 51, Slot.morphologicalRules->RealizationalRule and AllomorphCoOccurrenceRule.otherAllomorphs->Allomorph both lose their only declaring fixture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Revises this doc's own recommendation for languages/suffixing-extension-slot-ordering: an earlier draft recommended converting its family-blocking share, following the same pattern used for languages/fusional-realizational-morphology and edge-cases/morphotactic-attribute-breadth. That recommendation did not account for those two conversions already having landed by the time it was written -- both removed their own >=2-LexicalEntry-sharing-a-family material outright (HCLoader.cs has zero hits for "family"/"Family" across 2837 lines, confirmed by grep, so LexEntry.Family can never be set by any FieldWorks project), leaving suffixing-extension-slot-ordering as the ONLY fixture left in the suite exercising Word.CheckBlocking's family+stratum precondition (Word.cs:475-477). Converting it away, as originally recommended, would have completed the destruction of that regression witness -- the two prior conversions already cost 20 of 32 evidence-cards/ cells that used to name a family+stratum witness (the other 12 still resolve, to this very fixture). Keep it unconverted for all three of its defects, indefinitely, not merely "for now". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add ConformanceFixtureGateTests.EveryFixtureContributesAtLeastOneCoverageRow: a fixture whose words.yaml carries no "exercises:" tag anywhere silently contributes zero rows to conformance/coverage.csv, and nothing else catches it -- parity-check.py and CheckedInCoverageTablesAreUpToDate both only check that coverage.csv matches what WordsYamlLoader finds, which a fixture with zero exercises tags matches trivially. This is exactly how edge-cases/morphotactic-attribute-breadth silently lost its whole 28-row block (fixed by hand, ea3b211); the gap was still open for the other 35 fixtures. Checked all 35: edge-cases/simultaneous-epenthesis-cascade legitimately has none (expect_crash, single word, no completed parse ever exists to attribute a construct to -- named in the gate's allowlist with that reason). edge-cases/mpr-overwrite-order-dependence had zero "exercises:" tags since its own addition (f42d959) -- not legitimate, so its words.yaml now carries "MPR features/groups" throughout, and every downstream generated ledger (coverage.csv, rules.csv, generated/hc-conformance-manifest.v1.json, construct-claim-corroboration.tsv, grammar-coverage-ledger.tsv, interface-witness.tsv, evidence-cards/) is regenerated and every pinned count updated to match. The gate refuses vacuously three ways: asserts a discovered-fixture floor, asserts coverage.csv has at least one data row, and asserts every allowlist entry names a real discovered fixture id (so a rename/typo can't silently exempt nothing). Also: three committed citations of HCLoader.cs's line count as "2837" actually meant its non-blank line count -- verified against the FieldWorks checkout, the whole file is 2837 lines but only 2510 are non-blank; corrected to 2510 in conformance/docs/how-it-is-computed.md, conformance/docs/engine-only-fixture-retirement.md, and conformance/tools/generate-fieldworks-producibility.ps1 (the zero-hits-for-"family" conclusion those cite is independently re-verified and unchanged). And: CoverageGapRatchetTests.cs's "Raised 18 -> 21" comment claimed five fixtures were untouched since 43af40e/f42d9591; one of them, languages/suffixing-vowel-harmony, was in fact touched since then by f730e29 (metadata-only: fieldworks_producible front matter), which predates this branch's own start at 2c87903, so the comment now says that rather than overclaiming zero touches. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ere true My own correction sent the previous agent to 2,510 as if 2,837 were wrong; it is not. The file has 2,837 total lines and 2,510 non-blank. State both so neither reading looks like an error later.
…orks_producible Both CompoundingRules declared blockable="false", but this grammar declares no Family at all, so Word.CheckBlocking's family==null short-circuit makes Blockable's value unobservable either way (Word.cs:573-575). Dropped the attribute (reverting to the engine's own default) and re-verified against the C# founding oracle: 35/35 fixtures still pass, this fixture's own signatures unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crJoinInsert existed solely to host one isActive="no" CompoundingSubrule probe (HCLoader's compounding-rule loaders always synthesize exactly one, always-active subrule per rule, no per-subrule Disabled/isActive filter). Removed the rule and its sole negative control, "kaitu", rather than converting it. Re-verified against the C# founding oracle: 35/35 fixtures still pass, every remaining word's signature unchanged. Coverage lost, named plainly: CompoundingSubrule@isActive="no" (per-subrule deactivation) is no longer exercised by any fixture in the corpus. Not duplicated elsewhere (grep- confirmed). CompoundingRule@isActive="no" itself (crDecoy) is unaffected, still covered by kaptu. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mrSuf's MorphologicalInput.excludedMPRFeatures was stratum-level, and AffixProcessAllomorph.ExcludedMprFeatures is set at exactly one HCLoader site, inside LoadAffixTemplate's own slot-blocking loop -- producible only on a rule reached through an AffixTemplate Slot. This is the fixture both of this branch's earlier conversions modeled their own conversions on, and it was the one fixture left demonstrating the pattern itself unconverted. Moved mrSuf/mrSufAlt off the Stratum's own morphologicalRules attribute into one Slot of a new posMPR-gated AffixTemplate, tried as alternatives exactly as they were tried as alternatives at stratum level under this Stratum's own "unordered" order. Re-verified against the C# founding oracle: 35/35 fixtures still pass, every word's signature (including vokadan's own expect_fail) byte-for-byte unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Follows the three fieldworks_producible conversions in the preceding three commits (polysynthetic-stratal-derivation-chain, compounding-breadth, mpr-gated-exception). Regenerated every derived ledger these grammar edits touch: coverage.csv/rules.csv, semantic-coverage-baseline.txt (two fresh gaps: dtd:enum/CompoundingRule/blockable/false, dtd:enum/CompoundingSubrule/isActive/no -- both named, expected consequences of the conversions), interface-inventory.tsv, interaction-chains.tsv, dataflow-obligations.tsv, rule-interaction-pairs.tsv, engine-gate-inventory.tsv, evidence-cards/*, and the --write-coverage-traceability bundle (interface-witness.tsv, construct-claim- corroboration.tsv, grammar-coverage-ledger.tsv, fold-in-candidates.tsv). Updated every pinned-count test whose checked-in ledger moved, each with a one-line mechanism comment: CoverageGapRatchetTests (21->20, compounding-breadth's crJoinInsert removal shrinks the Ordering-pair inventory), ConstructClaimCorroborationTests (483->482, kaitu's own Confirmed isActive claim removed with it), GrammarCoverageGateTests (TraceFloor 69->68, polysynthetic's last CompoundingRule.blockable Trace witness removed), InterfaceWitnessLedgerTests (391->393) and GrammarCoverageLedgerTests (673->675, both from mpr-gated-exception's first AffixTemplate/Slot newly presenting two interfaces), and OrderingGeneratorTests (two pinned pair counts, 155->152 and 1492->1473, both from shrinking Stratum-level rule lists per EnumerateStratumPairs/EnumerateAdjacentPairs' own denominators). Full SIL.Machine.Morphology.HermitCrab.Tests project verified green (579 passed, 1 skipped ratchet warning, 0 failed) after these changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
engine-only-fixture-retirement.md's plan text said "no fixture stays deprecated-hc-only
past the end of this plan" -- measurement contradicts that (13 fixtures genuinely have no
FieldWorks-producible substitute), and pretending otherwise would mean deleting real
regressions Machine wants. WordsYamlLoader's front-matter key vocabulary is fixed and
unknown keys are hard errors (confirmed by reading it and words.schema.json), and neither
defines any "permanent"/"deprecated" key, so there is no new front-matter key to add.
Instead: each of the 13 fixtures the retirement doc recommends KEEP-ENGINE-ONLY gets an
explicit "PERMANENTLY KEEP-ENGINE-ONLY, not a temporary or deferred verdict" lead sentence
prepended to its own fieldworks_producible_notes, naming why no future conversion is
pending (structural HCLoader/LibLCM absence, or -- for fusional-realizational-morphology
and suffixing-extension-slot-ordering -- also that converting away the remainder would gut
the fixture's own purpose or destroy a corpus-wide regression witness). 12 of the 13 are
here; the 13th, bistratal-overlapping-segment-representation, is handled in the following
commit alongside its own oracle re-verification.
fusional-realizational-morphology's prior note read "out of scope for this task", which
reads as temporary/deferred even though the underlying reasons are structural; reworded to
state the permanent verdict plainly, and to make clear the FieldWorks-producibility
argument and the "don't gut what this fixture tests" argument are independent reasons.
The other 11 already conveyed permanence via direct technical language ("cannot arise from
a FieldWorks project", "has no HCLoader writer", "there is no ... state to produce") but
never stated the permanent-engine-only verdict as a category explicitly; the new lead
sentence makes that verdict greppable and consistent across all 13.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…gainst C# oracle This fixture's own header disclosed it was authored/verified only against pg_parse::Morpher (HC-Rust), never the C# founding oracle -- the exact oracle-discipline gap the retirement doc flagged for follow-up. The founding oracle is available on this machine (hc-conformance.exe, self-check mode). Re-ran it in isolation (a scratch copy of just this fixture) and as part of the full corpus's ordinary self-check sweep: both PASS with no --propose patch. NO DIVERGENCE FOUND -- des/sed's ordinary parses, basi/abis's SKIPPED status, and eds's expect_fail all hold exactly as originally transcribed from pg_parse. HC-Rust's own authoring of this fixture was correct; this is not an instance of the divergence hazard the oracle-discipline rule warns about. Updated the fixture's own header and fieldworks_producible_notes to record the re-verification, and folded in the same KEEP-ENGINE-ONLY permanence framing the preceding commit gave the other 12 engine-only fixtures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ing check Performed the one PanGloss-side check the retirement analysis could not do itself (read-only investigation of the PanGloss repo, this repo untouched by it). Finding: NO duplication. PlanComposed does compile edge-cases/morphotactic-attribute-breadth (to CellOutcome::OracleExact), but it never witnesses MprGroupOverwrite on it -- that fixture's only overwrite-shaped MPR group (groupDecoy) is declared isActive="no" and is filtered out before the grammar model is even built (rust/crates/pg-grammar/src/load.rs), so PlanComposed's capability::characterize() has nothing to observe there. PanGloss's own checked-in golden coverage ledger confirms this suite-wide: PlanComposed has never witnessed MprGroupOverwrite on any fixture but mpr-group-overwrite-without-realizational itself. Recommendation flips from DELETE-pending to KEEP: this fixture remains the sole FST-backend witness for MprGroupOverwrite under PlanComposed, on top of its own independent HC-conformance value. Deletion is still the repo owner's call, not this analysis's; the row and summary table are updated with citations so that call can be made without redoing the research. Also records this session's three executed CONVERT rows and updates the running false-fixture count (20 -> 17). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…limit My own verdict said LibLCM has exactly one character-definition table per project, so no FieldWorks project could ever produce these four fixtures. That is false. PhPhonData.PhonemeSetsOS is an unbounded owning sequence whose model comment says it is a collection 'to allow for phonological descriptions at multiple levels (strata)', and IMoStratum.PhonemesRA is a settable per-stratum reference. LibLCM was designed for this. What is actually missing is a producer and a consumer: HCLoader reads PhonemeSetsOS[0] and reuses that table for every stratum, never touching StrataOS/PhonemesRA, and the FieldWorks UI hardcodes index 0 and self-repairs to exactly one. So these stay engine-only TODAY, as the regression that would prove a future projection -- not as a permanent dead end. Oracle: 35 passed, 0 failed.
…impossibility Same flaw as the two-table rows, found by auditing the method rather than the answer. LexicalEntry.family: the grep for "family" in HCLoader proved only that the EXPORTER omits it. FieldWorks expresses the fact via LexEntryInflType/LexEntryRef, HCLoader already processes irregular variants (:626-807) through an MPR-feature substitute, and it carries two TODOs naming HC rule blocking as the intended fix (:744, :1726) -- neither containing the word "family", which is exactly why the grep missed them. SymbolicFeature.defaultSymbol: LibLCM's FsFeatDefn owns a Default property whose model comment reads "Feature of number might have a default value of singular" (MasterLCModel.xml:1342-1345). HCLoader never reads it. Both are loader gaps kept as the regression a future writer would need, not permanent dead ends. Oracle: 35 passed, 0 failed. Producibility + fixture gates: 11 passed.
…-side fix that closes it
Measured, not inferred. Synthetic p/q/s standing in for a regular root, its irregular past, and the
regular suffix, in raw HC XML mirroring what HCLoader emits for FieldWorks Variants:
base: 'ps' -> [P+SFX|ps] ACCEPTED <- the regular root still takes the regular affix
'qs' -> [] blocked <- the irregular entry correctly cannot double-mark
adhoc: 'ps' -> [] blocked <- one MoMorphAdhocProhib closes it
'q' -> [NULL+Q|q] unchanged <- the irregular form still parses
Cause: only the irregular entry gets the MPR feature (HCLoader.cs:746), and ExcludedMprFeatures
refuses only entries carrying it, so the untagged regular root is unconstrained.
So FieldWorks ships the "does not apply" mechanism -- ad hoc co-occurrence prohibitions, modelled,
UI-exposed, and projected (:341-350, :2163-2239) -- but the automatic Variants feature never invokes
it, and nothing tells the author they need to. That is the reportable defect.
Also corrects the family verdict a third time. LibLCM has NO Family concept (zero hits in
MasterLCModel.xml), so the CONSTRUCT really is unproducible and my first claim was right about it;
what was wrong was inferring FieldWorks cannot express irregular blocking at all. It can, twice
over. A future loader would synthesize LexFamily from LexEntryRef data, which is what HCLoader's
own TODOs at :744 and :1726 ask for.
Grammars are inlined in the doc, not committed as files: EveryGrammarTheCoverageGateReadsBelongsTo
ADiscoveredFixture correctly rejected them as non-fixture grammar.xml files when I first tried.
Oracle: 35 passed, 0 failed. Fixture/producibility/coverage gates: 12 passed.
…oader gap or data-model limit Grepping HCLoader for zero hits proves only that the loader omits a construct. Each of these rows now also says whether LibLCM models the concept and whether the FieldWorks UI exposes it: - CompoundingRule.outputProdRestrictionsMprFeatures: LOADER GAP. MoCompoundRule.ToProdRestrict is a dedicated LibLCM field whose model comment matches the engine semantics, the UI exposes it on both compound kinds, and HCLoader already projects the same field for derivational affixes. - HeadMorphologicalInput.requiredMPRFeatures: LOADER GAP. The head MSA's InflectionClass is modelled and shown in the compound-rule editor; HCLoader reads that MSA's POS and exception features but never its inflection class. - HeadMorphologicalInput.excludedMPRFeatures, ObligatorySyntacticFeatures (both outputObligatoryFeatures rows), morphologicalRuleOrder: DATA-MODEL LIMIT. No LibLCM field or parser parameter carries the concept. fusional-realizational-morphology drops its "PERMANENTLY" claim and classifies each blocker.
…the same node A morphological rule whose output is CopyFromInput only, with no InsertSegments, contributes no surface material. When such a rule applies immediately outside a rule whose own affix ends at the shape's last node, both rules' ApplyRhs calls claim that one node as their morph: the zero-width rule through the "no new output morphs" fallback, and the wrapped rule through the loop that re-marks every allomorph the current rule wraps. Word.MarkMorph reparents a node to whichever caller claims it last, so the wrapped rule won the node while the zero-width rule's morph had already been inserted ahead of it. AllomorphsInMorphOrder walks insertion order and reads each annotation's own Allomorph, so the wrapped rule's ID was rendered in the wrong place and lost. The fallback now runs after that loop instead of before it. It claims the same node either way and finds the same analyses; only the order of the two colliding MarkMorph calls changes, which is what the insertion-order tie-break needs to render the wrapped rule's ID before the wrapping rule's. Three tests: the dropped-ID case, plus controls where the zero-width rule wraps the bare root and where no zero-width rule can apply. The dropped-ID test fails without this change and passes with it; the controls pass either way. The full HermitCrab suite is 575 passed, 1 skipped, no failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016AzbiMyms1YL83LbdRNJwx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A morphological rule whose
MorphologicalOutputisCopyFromInputonly, with noInsertSegments, adds no surface material. When such a rule applies immediately outside a rule whose own affix ends at the shape's last node, the reported parse drops the inner rule's morpheme ID.Reproduction
Root
sag, and three rules on a two-symbolcatfeature:mrA2BumrB2AimrThirdCopyFromInputonly)Parsing
saguireports:The second is a mislabelling, not a second analysis.
mrThirdinserts nothing, soA2Balone would yieldsagu; that string cannot denote any real derivation. The derivation actually applieda2b, thenb2a, thenthirdoutermost, and should readSAG+A2B+B2A+THIRD. Both analyses are found correctly either way — only the rendering is wrong.Mechanism
Both rules end up claiming the same trailing shape node as their own morph.
mrThirdproduces no new morph nodes, soSynthesisAffixProcessAllomorphRuleSpec.ApplyRhs's fallback ("There are no new output morphs in a truncation rule...") marksoutput.Shape.Last.mrB2A's own morph is separately re-established a few lines later, by the loop that walksmatch.Input.Morphsand re-marks every allomorph the current rule wraps — for that same node.Word.MarkMorphreparents a node's self-annotation to whichever caller claims it last. The fallback ran before that loop, somrB2Awon the node whilemrThird's morph had already been inserted ahead of it in the annotation list.AllomorphsInMorphOrderwalks that order and reads each annotation's ownAllomorph, which is fixed at construction regardless of whether the annotation still has children — so the emptymrThirdmorph rendered wheremrB2Abelonged, andmrB2A's ID vanished.The change
The fallback now runs after the re-marking loop rather than before it. It claims the same node either way and finds the same analyses; only the order of the two colliding
MarkMorphcalls changes, which is what the insertion-order tie-break needs.Tests
Three cases: the dropped-ID case, plus controls where the zero-width rule wraps the bare root (no shared boundary, no drop) and where the zero-width rule cannot apply at all. The dropped-ID test fails without this change and passes with it; the controls pass either way. Run five times, same result each time.
Full HermitCrab suite: 575 passed, 1 skipped, 0 failed.
Separately:
BidirList's level randomization is observable, and it is not fixed hereWhile tracing this I found that the same process, same code, same input can produce different signatures across runs.
BidirList<TNode>(src/SIL.Machine/DataStructures/BidirList.cs:15) holdsprivate readonly Random _rand = new Random();, unseeded, used to assign skip-list levels. A skip list's level assignment should never affect level-0 traversal order of tied nodes, only lookup speed. Empirically it does here: pinning that field tonew Random(42), with no other change, gave the identical correct signature 12 times out of 12, where the unseeded field was correct only about half the time across 20 runs, across three distinct outputs.BidirListis the base ofAnnotationList,ShapeandOrderedBidirList, so this is engine-wide and predates this change. I have not attempted it: a wrong fix there risks correctness or performance across every construct that relies on annotation ordering. Reporting it with the reproduction instead, and happy to open a separate issue.I also have a conformance fixture for the dropped-ID bug, written against the correct signatures. I am deliberately not including it here, because that fixture exercises the same tied-node collision and is therefore unreliable until the
BidirListissue is resolved — a red CI you cannot fix is worse than no fixture. Glad to add it once that lands.Found via the Rust port (PanGloss), which renders this case correctly; the discrepancy is what surfaced it.
🤖 Generated with Claude Code
https://claude.ai/code/session_016AzbiMyms1YL83LbdRNJwx
This change is