test: Add 34 base 2023-09 conformance fixtures, strengthen 6 - #161
Open
leongdl wants to merge 1 commit into
Open
test: Add 34 base 2023-09 conformance fixtures, strengthen 6#161leongdl wants to merge 1 commit into
leongdl wants to merge 1 commit into
Conversation
34 new fixtures plus 6 strengthened: INT/FLOAT numeric boundary accepts, inclusive-boundary accepts, onExit-runs, cancelation defaults, SimpleAction argv desugaring, reference-grammar negatives, cross-type field misuse, EXPR/FEATURE_BUNDLE_1 gating from the base side, post-resolution name length, and off-diagonal forbidden lines added to six associative combination fixtures. Suite: 655 -> 687 passing. Spec-correct fixtures that FAIL against current implementations (the int64 acceptance family, type-name case gate, merge widening) are split to conformance-base-expected-failures. Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
34 new base 2023-09 fixtures plus 6 strengthened, from a systematic coverage audit of the base spec. Full suite goes 655 → 687 passing, 0 failing (plus 2 FEATURE_BUNDLE_1 fixtures, 57 → 57 incl. the new ones).
Areas covered
default/minValue/maxValue/allowedValuesaccepted at 2^63−1 and −2^63; FLOAT accepted at 1e308, rejected at 1e309/.inf/.nan; supplied INT value at 2^63−1 runs, at 2^63 rejected at job creation.>for>=passed the old suite.onExitruns: verified there was previously no base job fixture containing anonExitaction at all; marker-based fixture added.TERMINATE+notifyPeriodInSecondsrejected;cancelationwith nomoderejected; format-stringmoderejected without FEATURE_BUNDLE_1.{{MyParam}}and four malformed dotted segments rejected — the inner-segment parser was previously untested.minValueon STRING,minLengthon INT, unknown key in a parameter definition.letrejected in base step template and step script;bash:SimpleAction key rejected without FEATURE_BUNDLE_1.3.4.3associative-combination job fixtures gained off-diagonalforbidden:lines. Before this, an implementation computing a full cross product where the spec requires a zip passed all six.Deliberately excluded
Cases blocked on spec defects recorded in the audit (descending-range semantics where the spec's own example table contradicts both implementations and four existing fixtures;
<posintstring>gating;maxLength: 0) — those need spec decisions first, not fixtures.Companion PR
Spec-correct fixtures that FAIL against current implementations (the int64 template-validation acceptance family, the base type-name case gate, §1.2.1 merge widening) are in the expected-failures PR (branch
conformance-base-expected-failures).