test: three claims that were pinned by their shape and not by their answer - #15
Merged
Merged
Conversation
…nswer Asked of the merged spec: is every behaviour actually checked? Three were not. Each was a `hasproperty` or an `isa Function` — satisfied by an implementation that returns an empty result for every input — and one of them was hiding a live defect. **`reach_script` answered `:clean` for a script that reaches a mark.** `const RESULT = simulate(model)` is a script's *work* wearing a declaration's syntax, and it is how the line that produces the figure is actually written. It was evaluated in the scratch module and never analysed, so the thunk contained no call at all. The binding still has to be made — a later `struct` may use it — so the value is computed at top level and the right-hand side is analysed as well. The test now runs three scripts: one that reaches a mark, one of the same shape that does not, and the trivial one it had before. **A method-level mark had never been through a precompile cache.** `test/test_precompile.jl` is "the measurement that could kill the design", and it was reading `name`, `reason`, `since` and `tracking` — a `Symbol` and three strings. It never read `sig`, which is a `Type` in a `const` vector inside the cache image, and the fixture package carried no mark on somebody else's generic, whose signature names `typeof(Base.show)` and a type defined in the cached package. `MarkedPkg` now carries one of each kind and the probe reads all three, with two controls that a mark widened on its way through the cache would fail: `Base.show` is not experimental for anybody, and the sibling `show(::IO, ::Int)` is not marked. It also runs `reach` across the package boundary — a caller reaching a mark written while another process precompiled another package — with a same-depth control. Both pass; the claim is now measured rather than assumed. **Cross-package propagation** said `reach isa Function`, and the spec admitted it needed a fixture package. It now asserts the claim is checked where the fixture package lives, and checks the module-boundary version it can reach on its own. Renaming the precompilation testset while promoting it is what let the first two hide: the name that said "a method mark survives precompilation" was the only record that it did not. 1005 assertions, green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📚 Docs preview: https://codes.sota-shimozono.com/ExperimentalAPI.jl/previews/PR15/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asked of the just-merged spec: is every behaviour actually checked? Three were not. Each was a
haspropertyor anisa Function— satisfied by an implementation that returns an empty resultfor every input — and one of them was hiding a live defect.
reach_scriptanswered:cleanfor a script that reaches a markconst RESULT = simulate(model)is a script's work wearing a declaration's syntax, and it is howthe line that produces the figure is actually written. It was evaluated in the scratch module and
never analysed, so the thunk contained no call at all. The binding still has to be made — a later
structmay use it — so the value is computed at top level and the right-hand side is analysed aswell.
The test now runs three scripts: one that reaches a mark, one of the same shape that does not, and
the trivial one it had before.
A method-level mark had never been through a precompile cache
test/test_precompile.jlcalls itself "the measurement that could kill the design", and it wasreading
name,reason,sinceandtracking— aSymboland three strings. It never readsig, which is aTypein aconstvector inside the cache image, and the fixture packagecarried no mark on somebody else's generic, whose signature names
typeof(Base.show)and atype defined in the cached package.
MarkedPkgnow carries one of each kind. The probe reads all three, with two controls that a markwidened on its way through the cache would fail —
Base.showis not experimental for anybody, andthe sibling
show(::IO, ::Int)is not marked — and it runsreachacross the package boundarywith a same-depth control. Both pass: the claim is now measured rather than assumed.
Cross-package propagation said
reach isa FunctionThe spec admitted it needed a fixture package. It now asserts the claim is checked where that
package lives, and checks the module-boundary version it can reach on its own.
Renaming the precompilation testset while promoting it is what let the first two hide: the name
that said "a method mark survives precompilation" was the only record that it did not.
1005 assertions, green.
🤖 Generated with Claude Code