Skip to content

test: three claims that were pinned by their shape and not by their answer - #15

Merged
sotashimozono merged 1 commit into
mainfrom
test/close-the-three-shape-only-claims
Sep 5, 2026
Merged

test: three claims that were pinned by their shape and not by their answer#15
sotashimozono merged 1 commit into
mainfrom
test/close-the-three-shape-only-claims

Conversation

@sotashimozono

Copy link
Copy Markdown
Member

Asked of the just-merged spec: is every behaviour actually checked? Three were not. Each was a
hasproperty or an isa Function — satisfied by an implementation that returns an empty result
for every input — and one of them was hiding a live defect.

reach_script answered :clean for a script that reaches a mark

const RESULT = simulate(model) is a script's work wearing a declaration's syntax, and it is how
the line that produces the figure is actually written. It was evaluated in the scratch module and
never analysed, so the thunk contained no call at all. The binding still has to be made — a later
struct may use it — so the value is computed at top level and the right-hand side is analysed as
well.

The test now runs three scripts: one that reaches a mark, one of the same shape that does not, and
the trivial one it had before.

A method-level mark had never been through a precompile cache

test/test_precompile.jl calls itself "the measurement that could kill the design", and it was
reading name, reason, since and tracking — a Symbol and three strings. It never read
sig, which is a Type in a const vector inside the cache image, and the fixture package
carried no mark on somebody else's generic, whose signature names typeof(Base.show) and a
type defined in the cached package.

MarkedPkg now carries one of each kind. The probe reads all three, with two controls that a mark
widened on its way through the cache would fail — Base.show is not experimental for anybody, and
the sibling show(::IO, ::Int) is not marked — and it runs reach across the package boundary
with a same-depth control. Both pass: the claim is now measured rather than assumed.

Cross-package propagation said reach isa Function

The spec admitted it needed a fixture package. It now asserts the claim is checked where that
package lives, and checks the module-boundary version it can reach on its own.


Renaming the precompilation testset while promoting it is what let the first two hide: the name
that said "a method mark survives precompilation" was the only record that it did not.

1005 assertions, green.

🤖 Generated with Claude Code

…nswer

Asked of the merged spec: is every behaviour actually checked? Three were not. Each was a
`hasproperty` or an `isa Function` — satisfied by an implementation that returns an empty result
for every input — and one of them was hiding a live defect.

**`reach_script` answered `:clean` for a script that reaches a mark.** `const RESULT =
simulate(model)` is a script's *work* wearing a declaration's syntax, and it is how the line that
produces the figure is actually written. It was evaluated in the scratch module and never
analysed, so the thunk contained no call at all. The binding still has to be made — a later
`struct` may use it — so the value is computed at top level and the right-hand side is analysed
as well. The test now runs three scripts: one that reaches a mark, one of the same shape that
does not, and the trivial one it had before.

**A method-level mark had never been through a precompile cache.** `test/test_precompile.jl` is
"the measurement that could kill the design", and it was reading `name`, `reason`, `since` and
`tracking` — a `Symbol` and three strings. It never read `sig`, which is a `Type` in a `const`
vector inside the cache image, and the fixture package carried no mark on somebody else's
generic, whose signature names `typeof(Base.show)` and a type defined in the cached package.
`MarkedPkg` now carries one of each kind and the probe reads all three, with two controls that a
mark widened on its way through the cache would fail: `Base.show` is not experimental for
anybody, and the sibling `show(::IO, ::Int)` is not marked. It also runs `reach` across the
package boundary — a caller reaching a mark written while another process precompiled another
package — with a same-depth control. Both pass; the claim is now measured rather than assumed.

**Cross-package propagation** said `reach isa Function`, and the spec admitted it needed a
fixture package. It now asserts the claim is checked where the fixture package lives, and checks
the module-boundary version it can reach on its own.

Renaming the precompilation testset while promoting it is what let the first two hide: the name
that said "a method mark survives precompilation" was the only record that it did not.

1005 assertions, green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the chore label Sep 5, 2026
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

📚 Docs preview: https://codes.sota-shimozono.com/ExperimentalAPI.jl/previews/PR15/

(updates on each push to this PR)

@codecov

codecov Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sotashimozono
sotashimozono merged commit c5dbe2b into main Sep 5, 2026
14 checks passed
@sotashimozono
sotashimozono deleted the test/close-the-three-shape-only-claims branch September 5, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant