feat: report which experimental definitions a run entered - #9
Merged
Conversation
The package could say a name was unfinished. It could not say that *this run* went through one — which is the question a docstring structurally cannot answer, because it is asked after the run, about the run, often by somebody who wrote neither. `@experimental` now emits one statement into the body of a definition it attaches to: a set-once flag. `entered()` reads them back, and a summary at process exit reports them whether or not anyone asked. A definition the run never entered is absent, not reported with a count of zero. The cost, measured over 10M calls of a numeric body on Julia 1.12.2, minimum of 7-9 trials: | emitted into the body | 1 thread | 8 threads | counts correctly? | | nothing | 1.00x | 1.00x | - | | the flag emitted here | 1.03x | 0.985x | yes | | counter, plain shared Ref | 1.03x | 3.76x | **no** | | counter, global atomic | 1.17x | 4.87x | yes | | @warn, guarded, fires once | 5.65x | - | yes | Two rows decided it. A flag written once and only read afterwards stops dirtying the cache line, which is why it is free at eight threads while every counting scheme is not — and the plain counter is wrong as well as slow, recording 95,406,048 of 160,000,000 calls. The guarded `@warn` costs 5.65x even though it fires once: what stops the definition inlining is the call being in the body at all. Hence a summary at exit rather than a warning at the call. Scope, verified form by form rather than asserted: `function`, `f(x) = …`, parametric and return-type-annotated signatures are observed. A name list, `struct`, `const` and assignment are declaration-only. `@generated` is refused outright, as any macro-produced definition is — the docstring first claimed it was declaration-only, which measurement corrected. Eight spec behaviours reported Unexpected Pass and were promoted; nothing else moved, which is what the case matrix was for. One assertion was inverted rather than promoted: the spec used to require that the expansion be byte-identical to the bare definition, and now requires exactly one more statement whose head is `||` — a short-circuit read, not a store. `@wrapping` adds one statement too and is a store, so the control can fire. Three false claims went with it. "Calls are untouched", "emits the definition unchanged plus one push! at load time" and "costs nothing at run time" were true before this commit and are not now; they appeared in the README, the module docstring, the macro docstring and `docs/src/index.md`. Also: the README's primary example ran nothing — it referenced a `Model` and a `correction` that do not exist. It is now a program that runs, and `test/test_readme.jl` executes it verbatim and checks the output the README quotes, so it cannot rot. Confirmed by breaking the README and watching the test report `UndefVarError: Model not defined`. 174 behaviours, 61 operating. Suite: 532 pass, 152 broken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
📚 Docs preview: https://codes.sota-shimozono.com/ExperimentalAPI.jl/previews/PR9/ (updates on each push to this PR) |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
`windows-latest` only: git checks tracked text out with CRLF there, so the fence in
`findfirst("```julia\n", md)` never matched and the reader threw "README.md has no ```julia
block".
Same defect as the generated-table comparison one commit earlier, in a file written after that
fix and without it. Verified by converting README.md to CRLF locally and watching all three
testsets pass, rather than by pushing and waiting for the runner.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The package could say a name was unfinished. It could not say that this run went through one — which is the question a docstring structurally cannot answer, because it is asked after the run, about the run, often by somebody who wrote neither.
On by default. Silent when nothing marked was entered — loading a package that has marks prints nothing. Carries the reason, not just the symbol.
entered()returns the same thing as data.What it costs, and why the design is what it is
10M calls of
sqrt(abs(sin(x)cos(x) + exp(-|x|/1e6))), Julia 1.12.2, minimum of 7–9 trials:Ref@warn, guarded so it fires once@warn maxlog=1Two rows decided it:
@warncosts 5.65× even though it fires once — what stops the definition inlining is the call being in the body at all, not the warning being printed. That is why the notice is a summary at exit rather than a warning at the call.Counting, call sites and paths stay opt-in and are not built.
Scope, measured form by form
function f(x) … end,f(x) = …,f(x::T) where {T} = …,f(x)::R = …struct,const, assignment@generated functionThe
@generatedrow is a correction: the docstring first said "declaration only", and running it showed the macro rejects it and points at the name-list form.What the case matrix did
Eight behaviours reported
Unexpected Passand were promoted. Nothing else moved — no collateral failures, which is the whole reason the spec was written first.One assertion was inverted rather than promoted. The spec used to require the expansion be identical to the bare definition; it now requires exactly one more statement whose head is
||— a short-circuit read, not a store.WrapControl.@wrappingadds one statement too and it is a store, so the control can fire.Three claims that stopped being true
"Calls are untouched", "emits the definition unchanged plus one
push!at load time", and "costs nothing at run time" were accurate before this change and are not now. They appeared in the README, the module docstring, the macro docstring anddocs/src/index.md; all four are corrected.The README example now runs
It referenced a
Modeland acorrectionthat do not exist. It is now a program that runs, andtest/test_readme.jlexecutes the first ```julia block verbatim and checks the output the README quotes — so it cannot rot. Confirmed by breaking the README and watching the test reportUndefVarError:Model` not defined`.New docs page Observing, and
entered/Entry/marked_modules/detecting/summary_textare public and documented.174 behaviours, 61 operating (was 54). Suite: 532 pass, 152 broken, 0 failing.
🤖 Generated with Claude Code