diff --git a/README.md b/README.md index d02d4e1..851bfdf 100644 --- a/README.md +++ b/README.md @@ -67,10 +67,10 @@ it at run time. One flag per marked *name*, so two methods of a marked name shar ## It is also a check -Julia already checks half of the surface question: `Docs.undocumented_names` has been public API -in Base since 1.11, and `Aqua.test_undocumented_names` ships it as a test — every public name must -carry a docstring. What neither can express is the third answer: *this name is public, it has no -docstring, and that is deliberate, and here is why*. +**A mark is not a substitute for a docstring.** Every public name should have one; the mark is a +second, independent account — the docstring says what the name does, the mark says whether its +shape is settled. A marked name with no prose fails the check exactly as an unmarked one does, +and there is no switch that turns that off. A marker nobody compares against anything is a claim. Put this in `runtests.jl` and it becomes a contract: @@ -81,9 +81,8 @@ using MyPackage, ExperimentalAPI, Test ExperimentalAPI.test_surface(MyPackage) ``` -It fails, naming the symbol, when a public name has neither a docstring nor a mark — and also -when a mark points at a name that was never made public, which is the module contradicting -itself. +It fails, naming the symbol, when a public name has no docstring — and also when a mark points at +a name that was never made public, which is the module contradicting itself. Adopting it on a package that already has a backlog: @@ -116,14 +115,20 @@ Attached to a definition, or as a list of names defined elsewhere: ```julia # the definition site -@experimental "signature will be wrapped once the write-back refactor settles" \ -function ingest(config; doc, kwargs...) - # ... -end +@experimental( + "signature will be wrapped once the write-back refactor settles", + function ingest(config; doc, kwargs...) + # ... + end, +) # names an included file defines -@experimental "reads Test's internal result tree; not dogfooded in CI" \ - render_test_report dump_test_report load_test_dump +@experimental( + "reads Test's internal result tree; not dogfooded in CI", + render_test_report, + dump_test_report, + load_test_dump, +) # with the issue where the shape is being decided @experimental("export format is a guess until someone consumes it", @@ -199,7 +204,7 @@ public, which is the module contradicting itself and needs no reference to be wr | who may call a name | `export`, `public` (1.11) | orthogonal — a name can be public and unfinished | | a name on its way out | `@deprecate` | opposite direction | | type stability | DispatchDoctor | unrelated | -| every public name has a docstring | `Docs.undocumented_names` (Base 1.11+), `Aqua.test_undocumented_names` | the same check, plus a third answer | +| every public name has a docstring | `Docs.undocumented_names` (Base 1.11+), `Aqua.test_undocumented_names` | the same requirement, not a looser one — plus `foreign` and `dangling` | | generating documentation | Documenter | only ever checks whether prose exists | | run-time behaviour | — | one short-circuit read in the body; see the table above | diff --git a/docs/src/adopting.md b/docs/src/adopting.md index f864886..88a1268 100644 --- a/docs/src/adopting.md +++ b/docs/src/adopting.md @@ -15,7 +15,7 @@ julia> using MyPackage, ExperimentalAPI julia> audit(MyPackage) ``` -Two numbers matter. `unaccounted` is the backlog. `dangling` should be zero on day one, because +Two numbers matter. `undocumented` is the backlog. `dangling` should be zero on day one, because there are no marks yet. ## Turn the test on with the backlog listed @@ -44,19 +44,25 @@ means the only way the list changes is deliberately, and only downwards. ## Then, for each name, one decision -For every name in the list, exactly one of two things is true, and both are cheap: +For every name in the list, **write the docstring** — that part is not optional, and it is what +deletes the `skip` entry. Then ask a second question: is the shape settled? -- **it is settled** → write the docstring, delete the `skip` entry; -- **it is not settled** → say so, with the reason, and delete the `skip` entry: +- **settled** → nothing more to do; +- **not settled** → say so, with the reason, *in addition* to the docstring: ```julia -@experimental "reads Test's internal result tree; not dogfooded in CI" \ - render_test_report dump_test_report load_test_dump +@experimental( + "reads Test's internal result tree; not dogfooded in CI", + render_test_report, + dump_test_report, + load_test_dump, +) ``` -The second is not a lesser outcome. A name that is genuinely unfinished is *better* described by -a mark with a reason than by a docstring that has to pretend the shape is final — and it buys the -right to change it, which the docstring does not. +The mark is not a substitute for the prose and never was: the docstring says what the name does, +which is owed either way. What the mark adds is the thing a docstring cannot carry — a machine +can read it, so a run that goes through the name says so, and dropping the name later is not a +breaking change. ## What "declare it" is worth later @@ -71,4 +77,5 @@ Once the marks exist, three things follow that did not before: ## A note on where this ends The goal is not zero experimental names. A package with none is either finished or lying. The -goal is zero **unaccounted** names — no public name about which nothing at all has been said. +goal is zero **undocumented** names — every public name described, whether or not its shape has +settled. diff --git a/docs/src/checking.md b/docs/src/checking.md index a9662f2..c9c7a40 100644 --- a/docs/src/checking.md +++ b/docs/src/checking.md @@ -14,17 +14,21 @@ This is the part the rest of the package exists for. | bucket | meaning | |---|---| | `documented` | has a docstring | -| `declared` | has an [`@experimental`](@ref) mark (may also be documented) | +| `declared` | has an [`@experimental`](@ref) mark — an *additional* account, never a substitute | | `foreign` | public here, but bound in another package — not this module's to account for | -| `unaccounted` | **neither documented nor declared** — the finding | +| `undocumented` | **no docstring, marked or not** — the finding [`test_surface`](@ref) asserts empty | +| `unaccounted` | neither account at all — a subset of `undocumented`, and the worst case | | `dangling` | marked, but not public — the module contradicting itself | +A mark says the shape is unsettled. That is never a reason to say nothing about what the name +does, so a marked name with no prose is a finding exactly as an unmarked one is. + ```julia julia> audit(Archeion) Public surface of Archeion — 36 names documented 33 experimental 0 - unaccounted 3 ← neither documented nor @experimental + undocumented 3 ← no docstring FTPSTransport, pull_file, push_dir ``` @@ -86,10 +90,10 @@ The two checks are complementary and neither subsumes the other: makedocs(; modules = [MyPackage], checkdocs = :public) # every public name has a docstring ``` -Documenter's `checkdocs = :public` fails a build when a public name has no docstring — but it has -no notion of "declared unfinished instead", so a package that wants that third option needs both: -`checkdocs` for the names that must be documented, `test_surface` for the rule that lets a mark -stand in for a docstring. +Documenter's `checkdocs = :public` fails a build when a public name has no docstring, and +[`test_surface`](@ref) requires the same thing — the two agree, and running both is not a +contradiction. What `test_surface` adds is `foreign` (a re-exported name whose prose is somebody +else's job) and `dangling` (a mark on a name that was never made public), neither of which +Documenter has a notion of. -If you would rather run only one, run `test_surface`: it accepts everything `checkdocs = :public` -accepts, plus marks. +Run both. Neither is a looser version of the other. diff --git a/docs/src/declaring.md b/docs/src/declaring.md index 415c649..2d84ac4 100644 --- a/docs/src/declaring.md +++ b/docs/src/declaring.md @@ -11,14 +11,20 @@ definition** or **a list of names**. ```julia # attached to the definition: the mark and the thing it describes cannot drift apart -@experimental "signature will be wrapped once the write-back refactor settles" \ -function ingest(config; doc, kwargs...) - # ... -end +@experimental( + "signature will be wrapped once the write-back refactor settles", + function ingest(config; doc, kwargs...) + # ... + end, +) # a list, for names an included file defines -@experimental "reads Test's internal result tree; not dogfooded in CI" \ - render_test_report dump_test_report load_test_dump +@experimental( + "reads Test's internal result tree; not dogfooded in CI", + render_test_report, + dump_test_report, + load_test_dump, +) ``` The attached form is preferred where it fits, for the same reason a docstring goes above its @@ -89,10 +95,12 @@ questions, and nothing forces a choice between them: Ingest `doc` into the registry described by `config`. """ -@experimental "signature will be wrapped once the write-back refactor settles" \ -function ingest(config; doc, kwargs...) - # ... -end +@experimental( + "signature will be wrapped once the write-back refactor settles", + function ingest(config; doc, kwargs...) + # ... + end, +) ``` ## Marking is not making public diff --git a/docs/src/index.md b/docs/src/index.md index 700b6c9..4ba360f 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -95,7 +95,7 @@ was never made public, which is the module contradicting itself. | who may call a name | `export`, `public` (1.11) | orthogonal | | a name on its way out | `@deprecate` | opposite direction | | type stability | DispatchDoctor | unrelated | -| every public name has a docstring | `Docs.undocumented_names`, `Aqua.test_undocumented_names` | the same check, plus a third answer | +| every public name has a docstring | `Docs.undocumented_names`, `Aqua.test_undocumented_names` | the same requirement, not a looser one — plus `foreign` and `dangling` | | generating documentation | Documenter | only ever checks whether prose exists | | run-time behaviour | — | one short-circuit read in the body — see the table above | | how often a path ran | `Profile`, `@time` | not answered: the default layer knows *whether*, never how often | diff --git a/ext/ExperimentalAPITestExt.jl b/ext/ExperimentalAPITestExt.jl index 5ccd301..07fc4d4 100644 --- a/ext/ExperimentalAPITestExt.jl +++ b/ext/ExperimentalAPITestExt.jl @@ -16,21 +16,25 @@ function ExperimentalAPI.test_surface( # of findings, so on a clean module all of it collapses to nothing and the testset would # report `0 tests passed` — a green indistinguishable from the extension having failed to # load, or from `m` having no public names at all. These make the pass mean something. - @testset "nothing unaccounted for" begin - @test isempty(setdiff(a.unaccounted, skip)) + @testset "every public name has a docstring" begin + @test isempty(setdiff(a.undocumented, skip)) @test isempty(a.dangling) end # One testset per name, so a failing CI log names the symbol in its header rather than # printing a set difference the reader has to diff by eye. - @testset "$n is documented or @experimental" for n in setdiff(a.unaccounted, skip) - @test isdocumented(m, n) || isexperimental(m, n) + @testset "$n has a docstring" for n in setdiff(a.undocumented, skip) + @test isdocumented(m, n) end - # The allowlist can only shrink. An entry that has since been documented, declared or - # deleted fails here — otherwise adopting this on a package with a backlog would leave a - # list that silently stops describing anything, and a green suite would mean less every - # release. + # The allowlist can only shrink. An entry that has since been documented or deleted fails + # here — otherwise adopting this on a package with a backlog would leave a list that + # silently stops describing anything, and a green suite would mean less every release. + # + # `skip` is the only way to pass an undocumented name, and it is per-name and visible. + # There is deliberately no switch that turns the docstring requirement off wholesale: a + # mark records that a shape is unsettled, and that is never a reason to say nothing about + # what the name does. @testset "skip entry $n is still needed" for n in skip - @test n in a.unaccounted + @test n in a.undocumented end # Needs no oracle: the module marked a name it never made public. @testset "@experimental $n is public" for n in a.dangling diff --git a/src/ExperimentalAPI.jl b/src/ExperimentalAPI.jl index 6e708bf..c989380 100644 --- a/src/ExperimentalAPI.jl +++ b/src/ExperimentalAPI.jl @@ -20,8 +20,8 @@ The mark is three things, and the first is the reason to have it: * an **observation** — [`entered`](@ref) reports the marked definitions this run actually went through, and a summary says so at process exit whether or not anyone asked; * a **declaration** — the reason travels with the name, in the source, where the author is; - * a **check** — [`audit`](@ref) reports every public name that is neither documented nor - declared, so "document it or admit it is unfinished" becomes a test that fails. + * a **check** — [`audit`](@ref) reports every public name with no docstring, marked or not, so + "every public name is described" becomes a test that fails. # What it costs @@ -41,7 +41,7 @@ See [`@experimental`](@ref) for the form-by-form table. |---|---| | did this run go through unvalidated code? | `entered()` — and the summary at exit says so anyway | | what is unfinished here? | `experimental(M)` | -| what does this module owe nobody an explanation for? | `audit(M).unaccounted` — should be empty | +| which public names are undescribed? | `audit(M).undocumented` — should be empty | | is dropping this name breaking? | `compare(old_snapshot, M)` — see [`isbreaking`](@ref) | # What this is not @@ -57,10 +57,11 @@ See [`@experimental`](@ref) for the form-by-form table. # Scope of the check -[`audit`](@ref) compares `names(M)` — exported *and* `public` names — against two accounts: -a docstring, or a mark. It sees **names**, not signatures and not prose quality. A public name -with a docstring reading "TODO" is accounted for; a settled name whose method signature changed -under it is invisible here. See [`compare`](@ref) for the same limit on the release side. +[`audit`](@ref) compares `names(M)` — exported *and* `public` names — against two independent +accounts: a docstring, and a mark. They are not alternatives; the docstring is owed either way. +It sees **names**, not signatures and not prose quality. A public name with a docstring reading +"TODO" is accounted for; a settled name whose method signature changed under it is invisible here. +See [`compare`](@ref) for the same limit on the release side. """ module ExperimentalAPI @@ -84,8 +85,13 @@ include("release.jl") # a snapshot of the covenant, and what a diff of two of """ test_surface(m::Module; skip = Symbol[], outputlevel::Int = 0) -> Audit -Assert, as a `@testset`, that every public name of `m` is either documented or declared -[`@experimental`](@ref) — and that every mark applies to a name that is actually public. +Assert, as a `@testset`, that every public name of `m` has a **docstring** — and that every mark +applies to a name that is actually public. + +A mark is not an alternative to prose. `@experimental` records that a shape is unsettled, which is +never a reason to say nothing about what the name does, so a marked-but-undocumented name fails +this test exactly as an unmarked one does. There is no switch to turn that off; `skip` is the only +escape, and it is per-name, visible, and can only shrink. Available once `Test` is loaded (it lives in a package extension, so `ExperimentalAPI` itself never pulls `Test` into a runtime dependency). Put it in `runtests.jl`: @@ -96,9 +102,9 @@ using MyPackage, ExperimentalAPI, Test ExperimentalAPI.test_surface(MyPackage) ``` -`skip` is for adopting this on a package that already has a backlog: the listed names are -allowed to be unaccounted for. **A stale entry fails the test** — a name in `skip` that has since -been documented, declared, or removed is reported, so the list can only shrink. +`skip` is for adopting this on a package that already has a backlog: the listed names are allowed +to have no docstring. **A stale entry fails the test** — a name in `skip` that has since been +documented or removed is reported, so the list can only shrink. Returns the [`Audit`](@ref) on the normal return path whether the testset passed or not. `outputlevel ≥ 1` also prints it. diff --git a/src/audit.jl b/src/audit.jl index a74a291..0b12270 100644 --- a/src/audit.jl +++ b/src/audit.jl @@ -94,6 +94,7 @@ struct Audit foreign::Vector{Symbol} documented::Vector{Symbol} declared::Vector{Symbol} + undocumented::Vector{Symbol} unaccounted::Vector{Symbol} dangling::Vector{Symbol} end @@ -101,19 +102,27 @@ end """ audit(m::Module) -> Audit -List the public names of `m` that are **neither documented nor declared experimental**. +Report the public names of `m` that are missing prose, a mark, or both. ```julia -julia> audit(Pinax).unaccounted +julia> audit(Pinax).undocumented 15-element Vector{Symbol}: :completeness_overview :dump_test_report ⋮ ``` -Two accounts are accepted, and the choice between them is the author's: write the docstring, or -say [`@experimental`](@ref) and why. What is not accepted is saying nothing — which is the state -a public name is in by default, and the state no reader can distinguish from a settled one. +**A mark is not a substitute for a docstring.** The two are independent accounts of a name and +both are owed: the docstring says what it does, the mark says whether the shape is settled. A +public name that carries a mark and no prose appears in `undocumented` exactly as one with +neither does, and [`test_surface`](@ref) fails on it. + +| field | what it holds | +|---|---| +| `documented` | has a docstring | +| `declared` | has a mark — overlaps `documented`, and is not an alternative to it | +| `undocumented` | no docstring, mark or not. This is the one [`test_surface`](@ref) asserts empty | +| `unaccounted` | neither account — a subset of `undocumented`, and the worst case | Also reports `dangling`: marks on names that are not public. That check needs no reference implementation to be right, because the module is disagreeing with itself. @@ -135,21 +144,25 @@ function audit(m::Module) foreign = Symbol[] documented = Symbol[] declared = Symbol[] + undocumented = Symbol[] unaccounted = Symbol[] for n in surf if !_is_own(m, n) push!(foreign, n) - elseif isdocumented(m, n) + continue + end + n in marked && push!(declared, n) + if isdocumented(m, n) push!(documented, n) - n in marked && push!(declared, n) - elseif n in marked - push!(declared, n) else - push!(unaccounted, n) + push!(undocumented, n) + n in marked || push!(unaccounted, n) end end dangling = sort!(collect(setdiff(marked, surf))) - return Audit(m, surf, foreign, documented, declared, unaccounted, dangling) + return Audit( + m, surf, foreign, documented, declared, undocumented, unaccounted, dangling + ) end function Base.show(io::IO, a::Audit) @@ -160,8 +173,8 @@ function Base.show(io::IO, a::Audit) ": ", length(a.surface), " public, ", - length(a.unaccounted), - " unaccounted)", + length(a.undocumented), + " undocumented)", ) end @@ -169,6 +182,8 @@ function Base.show(io::IO, ::MIME"text/plain", a::Audit) println(io, "Public surface of ", a.mod, " — ", length(a.surface), " names") println(io, " documented ", lpad(length(a.documented), 4)) println(io, " experimental ", lpad(length(a.declared), 4)) + isempty(a.undocumented) || + println(io, " undocumented ", lpad(length(a.undocumented), 4)) isempty(a.foreign) || println( io, " foreign ", diff --git a/src/mark.jl b/src/mark.jl index 653dcaf..7b38831 100644 --- a/src/mark.jl +++ b/src/mark.jl @@ -100,14 +100,20 @@ list of names** already defined elsewhere: ```julia # attached to the definition -@experimental "signature will be wrapped once the write-back refactor settles" \ -function ingest(config; doc, kwargs...) - # ... -end +@experimental( + "signature will be wrapped once the write-back refactor settles", + function ingest(config; doc, kwargs...) + # ... + end, +) # declared for names defined in an included file -@experimental "reads Test's internal result tree; not dogfooded in CI" \ - render_test_report dump_test_report load_test_dump +@experimental( + "reads Test's internal result tree; not dogfooded in CI", + render_test_report, + dump_test_report, + load_test_dump, +) ``` Optional `since=` and `tracking=` come between the reason and the subject. `tracking` is what diff --git a/test/spec/README.md b/test/spec/README.md index ddc60c5..82e2dd7 100644 --- a/test/spec/README.md +++ b/test/spec/README.md @@ -45,7 +45,7 @@ that is entirely `@test_broken` is a claim written down, not a check being run. |---|---|---|---|---| | `test_spec_declare.jl` | 11 | 7 | 4 | what can carry a mark: function, method, struct, const, module, macro, extension | | `test_spec_dispatch.jl` | 14 | 4 | 10 | one call site, several methods, only some marked — the branch | -| `test_spec_docstring.jl` | 9 | 6 | 3 | a mark and a docstring are different accounts and must coexist | +| `test_spec_docstring.jl` | 9 | 8 | 1 | a mark and a docstring are different accounts and must coexist | | `test_spec_foreign.jl` | 14 | 5 | 9 | marking a method on somebody else's generic — the `QAtlas.fetch` case | | `test_spec_forms.jl` | 26 | 15 | 11 | the definition forms a real package hits on its second afternoon | | `test_spec_integration.jl` | 17 | 1 | 16 | where the mark has to surface: docs, Aqua, releases, provenance, CI | @@ -53,7 +53,7 @@ that is entirely `@test_broken` is a claim written down, not a check being run. | `test_spec_profile.jl` | 40 | 12 | 28 | what a real run went through, how often, and how much of it | | `test_spec_propagate.jl` | 20 | 2 | 18 | a caller that never names a marked thing still depends on it | | `test_spec_verify.jl` | 8 | 2 | 6 | how well is a marked thing exercised by the tests | -| **10 files** | **174** | **61** | **113** | | +| **10 files** | **174** | **63** | **111** | | The table is generated and pinned by `test/test_spec_table.jl`, which fails if it goes stale — diff --git a/test/spec/test_spec_docstring.jl b/test/spec/test_spec_docstring.jl index 0c1e992..aae9f93 100644 --- a/test/spec/test_spec_docstring.jl +++ b/test/spec/test_spec_docstring.jl @@ -81,16 +81,21 @@ end end @testset "a mark is not an excuse for missing prose" begin - # Scope: a marked name with no docstring is still undocumented. Today `declared` absorbs it. + # Scope: a marked name with no docstring is still undocumented. a = audit(Both) - @test_broken :marked_only in a.undocumented - @test_broken hasproperty(a, :undocumented) + @test hasproperty(a, :undocumented) + @test :marked_only in a.undocumented + # …and it is not in `unaccounted`, which stays the stricter bucket: neither account at all. + @test :marked_only ∉ a.unaccounted end -@testset "the check can require a docstring regardless of the mark" begin - # Scope: a package adopting both this and Aqua must not have to choose between them. - @test_broken ExperimentalAPI.test_surface(Both; require_docstring=true) isa - ExperimentalAPI.Audit +@testset "requiring a docstring is the default, not a knob" begin + # This was specified as `test_surface(m; require_docstring=true)`. The requirement changed + # after review: a switch that turns the docstring rule off wholesale is the looser practice + # the mark must not encourage. `skip` is the only escape — per-name, visible, and it can + # only shrink. So the assertion is that no such switch exists. + @test :require_docstring ∉ Base.kwarg_decl(only(methods(ExperimentalAPI.test_surface))) + @test :skip in Base.kwarg_decl(only(methods(ExperimentalAPI.test_surface))) end @testset "the reason is reachable from the rendered documentation" begin diff --git a/test/test_ext.jl b/test/test_ext.jl index 37f17c2..5fc0035 100644 --- a/test/test_ext.jl +++ b/test/test_ext.jl @@ -38,12 +38,16 @@ end failures(ts::Recorder) = [p for (p, r) in leaves(ts) if !(r isa Test.Pass)] failed_names(ts::Recorder) = Set(last(p) for p in failures(ts)) +# `marked` carries BOTH accounts, because a mark is no longer an alternative to prose. The +# fixture had only the mark, and it stopped being a clean module the moment that changed — +# which is the behaviour under test, so the fixture moved rather than the rule. module Clean using ExperimentalAPI export documented public marked "documented" documented(x) = x +"Marked, and still documented." @experimental "not settled" marked(x) = x end @@ -80,10 +84,10 @@ end test_surface(Dirty) end names = failed_names(ts) - @test "silent_one is documented or @experimental" in names - @test "silent_two is documented or @experimental" in names + @test "silent_one has a docstring" in names + @test "silent_two has a docstring" in names @test "@experimental not_public is public" in names - @test "nothing unaccounted for" in names # the summary pair fails too + @test "every public name has a docstring" in names # the summary pair fails too # The documented name is not among the complaints. @test !any(occursin("documented is", n) for n in names) @test length(names) == 4 @@ -94,7 +98,7 @@ end test_surface(Dirty; skip=[:silent_one, :silent_two]) end @test failed_names(ts) == - Set(["@experimental not_public is public", "nothing unaccounted for"]) + Set(["@experimental not_public is public", "every public name has a docstring"]) end @testset "a stale skip entry fails" begin @@ -119,7 +123,7 @@ end for m in (Clean, Dirty) local got = nothing ts = Test.@testset Recorder "r" begin - got = test_surface(m; skip=collect(audit(m).unaccounted)) + got = test_surface(m; skip=collect(audit(m).undocumented)) end @test got isa ExperimentalAPI.Audit @test got.mod === m diff --git a/test/test_readme.jl b/test/test_readme.jl index c509132..7e749a7 100644 --- a/test/test_readme.jl +++ b/test/test_readme.jl @@ -56,3 +56,78 @@ end end @test e !== nothing end + +# ── every julia code block in the docs ─────────────────────────────────────────────────────── +# +# Scope: a lint, not an execution. Most blocks reference a `MyPackage` that does not exist, so +# they cannot be run — but the defect that shipped here was not a runtime one. A trailing `\` +# used as a line continuation PARSES (Julia reads it as left-division) and fails only when the +# macro is expanded, so neither a parse check nor a `jldoctest` would have caught it. The check +# has to be for the character. + +"Every ```julia fence in `path`, as (line number, text) pairs." +function julia_blocks(path) + out = Tuple{Int,String}[] + inblock = false + start = 0 + buf = String[] + for (i, line) in enumerate(eachline(path)) + l = rstrip(line, ['\r']) + if inblock && startswith(strip(l), "```") + push!(out, (start, join(buf, "\n"))) + inblock = false + empty!(buf) + elseif inblock + push!(buf, l) + elseif strip(l) in ("```julia", "```jldoctest") + inblock = true + start = i + end + end + return out +end + +const _DOC_SOURCES = vcat( + [joinpath(@__DIR__, "..", "README.md")], + sort(readdir(joinpath(@__DIR__, "..", "docs", "src"); join=true)), + sort(readdir(joinpath(@__DIR__, "..", "src"); join=true)), +) + +@testset "no code block uses a trailing backslash as a line continuation" begin + # Julia has no line continuation. `@experimental "…" \` + a definition on the next line + # reaches the macro as `\(reason, def)` and dies with "nothing to mark"; the name-list form + # dies in `adjoint`. Both shipped, in eleven places, and survived a review round. + offenders = String[] + for path in _DOC_SOURCES + endswith(path, ".md") || endswith(path, ".jl") || continue + for (start, block) in julia_blocks(path) + for (k, line) in enumerate(split(block, "\n")) + endswith(rstrip(line), "\\") && + push!(offenders, "$(basename(path)):$(start + k)") + end + end + end + @test offenders == String[] +end + +@testset "…and the check can see one" begin + # Control: the scanner only looks inside fences, so it has to be shown to fire on a real one. + path = joinpath(mktempdir(), "sample.md") + write( + path, + join( + [ + "prose ending in a backslash \\", # outside a fence: not a finding + "```julia", + "@experimental \"why\" \\", + "f(x) = x", + "```", + ], + "\n", + ), + ) + blocks = julia_blocks(path) + @test length(blocks) == 1 + lines = split(blocks[1][2], "\n") + @test count(l -> endswith(rstrip(l), "\\"), lines) == 1 +end