diff --git a/CHANGELOG.md b/CHANGELOG.md index d0e2c00..4466700 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,14 +7,66 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -Accuracy fixes from the 2026-08-10 library audit: three numeric-claim -false-positive classes stopped, two silent false-negative classes closed -(short count labels, single-span/fenced CLI flags), markdown link titles -handled, and the semantic layer now judges against real source passages -instead of the content itself. Every fix carries a regression corpus case. +## [0.3.0] - 2026-08-11 + +**Beta.** The deterministic core is stable and the public API is now covered +by a compatibility promise (see the README status section). Two releases' +worth of accuracy work lands here: the 2026-08-10 audit fixes plus a +beta-review pass that closed two more silent false-negative classes in the +fence extractor and two link false positives. + +### Changed + +- **`Development Status :: 4 - Beta`** (was `2 - Pre-Alpha`). `verify`, + `VerifyContext`, `VerifyResult`, `Finding`, `FindingKind`, + `raise_if_failed` and the `Judge` protocol will not change shape without a + deprecation in a minor release. + +### Added + +- **`py.typed`.** The package is fully annotated but shipped no PEP 561 + marker, so type checkers ignored it in downstream projects. +- **Packaging guards.** `__version__` and the `pyproject.toml` version are + pinned equal by a test — the release flow bumps both by hand, and drift + would ship a wheel whose metadata disagrees with the runtime value. +- **README:** a per-checker table of what each checker settles and which + truth source it needs, plus an explicit known-limitations list. ### Fixed +- **Indented code fences are no longer invisible.** A fence nested under a + list item — how LLMs routinely write install steps — matched nothing, so + every import and shell flag inside one passed unchecked. The extractor is + now a line scanner: it accepts a fence at any indentation and strips that + indent from the body (uniformly indented code otherwise fails `ast.parse`, + which was the silent skip). This was the largest remaining hole. +- **Tilde fences (`~~~`) are extracted.** CommonMark-legal and previously + unrecognized — another silent pass. +- **Percent-encoded link targets resolve.** A link to a file whose name + contains a space is written `docs/my%20file.md`; that was checked + literally and flagged a file that exists. The raw form is still tried + first, so a file genuinely named `a%20b.md` resolves, and a decoded + target that escapes `project_root` is still caught. +- **Balanced parentheses in link targets are not truncated.** + `[doc](docs/a(1).md)` was cut to `docs/a(1)` and flagged. +- **Link syntax shown as an example is no longer checked as a link.** A doc + documenting its own conventions ("write it as `` `[text](target.md)` ``") + was flagged for a target it never claimed existed — no renderer resolves a + link inside a code span or fence. Links are now read from prose only — + spans of any delimiter width, so a doubled delimiter around a span that + itself contains backticks is masked whole rather than at its edges. Line + numbers are unaffected, and a real link sharing a line with an example is + still checked. Found by running verify over its own README. +- The fence scanner also enforces the rules the old regex ignored: an + unclosed fence is not a fence (its body was the rest of the document), a + closing run must match the opening character and length, and an inline + ``` ```code``` ``` span no longer opens one. +- Stripping fences before the flag scan now blanks the lines in place + rather than deleting them, so prose either side of a code block never + becomes adjacent when the checker looks backwards for a command name. + +### Fixed (2026-08-10 audit) + - **Comma-grouped numbers are one claim.** "1,234 tests" previously extracted the "234" fragment and flagged an error-severity count mismatch against `tests=1234`; `1,234` is now extracted as the single @@ -55,7 +107,7 @@ instead of the content itself. Every fix carries a regression corpus case. judge was provided"; the message now names the failing object and the protocol mismatch. -### Changed +### Changed (2026-08-10 audit) - Import resolution passes the module name to the child interpreter via `argv` instead of f-string interpolation into the `-c` program — diff --git a/README.md b/README.md index 583a359..c397266 100644 --- a/README.md +++ b/README.md @@ -41,12 +41,39 @@ if not result.ok: Together they bracket generation: rag verifies *"is this claim supported?"*; verify checks *"does this named thing exist?"* +## What each checker verifies + +| Checker | Claim it settles | Truth source you declare | +|---|---|---| +| imports | every import in a Python code fence resolves, by full dotted path | `env_python` | +| flags | every `--flag` in an inline span or shell fence appears in that command's `--help` | `help_commands` / `allowed_help_cmds` | +| links | every local markdown link target exists under the project | `project_root` | +| counts | every numeric claim matches the number it names | `count_sources` | + +Findings are `error` when a claim is refuted and `warning` when it cannot be +checked — an unverifiable claim is never a silent pass. `result.ok` is False +only on errors; `raise_if_failed(result)` turns it into a hard gate. + ## Status -Alpha — the deterministic core (imports, flags, links, counts) is shipped -and guarded by a labeled precision/recall corpus (gated ≥ 0.95 each) and -mutation testing (gated ≥ 0.75). The LLM semantic layer is optional via -the `[rag]` extra. +Beta — the deterministic core (imports, flags, links, counts) is stable and +guarded by a labeled precision/recall corpus (gated ≥ 0.95 each) and mutation +testing (gated ≥ 0.75). The public API above (`verify`, `VerifyContext`, +`VerifyResult`, `Finding`, `FindingKind`, `raise_if_failed`, and the `Judge` +protocol) is what beta covers: it will not change shape without a deprecation +in a minor release. The LLM semantic layer is optional via the `[rag]` extra +and is the least settled part of the surface. + +### Known limitations + +- Reference-style links (`[text][ref]`) are not resolved — only inline + `[text](target)` links are checked. +- Short flags (`-v`) are not checked; only `--long` forms are. +- A flag written as bare `` `--flag` `` in prose is attributed to the nearest + preceding word, so it may degrade to a warning rather than resolve. +- Counts are matched to a source by keyword overlap with the claim's + surrounding text; a numeric claim whose context names no source is skipped + rather than guessed at. ## License diff --git a/pyproject.toml b/pyproject.toml index 26405c6..7eb9f6b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "attune-verify" -version = "0.2.2" +version = "0.3.0" description = "Generation fact-checker for the attune-* family. Verifies named entities in LLM output actually exist — imports import, CLI flags are real, links resolve, counts match source." readme = {file = "README.md", content-type = "text/markdown"} requires-python = ">=3.10" @@ -17,7 +17,7 @@ keywords = [ "llm", "attune", "grounding", "faithfulness", ] classifiers = [ - "Development Status :: 2 - Pre-Alpha", + "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", @@ -50,6 +50,11 @@ Repository = "https://github.com/Smart-AI-Memory/attune-verify" [tool.setuptools.packages.find] where = ["src"] +[tool.setuptools.package-data] +# PEP 561: without this the marker is not installed and type checkers ignore +# the package's annotations entirely. +attune_verify = ["py.typed"] + [tool.pytest.ini_options] testpaths = ["tests"] asyncio_mode = "auto" diff --git a/src/attune_verify/__init__.py b/src/attune_verify/__init__.py index 44297e2..da933d9 100644 --- a/src/attune_verify/__init__.py +++ b/src/attune_verify/__init__.py @@ -23,7 +23,7 @@ raise_if_failed, ) -__version__ = "0.2.2" +__version__ = "0.3.0" __all__ = [ "verify", "VerifyContext", diff --git a/src/attune_verify/_extract.py b/src/attune_verify/_extract.py index ffe0c9e..f986636 100644 --- a/src/attune_verify/_extract.py +++ b/src/attune_verify/_extract.py @@ -34,14 +34,19 @@ class NumericClaim: line: Optional[int] = None -# The opening fence may carry an info string after the language word -# (```python title="ex.py") — [^\n]* consumes it so those fences are still -# extracted; only the leading word is the language. -_FENCE_RE = re.compile( - r"^```(\w*)[^\n]*\n(.*?)^```", - re.MULTILINE | re.DOTALL, -) -_LINK_RE = re.compile(r"\[([^\]]+)\]\(([^)]+)\)") +# An opening fence: optional indent, a run of 3+ backticks or tildes, then an +# info string. The info string may carry more than the language word +# (```python title="ex.py") — only the leading word is the language. +_FENCE_OPEN_RE = re.compile(r"^([ \t]*)(`{3,}|~{3,})(.*)$") +# A link target may contain one level of balanced parentheses — 'docs/a(1).md' +# is a legal CommonMark target, and [^)]+ truncated it to 'docs/a(1', flagging +# a file that exists. +_LINK_RE = re.compile(r"\[([^\]]+)\]\(((?:[^()]|\([^()]*\))*)\)") +# One inline code span — its contents are shown, not claimed. The delimiter is +# a run of backticks closed by a run of the same length, so a span that itself +# contains backticks (``` ``a `b` c`` ```) is masked whole rather than leaving +# its middle exposed as prose. +_INLINE_CODE_RE = re.compile(r"(? List[CodeFence]: - """Extract all fenced code blocks from markdown content.""" - fences = [] - for match in _FENCE_RE.finditer(content): - line = content[: match.start()].count("\n") + 1 - # A bare fence keeps language "" — downstream checkers decide how to - # treat untagged blocks (the import checker parses them speculatively). - fences.append( - CodeFence( - language=match.group(1), - content=match.group(2), - line=line, +@dataclass +class _FenceSpan: + """One fence located in the content, by 0-based line index.""" + + open_index: int + close_index: int + language: str + body: List[str] + + +def _iter_fence_spans(content: str) -> List[_FenceSpan]: + """Locate every closed code fence, line by line. + + A line scan rather than one regex, because a fence is defined by + properties a single pattern reads poorly: the closing run must use the + same character and be at least as long as the opening one, and an indented + fence (a code block nested under a list item — routine in LLM-written + docs) carries that indent into every body line. + + An unclosed fence is not a fence: its "body" is the rest of the document, + so treating it as code would drag ordinary prose into the checkers. + """ + lines = [line.rstrip("\r") for line in content.split("\n")] + spans: List[_FenceSpan] = [] + index = 0 + while index < len(lines): + opening = _FENCE_OPEN_RE.match(lines[index]) + if opening is None: + index += 1 + continue + indent, marker, info = opening.groups() + # A tilde fence's info string is unrestricted; a backtick fence's must + # not contain a backtick, else ``` `code` in prose ``` opens a fence. + if marker[0] == "`" and "`" in info: + index += 1 + continue + close_re = re.compile(rf"^[ \t]*{re.escape(marker[0])}{{{len(marker)},}}[ \t]*$") + close_index = next( + (j for j in range(index + 1, len(lines)) if close_re.match(lines[j])), + None, + ) + if close_index is None: + index += 1 + continue + spans.append( + _FenceSpan( + open_index=index, + close_index=close_index, + # A bare fence keeps language "" — downstream checkers decide + # how to treat untagged blocks (the import checker parses them + # speculatively). + language=_language_of(info), + body=[_strip_indent(line, len(indent)) for line in lines[index + 1 : close_index]], ) ) - return fences + index = close_index + 1 + return spans + + +def _language_of(info: str) -> str: + """Return the leading language word of a fence info string. + + An info string may carry more than the language (```python title="ex.py"), + and the language itself may be followed by punctuation. + """ + first = info.strip().split(maxsplit=1) + return re.match(r"\w*", first[0]).group(0) if first else "" + + +def _strip_indent(line: str, width: int) -> str: + """Remove up to ``width`` leading spaces/tabs — the fence's own indent. + + Without this, a fence nested under a list item yields uniformly indented + code that fails ``ast.parse``, so every import inside it went unchecked. + """ + removed = 0 + while removed < width and line[:1] in (" ", "\t"): + line = line[1:] + removed += 1 + return line + + +def extract_code_fences(content: str) -> List[CodeFence]: + """Extract all fenced code blocks from markdown content. + + Backtick and tilde fences are both recognized, at any indentation; a + fence's own indent is stripped from its body so nested blocks parse. + """ + return [ + CodeFence( + language=span.language, + content="".join(f"{line}\n" for line in span.body), + line=span.open_index + 1, + ) + for span in _iter_fence_spans(content) + ] + + +def strip_code_fences(content: str) -> str: + """Blank out every fence, keeping line count and prose offsets intact. + + Fence lines become empty rather than disappearing, so prose either side of + a block never becomes adjacent — a checker looking backwards for context + must not read across a code block it was told to ignore. + """ + lines = content.split("\n") + for span in _iter_fence_spans(content): + for index in range(span.open_index, span.close_index + 1): + lines[index] = "" + return "\n".join(lines) + + +def _mask_code(content: str) -> str: + """Blank code fences and inline spans, preserving every line break. + + Link syntax shown as an example — ``Write it as `[text](target.md)` `` — + is not a link: no renderer resolves it, so checking it flags a target that + was never claimed to exist. Masking keeps line offsets intact, so a link's + reported line number is still its line in the original content. + """ + masked = strip_code_fences(content) + return _INLINE_CODE_RE.sub(lambda m: " " * len(m.group(0)), masked) def extract_links(content: str) -> List[MarkdownLink]: - """Extract all markdown links from content. + """Extract all markdown links from prose. - Targets are normalized: an optional markdown title - (``docs/a.md "Read me"``) is stripped and ```` wrapping is - removed, so checkers see only the path. + Links inside code fences or inline code spans are example syntax, not + claims, and are skipped. Targets are normalized: an optional markdown + title (``docs/a.md "Read me"``) is stripped and ```` + wrapping is removed, so checkers see only the path. """ links = [] - for match in _LINK_RE.finditer(content): - line = content[: match.start()].count("\n") + 1 + prose = _mask_code(content) + for match in _LINK_RE.finditer(prose): + line = prose[: match.start()].count("\n") + 1 links.append( MarkdownLink( text=match.group(1), diff --git a/src/attune_verify/checkers/flags.py b/src/attune_verify/checkers/flags.py index ee9bfc4..00ed6e5 100644 --- a/src/attune_verify/checkers/flags.py +++ b/src/attune_verify/checkers/flags.py @@ -6,7 +6,7 @@ import subprocess from typing import Dict, FrozenSet, List, Optional -from attune_verify._extract import _FENCE_RE, extract_code_fences +from attune_verify._extract import extract_code_fences, strip_code_fences from attune_verify.result import Finding, FindingKind # One inline code span (`mytool --flag`); fences are handled separately. @@ -40,7 +40,7 @@ def check_flags( # Inline spans: `--flag` alone or a whole command in one span # (`mytool --flag`). Fence bodies are stripped first so they are never # double-scanned as inline code. - prose = _FENCE_RE.sub("", content) + prose = strip_code_fences(content) for match in _INLINE_CODE_RE.finditer(prose): span = match.group(1) for flag_match in _FLAG_TOKEN_RE.finditer(span): diff --git a/src/attune_verify/checkers/links.py b/src/attune_verify/checkers/links.py index f061d33..c4110b6 100644 --- a/src/attune_verify/checkers/links.py +++ b/src/attune_verify/checkers/links.py @@ -4,6 +4,7 @@ from pathlib import Path from typing import List, Optional +from urllib.parse import unquote from attune_verify._extract import MarkdownLink from attune_verify.result import Finding, FindingKind @@ -52,7 +53,7 @@ def check_links( # Site-absolute targets (/docs/page.md) mean root-relative in generated # docs; joining them raw would make Path use the filesystem root. rel = path_part.lstrip("/") if path_part.startswith("/") else path_part - resolved = (root / rel).resolve() + resolved = _resolve_target(root, rel) if not resolved.is_relative_to(root): # ../-traversal out of the declared truth boundary: the file may # exist on disk, but it cannot be verified AS a project link. @@ -80,3 +81,21 @@ def check_links( ) ) return findings + + +def _resolve_target(root: Path, rel: str) -> Path: + """Resolve a link target under root, honouring percent-encoding. + + A link to a file whose name contains a space is written ``a%20b.md``, and + checking that literally flagged a file that exists. The raw form is tried + first, so a file genuinely named ``a%20b.md`` still resolves; the decoded + form is the fallback, and is only preferred when it exists. + """ + resolved = (root / rel).resolve() + if resolved.exists(): + return resolved + decoded = unquote(rel) + if decoded == rel: + return resolved + decoded_path = (root / decoded).resolve() + return decoded_path if decoded_path.exists() else resolved diff --git a/src/attune_verify/py.typed b/src/attune_verify/py.typed new file mode 100644 index 0000000..e69de29 diff --git a/tests/corpus/cases.py b/tests/corpus/cases.py index 14c542e..d9c6868 100644 --- a/tests/corpus/cases.py +++ b/tests/corpus/cases.py @@ -313,4 +313,79 @@ def _py(code: str) -> str: content='See [the doc](docs/missing.md "Read me").', expected=(ExpectedFinding(FindingKind.DEAD_LINK, "docs/missing.md"),), ), + CorpusCase( + name="evasion_indented_fence_import", + label="evasion", + # A fence nested under a list item was invisible to the extractor in + # v0.2.x, so every import inside it passed unchecked — the exact + # silent-pass class this library exists to prevent, in the shape LLMs + # write installation docs. + content=( + "1. Install the package.\n" + "2. Then import it:\n" + "\n" + " ```python\n" + " import totally_fake_pkg_xyz_2026\n" + " ```\n" + ), + expected=(ExpectedFinding(FindingKind.UNRESOLVED_IMPORT, "totally_fake_pkg_xyz_2026"),), + ), + CorpusCase( + name="evasion_indented_fence_flag", + label="evasion", + # Same blind spot, flag side: an indented ```bash block under a step. + content="1. Run it:\n\n ```bash\n mytool --nonexistent\n ```\n", + help_commands={"mytool": "Options:\n --verbose Be loud\n"}, + expected=(ExpectedFinding(FindingKind.UNKNOWN_FLAG, "--nonexistent"),), + ), + CorpusCase( + name="evasion_tilde_fence_import", + label="evasion", + # ~~~ is a CommonMark fence the extractor never recognized. + content="~~~python\nimport totally_fake_pkg_xyz_2026\n~~~\n", + expected=(ExpectedFinding(FindingKind.UNRESOLVED_IMPORT, "totally_fake_pkg_xyz_2026"),), + ), + CorpusCase( + name="clean_link_percent_encoded_space", + label="clean", + # A file whose name contains a space is linked as %20; checking the + # literal string flagged a file that exists. + content="See [the doc](docs/my%20file.md).", + files=("docs/my file.md",), + ), + CorpusCase( + name="dead_link_percent_encoded_still_flagged", + label="hallucinated", + # Decoding must not cost recall. + content="See [the doc](docs/missing%20file.md).", + expected=(ExpectedFinding(FindingKind.DEAD_LINK, "docs/missing%20file.md"),), + ), + CorpusCase( + name="clean_link_syntax_shown_in_code_span", + label="clean", + # Docs that document link syntax were flagged for the example they + # show; no renderer resolves a link inside a code span. + content="Write it as `[text](docs/example.md)` in the body.", + ), + CorpusCase( + name="clean_link_syntax_shown_in_fence", + label="clean", + content="Example:\n\n```markdown\n[text](docs/example.md)\n```\n", + ), + CorpusCase( + name="dead_link_in_prose_beside_a_code_span_still_flagged", + label="hallucinated", + # Masking must not cost recall: a real link on the same line as an + # example is still checked. + content="Write it as `[text](target.md)` — see [the doc](docs/missing.md).", + expected=(ExpectedFinding(FindingKind.DEAD_LINK, "docs/missing.md"),), + ), + CorpusCase( + name="clean_link_balanced_parens", + label="clean", + # '[^)]+' truncated the target at the first ')', so 'docs/a(1).md' + # was checked as 'docs/a(1' and flagged. + content="See [the doc](docs/a(1).md).", + files=("docs/a(1).md",), + ), ) diff --git a/tests/test_behavioral.py b/tests/test_behavioral.py index 18d95ed..56faee1 100644 --- a/tests/test_behavioral.py +++ b/tests/test_behavioral.py @@ -23,6 +23,7 @@ extract_code_fences, extract_links, extract_numeric_claims, + strip_code_fences, ) from attune_verify.checkers.counts import check_counts from attune_verify.checkers.flags import _get_help, _guess_command, check_flags @@ -222,6 +223,47 @@ def test_links_site_absolute_target_is_root_relative(tmp_path): assert findings[0].severity == "error" # root/etc/passwd does not exist +def test_links_percent_encoded_target_resolves_to_the_real_file(tmp_path): + # A file whose name has a space is linked as %20; checking that literally + # flagged a file that exists. + (tmp_path / "my file.md").write_text("x", encoding="utf-8") + assert check_links([MarkdownLink(text="d", target="my%20file.md", line=1)], tmp_path) == [] + + +def test_links_percent_encoded_dead_target_still_flagged(tmp_path): + # Decoding must not cost recall. + findings = check_links([MarkdownLink(text="d", target="missing%20file.md", line=1)], tmp_path) + assert len(findings) == 1 + assert findings[0].severity == "error" + + +def test_links_literal_percent_in_filename_still_resolves(tmp_path): + # The raw form is tried first, so a file genuinely named 'a%20b.md' + # resolves and is not mistaken for an encoded 'a b.md'. + (tmp_path / "a%20b.md").write_text("x", encoding="utf-8") + assert check_links([MarkdownLink(text="d", target="a%20b.md", line=1)], tmp_path) == [] + + +def test_links_percent_encoded_traversal_is_still_caught(tmp_path): + # Decoding must not open an escape hatch out of the declared boundary. + outside = tmp_path / "outside.md" + outside.write_text("x", encoding="utf-8") + root = tmp_path / "project" + root.mkdir() + findings = check_links([MarkdownLink(text="up", target="%2e%2e/outside.md", line=1)], root) + assert len(findings) == 1 + assert findings[0].severity == "warning" + assert "outside project_root" in findings[0].detail + + +def test_links_balanced_parens_in_target_are_not_truncated(tmp_path): + # '[^)]+' stopped at the first ')', checking 'docs/a(1' instead. + (tmp_path / "a(1).md").write_text("x", encoding="utf-8") + links = extract_links("See [the doc](a(1).md).") + assert [link.target for link in links] == ["a(1).md"] + assert check_links(links, project_root=tmp_path) == [] + + # --------------------------------------------------------------------------- # Flag checker branches # --------------------------------------------------------------------------- @@ -366,6 +408,90 @@ def test_extract_code_fences_with_info_string(): assert "import os" in fences[0].content +def test_extract_code_fences_indented_under_list_item(): + # The fence's own indent must be stripped, or the body fails ast.parse + # and every import inside it goes unchecked. + content = "1. Then:\n\n ```python\n import os\n x = 1\n ```\n" + fences = extract_code_fences(content) + assert len(fences) == 1 + assert fences[0].language == "python" + assert fences[0].content == "import os\nx = 1\n" + assert fences[0].line == 3 + + +def test_extract_code_fences_tilde_fence(): + fences = extract_code_fences("~~~python\nimport os\n~~~\n") + assert len(fences) == 1 + assert fences[0].language == "python" + assert fences[0].content == "import os\n" + + +def test_extract_code_fences_marker_kinds_do_not_close_each_other(): + # A ``` line inside a ~~~ fence is body, not a closing marker. + fences = extract_code_fences("~~~\nnot a fence: ```\n~~~\n") + assert len(fences) == 1 + assert fences[0].content == "not a fence: ```\n" + + +def test_extract_code_fences_longer_closing_run_closes(): + # CommonMark: the closing run must be at least as long as the opening one. + fences = extract_code_fences("````python\nimport os\n`````\n") + assert len(fences) == 1 + assert fences[0].content == "import os\n" + + +def test_extract_code_fences_unclosed_fence_is_not_a_fence(): + # Otherwise the "body" is the rest of the document and prose gets checked + # as code. + assert extract_code_fences("```python\nimport os\nand then prose.\n") == [] + + +def test_extract_code_fences_inline_code_span_does_not_open_a_fence(): + # A backtick fence's info string may not contain a backtick — otherwise a + # line-leading ```span``` swallows the prose after it as a fence body. + assert extract_code_fences("```code``` is written inline.\nprose\n```\n") == [] + + +def test_extract_code_fences_tilde_info_string_may_contain_a_backtick(): + # The backtick rule is specific to backtick fences. + fences = extract_code_fences("~~~python `note`\nimport os\n~~~\n") + assert len(fences) == 1 + assert fences[0].language == "python" + + +def test_strip_code_fences_blanks_lines_without_moving_prose(): + content = "before\n```bash\nmytool --x\n```\nafter\n" + stripped = strip_code_fences(content) + assert "mytool" not in stripped + assert stripped.count("\n") == content.count("\n") + assert stripped.splitlines()[0] == "before" + assert stripped.splitlines()[4] == "after" + + +def test_extract_links_skips_code_spans_and_fences_but_keeps_line_numbers(): + content = ( + "intro\n" + "Write it as `[text](example.md)` in the body.\n" + "```markdown\n[shown](fenced.md)\n```\n" + "See [the doc](real.md).\n" + ) + links = extract_links(content) + assert [(link.target, link.line) for link in links] == [("real.md", 6)] + + +@pytest.mark.parametrize("ticks", ["`", "``", "```"]) +def test_extract_links_skips_spans_of_any_delimiter_width(ticks): + # A doubled delimiter is how a span containing backticks is written; a + # single-backtick rule masked its edges and left the middle exposed. + content = f"Write it as {ticks}[shown](example.md){ticks} — see [the doc](real.md)." + assert [link.target for link in extract_links(content)] == ["real.md"] + + +def test_extract_links_span_containing_backticks_is_masked_whole(): + content = "Inline ``code with `ticks` inside`` then [the doc](real.md)." + assert [link.target for link in extract_links(content)] == ["real.md"] + + def test_extract_links_captures_text_target_and_line(): links = extract_links("a\n[label](path/to.md)\n") assert len(links) == 1 diff --git a/tests/test_packaging.py b/tests/test_packaging.py new file mode 100644 index 0000000..ef32beb --- /dev/null +++ b/tests/test_packaging.py @@ -0,0 +1,52 @@ +"""Packaging guards: version single-sourcing and the typing marker. + +The release flow bumps two version sites by hand (``pyproject.toml`` and +``__init__.__version__``). A drift between them ships a wheel whose metadata +disagrees with the value users read at runtime, and nothing else would catch +it — so it is pinned here. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +import pytest + +import attune_verify + +_REPO_ROOT = Path(__file__).resolve().parent.parent +_PYPROJECT = _REPO_ROOT / "pyproject.toml" +_PACKAGE_ROOT = Path(attune_verify.__file__).resolve().parent + + +def _pyproject_field(name: str) -> str: + text = _PYPROJECT.read_text(encoding="utf-8") + match = re.search(rf'^{name} = "([^"]+)"', text, re.MULTILINE) + assert match is not None, f"{name} not found in pyproject.toml" + return match.group(1) + + +@pytest.mark.skipif(not _PYPROJECT.exists(), reason="running outside the source tree") +def test_version_matches_pyproject() -> None: + assert attune_verify.__version__ == _pyproject_field("version") + + +@pytest.mark.skipif(not _PYPROJECT.exists(), reason="running outside the source tree") +def test_development_status_classifier_is_declared_once() -> None: + # A stale second status classifier would misreport maturity on PyPI. + statuses = re.findall( + r'"Development Status :: ([^"]+)"', _PYPROJECT.read_text(encoding="utf-8") + ) + assert len(statuses) == 1, f"expected exactly one Development Status, got {statuses}" + + +def test_py_typed_marker_ships_with_the_package() -> None: + # Without the marker, PEP 561 tells type checkers to ignore the (fully + # annotated) package entirely. + assert (_PACKAGE_ROOT / "py.typed").is_file() + + +def test_public_api_is_importable_and_complete() -> None: + for name in attune_verify.__all__: + assert hasattr(attune_verify, name), f"__all__ names {name}, which is not exported"