Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ test: ## Run template tests
echo "uvx is required to run template tests. Install uv from https://docs.astral.sh/uv/getting-started/installation/" >&2; \
exit 1; \
fi
$(ACT_TEST_ENV) $(UV) --with pytest-copier --with pyyaml --with syrupy --with make-parser --with hypothesis pytest tests/
$(ACT_TEST_ENV) $(UV) --with pytest-copier --with pyyaml --with syrupy --with make-parser --with hypothesis --with mdast pytest tests/

help: ## Show available targets
@grep -E '^[a-zA-Z_-]+:.*?##' $(MAKEFILE_LIST) | \
Expand Down
12 changes: 11 additions & 1 deletion docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ make test
```

The target uses
`uvx --with pytest-copier --with pyyaml --with syrupy --with make-parser --with hypothesis pytest tests/`,
`uvx --with pytest-copier --with pyyaml --with syrupy --with make-parser --with hypothesis --with mdast pytest tests/`,
so Python test dependencies must be added to that invocation before tests
import them. Keep long runs logged through `tee` into `/tmp`, following the
example in `AGENTS.md`.
Expand All @@ -22,6 +22,13 @@ The tests render both library and application projects, run generated public
gates such as `make all`, validate generated Makefiles with `mbake`, and parse
generated `Cargo.toml` files as TOML.

Documentation snapshot tests parse Markdown files into normalized mdast JSON
before comparing snapshots. The normalization removes parser source positions
and treats soft line wrapping as formatting noise, while preserving document
structure such as headings, links, lists, tables, inline code, and code blocks.
Markdown formatting remains covered separately by `markdownlint-cli2` and
`mdtablefix`.
Comment on lines +25 to +30

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use en-GB spellings here.

Replace normalized/normalization with normalised/normalisation in both updated sections.

Triage: [type:spelling]
As per path instructions, use en-GB-oxendict spelling in docs/**.

♻️ Proposed fix
-Documentation snapshot tests parse Markdown files into normalized mdast JSON
-before comparing snapshots. The normalization removes parser source positions
+Documentation snapshot tests parse Markdown files into normalised mdast JSON
+before comparing snapshots. The normalisation removes parser source positions
@@
-- `tests/helpers/markdown_semantics.py` parses Markdown as normalized mdast JSON
+- `tests/helpers/markdown_semantics.py` parses Markdown as normalised mdast JSON

Also applies to: 98-99

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/developers-guide.md` around lines 25 - 30, Update the documentation
wording in the affected markdown sections to use en-GB spellings: replace
“normalized” with “normalised” and “normalization” with “normalisation” in the
prose that describes the Markdown snapshot process, including the reference to
tests/helpers/markdown_semantics.py. Keep the rest of the content unchanged and
ensure the terminology is consistent across both updated sections in
docs/developers-guide.md.

Source: Path instructions


Generated audit coverage is tested without network access by replacing Cargo
with a fake executable. The regression verifies that `make rust-audit` derives
the workspace root from `cargo metadata`, ignores manifests outside workspace
Expand Down Expand Up @@ -55,6 +62,7 @@ projects:
- `syrupy` for generated structured file snapshots in tests.
- `make-parser` for generated Makefile structure assertions in tests.
- `hypothesis` for generated-file schema helper property tests.
- `mdast` for parent documentation semantic snapshot tests.
- Rust and Cargo through `rustup`.
- cargo-nextest for generated fast test execution in CI, while generated
Makefiles still fall back to `cargo test` for contributors.
Expand Down Expand Up @@ -87,6 +95,8 @@ Reusable test support lives under `tests/helpers/`:
project command helpers.
- `tests/helpers/generated_files.py` centralizes generated text, TOML, and YAML
parsing with pytest failure messages.
- `tests/helpers/markdown_semantics.py` parses Markdown as normalized mdast JSON
for semantic documentation snapshots.
- `tests/helpers/tooling_contracts/` contains generated Makefile, Cargo,
documentation, CI, release, and coverage-action contract assertions.
- `tests/conftest.py` provides the session-scoped `act_ready` fixture, which
Expand Down
380 changes: 380 additions & 0 deletions docs/execplans/mdast-snapshot-assertions.md

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions template/Cargo.toml.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ categories = [{% for category in package_category_values %}"{{ category }}"{% if

[dependencies]

[dev-dependencies]
insta = { version = "1.48.0", features = ["json"] }
markdown = { version = "1.0.0", features = ["json"] }
serde_json = "1.0"

{% if flavour == 'app' -%}
# The package.metadata.binstall pkg-url intentionally mixes Jinja2 values
# rendered at project generation with single-brace cargo-binstall variables
Expand Down
13 changes: 13 additions & 0 deletions template/docs/developers-guide.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@ The main `.github/workflows/ci.yml` workflow deliberately does not run
`make test WITH_ACT=1`; the separate Act workflow runs those slower
container-backed checks in parallel.

Documentation snapshot tests live in `tests/documentation_snapshots.rs`. They
parse selected Markdown files into normalized mdast JSON before comparing
`insta` snapshots, so line wrapping and table alignment changes do not churn
semantic snapshots. Markdown formatting is still checked by `make markdownlint`
and formatted by `make fmt`. When a documentation meaning or structure change
is intentional, update the affected `tests/snapshots/*.snap` files with
`INSTA_UPDATE=always cargo test --test documentation_snapshots`, then review
the JSON diff before committing it.

Comment on lines +20 to +28

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Replace make fmt with make check-fmt.

The generated Makefile exposes check-fmt, not fmt, so this instruction points contributors at a dead target.

Proposed fix
- Markdown formatting is still checked by `make markdownlint` and formatted by `make fmt`.
+ Markdown formatting is still checked by `make markdownlint` and validated by `make check-fmt`.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Documentation snapshot tests live in `tests/documentation_snapshots.rs`. They
parse selected Markdown files into normalized mdast JSON before comparing
`insta` snapshots, so line wrapping and table alignment changes do not churn
semantic snapshots. Markdown formatting is still checked by `make markdownlint`
and formatted by `make fmt`. When a documentation meaning or structure change
is intentional, update the affected `tests/snapshots/*.snap` files with
`INSTA_UPDATE=always cargo test --test documentation_snapshots`, then review
the JSON diff before committing it.
Documentation snapshot tests live in `tests/documentation_snapshots.rs`. They
parse selected Markdown files into normalized mdast JSON before comparing
`insta` snapshots, so line wrapping and table alignment changes do not churn
semantic snapshots. Markdown formatting is still checked by `make markdownlint`
and validated by `make check-fmt`. When a documentation meaning or structure change
is intentional, update the affected `tests/snapshots/*.snap` files with
`INSTA_UPDATE=always cargo test --test documentation_snapshots`, then review
the JSON diff before committing it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@template/docs/developers-guide.md.jinja` around lines 20 - 28, The
documentation guidance incorrectly refers contributors to the nonexistent Make
target `make fmt`; update the text in the developers guide to point to the
actual formatting target exposed by the generated Makefile, `check-fmt`, and
keep the surrounding workflow references in `tests/documentation_snapshots.rs`
and `markdownlint` unchanged.

## Tooling

Development builds use Cranelift for debug code generation. On Linux targets,
Expand All @@ -27,6 +36,10 @@ LLVM-compatible linker behaviour.
Install `clang`, `lld`, `mold`, `python3`, and `cargo-audit` before running the
full generated workflow locally on Linux.

The documentation snapshot tests use the `markdown`, `serde_json`, and `insta`
dev-dependencies declared in `Cargo.toml`; no Node or npm Markdown snapshot
pipeline is required.

### Security audit ignores

Security audit jobs may set `CARGO_AUDIT_IGNORES` for narrowly scoped
Expand Down
106 changes: 106 additions & 0 deletions template/tests/documentation_snapshots.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
//! Semantic documentation snapshot tests.

use serde_json::Value;

fn parse_markdown_semantics(markdown: &str) -> Result<Value, Box<dyn std::error::Error>> {
let tree = markdown::to_mdast(markdown, &markdown::ParseOptions::gfm())
.map_err(|message| std::io::Error::other(message.to_string()))?;
let mut value = serde_json::to_value(tree)?;
normalise_node(&mut value);
Ok(value)
}

fn normalise_node(value: &mut Value) {
match value {
Value::Object(object) => {
let node_type = object
.get("type")
.and_then(Value::as_str)
.map(str::to_owned);
object.remove("position");
object.remove("line");
object.remove("column");
object.remove("offset");

if node_type.as_deref() == Some("text")
&& let Some(Value::String(text)) = object.get_mut("value")
{
*text = normalise_soft_text(text);
}

for child in object.values_mut() {
normalise_node(child);
}
}
Value::Array(children) => {
for child in children {
normalise_node(child);
}
}
_ => {}
}
}

fn normalise_soft_text(value: &str) -> String {
value.split_whitespace().collect::<Vec<_>>().join(" ")
}

fn parse_markdown_semantics_or_panic(markdown: &str) -> Value {
match parse_markdown_semantics(markdown) {
Ok(value) => value,
Err(error) => panic!("failed to parse Markdown semantics: {error}"),
}
}

#[test]
fn markdown_semantics_ignore_formatting_noise() {
let compact = "\
# Documentation

The generated project links to [docs](docs/contents.md) and explains
the workflow in one paragraph.

| Path | Purpose |
| - | - |
| `docs/contents.md` | Index |

- [x] Keep snapshots semantic
";
let reflowed = "\
# Documentation

The generated project links to [docs](docs/contents.md) and explains the
workflow in one paragraph.

| Path | Purpose |
| -------------------- | ------- |
| `docs/contents.md` | Index |

- [x] Keep snapshots semantic
";

assert_eq!(
parse_markdown_semantics_or_panic(compact),
parse_markdown_semantics_or_panic(reflowed)
);
}

#[test]
fn generated_documentation_matches_semantic_snapshots() {
let repository_layout = include_str!("../docs/repository-layout.md");
let repository_layout_snapshot = if repository_layout.contains(".github/workflows/release.yml")
{
"docs_repository_layout_app"
} else {
"docs_repository_layout_lib"
};

insta::assert_json_snapshot!(
"docs_contents",
parse_markdown_semantics_or_panic(include_str!("../docs/contents.md"))
);
insta::assert_json_snapshot!(
repository_layout_snapshot,
parse_markdown_semantics_or_panic(repository_layout)
);
}
Loading
Loading