Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/python_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,24 @@ jobs:
# -I keeps the checkout off sys.path, so this reads the installed package.
version="$("$RUNNER_TEMP/wheel-venv/bin/python" -I -c 'from bust._version import __VERSION__; print(__VERSION__)')"
PATH="$RUNNER_TEMP/wheel-venv/bin:$PATH" scripts/smoke-test.sh "$version"
# cloudbust runs bust in the browser. Install the wheel in Pyodide with only
# the core dependencies and check that it writes what CPython writes.
- name: Generate the examples with CPython
env:
SOURCE_DATE_EPOCH: 0
run: |
for example in example/*.json; do
"$RUNNER_TEMP/wheel-venv/bin/bust" "$example" -o "$RUNNER_TEMP/cpython" -p
done
- name: Generate the examples with Pyodide
working-directory: scripts/pyodide
env:
SOURCE_DATE_EPOCH: 0
run: |
npm ci
node run.mjs ../../dist/*.whl "$RUNNER_TEMP/pyodide"
- name: Compare
run: diff -r "$RUNNER_TEMP/cpython" "$RUNNER_TEMP/pyodide"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: dist
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,6 @@ example/*/doc/*.pdf

# bust writes its log to the working directory
debug.log

# npm install in scripts/pyodide
node_modules/
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ the git log and on [PyPI](https://pypi.org/project/bust/#history).
- `doc/<module>.md`, the documentation as Markdown, with the register list and a bits table per register, readable on GitHub.
- The documentation hosts the register map JSON Schema at https://bust.readthedocs.io/en/stable/register-map.schema.json. A register map may point to it with a `"$schema"` key, which gives completion, hover text and errors in editors with JSON Schema support. The editor adds the key to new register maps.
- `bust.bundle_zip` turns the result of `generate_files` into one zip, in the folder layout the CLI writes, with a `README.md` that lists what each file is.
- bust runs under Pyodide in the browser. Getting started lists the dependencies to install there, and CI checks that the wheel generates every example under Pyodide as it does under CPython.

### Changed
- The documentation and the editor overview show the bits that no field covers as `reserved` instead of `unused`, and list them among the fields with "Read as 0, writes have no effect."
Expand Down
14 changes: 13 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,21 @@ bust runs on Python 3.9 to 3.14. mypy checks against 3.9.

CI runs mypy and the tests on Python 3.9 to 3.14 (plus the next Python while it
is in pre-release), builds the docs, builds the package and smoke-tests the
wheel, runs the pre-commit hooks and simulates the generated VHDL. A pull
wheel, checks that the wheel generates the same files under Pyodide as under
CPython, runs the pre-commit hooks and simulates the generated VHDL. A pull
request needs all of it green.

To run the Pyodide check locally, with Node.js 18 or newer:

```bash
python -m build -w
cd scripts/pyodide
npm ci
node run.mjs ../../dist/bust-*.whl /tmp/bust-pyodide
```

`tests/test_pure_python.py` fails if a runtime dependency is not pure Python, since Pyodide cannot install it.

## Golden files

Most tests regenerate the outputs of the examples in `example/` and compare
Expand Down
13 changes: 13 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,19 @@ pip install bust
bust --version
```

### In the browser

bust runs under [Pyodide](https://pyodide.org), Python in the browser. Install it without its dependencies and add the ones that generating code needs:

```python
import micropip

await micropip.install(["json5", "Jinja2", "pylatexenc"])
await micropip.install("bust", deps=False)
```

That leaves out Textual, which only the editor (`-c`, `-e`) uses, and docopt, which only the command line uses. Generate the files with [`bust.generate_files`](command-line.md#from-python).

## Describe your registers

A register map is one JSON file with a `bus` section and a `module` section.
Expand Down
26 changes: 26 additions & 0 deletions scripts/pyodide/generate_examples.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""Generate every example register map with bust as Pyodide runs it.

run.mjs installs bust without its dependencies and imports this module. Only
the core dependencies are present, so generation fails if it needs any other.
"""

import glob
import os

import bust
from bust.utils.file_utils import write_string_to_file


def generate_examples(example_dir: str, output_dir: str) -> None:
"""Write what ``bust FILE -o output_dir -p`` writes for each example."""
json_paths = sorted(glob.glob(os.path.join(example_dir, "*.json")))
if not json_paths:
raise FileNotFoundError(f"No register maps in {example_dir}")
for json_path in json_paths:
print(f"Generating {json_path}")
with open(json_path, encoding="utf-8", newline="") as fp:
files = bust.generate_files(fp.read())
for relative_path, content in files.items():
directory, filename = os.path.split(os.path.join(output_dir, relative_path))
os.makedirs(directory, exist_ok=True)
write_string_to_file(content, filename, directory, True)
53 changes: 53 additions & 0 deletions scripts/pyodide/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions scripts/pyodide/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"name": "bust-pyodide-check",
"private": true,
"description": "Runs bust under Pyodide, as the browser does",
"type": "module",
"dependencies": {
"pyodide": "314.0.7"
}
}
58 changes: 58 additions & 0 deletions scripts/pyodide/run.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// Install a bust wheel in Pyodide the way the browser does and generate every
// example into OUTPUT_DIR, as `bust FILE -o OUTPUT_DIR -p` would.
// Usage: node scripts/pyodide/run.mjs WHEEL OUTPUT_DIR
import { mkdirSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { loadPyodide } from "pyodide";

const [wheel, outputDir] = process.argv.slice(2);
if (!wheel || !outputDir) {
console.error("Usage: node scripts/pyodide/run.mjs WHEEL OUTPUT_DIR");
process.exit(2);
}
const repo = fileURLToPath(new URL("../..", import.meta.url));
mkdirSync(outputDir, { recursive: true });

// SOURCE_DATE_EPOCH pins the time bust writes into the files.
const env = { HOME: "/home/pyodide" };
if (process.env.SOURCE_DATE_EPOCH !== undefined) {
env.SOURCE_DATE_EPOCH = process.env.SOURCE_DATE_EPOCH;
}
const pyodide = await loadPyodide({ env });
const mounts = {
"/repo": repo,
"/dist": path.dirname(path.resolve(wheel)),
"/out": path.resolve(outputDir),
};
for (const [mountPoint, root] of Object.entries(mounts)) {
pyodide.FS.mkdirTree(mountPoint);
pyodide.FS.mount(pyodide.FS.filesystems.NODEFS, { root }, mountPoint);
}

await pyodide.loadPackage(["micropip", "packaging"]);
const micropip = pyodide.pyimport("micropip");
await micropip.install.callKwargs(`emfs:/dist/${path.basename(wheel)}`, {
deps: false,
});
// The browser leaves out Textual (the editor) and docopt (the command line,
// and PyPI has no wheel of it). docs/getting-started.md lists the rest.
const coreRequirements = pyodide
.runPython(
`
from importlib.metadata import requires
from packaging.requirements import Requirement

[
str(requirement)
for requirement in map(Requirement, requires("bust"))
if requirement.marker is None and requirement.name not in ("textual", "docopt")
]
`,
)
.toJs();
await micropip.install(coreRequirements);

const sys = pyodide.pyimport("sys");
sys.path.append("/repo/scripts/pyodide");
pyodide.pyimport("generate_examples").generate_examples("/repo/example", "/out");
59 changes: 59 additions & 0 deletions tests/test_pure_python.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
"""bust runs in the browser under Pyodide, which installs only pure Python
wheels from PyPI. Every runtime dependency must stay pure Python."""

import email
from importlib import metadata
from typing import Dict, List

from packaging.requirements import Requirement
from packaging.utils import canonicalize_name

# Pyodide ships its own build of MarkupSafe, the one Jinja2 needs. MarkupSafe
# also runs without its C speedups, so it works everywhere else too.
PYODIDE_BUILDS = {"markupsafe"}


def runtime_requirements(name: str) -> List[Requirement]:
requirements = [Requirement(line) for line in metadata.requires(name) or []]
return [
requirement
for requirement in requirements
if requirement.marker is None or requirement.marker.evaluate({"extra": ""})
]


def runtime_dependencies() -> Dict[str, metadata.Distribution]:
"""Every distribution bust needs at run time, by canonical name."""
found: Dict[str, metadata.Distribution] = {}
pending = runtime_requirements("bust")
while pending:
name = canonicalize_name(pending.pop().name)
if name not in found:
found[name] = metadata.distribution(name)
pending.extend(runtime_requirements(name))
return found


def is_pure_python(distribution: metadata.Distribution) -> bool:
wheel = email.message_from_string(distribution.read_text("WHEEL") or "")
tags = wheel.get_all("Tag") or []
return wheel["Root-Is-Purelib"] == "true" and all(
tag.endswith("-none-any") for tag in tags
)


def test_the_walk_reaches_indirect_dependencies():
dependencies = runtime_dependencies()
assert {"jinja2", "markupsafe", "textual", "rich"} <= dependencies.keys()


def test_every_runtime_dependency_is_pure_python():
impure = [
name
for name, distribution in sorted(runtime_dependencies().items())
if name not in PYODIDE_BUILDS and not is_pure_python(distribution)
]
assert impure == [], (
f"{', '.join(impure)} is not pure Python, so bust cannot install it "
"in the browser under Pyodide"
)
Loading