refactor: drop bundled content — Python side becomes core-only - #6
Open
tbsvttr wants to merge 1 commit into
Open
refactor: drop bundled content — Python side becomes core-only#6tbsvttr wants to merge 1 commit into
tbsvttr wants to merge 1 commit into
Conversation
Per @scottbenton's review on the initial port, the biggest concern was Python-bindings content silently drifting from what actually ships in official-content / community-content. The initial port bundled a copy of every ruleset's JSON inside packages/<ruleset>/src/.../json/ — nice for `pip install datasworn-community-classic` ergonomics, but nothing kept those copies in sync with the JS side. Fix: adopt the "core-only" shape Scott suggested. This repo now publishes one PyPI package (datasworn-community-core) containing just the Pydantic models. Users load JSON however they want — HTTP fetch from the content repos, local file, npm tarball, whatever — and pass it to Ruleset.model_validate. ## What changed - **Deleted 9 per-ruleset workspace packages** — classic, delve, lodestar, starforged, sundered_isles, ancient_wonders, fe_runners, ironsmith, starsmith. Their JSON was drift-prone by design. - **Reduced root pyproject.toml workspace** to just `packages/core`. - **tests/conftest.py** — new shared `content_path` fixture that fetches main-branch JSON from official-content / community-content and caches under tests/.cache/. `DATASWORN_TESTS_OFFLINE=1` skips network. - **tests/test_load_rules_packages.py** — validates fetched content against the models. Currently xfail because models.py is on schema 0.1.0 while content is on 0.2.0 (also documented drift, see below). - **tests/test_type_alias_ergonomics.py** — trimmed. The end-to-end content tests moved to the new xfail set; kept here are three post-processor-invariant checks (RootModel[str] wrappers gone, discriminated-union bases resolved, `.id` reads as str). - **README** rewritten around "one package, one purpose": install core, load your own JSON, validate. Adds a **Regenerating models.py** section with the exact `datamodel-code-generator` invocation + post-process step for whoever picks up the drift fix. ## Known drift, called out The bundled models.py was generated from schema line 0.1.0. Content on main is 0.2.0. Real content validation therefore fails today — hence the 9 xfail markers on test_load_rules_packages.py. Fix = regenerate models.py from the current schema and re-run the post-processor; xfail markers should then flip green and be removed. Instructions in the README.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Directly addresses @scottbenton's review:
He's right. The initial port bundled each ruleset's JSON inside `packages//src/.../json/`, which was cute for `pip install datasworn-community-classic` ergonomics but structurally guaranteed drift.
This PR adopts Scott's suggested core-only shape: one PyPI package (`datasworn-community-core`) containing the Pydantic models, no bundled JSON. Users load content however they want — HTTP from `official-content` / `community-content`, local file, npm tarball, whatever — and hand it to `Ruleset.model_validate`.
What changed
Known drift — surfaced, not hidden
While rewriting the tests to fetch live content, I confirmed the exact drift Scott warned about. Loading current `main`-branch content today fails validation:
`models.py` here is on schema 0.1.0. Content on main is 0.2.0. So the 9 end-to-end validation tests in `tests/test_load_rules_packages.py` are marked `xfail` with a documented reason. When someone regenerates `models.py` against the current schema and re-runs the post-processor, those tests should flip green and the `xfail` markers should be removed.
The README's new Regenerating models.py section has the exact commands (`datamodel-code-generator` invocation + post-process step).
Test plan
Follow-ups (not this PR)