Skip to content

fix(level6): targeted API contract and checker quality fixes - #65

Merged
michalmaj merged 3 commits into
mainfrom
feature/level6-api-quality-fixes
Oct 1, 2026
Merged

michalmaj merged 3 commits into
mainfrom
feature/level6-api-quality-fixes

Conversation

@michalmaj

Copy link
Copy Markdown
Owner

Summary

Targeted fixes to level_6_api/ based on the accepted Level 6 audit. This is not a Level 6 rebuild — mission/project count, order, and the HTTP → schemas → services/domain → repository architecture are unchanged, and Level 6 stays sync-only (no async introduced).

  • P01 README: fixed the 404-vs-422 contradiction. An unknown monster_name outside the Literal[...] schema is a 422 (Pydantic validation, before the endpoint runs), not a 404. The README now explains why — the allowed set is enforced at the schema boundary, not at a runtime lookup — and explicitly warns against reading this as a universal rule, contrasting it with Project 02's real GET /monsters/{name} 404 case.
  • M06 README: promoted GET /monsters/{name} from an undocumented "Side Quest" to a required Task 5, matching the scaffold's existing TODO and check.py's existing hard requirement. Added the def_→defense schema-boundary naming note here, where the mapping first appears.
  • M01 check.py: replaced the "non-empty list" check with a real comparison against data/monsters.json. A hardcoded fake list now fails; any behaviorally-correct implementation (including one that never touches _service) passes.
  • M05 check.py: session endpoints are now isolated via a tempdir-backed SessionRepository through app.dependency_overrides (the exact technique the mission teaches), instead of leaking real files into data/sessions/.
  • M07 check.py (the core fix): added a second gate — after the student's suite passes against the correct app, it must also fail against six independent, single-behavior mutants covering the mission's six required tests (health status, monster list, battle winner, invalid-monster 404, session persistence, session-not-found 404). A client.get(...); assert True suite now fails. Verified against two structurally different correct suites (different names/order/assertion style) — both pass — and five negative/fake scenarios — all correctly rejected. Mutants are applied via exact string replacement and restored via try/finally with byte-identical verification; __pycache__ is cleared around every subprocess run.
  • P03 README: corrected the false "fill in the five Pydantic models" instruction — the file is pre-wired with four models — and fixed a second, previously-unnoticed bug where the MonsterOut table listed def_ instead of defense. Added a closing note pointing back to Level 5's Ruff habit.
  • P03 check.py: added a Gate 10 running ruff check rpg/api/ (scoped there after confirming a full correct solution is clean under that scope, while tests/ carries a pre-existing, unrelated lint error the student doesn't own). A real lint violation in student code now fails the boss fight; correct code still passes.

Explicitly not done (per audit scope)

No async, BackgroundTasks, middleware, lifecycle events, new service architecture, custom exception hierarchy, new missions, or Mypy/Pyright/coverage/CI added to Level 6.

Test plan

  • All 8 missions + 3 projects: scaffolds correctly FAIL
  • M01: real data → PASS, hardcoded fake → FAIL
  • M05: round-trip → PASS, no new files under data/sessions/, overrides cleared
  • M06: full 5-task solution → PASS, Tasks 1–4 only → FAIL with the new 404 message
  • M07: Gate 1 + Gate 2 verified with two independently-written correct suites (both PASS) and 5 fake/negative suites (all correctly FAIL at the expected mutant), run multiple times to confirm determinism
  • P01: unknown Literal monster → 422 (README now matches)
  • P03: full correct solution → PASS incl. new Ruff gate; injected lint violation → FAILs only the Ruff gate
  • tools/author_check.py → 727 checks passed, 0 errors
  • tools/course_status.py → runs clean
  • Working tree clean, no leftover __pycache__/.pytest_cache/session files/dependency overrides

🤖 Generated with Claude Code

- P01 README: fix the 404-vs-422 contradiction — an unknown monster_name
  outside the Literal schema is a 422 validation error, not a 404; explain
  why (schema-enforced set vs. runtime lookup) instead of stating it as a
  universal rule.
- M06 README: promote GET /monsters/{name} from an undocumented "Side
  Quest" to a required Task 5, matching the scaffold's existing TODO and
  check.py's existing hard requirement. Note the def_ -> defense renaming
  at the schema boundary where it first appears.
- M01 check.py: replace the non-empty-list-only GET /monsters check with a
  real comparison against data/monsters.json, so a hardcoded fake list
  fails while any behaviorally-correct implementation passes.
- M05 check.py: isolate the session endpoints with a tempdir-backed
  SessionRepository via app.dependency_overrides (the same technique the
  mission teaches) instead of writing real files into data/sessions/.
M07 check.py previously only verified structural markers (TestClient,
dependency_overrides, >=6 test functions, pytest passes) — a suite of
`client.get(...); assert True` tests satisfied every gate.

Add a second gate: after the student's suite passes against the correct
app, run it against six independent, single-behavior mutants covering the
mission's six required tests (health status, monster list, battle winner,
invalid-monster 404, session persistence, session-not-found 404). Each
mutant must be caught by the student's own suite, restored via try/finally
with byte-identical verification, with __pycache__ cleared around every
subprocess run.
- P03 README: the "Fill in rpg/api/schemas.py with the five Pydantic
  models" instruction was false — the file is pre-wired with four models.
  Reframe it as read-only reference material for the routers, and fix a
  second, previously-unnoticed doc bug (the MonsterOut table listed def_
  instead of the schema's actual defense field).
- P03 check.py: add a Gate 10 that runs `ruff check rpg/api/` (the
  directory the student actually implements) using the repo's root
  pyproject.toml — scoped there after confirming a full correct solution
  is clean under that scope while tests/ carries a pre-existing, unrelated
  lint error the student doesn't own. A real lint violation now fails the
  boss fight; correct code still passes.
@michalmaj
michalmaj merged commit b7dda0d into main Oct 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant