Skip to content

Document randomization options and display-order capture; clarify construct tags as provenance#69

Open
countercheck wants to merge 1 commit into
mainfrom
chore/randomization-construct-tags-docs
Open

Document randomization options and display-order capture; clarify construct tags as provenance#69
countercheck wants to merge 1 commit into
mainfrom
chore/randomization-construct-tags-docs

Conversation

@countercheck

Copy link
Copy Markdown
Owner

Summary

  • Adds the randomization + display-order capture story to the design doc as FR-13 with a new §3.5 subsection (three shipped mechanisms: Page.questionsOrder, Matrix.rowsOrder, choicesOrder; per-respondent render order captured at submit, threaded to fact_response_item.display_order; static-panel + survey-level page-order shuffling deferred to §5, also tracked as api+dbt: support static panel as a structural grouping element in the publish gate #68).
  • Sharpens construct tags as provenance, not pooling in CLAUDE.md (invariant 5, silent-defaults list) and in the researcher doc (§4a now says shared construct_block never licenses pooling — parent_question_id stays the explicit opt-in).
  • Extends CLAUDE.md's silent-defaults from three commitments to four (display-order capture: don't render randomized questions/choices without writing the realized order back; don't reconstruct display_order in SQL).
  • Adds patterns/randomization.json — page-level questionsOrder, matrix-level rowsOrder tagged with construct_block: "phq9" (exercises the matrix-container inheritance from §3.5), and a choicesOrder radiogroup with a downstream visibleIf showing value-based gating is order-invariant.

Pure docs + one new pattern fixture; no runtime/dbt/schema code touched. The capture model (shown_questions render-ordered + new shown_choice_orders jsonb on raw_responses + fact_response_item.display_order) is specified, not yet wired — wiring is a follow-up branch.

Why these specific design choices

  • shown_questions becomes definitively render-ordered (no new column). Forward-only spec change: pre-FR-13 responses have definition order = render order by construction.
  • shown_choice_orders (NEW jsonb on raw_responses), sparse, only for randomized choices. Backfill hook per A-2 — capture cheap now, surface in marts later.
  • display_order lands on fact_response_item, not fact_response (per @user choice). Same value across all option rows for the same (respondent, question_id, occurrence) — harmless denormalization across fan-out.
  • Block-aware randomization is emergent, not a new flag — a scale as a matrix with rowsOrder: 'random' and construct_block tag (inherited to row sub-questions per existing §3.5 rule) gives the "PHQ-9 items shuffle among themselves, stay contiguous" shape for free.
  • §4.11 / §4.12 reject SQL reconstruction of display order and recording randomization config on dim_question_version — same reasoning as routing reconstruction (§4.5) and definition-snapshot single-source.

What's intentionally deferred (design doc §5)

Test plan

  • uv run pytest api/tests/test_patterns.py — 37/37 pass (includes the new randomization.json through validate_definition + the round-trip oracle).
  • python3 scripts/check_invariants.py — all four invariants clean.
  • uv run ruff check . — passes.
  • CI: confirm the full suite (pre-commit, mypy, dbt, frontend) stays green.
  • Reviewer: sanity-check the shown_choice_orders shape and the display_order denormalization-onto-fan-out call — these are the load-bearing capture-model choices.

🤖 Generated with Claude Code

…struct tags as provenance

Design doc + companion docs gain a randomization-and-display-order story alongside
a sharper restatement of construct_block/_item as provenance (not a pooling key).
No runtime/dbt code changes — this is the design/spec layer for FR-13 and the
operational discipline that goes with it.

What the design doc says now (§3.5 "Randomization and display-order capture"):

- Three shipped mechanisms: Page.questionsOrder, Matrix.rowsOrder, choicesOrder.
  Block-aware randomization emerges from putting a scale in a matrix tagged with
  construct_block (no separate flag).
- Per-respondent render order captured at submit time. shown_questions clarified
  as render-ordered; new shown_choice_orders jsonb on raw_responses (sparse,
  only for randomized-choice questions). Threads to fact_response_item.display_order.
- Within-page / within-matrix / within-question randomization is order-invariant
  against visibleIf (values, not positions), so no new publish-gate lint there.
- §4.11 rejects SQL reconstruction of display order (same logic as routing, §4.5).
  §4.12 rejects recording randomization config on dim_question_version (the
  snapshot already has it; duplication risks drift).
- §5 deferrals: choice-order marts surface, static-panel container in the gate,
  survey-level page-order randomization.
- §6 adds undetected-order-effects + future page-order-vs-cross-page-visibleIf
  risks.

Construct tags get sharper language:

- §3.5 "Construct membership" already covered provenance-not-pooling; this
  change extends the same explicitness into CLAUDE.md (invariant 5 + silent-
  defaults) and the researcher doc (§4a now spells out that a shared construct
  tag never licenses pooling — it surfaces candidates).

CLAUDE.md gets the operational discipline:

- Invariant 3 extended to cover display-order capture.
- Silent-defaults list grows from three judgments to four (adds display-order
  capture: don't render randomized questions/choices without writing the
  realized order back; don't reconstruct display_order in SQL).
- Publish-gate + "Don't without asking" updated for the shipped surface vs the
  deferrals.

Researcher doc:

- "Three commitments" → "four commitments" (new §4d on display-order capture).
- §7 documents fact_response_item.display_order usage; §8 notes the three §5
  deferrals.

Pattern library:

- New patterns/randomization.json — page-level questionsOrder, matrix-level
  rowsOrder tagged with construct_block: "phq9" (exercises the matrix-container
  inheritance rule from §3.5), and a choicesOrder radiogroup with a downstream
  visibleIf to demonstrate value-based gating is order-invariant. Passes the
  publish gate (validate_definition + the round-trip oracle) via
  test_patterns.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant