feat(versioning): capture and expose version history for charts, dashboards, and datasets

[mikebridge]

SUMMARY

Adds backend plumbing to capture a version history for every save of a chart, dashboard, or dataset, and to expose that history via three new REST endpoints per entity (list, get, restore). No frontend in this PR. Second PR in the Versioning epic (sc-103156), depending on #39859 (composite-PK reshape on M2M association tables — sc-105349) and orthogonal to #39286 (sc-103157 soft-delete). See SIP-210 / issue #39492 for full design rationale.

🚧🚧🚧 This is still a draft/spike, not ready for final review. Branch contains two temp(*) commits (demo UI dropdowns + French i18n; URL-param stripping on restore navigation) that will revert before merge. 🚧🚧🚧

What changed:

• Continuum wiring. Adds sqlalchemy-continuum as a base dependency. Wired in superset/extensions/__init__.py with the validity strategy; a custom VersionTransactionFactory renames the transaction table to version_transaction (the word transaction is reserved in several dialects) and a VersioningFlaskPlugin supplies the acting user via get_user_id() (not Flask-Login's current_user) so CLI / Celery / JWT-auth API saves all attribute correctly.

• Six shadow tables, all Continuum-native:

  • parent shadows: dashboards_version, slices_version, tables_version
  • child shadows: table_columns_version, sql_metrics_version
  • M2M shadow: dashboard_slices_version

  Plus one version_transaction table (the per-flush "who/when/where" envelope) and one version_changes table (structured diff records, FK to version_transaction with ON DELETE CASCADE).

• Three endpoints per entity type (/chart, /dashboard, /dataset):

  • GET /api/v1/<resource>/<uuid>/versions/ — list history (version_number, version_uuid, issued_at, changed_by)
  • GET /api/v1/<resource>/<uuid>/versions/<version_uuid>/ — one snapshot (scalar fields; plus columns / metrics for datasets, slices for dashboards)
  • POST /api/v1/<resource>/<uuid>/versions/<version_uuid>/restore — restore entity to that version

• <version_uuid> is a deterministic UUIDv5 (fixed namespace, derived from the entity UUID + Continuum transaction id). Stable across replicas and retention pruning — the same transaction always produces the same version uuid, so API consumers can cache references safely.

• ETag headers (ETag: W/"<version_uuid>") on all three GET endpoints + the live entity GET. Foundation for optimistic-locking enforcement on writes (Phase 2); not enforced in this PR.

• Restore uses Continuum's native Reverter wrapped in a single_flush_scope context manager (suppresses autoflush inside the block, emits one trailing flush). The single-revert / single-flush shape was the spike outcome — earlier attempts at split-revert and JSON-snapshot tables were abandoned (see spike-continuum-restore.md and the revised ADR-004 in the spec folder).

• Baseline capture. First save under versioning of an entity that pre-existed the migration inserts a synthetic operation_type=0 row capturing the pre-edit state, attributed to the entity's existing changed_on / changed_by_fk. Listener runs before Continuum's own before_flush so the baseline transaction_id is lower than the edit's (correct ordering).

• No-op suppression. A SkipUnmodifiedPlugin marks Continuum Operations processed=True when post-flush column values are content-equal to the previous shadow row — including JSON-aware comparison for Dashboard.json_metadata that strips frontend-stamped audit sub-keys (map_label_colors, chart_configuration, …) so saves that only re-stamp those don't pollute history.

• Force-parent-dirty on child changes. A before_flush listener flags the versioned parent (SqlaTable) as dirty when only its versioned children (TableColumn / SqlMetric) changed, so child-only edits surface in the parent's version dropdown.

• Structured change records. Every save writes per-field diff records to version_changes keyed to the same transaction_id. Records carry kind / path / from_value / to_value — backbone for the Phase-2 UI's "Added column X" rendering, captured in V1 so the data is available from day one without a backfill.

• Retention is time-based, run by a Celery beat task. SUPERSET_VERSION_HISTORY_RETENTION_DAYS (default 90; 0 or None disables versioning entirely). Deletes shadow rows older than the cutoff while preserving the live row regardless of age. ON DELETE CASCADE on version_changes.transaction_id keeps diffs in sync. No write-path overhead; the prune is asynchronous.

• Composite PK reshape on M2M associations (sc-105349, PR refactor(db): composite PK on M2M association tables (sc-105349) #39859 — required for Continuum's M2M tracker to populate dashboard_slices_version correctly). The PRs are intended to merge in order refactor(db): composite PK on M2M association tables (sc-105349) #39859 → this one; the migration is included on this branch because the rebased history depends on it.

• Authorisation. Version endpoints reuse the resource's existing can_write permission. No new FAB permissions. Row-level access enforced via security_manager.raise_for_ownership(entity) in the restore command.

What is NOT versioned in v1 (see specs/sc-103156-entity-versioning/future-work.md):

• Tags, owners, roles — explicitly excluded from the versioned column set; restore leaves these at their live values.
• Per-chart position inside a dashboard — position_json is versioned as an opaque blob (restored wholesale on dashboard restore); finer-grained layout versioning is Phase 2.
• Auto-generated human-readable change summary text, change-type icons, search over diff content, AI attribution — all captured as Phase-2 frontend work.

Coordination with #39286 (sc-103157 soft-delete) — orthogonal in design; merge order can go either way. When sc-103157 merges, one small change hooks deleted_at into find_active_by_uuid() and the versioned models' Continuum exclude lists. Tracked as T043 in the spec.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A — backend-only. No UI is wired to the new endpoints yet. The temp(*) commits add non-final demo dropdowns and i18n strings for manual testing only; they will revert before merge.

TESTING INSTRUCTIONS

1. Save a chart a few times to generate history:

CHART_UUID=$(curl -s http://localhost:8088/api/v1/chart/?q='(page_size:1)' | jq -r '.result[0].uuid')
for i in 1 2 3; do
  curl -s -X PUT http://localhost:8088/api/v1/chart/$CHART_UUID -d "{\"slice_name\":\"Test v$i\"}" -H 'Content-Type: application/json'
done

2. List the history:

curl http://localhost:8088/api/v1/chart/$CHART_UUID/versions/ | jq

Expect an ordered array with version_number, version_uuid, issued_at, changed_by. Response will include an ETag: W/"<version_uuid>" header for the most recent version.

3. Fetch one version:

VERSION_UUID=$(curl -s http://localhost:8088/api/v1/chart/$CHART_UUID/versions/ | jq -r '.result[0].version_uuid')
curl -i http://localhost:8088/api/v1/chart/$CHART_UUID/versions/$VERSION_UUID/ | head -20

Expect 200 + ETag header. A second request with If-None-Match: "<that-etag>" returns 304.

4. Restore:

curl -X POST http://localhost:8088/api/v1/chart/$CHART_UUID/versions/$VERSION_UUID/restore

Expect 200. GET /api/v1/chart/$CHART_UUID should now reflect the restored state, and a new version row (the restore itself) appears in the version list.

5. Inspect structured diffs:

curl http://localhost:8088/api/v1/chart/$CHART_UUID/versions/$VERSION_UUID/ | jq '.result.changes'

Each change carries {kind, path, from_value, to_value}.

6. Repeat for dashboards and datasets using /api/v1/dashboard/ and /api/v1/dataset/. Datasets exercise child shadows (columns / metrics); dashboards exercise the M2M shadow (slices).

7. Run the test suite:

pytest tests/integration_tests/charts/version_history_tests.py \
       tests/integration_tests/dashboards/version_history_tests.py \
       tests/integration_tests/datasets/version_history_tests.py \
       tests/integration_tests/versioning/ -v

8. Run the performance validation harness (skipped in CI; run on demand):

SUPERSET_PERF_VALIDATION=1 pytest tests/integration_tests/versioning/perf_validation_tests.py -v -s

Asserts the three Success Criteria: list < 1 s, restore < 3 s, save p95 overhead < 50 ms.

ADDITIONAL INFORMATION

Migration list (in dependency order):

Migration	Operation	Reversible?
2bee73611e32	Composite PK reshape on dashboard_slices + 7 other association tables (sc-105349 / #39859)	Yes
56cd24c07170	Create version_transaction + parent shadow tables (dashboards_version, slices_version, tables_version)	Yes
e1f3c5a7b9d0	Create version_changes (structured diff records, ON DELETE CASCADE FK to version_transaction)	Yes
f7a2b3c4d5e6	Create child shadow tables (table_columns_version, sql_metrics_version) + M2M shadow (dashboard_slices_version)	Yes

All migrations are additive on the pre-existing slices / dashboards / tables / child tables — no existing columns altered. The composite-PK migration (2bee73611e32) reshapes the M2M association tables; round-trip tested on PostgreSQL, MySQL, and SQLite, including the MySQL FK / AUTO_INCREMENT quirks that required raw SQL workarounds (commits 56c36fde54, 65a3491861).

Write cost per save:

Table	Rows added
version_transaction	1
*_version parent shadow	0 if SkipUnmodifiedPlugin filtered the save; 1 if scalars changed
Child/M2M shadows	one per changed child/M2M entry
version_changes	one per atomic field-level change (zero for skipped saves)

No write-path retention overhead — pruning is asynchronous via Celery beat.

Performance:

The numbers below were captured before the ADR-004 reversal (JSON-snapshot → full Continuum). Architecture has changed since — child writes now go through Continuum shadows instead of dataset_snapshots / dashboard_snapshots JSON tables. Re-validation against the final architecture pending before review. Targets unchanged:

Criterion	Target
SC-002 list endpoint	< 1000 ms
SC-003 restore endpoint	< 3000 ms
SC-004 save p95 overhead	< 50 ms

Harness: SUPERSET_PERF_VALIDATION=1 pytest tests/integration_tests/versioning/perf_validation_tests.py -v -s.

• Has associated issue: sc-103156 / SIP-210 #39492
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

❌ Patch coverage is 41.22174% with 1222 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.04%. Comparing base (b0d7880) to head (df8097a).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...end/src/pages/ChartList/VersionHistoryDropdown.tsx	2.32%	126 Missing ⚠️
...src/pages/DashboardList/VersionHistoryDropdown.tsx	2.32%	126 Missing ⚠️
...d/src/pages/DatasetList/VersionHistoryDropdown.tsx	2.32%	126 Missing ⚠️
superset/tasks/version_history_retention.py	0.00%	117 Missing ⚠️
superset/versioning/diff.py	60.82%	87 Missing and 18 partials ⚠️
superset/versioning/queries.py	22.13%	95 Missing ⚠️
superset/versioning/api_helpers.py	32.05%	53 Missing ⚠️
superset/daos/dataset.py	11.32%	47 Missing ⚠️
superset/commands/version_restore.py	0.00%	36 Missing ⚠️
superset/versioning/baseline/children.py	27.65%	34 Missing ⚠️
... and 32 more

❗ There is a different number of reports uploaded between BASE (b0d7880) and HEAD (df8097a). Click for more details.

HEAD has 28 uploads less than BASE

Flag	BASE (b0d7880)	HEAD (df8097a)
python	22	3
presto	3	1
hive	3	1
unit	5	1
javascript	2	1

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #39603      +/-   ##
==========================================
- Coverage   64.30%   56.04%   -8.27%     
==========================================
  Files        2657     2686      +29     
  Lines      144060   146084    +2024     
  Branches    33216    33667     +451     
==========================================
- Hits        92641    81866   -10775     
- Misses      49797    63437   +13640     
+ Partials     1622      781     -841     

Flag	Coverage Δ
hive	39.26% <31.11%> (-0.19%)	⬇️
javascript	67.91% <3.51%> (-0.34%)	⬇️
mysql	?
postgres	?
presto	41.31% <50.14%> (+0.28%)	⬆️
python	42.59% <50.14%> (-17.14%)	⬇️
sqlite	?
unit	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	298a818
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a2b3a65211a7a000832bcc8
😎 Deploy Preview	https://deploy-preview-39603--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.