transform() now works for tables referenced by views

[simonw]

Closes:

• transform() should work for tables that are referenced by views #831

Claude Fable 5 (Claude Code for web) PR

What changed

transform_sql() now brackets its ALTER TABLE ... RENAME TO statements with PRAGMA legacy_alter_table=ON, followed by a closing pragma that restores the value the connection had when the SQL was generated (usually OFF, the SQLite default):

CREATE TABLE "t_new_x123" (...);
INSERT INTO "t_new_x123" (...) SELECT ... FROM "t";
DROP TABLE "t";
PRAGMA legacy_alter_table=ON;
ALTER TABLE "t_new_x123" RENAME TO "t";
PRAGMA legacy_alter_table=OFF;

In the keep_table= variant the pragma brackets both renames.

Why

Since SQLite 3.25, ALTER TABLE ... RENAME TO validates and rewrites references to the renamed table in every view definition. This caused two bugs:

1. Transform failed outright when a view referenced the table being transformed - the rename step raised error in view v: no such table: main.t, breaking every transform-based operation (retyping, renaming, dropping columns, changing primary keys, foreign key changes).
2. keep_table= silently repointed views at the backup table - when the rename succeeded, SQLite rewrote dependent views to select from the frozen backup copy instead of the live table, with no error or warning.

These renames are an internal implementation detail of transform(), so view definitions should not be touched. The pragmas are emitted from transform_sql() rather than handled inside transform(), so the documented "run these statements yourself" workflow and sqlite-utils transform --sql output remain standalone-correct.

Resulting semantics

• Views survive transform() with their SQL byte-for-byte unchanged, and keep reading from the live table even with keep_table=.
• A view referencing a column the transform renamed or dropped remains defined but raises no such column when next queried - inherent to SQLite views, whose SQL is stored as text.
• PRAGMA legacy_alter_table is restored to the value it had before the transform, noted in the docs.
• No new parameters, no API changes.

Reviewer notes

• Tests: seven new tests in tests/test_transform.py covering view preservation across rename/retype/pk-change/FK-change transforms, the renamed-column caveat, a view-on-view chain, the keep_table repointing regression, standalone execution of transform_sql() output, and transform inside an already-open transaction. The 13 parametrized expected-SQL lists gained the two pragma lines. Full suite: 1403 passed, 16 skipped.
• Docs: new "Tables referenced by views" section in docs/python-api.rst, cross-reference and updated --sql example in docs/cli.rst, changelog entry (in an "Unreleased" section - fold into the next release heading when cutting it).
• Also fixed a pre-existing missing blank line in docs/changelog.rst (before the .. _v4_1: label) that broke strict Sphinx builds.

🤖 Generated with Claude Code

---

📚 Documentation preview 📚: https://sqlite-utils--832.org.readthedocs.build/en/832/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.55%. Comparing base (a7b7349) to head (b8ed80b).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #832   +/-   ##
=======================================
  Coverage   95.55%   95.55%           
=======================================
  Files           9        9           
  Lines        3800     3805    +5     
=======================================
+ Hits         3631     3636    +5     
  Misses        169      169           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[simonw]

One consequence is that PRAGMA legacy_alter_table is reset to OFF (the SQLite default) after a transform, even if it was previously set to ON for the connection.

We can do better than that: spot what the previous value is at the start of the transform and set it back to that at the end.

[simonw]

Got a test failure against older SQLite 3.23.1 (prior to PRAGMA legacy_alter_table being added in 3.26):

    def test_transform_restores_legacy_alter_table_setting(fresh_db):
        dogs = fresh_db["dogs"]
        dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
        # Default is OFF, reset to OFF afterwards
        dogs.transform(types={"name": str})
>       assert fresh_db.execute("PRAGMA legacy_alter_table").fetchone()[0] == 0
E       TypeError: 'NoneType' object is not subscriptable