Skip to content

easypg exclude unique and foreign keys (falls back to all columns) - #363

Draft
wagnerd3 wants to merge 1 commit into
masterfrom
easypg_unique_constraint_snapshots
Draft

wagnerd3 wants to merge 1 commit into
masterfrom
easypg_unique_constraint_snapshots

Conversation

@wagnerd3

Copy link
Copy Markdown
Contributor

Follows this comment, where we got fed up with the fact that updates of the UNIQUE key cause DELETE+INSERT to be rendered instead of UPDATE: sapcc/limes#952 (comment)

@majewsky majewsky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this change in principle, but it has some corner cases where the new diffs are truncated a bit too much in my taste. For example, from running Keppel tests with this change applied in vendor/:

--- /tmp/easypg-diff1879045853/expected	2026-09-17 13:36:28.228011312 +0200
+++ /tmp/easypg-diff1879045853/actual	2026-09-17 13:36:28.228011312 +0200
@@ -1,3 +1,3 @@
 UPDATE manifests SET gc_status_json = '{"protected_by_subject":"sha256:b64a08e1f12b9283b59473a4cbacb51f1b744426e0ec9abdb7cdab6db55a8075"}' WHERE repo_id = 1 AND digest = 'sha256:4710c53d191f0016fd3f84340550e38652bf9c1c8284c133757b33509c8cd78d';
 UPDATE manifests SET gc_status_json = '{"relevant_policies":[{"match_repository":".*","time_constraint":{"on":"pushed_at","older_than":{"value":2,"unit":"h"}},"action":"delete"}]}' WHERE repo_id = 1 AND digest = 'sha256:b64a08e1f12b9283b59473a4cbacb51f1b744426e0ec9abdb7cdab6db55a8075';
-UPDATE repos SET next_gc_at = 7200 WHERE id = 1 AND account_name = 'test1' AND name = 'foo';
+UPDATE repos SET next_gc_at = 7200 WHERE id = 1;

The additional fields shown on the red side (coming from a UNIQUE (account_name, name) constraint) add useful context without being overtly verbose.

I propose adding a method (*Tracker) ConsiderColumnsAsKey(tableName string, columnNames ...string) (feel free to bikeshed the name) which could be called as tr.ConsiderColumnsAsKey("repos", "account_name", "name") when creating the tracker.

Or alternatively, tr.ConfigureTable("repos").ConsiderColumnsAsKey("account_name", "name") with a helper type in between just to make the API read well.

Comment thread easypg/snapshot.go Outdated
@wagnerd3

wagnerd3 commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

In this case, I would rather propose to offer a function tr.ConsiderUniqueConstraintAsKey, have this read automatically from the information_schema and just switch which columns are used to the old behavior? Are there any cases, where you would not want this to be used with the unique columns?

@majewsky

Copy link
Copy Markdown
Contributor

In this case, I would rather propose to offer a function tr.ConsiderUniqueConstraintAsKey, have this read automatically from the information_schema and just switch which columns are used to the old behavior? Are there any cases, where you would not want this to be used with the unique columns?

That would work, though it should be named ConsiderUniqueConstraintForKey because non-overlapping primary key fields should also count, like in the example shown.

@wagnerd3
wagnerd3 force-pushed the easypg_unique_constraint_snapshots branch from 7c567ff to d1d7ad5 Compare September 18, 2026 13:03
@wagnerd3
wagnerd3 requested a review from majewsky September 18, 2026 13:05
@wagnerd3
wagnerd3 force-pushed the easypg_unique_constraint_snapshots branch 2 times, most recently from 2baccaf to 19958d2 Compare September 18, 2026 13:07
@wagnerd3

Copy link
Copy Markdown
Contributor Author

During implementation, I realized that I would not like a function Tracker.ConsiderUniqueConstraintForKey for 2 reasons:

  • NewTracker returns Tracker, Assertable for the initial database status.o in order to create a tracker with UniqueConstraintForKey, one would always have to do

     tr, _ := NewTracker(...)
     tr, tr0 := tr.ConsiderUniqueConstraintForKey
    

    which I find a bit cumbersome

  • Also, you would not really want to switch ConsiderUniqueConstraintForKey in my eyes. Either, you want to set this at creation of the tracker, or not.

So I came up with the TrackerSetupOption interface and a variadic argument to NewTracker, which keeps the signature downwards compatible.

Please let me know your feedback.

Comment thread easypg/snapshot.go
// returned.
func (d dbSnapshot) ToSQL(prev dbSnapshot) string {
tableNames := make([]string, len(d))
tableNames := make([]string, 0, len(d))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My Copilot has flagged this, this did not cause wrong output but was still wrong.

@wagnerd3
wagnerd3 force-pushed the easypg_unique_constraint_snapshots branch from 19958d2 to 60b6fea Compare September 18, 2026 13:16
@wagnerd3
wagnerd3 force-pushed the easypg_unique_constraint_snapshots branch from 60b6fea to fe33427 Compare September 18, 2026 13:17
@github-actions

Copy link
Copy Markdown

Merging this branch will not change overall coverage

Impacted Packages Coverage Δ 🤖
github.com/sapcc/go-bits/easypg 0.00% (ø)

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/sapcc/go-bits/easypg/snapshot.go 0.00% (ø) 0 0 0
github.com/sapcc/go-bits/easypg/testhelpers.go 0.00% (ø) 0 0 0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants