Skip to content

Drop auction_prices table - #4852

Draft
jmg-duarte wants to merge 1 commit into
mainfrom
jmgd/drop
Draft

Drop auction_prices table#4852
jmg-duarte wants to merge 1 commit into
mainfrom
jmgd/drop

Conversation

@jmg-duarte

Copy link
Copy Markdown
Contributor

Description

Drops the now unused auction prices table, replaced by competition auctions, should save ~370GB (at least)

Changes

  • Drop the auction_prices table

@jmg-duarte
jmg-duarte requested a review from a team as a code owner September 2, 2026 16:49
@jmg-duarte
jmg-duarte marked this pull request as draft September 2, 2026 16:49
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Reminder: Please update the DB Readme and comment whether migrations are reversible (include rollback scripts if applicable).

  • If creating new tables, update the tables list.
  • When adding a new index, consider using CREATE INDEX CONCURRENTLY for tables involved in the critical execution path.
  • For breaking changes, remember that during rollout k8s starts the new autopilot, runs the Flyway migration, and only then shuts down the old pod. That overlap means the previous version can still be processing requests on the migrated schema, so make it compatible first and ship the breaking DB change in the following release.

Caused by:

@claude

claude Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Claude finished @jmg-duarte's task in 1m 15s —— View job


PR Review: Drop auction_prices table

  • Read the migration
  • Verify auction_prices is still referenced in code
  • Trace the write path
  • Post review

Verdict: ⚠️ Blocker found — see the inline comment.

The table is still being written to on the autopilot hot path. save_competition inserts into auction_prices on every auction:

run_loop.rs:642persistence::save_competitioncompetition.rs:51 auction_prices::insertINSERT INTO auction_prices (crates/database/src/auction_prices.rs:26).

That insert sits inside a try_join! propagated with ?, so after the migration runs it fails with relation "auction_prices" does not exist and rolls back competition post-processing for every auction. Migrations apply independently of the binary rollout, so this breaks regardless of deploy order.

Additionally, the in-repo deprecation note (crates/database/src/auction_prices.rs:1-3) says the table "can't currently be removed, since the solver team is still using it" — worth confirming that's stale (plus any analytics/replica readers) before dropping.

Suggested path: first land a PR removing the insert call site (competition.rs) and the insert/fetch helpers, then drop the table. The read helpers already query competition_auctions, so only the write side needs removing.
· branch jmgd/drop

@@ -0,0 +1,3 @@
-- Drop the `auction_prices` table. Its data duplicated the `price_tokens` /
-- `price_values` arrays of `competition_auctions`, which every reader now uses.
DROP TABLE IF EXISTS auction_prices;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: the table is still being written to. autopilot's save_competition still inserts into auction_prices on every auction:

  • crates/autopilot/src/database/competition.rs:51database::auction_prices::insert(...)INSERT INTO auction_prices (...) (crates/database/src/auction_prices.rs:26)
  • This runs on the run-loop hot path (crates/autopilot/src/run_loop.rs:642), inside a try_join! whose error is propagated with ?.

Once this migration runs, that INSERT will fail with relation "auction_prices" does not exist, rolling back the whole competition-save transaction and failing post-processing for every auction. Because migrations apply before/independently of the new binary rolling out, this breaks autopilot regardless of deploy ordering.

The write path (insert in crates/database/src/auction_prices.rs and its call site in competition.rs) must be removed before or together with dropping the table.

Also note the deprecation comment in crates/database/src/auction_prices.rs:1-3 states the table "can't currently be removed, since the solver team is still using it." If that's stale, please confirm with the solver team (and any analytics/replica consumers) before dropping — otherwise it's an external breakage too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Why did you decide to apply this change on top of main instead of the PR series that actually removes and refactors the related code?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to do a bit of git surgery on the other branches, I was just tired of it so I just took the code out and placed it outside. Eventually it gets merged to main anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants