fix: don't sum row counts across shards for EXECUTE on omnisharded tables - #1179
fix: don't sum row counts across shards for EXECUTE on omnisharded tables#1179mscrivo wants to merge 2 commits into
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
This touches the parser, which we are in the middle of a major rewrite of. This isn't likely to get reviewed until after that is done (we are hoping next week, but take that with a huge grain of salt) |
e423f26 to
33100b3
Compare
91017a3 to
32f1781
Compare
cefcf98 to
4fee48b
Compare
| /// SQL-level `PREPARE` statements sent over the simple protocol, | ||
| /// name -> statement text. Tracked so `EXECUTE` can be routed | ||
| /// based on the statement behind the name. | ||
| pub(super) simple: HashMap<String, String>, |
There was a problem hiding this comment.
I'm not sure this needs its own dedicated cache. You could extract the statement from PREPARE and store it in the global cache, e.g.:
PREPARE __sqlx_1 AS SELECT $1;when extracted becomes:
| Name | Statement |
|---|---|
__sqlx_1 |
SELECT $1 |
You can create a Parse from this and insert it into the regular cache, e.g.:
let parse = Parse::named("__sqlx_1", "SELECT $1");
prepared_statements.insert(&mut parse);When capturing EXECUTE, you can now fetch the original statement by calling:
let name = /* get name from EXECUTE using parser */;
prepared_statements.parse(name);We do this already in the rewriter here:
There was a problem hiding this comment.
thanks Lev! I'm reworking this now since I found some other issues as well. Sorry for the churn
| } | ||
|
|
||
| fn statement_omnisharded(query: &str, schema: &ShardingSchema) -> bool { | ||
| let Ok(ast) = pg_raw_parse::parse(query) else { |
There was a problem hiding this comment.
This should be using the statement cache, so we don't have to parse the statement on each execution.
|
It's a good start, but I think the correct solution here is to handle |
…bles EXECUTE of a server-side prepared statement was routed through the DDL catch-all, which broadcasts to all shards without the omnisharded flag. For statements that only touch omnisharded tables, the CommandComplete row counts from each shard were summed, so a DELETE/UPDATE of N rows reported N * shards rows to the client. Route PREPARE explicitly and store the statement behind the name in the prepared statements cache. EXECUTE resolves the name and parses the statement through the statement cache (once per unique statement, not per execution). Writes that only touch omnisharded tables carry the omnisharded flag, so cross-shard results are deduplicated, not aggregated. Reads are not flagged: EXECUTE always routes as a write, and an omnisharded write requires full shard coverage, which would reject shard directives on statements that can't diverge the shards. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers writes vs reads behind the name, VALUES, shard directives on both (allowed on reads, rejected on omnisharded writes), sharded tables and unknown statement names. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4fee48b to
1ad0939
Compare
@levkk I'll leave it with you and @sgrif at this point. I don't have enough context with the codebase or experience with rust, so I'm in over my head at this point :) It does demonstrate the outcome I'd like to see, so if you all close this PR and rework it entirely, that'd be fine with me as well. |
I'm not opposed to merging this (with maybe some minor changes) to unblock you guys! Let me know how soon you need this and we'll prioritize. We can always add support for sharded |
as soon as you're able I guess. We're not in any particular rush, but the problem on our end is that there's still a rather large bucket of test failures we're dealing with and so it makes it difficult to triage and categorize them. ie are they from this issue, another fixed or still open issue, etc. So the more we knock out, the easier that process becomes. |
Problem
EXECUTEof a server-side prepared statement targeting an omnisharded table returned the sum of row counts from all shards instead of the count from one shard:Plain
UPDATE/DELETE/INSERTon omnisharded tables already report the correct count (#752), butExecuteStmtfell through to the DDL catch-all, which broadcasts withRoute::write(Shard::All)and no omnisharded flag — soMultiShardaggregated theCommandCompletecounts.Fix
PREPAREexplicitly: still broadcast to all shards, but extract the statement behind the name and store it in the regular prepared statements cache (Parse::named(name, statement)+PreparedStatements::insert()), the same way theprepared_statements = "full"rewriter does. First PREPARE wins, matching the server, where a duplicate PREPARE fails.EXECUTEexplicitly: resolve the name viaPreparedStatements::parse()and inspect the statement through the statement cache, so each unique statement is parsed once, not per execution. If it's a write that only touches omnisharded tables, the route carries the omnisharded flag andMultiSharddeduplicates results across shards instead of aggregating them.EXECUTEare not flagged:EXECUTEalways routes as a write (the statement only exists on the connectionsPREPAREreached), and an omnisharded write requires full shard coverage — flagging reads would reject shard directives on statements that can't diverge the shards.RouterContext::with_prepared_statements().This also extends the omni-write-in-direct-to-shard-transaction guard (#1086) and the omni-write-with-directive guard to
EXECUTEof omnisharded writes, since both key off the same route flag.Testing
parser/query/test/test_execute.rs(fail without the fix): omnisharded UPDATE/DELETE/INSERT behindEXECUTEcarry the flag; SELECT/VALUES/sharded/unknown names don't; shard directives are allowed on reads and rejected on omnisharded writes.EXECUTE upd('x')returnsUPDATE 3instead ofUPDATE 6, and the write still reaches every shard. Same forDELETE.🤖 Generated with Claude Code