Skip to content

Presume a deadlock only with a holder that held through a grace period - #170

Open
benh wants to merge 2 commits into
mainfrom
txn-wait-die-turnover
Open

Presume a deadlock only with a holder that held through a grace period#170
benh wants to merge 2 commits into
mainfrom
txn-wait-die-turnover

Conversation

@benh

@benh benh commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

A follow-up to #166. The presumed-deadlock check now judges only a holder that has held the lock for a whole grace period while the waiter waited, so a transaction queued behind a lock that is changing hands is no longer asked to retry.

The problem

Wait-die's check ran each time a grace period elapsed and looked at whoever held the lock at that instant, with no memory of earlier checks. It could not tell "this older transaction has held the lock the whole time I have waited" from "the lock has changed hands three times and an older transaction happens to hold it right now". So on a busy state, a younger transaction queued in FIFO order behind a run of short older holders waited past one grace period, saw an older holder at its check, and died with TransactionShouldRetry, reason PRESUMED_DEADLOCK, although the lock was turning over and it was about to be granted. Under load that is exactly the transaction that gets killed repeatedly, and the carried age only helps once its retry is older than the new arrivals. The sibling check in claim_ownership had the same blind spot.

The change

_presume_deadlock_on_grace builds the on_grace hook for a lock wait. It snapshots the lock's holders as the wait begins and, each time a grace period elapses, judges only the holders present both now and at the previous snapshot, then replaces the snapshot. A holder in both has held the lock for a whole grace period while the waiter waited, which is what a deadlock looks like, and the existing presumption applies: if it is older, the waiter dies. A holder in only the current set arrived since the last check, so the lock is moving and the waiter keeps waiting. Both lock-wait sites, the acquire in _transaction_participant_start and the upgrade in _upgrade_lock, use it, and _abort_if_presumed_deadlock takes the holders to judge rather than reading the lock itself.

claim_ownership does the same with the owner id chain: its hook remembers the chain at the previous check and runs the sibling check only when the chain is unchanged, so ownership moving between nested transactions keeps the caller waiting.

Detection latency for a real deadlock is unchanged. The first snapshot is taken as the wait begins, so a holder that is stuck on the waiter is in both sets at the first check and is caught one grace period in, as before.

Tests

  • New in transaction_tests: four older roots each hold a General state's exclusive lock for most of a grace period, queued in order, and a youngest root queues behind them. The youngest waits out more than a grace period, is granted, and is never asked to retry. Before this change it died at its first check.
  • The existing deadlock tests, two opposite transfers and two nested siblings, still resolve in well under the deadline, which shows the first check still catches a real deadlock.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EK2ybbxpNK5L7S5exgJ8fq

@aviator-app

aviator-app Bot commented Sep 13, 2026

Copy link
Copy Markdown

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue-ready label.


See the real-time status of this PR on the Aviator webapp.
Use the Aviator Chrome Extension to see the status of your PR within GitHub.

@github-actions

Copy link
Copy Markdown
Contributor

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Wait-die's check ran each time a grace period elapsed and judged
whoever held the lock at that instant, with no memory of earlier
checks. It could not tell an older transaction that had held the lock
the whole time the waiter waited from a lock that had changed hands
several times and happened to be held by an older transaction just
then. So on a busy state a younger transaction queued behind a run of
short older holders waited past one grace period, saw an older holder
at its check, and died with `TransactionShouldRetry`, reason
`PRESUMED_DEADLOCK`, although the lock was turning over and it was
about to be granted.

The hook for a lock wait now snapshots the lock's holders as the wait
begins and, each time a grace period elapses, judges only the holders
present both now and at the previous snapshot. A holder in both has
held the lock for a whole grace period while the waiter waited, which
is what a deadlock looks like, and the presumption applies as before.
A holder in only the current set arrived since the last check, so the
lock is moving and the waiter keeps waiting. `claim_ownership` does the
same with the owner id chain, running the sibling check only when the
chain is unchanged since the previous grace period.

Detection latency for a real deadlock is unchanged: the first snapshot
is taken as the wait begins, so a holder stuck on the waiter is in
both sets at the first check.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK2ybbxpNK5L7S5exgJ8fq
@benh
benh force-pushed the txn-wait-die-turnover branch from e315858 to 934e478 Compare September 13, 2026 20:01
Two shared holders that both try to upgrade are a deadlock by
construction: each waits for the other's shared hold to go. The lock
already refused the second upgrade at once, but with a plain
`Unavailable`, so the aborted transaction's retry carried no age, was
the youngest again, and backed off first as if the cause were load.

The lock now refuses it with `TransactionShouldRetry`, reason
`PRESUMED_DEADLOCK`, carrying the transaction's age, and logs which
transaction it lost to, the same way a deadlock found after the grace
period is reported; a caller that passed no participant is still
refused with `Unavailable`. The second to ask is still the one to go;
choosing the younger would mean aborting the upgrader already parked
in the lock from outside, which is the wound-wait machinery this
design leaves out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EK2ybbxpNK5L7S5exgJ8fq
@benh
benh force-pushed the txn-wait-die-turnover branch from 934e478 to e34a7dc Compare September 13, 2026 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant