Skip to content

One submission, one Runner, and a claim nobody heard about is free - #91

Merged
kisielewski merged 6 commits into
mainfrom
fix/the-queue-review-findings
Sep 4, 2026
Merged

One submission, one Runner, and a claim nobody heard about is free#91
kisielewski merged 6 commits into
mainfrom
fix/the-queue-review-findings

Conversation

@kisielewski

Copy link
Copy Markdown
Member

Nine findings from a review of "does a submission reach one waiting Runner that handles its type". The answer to the question as asked is yes on both halves — two conformance cases prove it, each with a sabotage — and one finding made it no for a submission as opposed to a job.

The two halves hold

Type, external and tag matching is the claim's own predicate; exclusivity is FOR UPDATE OF j SKIP LOCKED held across the state write. The first test had to be written twice: the obvious shape — both Runners waiting, the capable one asserted to win — passed with the type filter defeated, because the capable Runner won the race anyway. Only the wrong Runner waits now, and the job is still there afterwards.

A submission is not a job

QueueRejudgeAsync added an attempt unconditionally, so a rejudge before the first finished left two claimable rows and SKIP LOCKED correctly handed them to two Runners — the same source judged twice, and on an External Runner the same solution submitted twice to somebody else's site. Invisible, because scoring reads the highest attempt; a passing test asserted the state as correct.

The fix is in two parts because the cases are not alike. A queued sibling is superseded outright — free, because a claim marks a job running inside the transaction that takes it, so nobody holds one. A running sibling is left alone and the new attempt waits behind it on a fourth claim filter: the keeper never aborts an evaluation, so cancelling would mean a sandbox finishing and discarding, or a UVa submission already spent. Finishing now nudges, which it did not — a completion was not work for anybody.

Sixth state Superseded rather than reusing Cancelled: nothing is released, so the only argument for reuse was the size of the change, against telling a manager somebody cancelled an attempt nobody cancelled.

A nudge reaches every waiter

The signal is captured before the look, so a submission committed while a claim is unwinding is not fired into an empty room. Four writers that queued work silently now nudge: a single rejudge (its sibling three screens down always did), reopening after a drain, retagging a Runner, activity or round, and an applied aj-admin config apply.

A claim nobody was heard from about is free

A claim is committed before its answer is written, so a lost answer left a job leased to nobody that the Runner could not release, for the full lease, having spent one of five attempts. The handout now runs on the process's lifetime rather than the request's and is undone when the caller is seen to go; what that cannot see — a black-holed connection — the reaper refunds on AcknowledgedAt being null, bounded like Releases.

Registration proves the key

POST /runner/register was anonymous for keys the Server already knew, and a public key is public by construction. Whoever had one could rewrite problemTypes and external — two of the filters above — with the approval left where it was. An existing test performed that rewrite. The nonce is signed rather than the body.

Three more

  • Reporting was an oracle: the repeat is answered before the lease, which is right, but it was answered before the owner too, so any approved Runner naming a finished job and any well-formed GUID got its result id and state. The fix guards the branch and leaves the refusals in order — moving the refusal up was the first attempt and told a Runner whose lease merely expired that the job "belongs to another Runner", when the reaper had left it belonging to nobody.
  • A trial's package outlived its lease by up to a reaper tick: the check asked only State == Running while its own comment promised the deadline.
  • Nothing recorded which Runner uploaded a file, so every Runner's uploads were one anonymous pool and any of them could attach another's. The uploader is now an argument the compiler puts to all six writers.

Plus: a unique-index violation is a 409 rather than a 500.

Merge order

AlgoJudge-Runner and AlgoJudge-External-Runner are merged; this must land after them, because a Server requiring a signed re-registration crash-loops an older Runner image and the refusal is not one a Runner retries.

808 of 810 tests, 2 skipped. Two migrations. openapi.json regenerated from the container and carrying only the two new register fields.

Two conformance cases for what a broadcast nudge could have broken. The
first was written twice: the obvious shape passed with the type filter
defeated, because the capable Runner won the race anyway.
The signal is captured before the look rather than after it, so a
submission committed while a claim is unwinding is not fired into an
empty room. Four writers that queued work silently now nudge: a single
rejudge, reopening after a drain, retagging a Runner or an activity, and
an applied configuration change.
A rejudge supersedes a queued attempt and waits behind a running one, so
two Runners can no longer judge one submission at once. A handout runs on
the process's lifetime rather than the request's and is undone when the
caller is seen to go; what that cannot catch, the reaper refunds. A
registration for a fingerprint the Server already holds must be signed.
The repeat is still answered before the lease, and now only for the
Runner whose result it is: any approved Runner naming a finished job and
any well-formed GUID was handed that job's result id and state. The
trial's file check gained the lease term its own comment had been
promising.
Files carried no uploader when a Runner put them there, because the
commit asked a session service and a Runner holds a token. So every
Runner's scratch uploads were one anonymous pool and any of them could
attach another's. The uploader is now an argument the compiler puts to
every writer, and the attach predicate asks whose it is.
@kisielewski
kisielewski merged commit e01247c into main Sep 4, 2026
3 checks passed
@kisielewski
kisielewski deleted the fix/the-queue-review-findings branch September 4, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant