fix(worker): the cron could rebuild a repository the button never could

[sshlg]

What was measured

run_repo_index_task is the repo-index pipeline. Two ARQ jobs call it, each carrying its own ceiling:

Entry	ARQ job	Ceiling
nightly cron	run_daily_project_knowledge_sync	7200 s
"Re-index repository", POST /api/projects/{id}/index	run_repo_index	1800 s

Both numbers from indexing_runs, same repository (9 981 files):

08-25 22:00   completed      42.4 min    nightly
08-27 09:30   TimeoutError   30.0 min    manual

1800.02s ! 33f44e65d24b48bd9795717bc21b0285:run_repo_index failed, TimeoutError
  pipeline_runner.py:1468 in _run_code_symbol_embed

2 544 s of work fits under one ceiling and can never fit under the other. The repository rebuilt itself unattended at 3 a.m. and failed every time a person asked — the worse half, because the button is what an operator presses after shipping a fix.

Why it survived a previous diagnosis

AUD-0819-20 gave the job its own knob on 2026-08-19 for this exact failure, then left the default at 1800 — the value just measured as too small — with a note that a repository needing longer could say so without a code edit. Nothing in production said so; neither REPO_INDEX_JOB_TIMEOUT_SECONDS nor its sibling is set. A knob defaulting to the known-bad value moves a defect rather than closing it, and a test pinning {"timeout": 1800} as a literal made that default look deliberate.

The same day's memory fix had made it worse and the link was never drawn: EMBEDDING_UPSERT_BATCH_SIZE went 200 → 8, buying ~552 MiB with ~17 % more wall clock — spent inside the step the ceiling was cutting off. That fix's caution (raising the ceiling helps only a worker that is not swapping) was right, and its precondition is now met and measured: Standard-2X, a full rebuild with zero R14 and zero R15 (CB-OPS1, closed on that evidence).

The change

repo_index_job_timeout_seconds defaults to 3600. The invariants are tests, not comments:

• it must clear the measured rebuild with 25 % headroom;
• it must stay below daily_knowledge_sync_job_timeout_seconds, which contains it plus a DB index plus a code↔DB sync — asserted together with the fact that the containment still exists, so the ordering cannot quietly become arbitrary;
• no long job may be registered without its own knob again — the shape that produced AUD-0819-20;
• the non-positive guard still refuses to boot.

The existing registration test now compares against the setting rather than a literal.

Checked rather than assumed

The killed run's checkpoint recorded all five completed steps in indexing_checkpoint_step, so a resume skips them — the empty completed_steps column on the row is legacy and no longer written (checkpoint_service.py:10-15). No second defect there.

Verification

• pytest tests/ — full backend suite green (6 952 passed / 4 skipped / 1 xfailed locally).
• ruff format --check, ruff check, mypy app/ — clean.
• Post-deploy, per CB-OPS2: force one manual full re-index and require it to reach pipeline_end. The row is struck on that evidence, not on this merge.