Story #2609 :: Task: Run the achievement backfill from the weekly release pipeline

[herzog0]

Issue: #2609

⚠️ Base branch is teo/2541-source-library-review

Summary & Context

Lets the achievement backfill run unattended. This is deliberately the last step of the ingestion
half: by now every source is wired and reviewed, and the previous eight PRs can all be exercised
by hand first.

• Figma link: n/a
• Link to components/page: n/a

Changes

• release_tasks: a Backfilling achievements action, last in the list, so every source it
  reads has already been refreshed by the steps above it.
• update_authors_and_maintainers: backfills only the three sources whose upstream data that
  task just changed (library-authoring, library-maintenance, library-versioning). A blanket
  backfill would also sweep the commit and review tables it never touches.
• update_commits: a comment recording why there is deliberately no backfill call there - it
  runs as a step of release_tasks, which sweeps everything once at the end, so calling it here
  too would walk the whole Commit table twice per release.
• libraries/tests/test_tasks.py: three tests asserting the wiring, which was previously three
  untested call_command lines.

‼️ Risks & Considerations ‼️

• This is the PR that makes badges appear in production without anyone pressing anything.
  Everything it runs is additive - backfill_achievements is sync_source with remove=False
  and cannot delete a grant or revoke a badge - so the failure mode is over-granting, not
  data loss.
• Blocking product decision: the seeded thresholds are the original numbers, not the revised
  set. It must be settled before this runs for real, because backfill
  awards badges against whatever thresholds are live at that moment, and lowering a threshold
  afterwards is easy while raising one is not (retiring and replacing a tier keeps the members who
  already met the old number). Tracked by the threshold-confirmation ticket.
• Cost: one full walk of every source per release. The commit table is the long pole. Measured at
  a few seconds against a full copy of the Boost data.
• If the catalogue is not seeded, a scheduled sweep reports the missing slugs on stderr and keeps
  going rather than failing the release job. Only an explicitly named missing source is fatal.

Screenshots

n/a - no UI.

Peer-review testing steps

This PR adds no UI of its own; what it changes is what three existing admin buttons do afterwards, and
the sync run log is where you see it. Setup: just load_production_data, just migrate,
docker compose up, and note the current last row id on /admin/badges/achievementsyncrun/ so you can
tell new rows from old ones.

1. The narrow path. /admin/libraries/library/ -> Update Authors & Maintainers. When the
   worker finishes, /admin/badges/achievementsyncrun/ has exactly three new rows -
   library-authoring, library-maintenance, library-versioning - and no code-commits or
   library-review row. That absence is the review: this task only touches library authorship, so a
   blanket backfill would walk the commit and review tables for nothing.
2. Those rows read command with no actor, because the libraries changelist buttons are the eight
   legacy GET views that were deliberately left unconverted. Worth knowing when reading the log; not
   something this PR changes.
3. The deliberate omission. /admin/libraries/commit/ -> Update Commits. No new sync run row
   appears. That is intentional and commented in the code: this task also runs as a step of
   release_tasks, which sweeps everything once at the end, so backfilling here too would walk the
   whole Commit table twice per release.
4. The full pipeline. /admin/libraries/releasereport/ -> Get Release Report Data, which is the
   release_tasks entry point. Only against a scratch database with GitHub credentials configured: it
   imports versions, libraries, commits, issues, Slack and mailing-list data first, and its own tooltip
   warns it can take hours. When it lands, the log has one row per source with trigger pipeline, all
   timestamped after the import steps. Ordering matters - a backfill that ran before the imports would
   grant against yesterday's data, which is why it is the last action in the list.
5. It cannot revoke anything. Across every row those runs produced, Removed is 0 and Members
   changed only ever counts additions. Then filter /admin/badges/userbadge/ to Revoked and
   confirm nothing was revoked while the pipeline ran. backfill_achievements is sync_source with
   remove=False, so the failure mode here is over-granting, never data loss.
6. A misconfigured badge does not fail the release either. Tick delete on every tier of one badge on
   /admin/badges/badge/ - the ladder column then reads No tiers - awards nothing - and run
   Update Authors & Maintainers again. Grants are still recorded, no badge is awarded, and the
   other sources complete normally. (The stronger case, a source whose Achievement row is missing
   entirely, is not reachable from the admin on purpose: achievement deletion is refused. It is covered
   in tests, and the behaviour is stderr-and-continue so a taxonomy mistake cannot kill the release job.)
7. Before this is allowed to run in production, read the ladders on /admin/badges/badge/ and
   confirm each one is the number the client intends. The sweep awards against whatever is live at that
   moment, and lowering a threshold later is easy while raising one is not.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 251c71e2-3bfa-41f8-bcea-b755147a9dbd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[jlchilders11]

One major concern with the release tasks, which I have been bitten by before. Happy to discuss solutions!

[jlchilders11]

For actions which call management commands, the Action returns and the Manager moves on once the managment command is called. For commands that then call tasks, this is instantaneous. This means that we have no real guarantee that the previous actions have finished resolving before this is called (and in fact likely are not, since the celery worker is weaker than the main thread).