Skip to content

Commit 70b078a

Browse files
committed
ci(board-snapshot): mirror the archive to Cloudflare R2 after the branch push
The archive branch survives an account suspension but not an action against the repository itself. Add one step after the archive commit that mirrors the archive checkout to an R2 prefix with --delete, and writes one tarball per UTC day outside that prefix, since R2 has no bucket versioning and a --delete sync keeps no history. Until the four repository secrets exist the step prints one notice and exits 0; a configured upload that fails goes red. pull_request runs never upload, the same guard the archive commit already carries. Claude-Session: https://claude.ai/code/session_01DAcomhvR9kKizeYgg89Vo8 Co-authored-by: Claude <noreply@anthropic.com>
1 parent 65767d2 commit 70b078a

1 file changed

Lines changed: 109 additions & 0 deletions

File tree

.github/workflows/board-snapshot.yml

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,113 @@ jobs:
274274
echo "committed=${files}" >> "$GITHUB_OUTPUT"
275275
echo "pushed ${files} changed file(s) to board-archive"
276276
277+
# ## The copy that outlives the platform, not just the account
278+
#
279+
# The archive branch above survives an ACCOUNT suspension, because a
280+
# branch belongs to the repository rather than to a user. It does not
281+
# survive an action against the organisation or the repository itself,
282+
# and that is the remaining hole. The maintainer's answer, verbatim and
283+
# untranslated, in chat on 2026-09-13:
284+
#
285+
# 「那如果推 s3 呢?」 「Cloudflare R2」
286+
#
287+
# So this step mirrors the archive CHECKOUT — the same tree the commit
288+
# above just pushed — into one R2 prefix. Two things follow from R2
289+
# having NO bucket versioning:
290+
#
291+
# - `--delete` makes the prefix equal to the branch tip and nothing
292+
# else, so a restore never reads a file the board no longer has;
293+
# - a mirror therefore keeps no HISTORY at all, so once per UTC day the
294+
# tree is also written as `snapshots/YYYY-MM-DD.tar.gz`. That key
295+
# lives OUTSIDE the mirrored prefix on purpose: inside it, the very
296+
# next `--delete` sync would remove it as an object with no local
297+
# counterpart, and the history would be exactly one day long.
298+
#
299+
# The daily tarball is decided by asking the bucket for today's key
300+
# rather than by the clock, so a missed 02:07 run, a re-run and a
301+
# `workflow_dispatch` all converge on one tarball per day.
302+
#
303+
# ## Unset secrets are a notice, not a failure
304+
#
305+
# The bucket and its scoped API token are the maintainer's to create.
306+
# Until all four secrets exist this step prints one notice and exits 0,
307+
# so this workflow keeps archiving to the branch and nothing here has to
308+
# be merged in the same hour as the bucket. A CONFIGURED upload that then
309+
# fails is the opposite case and goes red: a backup that silently stops
310+
# copying is the failure mode this whole file exists to prevent.
311+
#
312+
# ⛔ The four secrets are read, never echoed, and never written to the
313+
# summary; the run below also never touches the board, `main`, or the
314+
# archive branch — it only reads the checkout that is already on disk.
315+
- name: Upload the archive to R2
316+
id: r2
317+
if: github.event_name != 'pull_request'
318+
env:
319+
AWS_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
320+
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
321+
# R2 is S3-compatible with one fixed pseudo-region.
322+
AWS_DEFAULT_REGION: auto
323+
# S3-compatible endpoints other than S3 itself are the documented
324+
# case for this setting: the CLI then sends an integrity checksum
325+
# only where the API requires one, instead of on every request.
326+
AWS_REQUEST_CHECKSUM_CALCULATION: when_required
327+
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
328+
R2_BUCKET: ${{ secrets.R2_BUCKET }}
329+
run: |
330+
set -euo pipefail
331+
332+
if [ -z "${AWS_ACCESS_KEY_ID:-}" ] || [ -z "${AWS_SECRET_ACCESS_KEY:-}" ] \
333+
|| [ -z "${R2_ACCOUNT_ID:-}" ] || [ -z "${R2_BUCKET:-}" ]; then
334+
echo "status=skipped — no R2 credentials configured in this repository" >> "$GITHUB_OUTPUT"
335+
echo "::notice::R2 upload skipped: R2_ACCOUNT_ID, R2_BUCKET, R2_ACCESS_KEY_ID and R2_SECRET_ACCESS_KEY are not all set as repository secrets; this run is archived on the board-archive branch only."
336+
exit 0
337+
fi
338+
339+
endpoint="https://${R2_ACCOUNT_ID}.r2.cloudflarestorage.com"
340+
mirror="s3://${R2_BUCKET}/objectstack/board/"
341+
342+
# `archive/` is a linked worktree, so its `.git` is a FILE and the
343+
# pattern `.git/*` does not match it; both spellings are excluded.
344+
# No pipe between the command and `$?` — see the snapshot step.
345+
set +e
346+
aws s3 sync archive/ "$mirror" \
347+
--endpoint-url "$endpoint" \
348+
--delete --no-progress \
349+
--exclude '.git' --exclude '.git/*' \
350+
> "$RUNNER_TEMP/r2-sync.log" 2>&1
351+
code=$?
352+
set -e
353+
if [ "$code" != "0" ]; then
354+
cat "$RUNNER_TEMP/r2-sync.log" >&2
355+
echo "status=FAILED — aws s3 sync exited $code" >> "$GITHUB_OUTPUT"
356+
echo "::error::aws s3 sync to R2 exited $code. This run IS archived on board-archive; the out-of-GitHub copy is now behind and every later run will stay behind until this is fixed."
357+
exit 1
358+
fi
359+
# One line per object moved. The first configured run moves the whole
360+
# board and every later run moves a handful, so the log is truncated
361+
# here and the counts below are taken over all of it.
362+
transfers=$(wc -l < "$RUNNER_TEMP/r2-sync.log")
363+
head -n 50 "$RUNNER_TEMP/r2-sync.log"
364+
if [ "$transfers" -gt 50 ]; then
365+
echo "... and $((transfers - 50)) further transfer line(s), not printed."
366+
fi
367+
uploaded=$(grep -c '^upload:' "$RUNNER_TEMP/r2-sync.log" || true)
368+
deleted=$(grep -c '^delete:' "$RUNNER_TEMP/r2-sync.log" || true)
369+
370+
day=$(date -u +%Y-%m-%d)
371+
tarball="s3://${R2_BUCKET}/objectstack/snapshots/${day}.tar.gz"
372+
if aws s3 ls "$tarball" --endpoint-url "$endpoint" > /dev/null 2>&1; then
373+
tarred="already written for ${day}"
374+
else
375+
tar -czf "$RUNNER_TEMP/${day}.tar.gz" --exclude=.git -C archive .
376+
aws s3 cp "$RUNNER_TEMP/${day}.tar.gz" "$tarball" \
377+
--endpoint-url "$endpoint" --no-progress
378+
tarred="written for ${day}"
379+
fi
380+
381+
echo "status=synced ${uploaded} object(s), deleted ${deleted}; daily tarball ${tarred}" >> "$GITHUB_OUTPUT"
382+
echo "R2: synced ${uploaded} object(s), deleted ${deleted}; daily tarball ${tarred}"
383+
277384
- name: Publish the run to the summary
278385
if: always()
279386
run: |
@@ -282,6 +389,8 @@ jobs:
282389
echo
283390
echo "Committed: ${{ steps.commit.outputs.committed || 'nothing (the run stopped before the commit step)' }}"
284391
echo
392+
echo "R2: ${{ steps.r2.outputs.status || 'not attempted (a pull_request run, or the job stopped before the upload step)' }}"
393+
echo
285394
echo '```'
286395
cat "$RUNNER_TEMP/snapshot.md" 2>/dev/null || echo '(no report produced)'
287396
echo '```'

0 commit comments

Comments
 (0)