Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion backuplsststack.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,10 @@ cd "$ROOT_DIR" || fail "Can't find ${ROOT_DIR}"
[[ -d "$LSSTSW_DIR/stack" ]] || fail "Can't find stack dir"
[[ -d "$LSSTSW_DIR/build" ]] || fail "Can't find build dir"

tar --zstd -cf "${TARGET}" lsstsw
# lsstDoxygen is not an EUPS product -- create_xlinkdocs.sh clones the doc repo
# and generates HTML under lsstsw/build/, which bloats the tarball and shows up
# as a package to consumers that enumerate lsstsw/build/*.
tar --zstd -cf "${TARGET}" --exclude=lsstsw/build/lsstDoxygen lsstsw

gcloud storage cp "${TARGET}" "${GCP_BUCKET}"

Expand Down
38 changes: 38 additions & 0 deletions promotelsststack.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash

set -xeo pipefail

# Renames an lsstsw cache tarball in GCP_BUCKET from one tag to another. Used by
# the release pipelines: run-rebuild uploads the cache under a temporary
# per-build tag and this promotes it to the shared tag (eg. d_latest) once the
# release has succeeded. The move is server side, so the multi-GB tarball is
# never transferred. Every architecture variant present under the source tag is
# promoted.

GCP_BUCKET="gs://eups-lsstsw-cache"

fail() {
echo "$1" >&2
exit 1
}

[[ $# -eq 2 ]] || fail "usage: $0 <src_tag> <dst_tag>"
SRC_TAG=$1
DST_TAG=$2

[[ -n "$SRC_TAG" ]] || fail "src tag is empty"
[[ -n "$DST_TAG" ]] || fail "dst tag is empty"

[[ "$SRC_TAG" != "$DST_TAG" ]] || fail "src and dst tags are identical: ${SRC_TAG}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could also check if src/dst tag are not empty

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.


# `ls` exits non-zero when nothing matches; report that as our own error rather
# than dying inside the command substitution.
objects=$(gcloud storage ls "${GCP_BUCKET}/${SRC_TAG}_*_lsstsw.tar.zst" || true)
[[ -n "$objects" ]] || fail "no cache tarball found for tag ${SRC_TAG}"

echo "$objects" | while read -r src; do
arch=$(basename "$src")
arch=${arch#"${SRC_TAG}_"}
arch=${arch%_lsstsw.tar.zst}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can arch be empty?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think so. Would be an issue upstream if that were to happen.

gcloud storage mv "$src" "${GCP_BUCKET}/${DST_TAG}_${arch}_lsstsw.tar.zst"
done
Loading