From d6ba9ff859371920c8312b6826231d3206807a66 Mon Sep 17 00:00:00 2001 From: Andrew Kabas Date: Wed, 23 Sep 2026 11:32:44 -0400 Subject: [PATCH 1/2] Document the release process The README's release section was four paragraphs that covered the happy path and little else. It omitted every step and hazard that has actually bitten us, and one of its two verification links (oss.sonatype.org) now returns 404 since legacy OSSRH was sunset. Add docs/releasing.md covering the full process: prerequisites, how the version number is chosen, the step-by-step run, what the workflow does under the hood, known issues, recovery, and the manual fallback. Reduce the README section to the quick version plus a pointer. The gaps worth calling out, all of which have cost us time before: - The -SNAPSHOT version in pom.xml is not an input. v0.10.29 was cut from a tree reading 0.10.28-SNAPSHOT and published 0.10.29 silently. The next development version is derived from the release version too, not from the POM. - The version input is unvalidated and Maven Central is immutable, so a typo is permanent. This is the highest-risk step and had no warning. - The workflow does not create the GitHub Release. Done by hand, it has twice been left flagged as a pre-release (#1011, and v0.10.30 for three months), so the releases page advertised a stale version. - Auto-generated notes only enumerate PRs, so a squashed cycle collapses to one bullet -- v0.10.30's Java 8 drop and SLF4J 2.x migration went unmentioned. - is restored from the backup POM rather than computed, so a revert can bake in a stale literal that then replays forever (#1044). - The v0.10.29 tag does not point at the commit its artifacts were built from. - e2e tests do not run during a release, only in PR CI. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 25 ++--- docs/releasing.md | 247 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 260 insertions(+), 12 deletions(-) create mode 100644 docs/releasing.md diff --git a/README.md b/README.md index 8d659edd..4eff2934 100644 --- a/README.md +++ b/README.md @@ -295,22 +295,23 @@ expected to honor this code. ## Release -Trigger the [release](https://github.com/spotify/dbeam/actions/workflows/release.yml) workflow manually. This workflow requires a -single input, `version`, which should be set to the desired semantic version in the format `{major_version}.{minor_version}.{patch_version}`. -It will update versions in all `pom.xml` files, push a tag `vx.y.z`, package, sign, and deploy artifacts to Sonatype, and finally bump all -`pom.xml`s to the next development SNAPSHOT version. +Trigger the [release](https://github.com/spotify/dbeam/actions/workflows/release.yml) workflow manually, with `version` +set to the desired semantic version (for example `0.10.31`, no `v` prefix): -You can check the deployment in the following links: +```shell +gh workflow run release.yml --repo spotify/dbeam -f version=0.10.31 +``` -- https://github.com/spotify/dbeam/actions -- https://oss.sonatype.org/#nexus-search;quick~dbeam-core +It updates the versions in all `pom.xml` files, pushes a tag `vx.y.z`, packages, signs and deploys the artifacts to +Maven Central, then bumps all `pom.xml`s to the next development SNAPSHOT version. Publication is automatic — there is +no staging repository to close. Verify at +[repo1.maven.org](https://repo1.maven.org/maven2/com/spotify/dbeam-core/maven-metadata.xml). -You can also do a manual release. First, export env variables $SONATYPE_USERNAME, $SONATYPE_PASSWORD (for information on generating a token see [here](https://help.sonatype.com/en/user-tokens.html)), $MAVEN_GPG_KEY_NAME. -Then, you can run `maven release` to deploy to Sonatype and automatically push commits bumping the project version: +The workflow does **not** create the GitHub Release; that is a manual step afterwards. -```shell -mvn -s sonatype-settings.xml -DreleaseVersion={NEW_VERSION} release:prepare release:perform # Run with -DdryRun=true first to validate pom modification -``` +**Read [docs/releasing.md](docs/releasing.md) before cutting a release.** It covers how the version number is chosen +(the `-SNAPSHOT` in `pom.xml` is *not* what gets released), the full step-by-step process, the known issues and +gotchas, how to recover from a failed release, and the manual fallback. ## Future roadmap diff --git a/docs/releasing.md b/docs/releasing.md new file mode 100644 index 00000000..c09af086 --- /dev/null +++ b/docs/releasing.md @@ -0,0 +1,247 @@ +# Releasing DBeam + +DBeam publishes `com.spotify:dbeam-core`, `dbeam-bom` and `dbeam-parent` to Maven Central via the +Sonatype Central Portal. Releases are cut by the +[Sonatype Release](https://github.com/spotify/dbeam/actions/workflows/release.yml) workflow, driven +by `maven-release-plugin`. + +**Maven Central is immutable.** A published version can never be edited, replaced or deleted. Every +step below exists to catch mistakes before that point. + +## Prerequisites + +- **Write access** to `spotify/dbeam`. The `main_env` environment has no approval rules, so anyone + with write access can publish a release unilaterally. +- Nothing to install locally for the normal path — the workflow does everything. + +Credentials live as `main_env` environment secrets and are already configured: + +| Secret | Used for | +| --- | --- | +| `SONATYPE_USERNAME`, `SONATYPE_TOKEN` | Central Portal user token | +| `GPG_KEY`, `GPG_PASSPHRASE` | Artifact signing | +| `GPG_KEY_NAME` | Only used by the (disabled) deploy job in `maven.yml` | + +## Choosing a version number + +The `version` input you type is passed as `-DreleaseVersion` and **fully determines what is +published**. Two consequences that surprise people: + +- **The `-SNAPSHOT` version in `pom.xml` is ignored.** It is a placeholder, not an input. The + v0.10.29 release was cut from a tree that said `0.10.28-SNAPSHOT` and published `0.10.29` with no + warning. You do not need to edit the POM before releasing, and doing so changes nothing. +- **The next development version is derived from your input, not from the POM.** In batch mode the + plugin increments the last numeric segment of the release version. `0.10.31` leaves master on + `0.10.32-SNAPSHOT`. There is no workflow input to override this; releasing `0.11.0` lands master + on `0.11.1-SNAPSHOT`, so if you want a different next line you need a follow-up commit. + +There is **no validation** of the input. A typo (`1.0.31`, `0.1.031`) is accepted and published +permanently. Read the value twice before submitting. + +Use the version to signal compatibility. 0.10.30 raised `maven.compiler.release` from 8 to 11, +dropping Java 8 support in a patch release — avoid repeating that. + +## Step by step + +### 1. Pre-flight + +- `master` is green in CI. +- Everything you intend to ship is merged. `git log v..master --oneline`. +- Review dependency changes since the last release for anything consumer-visible (a JDK baseline + change, a major bump in Beam / Avro / SLF4J). These belong in the release notes and may change + your version number. + +### 2. Trigger the release + +From the [Actions tab](https://github.com/spotify/dbeam/actions/workflows/release.yml), or: + +```shell +gh workflow run release.yml --repo spotify/dbeam -f version=0.10.31 +``` + +Always trigger from `master`. The workflow has no concurrency guard — do not start a second run +while one is in flight. + +### 3. Watch it + +```shell +gh run watch --repo spotify/dbeam $(gh run list --repo spotify/dbeam --workflow release.yml --limit 1 --json databaseId --jq '.[0].databaseId') +``` + +Expect ~2 commits and 1 tag pushed to master, then a deploy. If it fails, go to +[Recovery](#recovery) before retrying — a partially completed `release:prepare` leaves state behind. + +### 4. Verify on Maven Central + +Publication is automatic (`autoPublish=true`); there is no staging repository to close. Artifacts +appear within ~10-30 minutes. + +```shell +curl -s https://repo1.maven.org/maven2/com/spotify/dbeam-core/maven-metadata.xml | grep -E '|' +curl -s https://repo1.maven.org/maven2/com/spotify/dbeam-core/0.10.31/ | grep -o 'dbeam-core[^"]*' +``` + +Each of `dbeam-core`, `dbeam-bom` and `dbeam-parent` should have its `.pom`, and `dbeam-core` should +additionally have the main jar, `-sources.jar` and `-javadoc.jar`, each with `.asc`, `.md5`, `.sha1`, +`.sha256` and `.sha512`. + +> The README previously pointed at `https://oss.sonatype.org/` for this. That host is gone (HTTP +> 404) — legacy OSSRH was sunset in 2025. Use `repo1.maven.org` or the Central Portal. + +### 5. Create the GitHub Release + +**The workflow does not do this.** The tag is pushed but no release object is created. + +```shell +gh release create v0.10.31 --repo spotify/dbeam --verify-tag --generate-notes --latest +``` + +Then **check the pre-release flag and edit the notes** — see [Known issues](#known-issues) 2 and 3. + +### 6. Post-release + +- Confirm master is on the expected `-SNAPSHOT` and that `` still reads `HEAD`. +- Announce if the release carries anything consumer-visible. + +## What the workflow actually does + +`release.yml` checks out master, configures the `github-actions[bot]` identity, sets up **JDK 11** +and imports the GPG key, then runs a single command: + +```shell +mvn -B release:prepare release:perform -DreleaseVersion= -Darguments="-Dgpg.passphrase=..." +``` + +`release:prepare`: + +1. Backs up the POMs to `pom.xml.releaseBackup` and writes `release.properties`. +2. Rewrites `` to the release version in all modules (`autoVersionSubmodules=true`) and + `` to `v` (`tagNameFormat`). Commits as `prepare release vX`. +3. Runs the default preparation goals, `clean verify` — the full unit test suite. **The e2e suite + does not run**; `e2e/e2e.sh` is invoked only by `maven.yml` on PRs and pushes. +4. Tags and pushes. +5. Rewrites `` to the next `-SNAPSHOT` and **restores the `` section from the backup**. + Commits as `prepare for next development iteration`. + +`release:perform` clones the tag into `target/checkout` and runs `deploy` there with the `release` +profile active (`useReleaseProfile=false`, `releaseProfiles=release`). That profile GPG-signs every +artifact, attaches javadoc, and deploys through `central-publishing-maven-plugin` with +`autoPublish=true`. Tests run a second time here. + +Not published: the `pack` profile's shaded fat jar (CI-only) and `-SNAPSHOT` builds (the +`snapshotRepository` is commented out in `distributionManagement`). + +## Known issues + +### 1. The `version` input is unvalidated and Central is immutable + +The single highest-risk step. Nothing checks that your input is well-formed or greater than the +current version. See [Recovery](#recovery) — there is no undo. + +### 2. The GitHub Release is not created, and the pre-release flag is a repeat offender + +The workflow pushes a tag and stops. When the release is created by hand it has twice been left +flagged as a pre-release — PR #1011 was literally titled "revert prerelease", and v0.10.30 sat +flagged for three months, so the releases page and the README tag badge advertised v0.10.29 while +Central served v0.10.30. Pass `--latest` and verify with: + +```shell +gh release view v0.10.31 --repo spotify/dbeam --json tagName,isPrerelease,isLatest +``` + +### 3. Auto-generated release notes only list pull requests + +`--generate-notes` enumerates merged PRs. When a release cycle is squashed into one large PR the +notes collapse to a single line — v0.10.30's ~30 dependency, JUnit 5 and Java 25 commits all landed +in #1039, and the generated notes were one bullet that mentioned neither the Java 8 drop nor the +SLF4J 2.x migration. For a release like that, write the notes from `git log` instead. + +### 4. `` is a self-perpetuating loop + +The development phase **restores** `` from the backup POM rather than computing it, so +whatever master holds, master keeps holding. Master must hold the sentinel `HEAD`. During the +v0.10.29 cycle a revert put the literal `v0.10.29` there and it was replayed through v0.10.30 +(fixed in #1044). + +This matters because a standalone `release:perform` resolves its checkout from `release.properties` +and then falls back to the POM's `` section — with a stale tag it would check out the wrong +commit and try to redeploy an already-published version. **If you ever revert a release commit, +check that `` is back to `HEAD`.** + +### 5. The `v0.10.29` tag does not point at the released commit + +It points at `e032ae1`, but the published 0.10.29 artifacts were built from `558b87a` (the Central +POM contains `central-publishing-maven-plugin`, added later in #1012). Do not trust +tag-to-artifact correspondence for that one version, and treat `v0.10.29...vX` compare ranges as +over-reporting. The tag was left in place deliberately: force-moving a published tag breaks anyone +who pinned it. + +### 6. Release builds on JDK 11, but v0.10.30 was cut locally on JDK 21 + +v0.10.30's manifest shows `Build-Jdk-Spec: 21` because it was released from a laptop rather than +through the workflow. The bytecode target is unaffected (`maven.compiler.release=11`), but for +reproducible provenance, cut releases through the workflow. + +### 7. Signing depends on one person's personal key + +Artifacts are signed with RSA-4096 key `E372CF3377C63C2F79EC4D13464754B85A79AB63` +(`Luis Bianchin `, created 2021-04-16, no expiry). It does not expire, but +it is tied to an individual rather than to the project. + +### 8. Stale configuration to ignore or clean up + +- `maven.yml`'s `deploy` job is permanently disabled (`if: false`) and references a + `github-settings.xml` that does not exist in the repo. +- `sonatype-settings.xml`'s header comment describes Travis and `.travis.yml`; neither exists. +- `nexus-staging-maven-plugin` is still declared solely to disable itself, which keeps attracting + Dependabot PRs (#1013). +- `.github/release-drafter.yml` exists with no workflow to run it, so it does nothing. +- `distributionManagement` still points at the dead `oss.sonatype.org` endpoints; deployment + actually goes through `central-publishing-maven-plugin`, so these are inert. + +## Recovery + +**A published version cannot be withdrawn.** With `autoPublish=true` there is no staging window in +which to drop a deployment. If a bad version reaches Central, the only remedy is to publish a +corrected one and document the bad version in its release notes. + +Before it reaches Central, the release is recoverable. From a local clone: + +```shell +# Undo prepare's POM rewrites and commits (only works while release.properties is present) +mvn release:rollback + +# Remove the tag if prepare pushed it +git tag -d v0.10.31 && git push --delete origin v0.10.31 + +# Reset master if commits were pushed +git reset --hard && git push --force-with-lease origin master + +# Clear leftover plugin state +mvn release:clean +``` + +After any such revert, verify two things before retrying: `` is the intended `-SNAPSHOT`, +and `` is `HEAD` (known issue 4). + +If the tag was pushed and the build succeeded but the deploy failed, you can retry just the deploy +rather than redoing the whole cycle: + +```shell +mvn release:perform -Dtag=v0.10.31 -DconnectionUrl=scm:git:https://github.com/spotify/dbeam.git +``` + +## Manual release (fallback) + +Only for when the workflow itself is broken. Requires the GPG private key locally plus +`SONATYPE_USERNAME`, `SONATYPE_PASSWORD` (a Central Portal +[user token](https://help.sonatype.com/en/user-tokens.html)) and `MAVEN_GPG_KEY_NAME` in the +environment. + +```shell +mvn -s sonatype-settings.xml -DreleaseVersion=0.10.31 -DdryRun=true release:prepare # validate first +mvn -s sonatype-settings.xml -DreleaseVersion=0.10.31 release:prepare release:perform +``` + +Note this differs from the workflow: without `-B` the plugin runs **interactively** and will prompt +for the development version, and the build uses whatever JDK you have locally. From 44cf85fe6caddcf29ff318d87efffb47f7bf5a30 Mon Sep 17 00:00:00 2001 From: Andrew Kabas Date: Wed, 23 Sep 2026 14:34:47 -0400 Subject: [PATCH 2/2] Clarify: no v prefix in version input Co-Authored-By: Claude Opus 4.6 --- README.md | 2 +- docs/releasing.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 4eff2934..f2296c6b 100644 --- a/README.md +++ b/README.md @@ -296,7 +296,7 @@ expected to honor this code. ## Release Trigger the [release](https://github.com/spotify/dbeam/actions/workflows/release.yml) workflow manually, with `version` -set to the desired semantic version (for example `0.10.31`, no `v` prefix): +set to the desired semantic version (for example `0.10.31` — no `v` prefix, that is added by `tagNameFormat`): ```shell gh workflow run release.yml --repo spotify/dbeam -f version=0.10.31 diff --git a/docs/releasing.md b/docs/releasing.md index c09af086..02cda166 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -59,6 +59,10 @@ From the [Actions tab](https://github.com/spotify/dbeam/actions/workflows/releas gh workflow run release.yml --repo spotify/dbeam -f version=0.10.31 ``` +**No `v` prefix.** The input sets the POM `` directly. The `v` is added by `tagNameFormat` +(`v@{project.version}`), so `0.10.31` produces tag `v0.10.31`. Typing `v0.10.31` would set the POM +version to `v0.10.31` and the tag to `vv0.10.31`. + Always trigger from `master`. The workflow has no concurrency guard — do not start a second run while one is in flight.