From 4706425385651b0c37a97cfb947ed2f557936219 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:07:34 +0000 Subject: [PATCH 1/7] Publish previews from a tag on main, before Maven Central exists `org.chdb` is not on Central yet, so there is nowhere for a user to resolve the driver from. This publishes a release as GitHub Release assets instead: one zip per platform, holding the jars and the POMs the build produced, and scripts/install-preview.sh installs one into a local Maven repository with `install-file`. The workflow is the Central path with the last step swapped -- the same four runners, the same build-native.sh, the same integration tests against the staged package -- and it keeps release.yml's rule that the version lives in git: no `versions:set` in CI, the POMs at the tagged commit must already carry the tag's version, scripts/check-release-tag.sh must agree that the tag names that commit, the commit must be an ancestor of main, and `build` must have concluded successfully for it. Each of those checks names a way the first preview went wrong. v1.0.0-preview.1 was tagged on a branch 67 commits behind main, so the published binaries were missing thirteen merged fixes -- non-streamable statements, the non-ASCII storage path, the shutdown hook, stream handle ownership, RowBinary types -- and its own install command pointed at a script that existed only on that branch. A preview tag is also excluded from release.yml's trigger. `v*` matched it, and preflight would have accepted it, so the Central path would have staged and signed a preview and offered it in the portal, where a release cannot be unpublished. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/preview-release.yml | 293 ++++++++++++++++++++++++++ .github/workflows/release.yml | 9 +- docs/publishing.md | 42 ++++ docs/release-readiness.md | 7 + scripts/install-preview.sh | 194 +++++++++++++++++ scripts/package-preview.sh | 127 +++++++++++ 6 files changed, 671 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/preview-release.yml create mode 100755 scripts/install-preview.sh create mode 100755 scripts/package-preview.sh diff --git a/.github/workflows/preview-release.yml b/.github/workflows/preview-release.yml new file mode 100644 index 0000000..8e2eb26 --- /dev/null +++ b/.github/workflows/preview-release.yml @@ -0,0 +1,293 @@ +name: preview release + +# Publishes a preview as GitHub Release assets, one zip per platform, because there is nowhere +# else to put it yet: `org.chdb` is not on Maven Central (docs/release-readiness.md step 1) and +# the native packages are 112–167 MB each, which is past what a repository file may be. +# +# It is the Central release path with the last step swapped, not a second way to build: +# the same four runners, the same build-native.sh, the same integration tests against the +# staged package. Where release.yml signs and uploads a portal bundle, this uploads zips and +# scripts/install-preview.sh puts them in a user's local Maven repository. +# +# The rule release.yml's header calls decision 1 holds here too, and for the same reason: the +# version lives in git, not in CI. This workflow never runs `versions:set`. A preview is a +# commit whose POMs already carry `.-preview.` and a tag naming it, so +# `git show v26.7.3.1-preview.1` says exactly what was published. The first preview was cut +# without that rule and shipped a tree 67 commits behind main under a tag nobody could relate +# to a commit on main; every check in `preflight` below exists because of it. +on: + push: + tags: ["v*-preview.*"] + workflow_dispatch: + inputs: + tag: + description: An existing preview tag to build and publish + required: true + type: string + +concurrency: + group: preview-release-${{ inputs.tag || github.ref_name }} + cancel-in-progress: false + +permissions: + contents: read + actions: read + +env: + MAVEN_ARGS: "--batch-mode --no-transfer-progress" + +jobs: + preflight: + name: preflight + runs-on: ubuntu-latest + outputs: + tag: ${{ steps.resolve.outputs.tag }} + version: ${{ steps.resolve.outputs.version }} + sha: ${{ steps.resolve.outputs.sha }} + steps: + - uses: actions/checkout@v4 + with: + # On a dispatch `github.sha` is the branch the run was started from, which is not + # what gets built. Everything downstream keys off the tag, resolved here once. + ref: ${{ inputs.tag || github.ref }} + + - uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: "11" + cache: maven + + - name: Resolve the tag and check it against the POMs + id: resolve + env: + TAG: ${{ inputs.tag || github.ref_name }} + run: | + set -euo pipefail + + case "$TAG" in + v*-preview.*) ;; + *) + echo "::error::expected a tag like v26.7.3.1-preview.1, got $TAG" + exit 1 + ;; + esac + TAG_VERSION="${TAG#v}" + + # Read the version from Maven rather than with sed, so an inherited or + # property-substituted version reads correctly. + VERSION=$(mvn $MAVEN_ARGS -q -DforceStdout help:evaluate -Dexpression=project.version) + if [ "$TAG_VERSION" != "$VERSION" ]; then + echo "::error::tag $TAG implies version $TAG_VERSION but the POMs say $VERSION. Commit the preview version, then tag that commit." + exit 1 + fi + case "$VERSION" in + *-SNAPSHOT) + echo "::error::the POMs are at $VERSION. A published preview needs a non-SNAPSHOT version committed and tagged." + exit 1 + ;; + esac + + SHA=$(git rev-parse HEAD) + echo "tag $TAG, version $VERSION, commit $SHA" + { + echo "tag=$TAG" + echo "version=$VERSION" + echo "sha=$SHA" + } >> "$GITHUB_OUTPUT" + + - name: The tag must point at the commit being built + # The same script release.yml uses, for the same guarantee: a published artifact names + # bytes a commit describes. Here it also catches the failure that produced the first + # preview -- a tag on a side branch, built and published as though it were the release + # line -- because the commit it names has to be the one this run checked out. + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + scripts/check-release-tag.sh \ + "${{ github.repository }}" \ + "${{ steps.resolve.outputs.version }}" \ + "${{ steps.resolve.outputs.sha }}" \ + | tee -a "$GITHUB_STEP_SUMMARY" + + - name: The tagged commit must be on main + # A preview is installed by users; it is not a scratch build. `build` running green on + # a branch says the tree works, not that it is the tree main has -- the first preview + # was green on its own branch while missing thirteen merged fixes. + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + if ! gh api "repos/${{ github.repository }}/compare/main...${{ steps.resolve.outputs.sha }}" \ + --jq '.status' | grep -qx 'identical\|behind'; then + echo "::error::${{ steps.resolve.outputs.sha }} is not an ancestor of main. Merge the release commit to main, then tag it there." + exit 1 + fi + + - name: The `build` workflow must have passed for this commit + # `build` filters on branches, so a tag push does not run it. + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + CONCLUSION=$(gh api \ + "repos/${{ github.repository }}/actions/runs?head_sha=${{ steps.resolve.outputs.sha }}&per_page=100" \ + --jq '[.workflow_runs[] | select(.name == "build")] | first | .conclusion // "none"') + echo "build workflow for ${{ steps.resolve.outputs.sha }}: $CONCLUSION" + if [ "$CONCLUSION" != "success" ]; then + echo "::error::the build workflow for this commit concluded '$CONCLUSION'. Let the matrix go green on main, then tag that commit." + exit 1 + fi + + - name: The engine version this preview carries + run: | + set -euo pipefail + ENGINE=$(sed -n 's/^engine.version=//p' scripts/engine.properties | head -1) + echo "engine $ENGINE, pinned by SHA-256 in scripts/engine.properties" >> "$GITHUB_STEP_SUMMARY" + + # One job per platform, on its own runner, because build-native.sh refuses to cross-build. + bundle: + name: bundle ${{ matrix.platform }} + needs: preflight + runs-on: ${{ matrix.runner }} + strategy: + fail-fast: false + matrix: + include: + - platform: linux-x86_64-gnu + runner: ubuntu-22.04 + - platform: linux-aarch64-gnu + runner: ubuntu-22.04-arm + - platform: macos-aarch64 + runner: macos-15 + # macos-15-intel, not macos-15: the plain label is Apple Silicon. + - platform: macos-x86_64 + runner: macos-15-intel + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ needs.preflight.outputs.sha }} + + - uses: actions/setup-java@v4 + with: + # The floor, deliberately: the shim is built against Java 11's jni.h and the driver + # compiled by Java 11's javac, which is what makes maven.compiler.release=11 a fact. + distribution: temurin + java-version: "11" + cache: maven + + - name: Cache the pinned engine download + uses: actions/cache@v4 + with: + path: target/engine/download + key: chdb-engine-${{ matrix.platform }}-${{ hashFiles('scripts/engine.properties') }} + + - name: Compile the Java side and run its unit tests + run: mvn $MAVEN_ARGS -pl chdb-jdbc -am test + + - name: Fetch the engine, build the shim, stage the platform package + run: | + case "${{ matrix.platform }}" in + linux-*) scripts/build-native-in-container.sh ${{ matrix.platform }} ;; + *) scripts/build-native.sh ${{ matrix.platform }} ;; + esac + + - name: Integration tests against the staged package + # The bytes about to be zipped are new bytes, whatever `build` concluded for the commit. + run: | + set -euo pipefail + case "${{ matrix.platform }}" in + macos-*) OS=macos ;; + *) OS=linux ;; + esac + case "${{ matrix.platform }}" in + *aarch64*) ARCH=aarch64 ;; + *) ARCH=x86_64 ;; + esac + LIBS="$PWD/chdb-native-${{ matrix.platform }}/target/native/META-INF/chdb/native/$OS/$ARCH" + mvn $MAVEN_ARGS -pl chdb-integration-tests -am verify \ + -Dchdb.it.platform=${{ matrix.platform }} \ + -Dchdb.it.library.path="$LIBS" + + - name: Package the JARs and the preview bundle + run: | + set -euo pipefail + mvn $MAVEN_ARGS -pl chdb-jdbc,chdb-native-${{ matrix.platform }} package -DskipTests + scripts/package-preview.sh "${{ needs.preflight.outputs.version }}" \ + '${{ matrix.platform }}' dist + + - name: Install the bundle the way a user will, and query through it + # End to end over the artifact itself: install-preview.sh cannot be run against an + # unpublished release, so the same install-file path is exercised against the local + # zip. This is the step that would have caught a bundle whose POMs do not resolve. + run: | + set -euo pipefail + VERSION='${{ needs.preflight.outputs.version }}' + PLATFORM='${{ matrix.platform }}' + WORK="$RUNNER_TEMP/preview-install" + rm -rf "$WORK" && mkdir -p "$WORK" + unzip -q "dist/chdb-java-${VERSION}-${PLATFORM}.zip" -d "$WORK" + BUNDLE="$WORK/chdb-java-${VERSION}-${PLATFORM}" + REPO="$RUNNER_TEMP/preview-m2" + for spec in \ + "$BUNDLE/maven-poms/chdb-java-parent.pom:$BUNDLE/maven-poms/chdb-java-parent.pom:pom" \ + "$BUNDLE/lib/chdb-jdbc-${VERSION}.jar:$BUNDLE/maven-poms/chdb-jdbc.pom:jar" \ + "$BUNDLE/lib/chdb-native-${PLATFORM}-${VERSION}.jar:$BUNDLE/maven-poms/chdb-native-${PLATFORM}.pom:jar" \ + "$BUNDLE/maven-poms/chdb-bom.pom:$BUNDLE/maven-poms/chdb-bom.pom:pom"; do + FILE="${spec%%:*}"; REST="${spec#*:}"; POM="${REST%%:*}"; PACKAGING="${REST##*:}" + mvn $MAVEN_ARGS -q -Dmaven.repo.local="$REPO" \ + org.apache.maven.plugins:maven-install-plugin:3.1.2:install-file \ + -Dfile="$FILE" -DpomFile="$POM" -Dpackaging="$PACKAGING" -DgeneratePom=false + done + mvn $MAVEN_ARGS -pl chdb-examples -am compile -DskipTests + java -cp "$REPO/org/chdb/chdb-jdbc/${VERSION}/chdb-jdbc-${VERSION}.jar:$REPO/org/chdb/chdb-native-${PLATFORM}/${VERSION}/chdb-native-${PLATFORM}-${VERSION}.jar:chdb-examples/target/classes" \ + -Dchdb.cache.dir="$PWD/target/preview-cache" \ + org.chdb.examples.QuickStart + + - name: Upload the platform bundle + uses: actions/upload-artifact@v4 + with: + name: preview-${{ matrix.platform }} + path: dist/*.zip + if-no-files-found: error + retention-days: 14 + + publish: + name: publish + needs: [preflight, bundle] + runs-on: ubuntu-latest + permissions: + contents: write + actions: read + steps: + - name: Download every platform bundle + uses: actions/download-artifact@v4 + with: + pattern: preview-* + path: release-assets + merge-multiple: true + + - name: Checksum what is about to be uploaded + working-directory: release-assets + run: | + set -euo pipefail + test "$(ls -1 -- *.zip | wc -l)" -eq 4 || { echo "::error::expected four platform bundles"; exit 1; } + sha256sum -- *.zip | tee SHA256SUMS >> "$GITHUB_STEP_SUMMARY" + + - name: Publish the release + # --prerelease, always: a preview must not become the repository's "Latest release". + # --verify-tag so a deleted or moved tag between preflight and here fails the upload + # rather than creating a tag of its own. + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ needs.preflight.outputs.tag }} + run: | + set -euo pipefail + if gh release view "$TAG" --repo "${{ github.repository }}" >/dev/null 2>&1; then + gh release edit "$TAG" --repo "${{ github.repository }}" --prerelease + else + gh release create "$TAG" --repo "${{ github.repository }}" \ + --title "chdb-java $TAG" --prerelease --verify-tag --generate-notes + fi + gh release upload "$TAG" --repo "${{ github.repository }}" \ + release-assets/*.zip release-assets/SHA256SUMS --clobber diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 96cc6b6..8381e42 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -62,7 +62,14 @@ on: # A release is a tag on a commit whose POMs already carry the release version. `build` does # not run on tags (it filters on branches), which is why preflight checks that `build` # succeeded for this exact commit rather than assuming a tag implies a tested tree. - tags: ["v*"] + tags: + - "v*" + # Never a preview. `v26.7.3.1-preview.1` matches `v*`, and without this exclusion a + # preview tag would start the Central path as well: preflight would accept it -- the + # POMs carry that exact version and the tag names the commit -- and the run would go + # on to stage, sign and offer a preview as a release bundle in the portal, where a + # release cannot be unpublished. Previews are published by preview-release.yml. + - "!v*-preview.*" workflow_dispatch: inputs: channel: diff --git a/docs/publishing.md b/docs/publishing.md index a5fe316..a019c97 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -4,6 +4,9 @@ Everything needed to put `org.chdb:chdb-jdbc` on Maven Central, in the order it marked by who can do it. The short version: the mechanics are done and tested, the namespace needs one DNS record, and one licence question needs somebody with authority to answer it. +Until those land there is nothing for `mvn` to resolve, so releases go out as preview bundles +on GitHub instead — section 8. + --- ## Status @@ -17,6 +20,7 @@ needs one DNS record, and one licence question needs somebody with authority to | Third-party licence inventory | **done** — generated from the engine, shipped in the package, drift-tested | | `org.chdb` namespace | **not started** — needs a DNS TXT record, and there is no fallback | | A way to stage all four platforms for one release | **done** — `.github/workflows/release.yml`, issue #15 | +| A way to ship before the namespace exists | **done** — preview bundles, section 8 | | Proof that the artifacts work when consumed from a repository | **done, locally** — `scripts/verify-consumer.sh`, issue #10 | | GPG key for the project | **not started** — needs a decision about whose key | | Position on the LGPL components | **not started** — needs chdb-io | @@ -384,3 +388,41 @@ Each step can invalidate the next, so: | A publishing-limit exception | Sonatype support | yes, if the numbers need it | | **Position on the twelve copyleft components** | **chdb-io / ClickHouse legal** | **yes — the only real gate** | | Whether chDB is "commercial" for Central | whoever owns the Sonatype relationship | yes | + +--- + +## 8. Previews, until Central exists + +A preview is this same release, published somewhere a user can reach today. Same four +runners, same `build-native.sh`, same integration tests against the staged package; the +difference is only where the bytes go, and it is one workflow apart: +`.github/workflows/preview-release.yml` uploads one zip per platform to a GitHub Release, +and `scripts/install-preview.sh` installs a zip into a local Maven repository with +`install-file`. The POMs inside the bundle are the POMs the build produced, so what a user +installs is what CI packaged rather than something reconstructed from a template. + +Why a GitHub Release and not files in the repository: a native package is 112–167 MB, past +what a repository file may be. + +Cutting one is the same shape as cutting a release, because it *is* one: + +```bash +mvn versions:set -DnewVersion=26.7.3.1-preview.1 -DgenerateBackupPoms=false +git commit -am "Set the version for the 26.7.3.1-preview.1 release" +# merge to main, let build.yml go green on the merge commit +git tag -a v26.7.3.1-preview.1 -m "chdb-java 26.7.3.1-preview.1" +git push origin v26.7.3.1-preview.1 +``` + +`preview-release.yml` refuses the tag unless the POMs at that commit carry exactly that +version, the tag points at that commit, the commit is an ancestor of `main`, and `build` +concluded successfully for it. Those four checks are not theatre: the first preview, +`v1.0.0-preview.1`, was tagged on a side branch 67 commits behind `main` and published +binaries missing thirteen merged fixes, under a README whose install command pointed at a +script that only existed on that branch. It was deleted rather than superseded. Afterwards, +the ordinary "back to development" commit returns the POMs to a `-SNAPSHOT`. + +A preview tag is excluded from `release.yml`'s trigger, so it can never start the Central +path, and the release it creates is always marked pre-release, so it never becomes the +repository's "Latest release". + diff --git a/docs/release-readiness.md b/docs/release-readiness.md index 53f3bab..a60ce8e 100644 --- a/docs/release-readiness.md +++ b/docs/release-readiness.md @@ -136,6 +136,13 @@ in CI. Afterwards, the ordinary "back to development" commit returns the POMs to a `-SNAPSHOT`. +**Before any of this, a preview can go out.** Step 1 gates Central, not shipping: a preview is +the same commit, the same four-runner build and the same integration tests, published as +GitHub Release assets and installed with `scripts/install-preview.sh`. It needs no namespace, +no GPG key and no portal account, so it is the only thing on this page that can happen today. +The version is the release's with a `-preview.` qualifier — `26.7.3.1-preview.1` — and the +runbook is [docs/publishing.md](publishing.md) section 8. + --- ## Track 1 — Signing and publishing mechanics diff --git a/scripts/install-preview.sh b/scripts/install-preview.sh new file mode 100755 index 0000000..e018190 --- /dev/null +++ b/scripts/install-preview.sh @@ -0,0 +1,194 @@ +#!/usr/bin/env bash +# +# Installs a published preview into a local Maven repository. +# +# Until `org.chdb` exists on Maven Central there is nowhere for `mvn` to resolve the driver +# from, so a preview ships as one GitHub Release asset per platform: a zip holding the driver +# jar, the native package for that platform, and the four POMs the build produced. This script +# downloads the asset for the host it runs on, verifies it against the release's SHA256SUMS, +# and installs the contents with `install-file` so an ordinary `` resolves. +# +# It is deliberately the same bytes a Central release would carry, installed by hand rather +# than rebuilt: the POMs come out of the bundle rather than being generated here, so a preview +# a user installs is the artifact CI packaged, not an approximation of it. +# +# Usage: +# install-preview.sh v26.7.3.1-preview.1 [--repo OWNER/REPO] [--maven-repo PATH] +# +# Needs curl, unzip and mvn. Writes nothing outside the local Maven repository and one +# temporary directory. + +set -euo pipefail + +usage() { + cat >&2 <<'EOF' +Usage: install-preview.sh [--repo OWNER/REPO] [--maven-repo PATH] + +Downloads the matching platform bundle from a GitHub Release and installs its +POMs and JARs into the local Maven repository. +EOF +} + +die() { + printf 'error: %s\n' "$1" >&2 + exit 1 +} + +TAG='' +REPO='chdb-io/chdb-java' +MAVEN_REPO=${MAVEN_REPO:-"$HOME/.m2/repository"} + +while [[ $# -gt 0 ]]; do + case "$1" in + -h|--help) + usage + exit 0 + ;; + --repo) + [[ $# -ge 2 ]] || die "--repo requires OWNER/REPO" + REPO=$2 + shift 2 + ;; + --repo=*) + REPO=${1#*=} + shift + ;; + --maven-repo) + [[ $# -ge 2 ]] || die "--maven-repo requires a path" + MAVEN_REPO=$2 + shift 2 + ;; + --maven-repo=*) + MAVEN_REPO=${1#*=} + shift + ;; + --*) + die "unknown option: $1" + ;; + '') + die "release tag cannot be empty" + ;; + *) + [[ -z "$TAG" ]] || die "only one release tag may be supplied" + TAG=$1 + shift + ;; + esac +done + +[[ -n "$TAG" ]] || { usage; exit 2; } +[[ "$REPO" =~ ^[^/]+/[^/]+$ ]] || die "repository must look like OWNER/REPO: $REPO" +case "$TAG" in + v*-preview.*) ;; + *) die "release tag must look like v26.7.3.1-preview.1: $TAG" ;; +esac +VERSION=${TAG#v} + +case "$(uname -s):$(uname -m)" in + Darwin:arm64|Darwin:aarch64) + PLATFORM='macos-aarch64' + ;; + Darwin:x86_64|Darwin:amd64) + PLATFORM='macos-x86_64' + ;; + Linux:aarch64|Linux:arm64) + PLATFORM='linux-aarch64-gnu' + ;; + Linux:x86_64|Linux:amd64) + PLATFORM='linux-x86_64-gnu' + ;; + *) + die "unsupported host $(uname -s)/$(uname -m); chdb-java preview supports Linux and macOS on x86_64/aarch64" + ;; +esac + +for command_name in curl unzip mvn; do + command -v "$command_name" >/dev/null 2>&1 || die "required command is not installed: $command_name" +done + +ASSET="chdb-java-${VERSION}-${PLATFORM}.zip" +BASE_URL="https://github.com/${REPO}/releases/download/${TAG}" +WORK=$(mktemp -d "${TMPDIR:-/tmp}/chdb-java-preview.XXXXXX") +trap 'rm -rf "$WORK"' EXIT + +# curl's own "error: 22" says nothing about which of the two plausible mistakes was made -- +# a tag that does not exist, or a release that never carried this platform -- and both are +# ordinary enough to name. +download() { + local name=$1 + curl --fail --location --retry 3 --silent --show-error \ + --output "$WORK/$name" "$BASE_URL/$name" && return 0 + die "cannot download $name from $BASE_URL. +Check that the release exists and publishes this asset: https://github.com/${REPO}/releases/tag/${TAG}" +} + +download "$ASSET" +download SHA256SUMS + +EXPECTED=$(awk -v asset="$ASSET" '$2 == asset || $3 == asset { print $1; exit }' "$WORK/SHA256SUMS") +[[ "$EXPECTED" =~ ^[[:xdigit:]]{64}$ ]] || die "no valid checksum found for $ASSET" + +if command -v sha256sum >/dev/null 2>&1; then + ACTUAL=$(sha256sum "$WORK/$ASSET" | awk '{print $1}') +else + command -v shasum >/dev/null 2>&1 || die "sha256sum or shasum is required to verify the bundle" + ACTUAL=$(shasum -a 256 "$WORK/$ASSET" | awk '{print $1}') +fi +[[ "$EXPECTED" == "$ACTUAL" ]] || die "checksum mismatch for $ASSET" + +unzip -q "$WORK/$ASSET" -d "$WORK/extracted" +BUNDLE_ROOT="$WORK/extracted/chdb-java-${VERSION}-${PLATFORM}" +[[ -d "$BUNDLE_ROOT" ]] || die "unexpected bundle layout in $ASSET" + +PARENT_POM="$BUNDLE_ROOT/maven-poms/chdb-java-parent.pom" +DRIVER_POM="$BUNDLE_ROOT/maven-poms/chdb-jdbc.pom" +NATIVE_POM="$BUNDLE_ROOT/maven-poms/chdb-native-${PLATFORM}.pom" +BOM_POM="$BUNDLE_ROOT/maven-poms/chdb-bom.pom" +PREVIEW_PROPERTIES="$BUNDLE_ROOT/preview.properties" +DRIVER_JAR=$(find "$BUNDLE_ROOT/lib" -maxdepth 1 -type f -name 'chdb-jdbc-*.jar' -print -quit) +NATIVE_JAR=$(find "$BUNDLE_ROOT/lib" -maxdepth 1 -type f -name "chdb-native-${PLATFORM}-*.jar" -print -quit) + +for required_file in "$PARENT_POM" "$DRIVER_POM" "$NATIVE_POM" "$BOM_POM" "$PREVIEW_PROPERTIES" "$DRIVER_JAR" "$NATIVE_JAR"; do + [[ -n "$required_file" && -f "$required_file" ]] || die "bundle is missing $required_file" +done + +read_property() { + awk -F= -v key="$1" '$1 == key { print $2; exit }' "$PREVIEW_PROPERTIES" +} + +GROUP_ID=$(read_property chdb.java.preview.groupId) +[[ -n "$GROUP_ID" ]] || die "bundle does not declare a Maven groupId" + +# The asset name is chosen by this script from the tag, so a release that uploaded a bundle +# built at a different version would be installed under a version it does not carry. +BUNDLE_VERSION=$(read_property chdb.java.preview.version) +[[ "$BUNDLE_VERSION" == "$VERSION" ]] || \ + die "bundle declares version $BUNDLE_VERSION but $TAG names $VERSION; the release asset does not match its tag" + +install_file() { + local file=$1 + local pom=$2 + local packaging=${3:-jar} + + ( + # Do not let Maven inspect the caller's project POM. The preview POMs are the only metadata + # this operation needs, and a caller may be installing from an unrelated Maven project. + cd "$WORK" + mvn -q -B -Dmaven.repo.local="$MAVEN_REPO" \ + org.apache.maven.plugins:maven-install-plugin:3.1.2:install-file \ + -Dfile="$file" \ + -DpomFile="$pom" \ + -Dpackaging="$packaging" \ + -DgeneratePom=false + ) +} + +# Install the parent first because the module POMs retain their normal parent relationship. +install_file "$PARENT_POM" "$PARENT_POM" pom +install_file "$DRIVER_JAR" "$DRIVER_POM" +install_file "$NATIVE_JAR" "$NATIVE_POM" +install_file "$BOM_POM" "$BOM_POM" pom + +printf 'Installed chdb-java preview %s (%s) into %s\n' "$TAG" "$PLATFORM" "$MAVEN_REPO" +printf 'Driver: %s:chdb-jdbc:%s\n' "$GROUP_ID" "$VERSION" +printf 'Native: %s:chdb-native-%s:%s\n' "$GROUP_ID" "$PLATFORM" "$VERSION" diff --git a/scripts/package-preview.sh b/scripts/package-preview.sh new file mode 100755 index 0000000..a8f9f95 --- /dev/null +++ b/scripts/package-preview.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash +# +# Packages one platform's preview bundle: the zip that .github/workflows/preview-release.yml +# uploads as a GitHub Release asset and scripts/install-preview.sh installs from. +# +# A preview bundle is not a second build of anything. It is the jars Maven just produced plus +# the POMs that describe them, zipped, so that the bytes a user installs are the bytes CI +# tested. That is also why the POMs are copied rather than generated: `install-file` with a +# generated POM would drop the parent relationship and the dependency the native package +# declares on the driver. +# +# The layout below is the installer's contract; changing it means changing both. +# +# chdb-java--/ +# lib/chdb-jdbc-.jar +# lib/chdb-native--.jar +# maven-poms/{chdb-java-parent,chdb-jdbc,chdb-native-,chdb-bom}.pom +# preview.properties groupId, version and platform, for the installer to read back +# LICENSE, README.md +# +# Usage: +# scripts/package-preview.sh + +set -euo pipefail + +usage() { + printf 'Usage: %s \n' "$(basename "$0")" >&2 + printf 'Platforms: macos-aarch64, macos-x86_64, linux-aarch64-gnu, linux-x86_64-gnu\n' >&2 +} + +die() { + printf 'error: %s\n' "$1" >&2 + exit 1 +} + +if [[ $# -ne 3 ]]; then + usage + exit 2 +fi + +VERSION=$1 +PLATFORM=$2 +OUTPUT_DIR=$3 +ROOT=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) + +case "$VERSION" in + ''|*[!A-Za-z0-9._-]*) die "version contains unsupported characters: $VERSION" ;; +esac + +case "$PLATFORM" in + macos-aarch64|macos-x86_64|linux-aarch64-gnu|linux-x86_64-gnu) ;; + *) die "unsupported platform: $PLATFORM" ;; +esac + +case "$PLATFORM" in + macos-aarch64) NATIVE_OS=macos; NATIVE_ARCH=aarch64; JNI_SUFFIX=dylib ;; + macos-x86_64) NATIVE_OS=macos; NATIVE_ARCH=x86_64; JNI_SUFFIX=dylib ;; + linux-aarch64-gnu) NATIVE_OS=linux; NATIVE_ARCH=aarch64; JNI_SUFFIX=so ;; + linux-x86_64-gnu) NATIVE_OS=linux; NATIVE_ARCH=x86_64; JNI_SUFFIX=so ;; +esac + +DRIVER_JAR="$ROOT/chdb-jdbc/target/chdb-jdbc-${VERSION}.jar" +NATIVE_JAR="$ROOT/chdb-native-${PLATFORM}/target/chdb-native-${PLATFORM}-${VERSION}.jar" +PARENT_POM="$ROOT/pom.xml" +DRIVER_POM="$ROOT/chdb-jdbc/pom.xml" +NATIVE_POM="$ROOT/chdb-native-${PLATFORM}/pom.xml" +BOM_POM="$ROOT/chdb-bom/pom.xml" + +for required_file in "$DRIVER_JAR" "$NATIVE_JAR" "$PARENT_POM" "$DRIVER_POM" "$NATIVE_POM" "$BOM_POM" "$ROOT/LICENSE" "$ROOT/README.md"; do + [[ -f "$required_file" ]] || die "required file does not exist: $required_file" +done + +GROUP_ID=$(awk ' + // { + sub(/^.*/, "") + sub(/<\/groupId>.*$/, "") + print + exit + } +' "$PARENT_POM") +[[ -n "$GROUP_ID" ]] || die "could not determine the Maven groupId from $PARENT_POM" + +mkdir -p "$OUTPUT_DIR" +OUTPUT_DIR=$(cd -- "$OUTPUT_DIR" && pwd) +ASSET="$OUTPUT_DIR/chdb-java-${VERSION}-${PLATFORM}.zip" +WORK=$(mktemp -d "${TMPDIR:-/tmp}/chdb-java-preview.XXXXXX") +trap 'rm -rf "$WORK"' EXIT + +BUNDLE_ROOT="$WORK/chdb-java-${VERSION}-${PLATFORM}" +mkdir -p "$BUNDLE_ROOT/lib" "$BUNDLE_ROOT/maven-poms" + +cp "$DRIVER_JAR" "$BUNDLE_ROOT/lib/" +cp "$NATIVE_JAR" "$BUNDLE_ROOT/lib/" +cp "$PARENT_POM" "$BUNDLE_ROOT/maven-poms/chdb-java-parent.pom" +cp "$DRIVER_POM" "$BUNDLE_ROOT/maven-poms/chdb-jdbc.pom" +cp "$NATIVE_POM" "$BUNDLE_ROOT/maven-poms/chdb-native-${PLATFORM}.pom" +cp "$BOM_POM" "$BUNDLE_ROOT/maven-poms/chdb-bom.pom" +cp "$ROOT/LICENSE" "$BUNDLE_ROOT/LICENSE" +cp "$ROOT/README.md" "$BUNDLE_ROOT/README.md" + +printf 'chdb.java.preview.groupId=%s\nchdb.java.preview.version=%s\nchdb.java.preview.platform=%s\n' \ + "$GROUP_ID" "$VERSION" "$PLATFORM" > "$BUNDLE_ROOT/preview.properties" + +if ! command -v jar >/dev/null 2>&1; then + die "the Java jar tool is required to create the preview bundle" +fi +if ! command -v unzip >/dev/null 2>&1; then + die "unzip is required to validate the native JAR" +fi + +NATIVE_ROOT="META-INF/chdb/native/${NATIVE_OS}/${NATIVE_ARCH}" +NATIVE_CONTENTS="$WORK/native-jar-contents.txt" +unzip -l "$NATIVE_JAR" > "$NATIVE_CONTENTS" +for entry in \ + "$NATIVE_ROOT/libchdb.so" \ + "$NATIVE_ROOT/libchdb_java_jni.${JNI_SUFFIX}" \ + "$NATIVE_ROOT/manifest.properties" \ + "$NATIVE_ROOT/sha256sums.txt" \ + "META-INF/sbom/bom.json"; do + grep -Fq "$entry" "$NATIVE_CONTENTS" || die "native JAR is missing $entry: $NATIVE_JAR" +done +grep -Fq 'META-INF/licenses/' "$NATIVE_CONTENTS" || die "native JAR is missing its license inventory: $NATIVE_JAR" + +rm -f "$ASSET" +jar --create --file="$ASSET" -C "$WORK" "$(basename "$BUNDLE_ROOT")" + +printf 'Created %s (%s bytes)\n' "$ASSET" "$(wc -c < "$ASSET" | tr -d ' ')" From e1027e8d3a1afebe7b0da16ce61b19cb80140ebf Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:07:34 +0000 Subject: [PATCH 2/7] Say in the README how to install a preview, and that Central does not resolve yet The Maven coordinates at the top of Installing are the ones a user will eventually write, and today they resolve to nothing. Say so where they are, and give the path that works: the installer script, what it verifies, and the coordinates it prints. Versioning gains the preview qualifier -- `26.7.3.1-preview.1` is the first preview of `26.7.3.1` and sorts below it -- so a preview stays the same scheme from a different place rather than a second one. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ee093ac..1708adb 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,9 @@ streaming, forward-only result sets over ClickHouse SQL. > **Status: pre-release.** V1 is under construction against the work plan in > [`CHDB_JAVA_V1_WORK_PLAN.md`](CHDB_JAVA_V1_WORK_PLAN.md). Nothing is published to Maven -> Central yet, and the public API is not frozen. See [What works today](#what-works-today). +> Central yet, and the public API is not frozen. Until the namespace exists, releases are +> published as [preview bundles](#a-preview-from-a-github-release) you install into your local +> Maven repository. See [What works today](#what-works-today). ```java try (Connection connection = DriverManager.getConnection("jdbc:chdb::memory:"); @@ -62,6 +64,34 @@ on. The native package pulls in the driver, so declaring it alone is enough. ``` +**These coordinates do not resolve yet.** `org.chdb` is not on Maven Central — the namespace +needs a DNS record and a licence decision first, tracked in +[docs/release-readiness.md](docs/release-readiness.md). Until then, install a preview. + +### A preview from a GitHub Release + +A preview is the same four artifacts a Central release would carry — the driver, one native +package, the parent POM and the BOM — published as one zip per platform and installed into +your local Maven repository: + +```bash +curl -fsSL -o /tmp/install-chdb-java-preview.sh \ + https://raw.githubusercontent.com/chdb-io/chdb-java/main/scripts/install-preview.sh +chmod +x /tmp/install-chdb-java-preview.sh +/tmp/install-chdb-java-preview.sh v26.7.3.1-preview.1 +``` + +It downloads the bundle for the platform it runs on, checks it against the release's +`SHA256SUMS`, installs it with `mvn install-file`, and prints the coordinates. Nothing is +fetched at runtime afterwards: the engine is inside the native package. Pass +`--maven-repo /path/to/repository` to install somewhere other than `~/.m2/repository`, and +`--repo OWNER/REPO` to install from a fork's releases. + +Then declare the dependency at the preview's version — the same XML as above with +`26.7.3.1-preview.1`. [Releases](https://github.com/chdb-io/chdb-java/releases) lists the +preview tags; each one is a commit on `main`, built and tested on all four platforms by +[`preview-release.yml`](.github/workflows/preview-release.yml). + Building for several platforms — a CI matrix, or a distribution your users install on either architecture — declare the driver plus each native package you need: @@ -109,6 +139,10 @@ another, or from an RC to a stable release, counts as a new engine and resets th revision to `1`: that is why the move off `26.7.2-rc.2.1` is `26.7.3.1` and not `26.7.2-rc.2.2`. This is not SemVer; see [work plan §4.3](CHDB_JAVA_V1_WORK_PLAN.md). +A preview of a release carries that release's version with a `-preview.` qualifier, so +`26.7.3.1-preview.1` is the first preview of `26.7.3.1` and sorts below it in Maven's ordering. +Previews are the same coordinates from a different place, not a different version scheme. + ## Connecting ``` From 63222f4821b0e46b3fb925bd73f97346556553a8 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:25:46 +0000 Subject: [PATCH 3/7] Version the binding on its own, not on the engine it embeds MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `.` read the wrong way round in both directions. The move from 26.7.2-rc.2.1 to 26.7.3.1 looked like a major change and was none of the binding's doing, while a break in the Java API could ship as a trailing `.2` that nothing in the number marked as breaking. A version is read by consumers, and the question they ask it is about the Java API. So: SemVer for the binding, `1.0.0` first, `1.0.0-preview.` for a preview. The engine version is not dropped, it is moved to where it can be read rather than parsed out of a string -- `engine.version` in each native package's manifest.properties, the pinned baseline and its SHA-256 in scripts/engine.properties, and the release notes. The ABI check already refuses any engine but the one a package was built against, so the pairing was never the artifact name's job. Work plan §4.3 is rewritten rather than annotated, because a versioning rule with two answers in it is worse than either. The README also says plainly that `org.chdb` itself is provisional and may end up as `com.clickhouse`. A preview survives that: the bundle carries the POMs it was built with and the installer reads the group id out of them, so a namespace decision changes what a user declares, not whether an install keeps working. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/release.yml | 2 +- CHDB_JAVA_V1_WORK_PLAN.md | 47 +++++++++++++++++++++-------------- README.md | 34 +++++++++++++------------ docs/publishing.md | 15 +++++------ docs/release-readiness.md | 2 +- scripts/install-preview.sh | 4 +-- 6 files changed, 59 insertions(+), 45 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8381e42..f33a184 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -64,7 +64,7 @@ on: # succeeded for this exact commit rather than assuming a tag implies a tested tree. tags: - "v*" - # Never a preview. `v26.7.3.1-preview.1` matches `v*`, and without this exclusion a + # Never a preview. `v1.0.0-preview.1` matches `v*`, and without this exclusion a # preview tag would start the Central path as well: preflight would accept it -- the # POMs carry that exact version and the tag names the commit -- and the run would go # on to stage, sign and offer a preview as a release bundle in the portal, where a diff --git a/CHDB_JAVA_V1_WORK_PLAN.md b/CHDB_JAVA_V1_WORK_PLAN.md index 0fed8dc..fdb5c42 100644 --- a/CHDB_JAVA_V1_WORK_PLAN.md +++ b/CHDB_JAVA_V1_WORK_PLAN.md @@ -201,36 +201,47 @@ META-INF/sbom/ ### 4.3 Versioning rules -The scheme is "full engine version plus binding revision": +The Java binding carries its own SemVer version, independent of the engine it embeds: ```text -. +MAJOR.MINOR.PATCH ``` -- `engine-version` keeps the chDB Core release version verbatim, `rc` qualifier included. -- `binding-revision` is the trailing positive integer covering Java, JNI, loader and platform packaging revisions. It restarts at `1` for every new engine version. -- This is an engine-aligned scheme. Do not read it as Java SemVer. +The first release is `1.0.0`. What the number describes is the Java API — a `MAJOR` bump is a +breaking change to it, `MINOR` adds, `PATCH` fixes — which is the question a consumer asks a +version, and the only one it can answer. + +The engine version is not encoded in it. It is recorded where a machine can read it and a +person can look it up: `engine.version` in each native package's `manifest.properties`, the +pinned baseline plus SHA-256 in `scripts/engine.properties`, and the release notes. The +driver still refuses to load an engine that is not the one it was built against, so the +pairing is enforced by the ABI check rather than by the artifact's name. + +This replaces an engine-aligned scheme, `.` — `26.7.3.1` +for the first binding on engine 26.7.3. It read the wrong way round in both directions: the +move from `26.7.2-rc.2.1` to `26.7.3.1` looked like a major change and was none of the +binding's doing, while an actual break in the Java API could ship as a trailing `.2` that +nothing about the number marked as breaking. Examples: | Case | Maven version | Meaning | |---|---|---| -| First binding against stable engine 26.7.0 | `26.7.0.1` | engine=`26.7.0`, binding revision=`1` | -| Java/JNI/loader fix only, same stable engine | `26.7.0.2` | engine unchanged, binding revision incremented | -| First binding against engine 26.7.2-rc.2 | `26.7.2-rc.2.1` | engine=`26.7.2-rc.2`, binding revision=`1` | -| Re-release on the same rc.2 after an addon change | `26.7.2-rc.2.2` | engine unchanged, binding revision incremented | -| Engine moves to rc.3 | `26.7.2-rc.3.1` | new engine, binding revision back to `1` | -| Engine reaches stable 26.7.2 | `26.7.2.1` | first binding release on that stable engine | +| First release | `1.0.0` | the API this document specifies | +| Fix in the driver, loader or JNI shim | `1.0.1` | no API change | +| New engine baseline, no Java API change | `1.1.0` | engine recorded in the manifest, not here | +| Breaking change to the Java API | `2.0.0` | the only thing a major bump means | +| Preview of `1.0.0` | `1.0.0-preview.1` | sorts below `1.0.0`; see [docs/publishing.md](docs/publishing.md) section 8 | +| Release candidate for `1.0.0` | `1.0.0-rc.1` | sorts below `1.0.0` and above nothing else | Release rules: -- A release artifact in a Maven repository is immutable. Never overwrite `26.7.2-rc.2.1`. Any addon, JNI, Java, POM, loader, checksum or single-platform fix ships as `.2`. +- A release artifact in a Maven repository is immutable. Never overwrite `1.0.0`. Any addon, JNI, Java, POM, loader, checksum or single-platform fix ships as `1.0.1`. - Within one binding release, `chdb-jdbc`, the four platform packages and `chdb-bom` carry exactly the same version. They ship together even when some platform content did not change, so the BOM and the platform packages never end up on mixed versions. -- Moving the engine from one RC to another, or from an RC to a stable release, counts as a new engine version, and the binding revision restarts at `1`. -- A Java artifact built on an engine RC is itself a preview and cannot be the engine dependency of V1 GA. V1 GA has to bind a stable chDB Core release. -- Development builds may use `26.7.2-rc.2.2-SNAPSHOT`, but `SNAPSHOT` never enters a Maven Central release. -- If the Java binding on a stable engine needs its own release candidates, use `26.7.0.1-rc.1`, `26.7.0.1-rc.2`, with `26.7.0.1` as the final GA. A candidate and the GA never reuse the same immutable artifact. -- Any engine change produces a new Maven version and a full platform test run. +- Any engine change still produces a new binding release and a full four-platform test run. It is a `MINOR` bump when it changes what the driver can do and a `PATCH` when it does not; what it is never is invisible, because `manifest.properties` and the release notes name the engine. +- A Java artifact built on an engine RC is itself a preview: it ships as `-preview.` and cannot be V1 GA. V1 GA has to bind a stable chDB Core release. +- Development builds use `-SNAPSHOT`, which never enters a Maven Central release. +- A pre-release qualifier is `-preview.` or `-rc.`, both of which Maven orders below the release they qualify. A candidate and the GA never reuse the same immutable artifact. To remove string-parsing ambiguity, every artifact manifest records these separately: @@ -524,7 +535,7 @@ Exit condition: a Java user who knows nothing about the implementation can insta - [ ] Freeze the public Java API and the JNI ABI. - [ ] Write the release notes and the known limitations. - [ ] Publish an RC and hold a soak and external validation window of at least one week. -- [ ] Once every V1 release gate passes, publish the first release aligned to a stable engine, for example `26.7.0.1`. +- [ ] Once every V1 release gate passes, publish the first release, `1.0.0`, built on a stable engine. ## 6. Milestones diff --git a/README.md b/README.md index 1708adb..7cfe07a 100644 --- a/README.md +++ b/README.md @@ -60,13 +60,16 @@ on. The native package pulls in the driver, so declaring it alone is enough. org.chdb chdb-native-linux-x86_64-gnu - 26.7.3.1 + 1.0.0 ``` -**These coordinates do not resolve yet.** `org.chdb` is not on Maven Central — the namespace -needs a DNS record and a licence decision first, tracked in -[docs/release-readiness.md](docs/release-readiness.md). Until then, install a preview. +**These coordinates do not resolve yet, and `org.chdb` is provisional.** Nothing is on Maven +Central: the namespace needs a DNS record and a licence decision first, tracked in +[docs/release-readiness.md](docs/release-readiness.md), and the published group id may end up +being `com.clickhouse` instead. A preview does not depend on that being settled — the bundle +carries the POMs it was built with and the installer reads the group id out of them — so a +change of namespace changes what you declare, not whether an installed preview keeps working. ### A preview from a GitHub Release @@ -78,7 +81,7 @@ your local Maven repository: curl -fsSL -o /tmp/install-chdb-java-preview.sh \ https://raw.githubusercontent.com/chdb-io/chdb-java/main/scripts/install-preview.sh chmod +x /tmp/install-chdb-java-preview.sh -/tmp/install-chdb-java-preview.sh v26.7.3.1-preview.1 +/tmp/install-chdb-java-preview.sh v1.0.0-preview.1 ``` It downloads the bundle for the platform it runs on, checks it against the release's @@ -88,7 +91,7 @@ fetched at runtime afterwards: the engine is inside the native package. Pass `--repo OWNER/REPO` to install from a fork's releases. Then declare the dependency at the preview's version — the same XML as above with -`26.7.3.1-preview.1`. [Releases](https://github.com/chdb-io/chdb-java/releases) lists the +`1.0.0-preview.1`. [Releases](https://github.com/chdb-io/chdb-java/releases) lists the preview tags; each one is a commit on `main`, built and tested on all four platforms by [`preview-release.yml`](.github/workflows/preview-release.yml). @@ -101,7 +104,7 @@ architecture — declare the driver plus each native package you need: org.chdb chdb-bom - 26.7.3.1 + 1.0.0 pom import @@ -132,16 +135,15 @@ unpacked. There is no all-platforms package, on purpose: it would be the sum of ### Versioning -`.`, so `26.7.3.1` is the first Java release built -against engine 26.7.3 — the engine version is carried through verbatim, `rc` qualifier -included when there is one. A new engine always means a new version, and moving from one RC to -another, or from an RC to a stable release, counts as a new engine and resets the binding -revision to `1`: that is why the move off `26.7.2-rc.2.1` is `26.7.3.1` and not `26.7.2-rc.2.2`. -This is not SemVer; see [work plan §4.3](CHDB_JAVA_V1_WORK_PLAN.md). +The binding is versioned on its own, in SemVer: `1.0.0` is the first release, a major bump +means a breaking change to the Java API, and the engine version is not part of it. Which +engine a package embeds is recorded in its `manifest.properties`, pinned by SHA-256 in +[`scripts/engine.properties`](scripts/engine.properties), and named in the release notes — +`26.7.3` today. The driver refuses to load any other engine build, so that pairing is checked +rather than implied by a version string. See [work plan §4.3](CHDB_JAVA_V1_WORK_PLAN.md). -A preview of a release carries that release's version with a `-preview.` qualifier, so -`26.7.3.1-preview.1` is the first preview of `26.7.3.1` and sorts below it in Maven's ordering. -Previews are the same coordinates from a different place, not a different version scheme. +A preview is that version with a `-preview.` qualifier: `1.0.0-preview.1` sorts below +`1.0.0` in Maven's ordering. Same coordinates, same scheme, published somewhere else. ## Connecting diff --git a/docs/publishing.md b/docs/publishing.md index a019c97..c144978 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -43,7 +43,7 @@ scripts/build-native.sh macos-x86_64 # on an Intel Mac JAVA_HOME=/path/to/linux-jdk scripts/build-native-in-container.sh linux-x86_64-gnu JAVA_HOME=/path/to/linux-jdk scripts/build-native-in-container.sh linux-aarch64-gnu -mvn versions:set -DnewVersion=26.7.0.1 # a release, not the -SNAPSHOT in the POM +mvn versions:set -DnewVersion=1.0.0 # a release, not the -SNAPSHOT in the POM mvn -Prelease deploy ``` @@ -107,7 +107,7 @@ workflow's own header and summarised here: What the workflow does instead is record what it shipped: every staged library's SHA-256 goes into the job summary and into `manifest.properties` inside the JAR. -The version matters too: `deploy` on the `26.7.0.1-SNAPSHOT` currently in the POM publishes a +The version matters too: `deploy` on the `-SNAPSHOT` currently in the POM publishes a snapshot, which goes to a different place and is never validated or promoted. A Central release bundle needs a non-`SNAPSHOT` version. @@ -407,11 +407,11 @@ what a repository file may be. Cutting one is the same shape as cutting a release, because it *is* one: ```bash -mvn versions:set -DnewVersion=26.7.3.1-preview.1 -DgenerateBackupPoms=false -git commit -am "Set the version for the 26.7.3.1-preview.1 release" +mvn versions:set -DnewVersion=1.0.0-preview.1 -DgenerateBackupPoms=false +git commit -am "Set the version for the 1.0.0-preview.1 release" # merge to main, let build.yml go green on the merge commit -git tag -a v26.7.3.1-preview.1 -m "chdb-java 26.7.3.1-preview.1" -git push origin v26.7.3.1-preview.1 +git tag -a v1.0.0-preview.1 -m "chdb-java 1.0.0-preview.1" +git push origin v1.0.0-preview.1 ``` `preview-release.yml` refuses the tag unless the POMs at that commit carry exactly that @@ -419,7 +419,8 @@ version, the tag points at that commit, the commit is an ancestor of `main`, and concluded successfully for it. Those four checks are not theatre: the first preview, `v1.0.0-preview.1`, was tagged on a side branch 67 commits behind `main` and published binaries missing thirteen merged fixes, under a README whose install command pointed at a -script that only existed on that branch. It was deleted rather than superseded. Afterwards, +script that only existed on that branch. Nothing had downloaded it, so it was deleted and the +number reused rather than left standing as the repository's newest release. Afterwards, the ordinary "back to development" commit returns the POMs to a `-SNAPSHOT`. A preview tag is excluded from `release.yml`'s trigger, so it can never start the Central diff --git a/docs/release-readiness.md b/docs/release-readiness.md index a60ce8e..c6250a2 100644 --- a/docs/release-readiness.md +++ b/docs/release-readiness.md @@ -140,7 +140,7 @@ Afterwards, the ordinary "back to development" commit returns the POMs to a `-SN the same commit, the same four-runner build and the same integration tests, published as GitHub Release assets and installed with `scripts/install-preview.sh`. It needs no namespace, no GPG key and no portal account, so it is the only thing on this page that can happen today. -The version is the release's with a `-preview.` qualifier — `26.7.3.1-preview.1` — and the +The version is the release's with a `-preview.` qualifier — `1.0.0-preview.1` — and the runbook is [docs/publishing.md](publishing.md) section 8. --- diff --git a/scripts/install-preview.sh b/scripts/install-preview.sh index e018190..2f4d91f 100755 --- a/scripts/install-preview.sh +++ b/scripts/install-preview.sh @@ -13,7 +13,7 @@ # a user installs is the artifact CI packaged, not an approximation of it. # # Usage: -# install-preview.sh v26.7.3.1-preview.1 [--repo OWNER/REPO] [--maven-repo PATH] +# install-preview.sh v1.0.0-preview.1 [--repo OWNER/REPO] [--maven-repo PATH] # # Needs curl, unzip and mvn. Writes nothing outside the local Maven repository and one # temporary directory. @@ -80,7 +80,7 @@ done [[ "$REPO" =~ ^[^/]+/[^/]+$ ]] || die "repository must look like OWNER/REPO: $REPO" case "$TAG" in v*-preview.*) ;; - *) die "release tag must look like v26.7.3.1-preview.1: $TAG" ;; + *) die "release tag must look like v1.0.0-preview.1: $TAG" ;; esac VERSION=${TAG#v} From 00140869ca58837e8141c6b72dabb354fe509ffb Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:25:46 +0000 Subject: [PATCH 4/7] Consume a preview bundle from a real project, and check out before publishing Two gaps in the preview workflow, both found in review. The bundle check installed the artifacts and then ran a class off a hand-built `-cp`. That proves the JARs execute and says nothing about whether Maven can resolve them, which is the half `install-file` can break: the parent relationship, the BOM's dependencyManagement, and the native package's dependency on the driver are all metadata a classpath never reads. scripts/verify-preview-bundle.sh builds a project outside the checkout instead, declaring the native package and no version and resolving from nothing but the repository the bundle was installed into, then runs an in-memory query and one that has to survive a reopen. It also asserts every org.chdb artifact on the classpath came from that repository. Running it locally against the published bundle turned up a bug in it worth keeping the note: `jar --create` writes its own META-INF beside the bundle directory, so "the first directory in the zip" is not the bundle. The publish job had no checkout. Every gh call in it passes --repo, so it had a good chance of working, but `gh release create` with no --repo and no checkout is exactly how the first attempt at publishing a preview failed -- `fatal: not a git repository` -- and a job that publishes is the wrong place to rely on a flag being right. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/preview-release.yml | 54 +++---- scripts/verify-preview-bundle.sh | 202 ++++++++++++++++++++++++++ 2 files changed, 225 insertions(+), 31 deletions(-) create mode 100755 scripts/verify-preview-bundle.sh diff --git a/.github/workflows/preview-release.yml b/.github/workflows/preview-release.yml index 8e2eb26..d1a7250 100644 --- a/.github/workflows/preview-release.yml +++ b/.github/workflows/preview-release.yml @@ -11,10 +11,11 @@ name: preview release # # The rule release.yml's header calls decision 1 holds here too, and for the same reason: the # version lives in git, not in CI. This workflow never runs `versions:set`. A preview is a -# commit whose POMs already carry `.-preview.` and a tag naming it, so -# `git show v26.7.3.1-preview.1` says exactly what was published. The first preview was cut -# without that rule and shipped a tree 67 commits behind main under a tag nobody could relate -# to a commit on main; every check in `preflight` below exists because of it. +# commit whose POMs already carry `-preview.` and a tag naming it, so +# `git show ` says exactly what was published. The first attempt at v1.0.0-preview.1 was +# cut without that rule: `versions:set` in CI on a side branch 67 commits behind main, under +# a tag nothing on main could account for. Every check in `preflight` below exists because of +# it, and it was withdrawn rather than superseded, so the tag name is in use again here. on: push: tags: ["v*-preview.*"] @@ -67,7 +68,7 @@ jobs: case "$TAG" in v*-preview.*) ;; *) - echo "::error::expected a tag like v26.7.3.1-preview.1, got $TAG" + echo "::error::expected a tag like v1.0.0-preview.1, got $TAG" exit 1 ;; esac @@ -216,33 +217,14 @@ jobs: scripts/package-preview.sh "${{ needs.preflight.outputs.version }}" \ '${{ matrix.platform }}' dist - - name: Install the bundle the way a user will, and query through it - # End to end over the artifact itself: install-preview.sh cannot be run against an - # unpublished release, so the same install-file path is exercised against the local - # zip. This is the step that would have caught a bundle whose POMs do not resolve. + - name: Consume the bundle from a project outside this checkout + # Not a classpath smoke test: a real Maven project, resolving from nothing but the + # repository the bundle was installed into, declaring the native package and letting + # the BOM and the POMs supply the rest. That is the half `install-file` can break and + # a hand-built `-cp` cannot see. run: | - set -euo pipefail - VERSION='${{ needs.preflight.outputs.version }}' - PLATFORM='${{ matrix.platform }}' - WORK="$RUNNER_TEMP/preview-install" - rm -rf "$WORK" && mkdir -p "$WORK" - unzip -q "dist/chdb-java-${VERSION}-${PLATFORM}.zip" -d "$WORK" - BUNDLE="$WORK/chdb-java-${VERSION}-${PLATFORM}" - REPO="$RUNNER_TEMP/preview-m2" - for spec in \ - "$BUNDLE/maven-poms/chdb-java-parent.pom:$BUNDLE/maven-poms/chdb-java-parent.pom:pom" \ - "$BUNDLE/lib/chdb-jdbc-${VERSION}.jar:$BUNDLE/maven-poms/chdb-jdbc.pom:jar" \ - "$BUNDLE/lib/chdb-native-${PLATFORM}-${VERSION}.jar:$BUNDLE/maven-poms/chdb-native-${PLATFORM}.pom:jar" \ - "$BUNDLE/maven-poms/chdb-bom.pom:$BUNDLE/maven-poms/chdb-bom.pom:pom"; do - FILE="${spec%%:*}"; REST="${spec#*:}"; POM="${REST%%:*}"; PACKAGING="${REST##*:}" - mvn $MAVEN_ARGS -q -Dmaven.repo.local="$REPO" \ - org.apache.maven.plugins:maven-install-plugin:3.1.2:install-file \ - -Dfile="$FILE" -DpomFile="$POM" -Dpackaging="$PACKAGING" -DgeneratePom=false - done - mvn $MAVEN_ARGS -pl chdb-examples -am compile -DskipTests - java -cp "$REPO/org/chdb/chdb-jdbc/${VERSION}/chdb-jdbc-${VERSION}.jar:$REPO/org/chdb/chdb-native-${PLATFORM}/${VERSION}/chdb-native-${PLATFORM}-${VERSION}.jar:chdb-examples/target/classes" \ - -Dchdb.cache.dir="$PWD/target/preview-cache" \ - org.chdb.examples.QuickStart + scripts/verify-preview-bundle.sh \ + "dist/chdb-java-${{ needs.preflight.outputs.version }}-${{ matrix.platform }}.zip" - name: Upload the platform bundle uses: actions/upload-artifact@v4 @@ -260,6 +242,16 @@ jobs: contents: write actions: read steps: + # gh resolves the repository from git when it can, and this job has no checkout of its + # own -- which is what failed the first attempt at publishing a preview: `gh release + # create` exited with `fatal: not a git repository`. Every call below passes --repo, so + # the checkout is belt and braces rather than the fix, and it also gives --generate-notes + # the history it reads. + - uses: actions/checkout@v4 + with: + ref: ${{ needs.preflight.outputs.sha }} + fetch-depth: 0 + - name: Download every platform bundle uses: actions/download-artifact@v4 with: diff --git a/scripts/verify-preview-bundle.sh b/scripts/verify-preview-bundle.sh new file mode 100755 index 0000000..d282050 --- /dev/null +++ b/scripts/verify-preview-bundle.sh @@ -0,0 +1,202 @@ +#!/usr/bin/env bash +# +# Consumes a preview bundle the way a user will: installs it into an empty local Maven +# repository and builds a project that is not part of this reactor against it. +# +# The distinction matters, and an earlier version of this check missed it. Installing the +# bundle and then running a class off a hand-built `-cp` proves the JARs execute; it says +# nothing about whether Maven can resolve them. Everything a consumer depends on that a +# classpath does not exercise lives in the POMs -- the parent relationship, the BOM's +# dependencyManagement, the native package's dependency on the driver -- and all three are +# things `install-file` can get wrong. So the consumer here is a real project, outside this +# checkout, resolving from nothing but the repository the bundle was installed into: +# +# - it declares the native package and nothing else, so `chdb-jdbc` has to arrive +# transitively through that POM; +# - it imports chdb-bom and omits every version, so the BOM has to be readable and complete; +# - it runs a query, on-disk and in-memory, so the engine inside the JAR has to unpack and +# load from a local repository path rather than from a build directory; +# - and every org.chdb artifact on the resulting classpath is checked to have come from that +# repository, because `org.chdb` not being on Central is a fact about today, not a +# guarantee. +# +# Usage: +# scripts/verify-preview-bundle.sh +# +# Needs unzip, mvn and a JDK. Writes only into a temporary directory, kept on failure. + +set -euo pipefail + +die() { printf 'verify-preview-bundle: %s\n' "$*" >&2; exit 1; } +step() { printf '\n== %s\n' "$*"; } +ok() { printf ' ok: %s\n' "$*"; } + +BUNDLE_ZIP=${1:-} +[[ -n "$BUNDLE_ZIP" ]] || { printf 'usage: %s \n' "$0" >&2; exit 2; } +[[ -f "$BUNDLE_ZIP" ]] || die "no such bundle: $BUNDLE_ZIP" +BUNDLE_ZIP=$(cd "$(dirname "$BUNDLE_ZIP")" && printf '%s/%s' "$(pwd)" "$(basename "$BUNDLE_ZIP")") + +for command_name in unzip mvn java; do + command -v "$command_name" >/dev/null 2>&1 || die "required command is not installed: $command_name" +done + +MVN_FLAGS=(--batch-mode --no-transfer-progress) +WORK="$(cd "$(mktemp -d "${TMPDIR:-/tmp}/chdb-verify-preview.XXXXXX")" && pwd)" +cleanup() { + status=$? + if [ "$status" -eq 0 ]; then + rm -rf "$WORK" + else + printf '\nverify-preview-bundle: left %s in place for diagnosis\n' "$WORK" >&2 + fi +} +trap cleanup EXIT + +M2="$WORK/m2" +CONSUMER="$WORK/consumer" +mkdir -p "$M2" "$CONSUMER/src/main/java" + +step "Unpacking the bundle" +unzip -q "$BUNDLE_ZIP" -d "$WORK/extracted" +# By the directory that holds preview.properties, not by taking the first one: `jar --create` +# writes a META-INF/MANIFEST.MF of its own beside the bundle directory. +PROPERTIES=$(find "$WORK/extracted" -mindepth 2 -maxdepth 2 -name preview.properties -print -quit) +[[ -n "$PROPERTIES" ]] || die "the bundle has no preview.properties" +BUNDLE_ROOT=$(dirname "$PROPERTIES") + +read_property() { awk -F= -v key="$1" '$1 == key { print $2; exit }' "$PROPERTIES"; } +GROUP_ID=$(read_property chdb.java.preview.groupId) +VERSION=$(read_property chdb.java.preview.version) +PLATFORM=$(read_property chdb.java.preview.platform) +[[ -n "$GROUP_ID" && -n "$VERSION" && -n "$PLATFORM" ]] || die "preview.properties is incomplete" +GROUP_PATH=${GROUP_ID//./\/} +ok "$GROUP_ID:$VERSION for $PLATFORM" + +# The bundle is per-platform, and the whole point of the run is to execute a query. +case "$(uname -s):$(uname -m)" in + Darwin:arm64|Darwin:aarch64) HOST=macos-aarch64 ;; + Darwin:x86_64|Darwin:amd64) HOST=macos-x86_64 ;; + Linux:aarch64|Linux:arm64) HOST=linux-aarch64-gnu ;; + Linux:x86_64|Linux:amd64) HOST=linux-x86_64-gnu ;; + *) die "unsupported host $(uname -s)/$(uname -m)" ;; +esac +[[ "$PLATFORM" == "$HOST" ]] || die "this is a $PLATFORM bundle and the host is $HOST" + +step "Installing it into an empty local repository" +install_file() { + ( cd "$WORK" && mvn "${MVN_FLAGS[@]}" -q -Dmaven.repo.local="$M2" \ + org.apache.maven.plugins:maven-install-plugin:3.1.2:install-file \ + -Dfile="$1" -DpomFile="$2" -Dpackaging="${3:-jar}" -DgeneratePom=false ) +} +POMS="$BUNDLE_ROOT/maven-poms" +install_file "$POMS/chdb-java-parent.pom" "$POMS/chdb-java-parent.pom" pom +install_file "$BUNDLE_ROOT/lib/chdb-jdbc-${VERSION}.jar" "$POMS/chdb-jdbc.pom" +install_file "$BUNDLE_ROOT/lib/chdb-native-${PLATFORM}-${VERSION}.jar" "$POMS/chdb-native-${PLATFORM}.pom" +install_file "$POMS/chdb-bom.pom" "$POMS/chdb-bom.pom" pom +ok "parent, driver, native package and BOM installed" + +step "Building a project outside this checkout against it" +# One dependency, no version: the BOM has to supply the version and the native package's POM +# has to bring in the driver. A project that named both, with versions, would pass while the +# metadata this check is about was broken. +cat > "$CONSUMER/pom.xml" < + 4.0.0 + org.chdb.verify + preview-consumer + 1 + + 11 + UTF-8 + + + + + ${GROUP_ID} + chdb-bom + ${VERSION} + pom + import + + + + + + ${GROUP_ID} + chdb-native-${PLATFORM} + + + +POM + +cat > "$CONSUMER/src/main/java/PreviewConsumer.java" <<'JAVA' +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.Statement; + +/** A user's first five minutes: no Class.forName, one in-memory query, one that persists. */ +public final class PreviewConsumer { + public static void main(String[] args) throws Exception { + try (Connection connection = DriverManager.getConnection("jdbc:chdb::memory:"); + Statement statement = connection.createStatement(); + ResultSet results = statement.executeQuery("SELECT 42, chdb()")) { + results.next(); + if (results.getInt(1) != 42) { + throw new IllegalStateException("SELECT 42 returned " + results.getInt(1)); + } + System.out.println("engine " + results.getString(2)); + } + + Path storage = Path.of(args[0]); + try (Connection connection = DriverManager.getConnection("jdbc:chdb:" + storage); + Statement statement = connection.createStatement()) { + statement.execute("CREATE DATABASE IF NOT EXISTS verify"); + statement.execute("CREATE TABLE IF NOT EXISTS verify.t (id UInt32) ENGINE = MergeTree ORDER BY id"); + statement.execute("INSERT INTO verify.t SELECT number FROM numbers(1000)"); + } + try (Connection connection = DriverManager.getConnection("jdbc:chdb:" + storage); + Statement statement = connection.createStatement(); + ResultSet results = statement.executeQuery("SELECT count() FROM verify.t")) { + results.next(); + if (results.getLong(1) != 1000) { + throw new IllegalStateException("reopened storage holds " + results.getLong(1) + " rows"); + } + } + System.out.println("PREVIEW BUNDLE OK"); + } +} +JAVA + +( cd "$CONSUMER" && mvn "${MVN_FLAGS[@]}" -q -Dmaven.repo.local="$M2" package ) \ + > "$WORK/build.log" 2>&1 || { tail -40 "$WORK/build.log"; die "the consumer project did not build; see $WORK/build.log"; } +ok "resolved and compiled with only the BOM and the native package declared" + +( cd "$CONSUMER" && mvn "${MVN_FLAGS[@]}" -q -Dmaven.repo.local="$M2" \ + dependency:build-classpath -Dmdep.outputFile="$WORK/cp.txt" ) \ + > "$WORK/resolve.log" 2>&1 || { tail -40 "$WORK/resolve.log"; die "could not resolve the consumer's classpath"; } + +# The driver has to be here through the native package's POM, not because it was asked for. +grep -q "chdb-jdbc-${VERSION}.jar" "$WORK/cp.txt" \ + || die "chdb-jdbc is not on the classpath: the native package's POM did not bring it in" +ok "chdb-jdbc arrived transitively" + +# Everything under the group has to have come from the repository the bundle was installed +# into. A copy elsewhere serving these coordinates would make the whole run meaningless. +while IFS= read -r entry; do + case "$entry" in + *"/${GROUP_PATH}/"*) + [[ "$entry" == "$M2/"* ]] || die "$entry did not come from $M2" + ;; + esac +done < <(tr ':' '\n' < "$WORK/cp.txt") +ok "every $GROUP_ID artifact resolved from the bundle's repository" + +step "Running a query through it" +OUTPUT=$( cd "$CONSUMER" && java -cp "target/classes:$(cat "$WORK/cp.txt")" \ + -Dchdb.cache.dir="$WORK/native-cache" PreviewConsumer "$WORK/storage" ) +printf '%s\n' "$OUTPUT" +grep -q 'PREVIEW BUNDLE OK' <<<"$OUTPUT" || die "the consumer did not complete its queries" + +printf '\nverify-preview-bundle: %s is installable and works, %s\n' "$(basename "$BUNDLE_ZIP")" "$PLATFORM" From d1cb70b26327e215227c5ede9eb70cd92bee9b06 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:25:48 +0000 Subject: [PATCH 5/7] Set the version for the 1.0.0-preview.1 release The tag has to name bytes a commit describes, so the version is committed here rather than set in CI. The back-to-development commit returns the POMs to a -SNAPSHOT after the tag is pushed. Co-Authored-By: Claude Opus 5 (1M context) --- chdb-bom/pom.xml | 2 +- chdb-examples/pom.xml | 2 +- chdb-integration-tests/pom.xml | 2 +- chdb-jdbc/pom.xml | 2 +- chdb-native-linux-aarch64-gnu/pom.xml | 2 +- chdb-native-linux-x86_64-gnu/pom.xml | 2 +- chdb-native-macos-aarch64/pom.xml | 2 +- chdb-native-macos-x86_64/pom.xml | 2 +- pom.xml | 2 +- 9 files changed, 9 insertions(+), 9 deletions(-) diff --git a/chdb-bom/pom.xml b/chdb-bom/pom.xml index 033a9b4..d27ae3c 100644 --- a/chdb-bom/pom.xml +++ b/chdb-bom/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-bom diff --git a/chdb-examples/pom.xml b/chdb-examples/pom.xml index 284c0a2..9135439 100644 --- a/chdb-examples/pom.xml +++ b/chdb-examples/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-examples diff --git a/chdb-integration-tests/pom.xml b/chdb-integration-tests/pom.xml index 4a774bd..b5cdb5a 100644 --- a/chdb-integration-tests/pom.xml +++ b/chdb-integration-tests/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-integration-tests diff --git a/chdb-jdbc/pom.xml b/chdb-jdbc/pom.xml index 36acf8c..02a86c5 100644 --- a/chdb-jdbc/pom.xml +++ b/chdb-jdbc/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-jdbc diff --git a/chdb-native-linux-aarch64-gnu/pom.xml b/chdb-native-linux-aarch64-gnu/pom.xml index f2904da..989551c 100644 --- a/chdb-native-linux-aarch64-gnu/pom.xml +++ b/chdb-native-linux-aarch64-gnu/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-native-linux-aarch64-gnu diff --git a/chdb-native-linux-x86_64-gnu/pom.xml b/chdb-native-linux-x86_64-gnu/pom.xml index 135afaa..408fe78 100644 --- a/chdb-native-linux-x86_64-gnu/pom.xml +++ b/chdb-native-linux-x86_64-gnu/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-native-linux-x86_64-gnu diff --git a/chdb-native-macos-aarch64/pom.xml b/chdb-native-macos-aarch64/pom.xml index 20e3568..6dfd182 100644 --- a/chdb-native-macos-aarch64/pom.xml +++ b/chdb-native-macos-aarch64/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-native-macos-aarch64 diff --git a/chdb-native-macos-x86_64/pom.xml b/chdb-native-macos-x86_64/pom.xml index 71be5b5..4d600dc 100644 --- a/chdb-native-macos-x86_64/pom.xml +++ b/chdb-native-macos-x86_64/pom.xml @@ -7,7 +7,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 chdb-native-macos-x86_64 diff --git a/pom.xml b/pom.xml index 60ed114..4e0c499 100644 --- a/pom.xml +++ b/pom.xml @@ -6,7 +6,7 @@ org.chdb chdb-java-parent - 26.7.3.1-SNAPSHOT + 1.0.0-preview.1 pom chDB Java Binding (parent) From f8822b98e94545fc38e997c0d5156761c1d93556 Mon Sep 17 00:00:00 2001 From: Shawn Chen Date: Mon, 21 Sep 2026 00:39:22 +0000 Subject: [PATCH 6/7] Harden the preview publish step and the installer Five things review found, each able to publish or install something wrong. The publish job re-resolves the tag immediately before uploading and fails unless it still points at the commit the bundles were built from. `--verify-tag` only asks whether the tag exists, and staging takes tens of minutes -- long enough for a tag to move. `gh release edit` passes `--draft=false`, so an existing draft is published rather than quietly taking the uploads and staying invisible. The integration-test step closes stdin, as build.yml already does: chDB reads a non-TTY stdin with bytes on it as external data for an INSERT. verify-preview-bundle.sh does the same, because its consumer inserts. In the installer, `$HOME` is no longer dereferenced under `set -u` before arguments are parsed, and a relative `--maven-repo` is made absolute before Maven runs from the temporary directory -- it used to install into a path the EXIT trap deleted, then report success. Comments here are cut to what is not obvious from the code. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/preview-release.yml | 77 ++++++++++++--------------- scripts/install-preview.sh | 34 ++++-------- scripts/package-preview.sh | 23 +++----- scripts/verify-preview-bundle.sh | 45 +++++----------- 4 files changed, 66 insertions(+), 113 deletions(-) diff --git a/.github/workflows/preview-release.yml b/.github/workflows/preview-release.yml index d1a7250..1313adf 100644 --- a/.github/workflows/preview-release.yml +++ b/.github/workflows/preview-release.yml @@ -1,21 +1,13 @@ name: preview release -# Publishes a preview as GitHub Release assets, one zip per platform, because there is nowhere -# else to put it yet: `org.chdb` is not on Maven Central (docs/release-readiness.md step 1) and -# the native packages are 112–167 MB each, which is past what a repository file may be. +# Publishes a preview as GitHub Release assets, one zip per platform, because `org.chdb` is +# not on Maven Central yet and a native package is too large to commit. # -# It is the Central release path with the last step swapped, not a second way to build: -# the same four runners, the same build-native.sh, the same integration tests against the -# staged package. Where release.yml signs and uploads a portal bundle, this uploads zips and -# scripts/install-preview.sh puts them in a user's local Maven repository. -# -# The rule release.yml's header calls decision 1 holds here too, and for the same reason: the -# version lives in git, not in CI. This workflow never runs `versions:set`. A preview is a -# commit whose POMs already carry `-preview.` and a tag naming it, so -# `git show ` says exactly what was published. The first attempt at v1.0.0-preview.1 was -# cut without that rule: `versions:set` in CI on a side branch 67 commits behind main, under -# a tag nothing on main could account for. Every check in `preflight` below exists because of -# it, and it was withdrawn rather than superseded, so the tag name is in use again here. +# Same four runners, same build-native.sh and same integration tests as release.yml; only the +# last step differs. The version lives in git here too: no `versions:set` in CI, so the POMs +# at the tagged commit already carry the tag's version. The first attempt at v1.0.0-preview.1 +# broke that rule and shipped a side branch 67 commits behind main, which is what every check +# in `preflight` is for. It was withdrawn, so the tag name is in use again here. on: push: tags: ["v*-preview.*"] @@ -48,8 +40,7 @@ jobs: steps: - uses: actions/checkout@v4 with: - # On a dispatch `github.sha` is the branch the run was started from, which is not - # what gets built. Everything downstream keys off the tag, resolved here once. + # On a dispatch `github.sha` is the branch, not what gets built. ref: ${{ inputs.tag || github.ref }} - uses: actions/setup-java@v4 @@ -74,8 +65,7 @@ jobs: esac TAG_VERSION="${TAG#v}" - # Read the version from Maven rather than with sed, so an inherited or - # property-substituted version reads correctly. + # From Maven, so an inherited or property-substituted version reads correctly. VERSION=$(mvn $MAVEN_ARGS -q -DforceStdout help:evaluate -Dexpression=project.version) if [ "$TAG_VERSION" != "$VERSION" ]; then echo "::error::tag $TAG implies version $TAG_VERSION but the POMs say $VERSION. Commit the preview version, then tag that commit." @@ -97,10 +87,7 @@ jobs: } >> "$GITHUB_OUTPUT" - name: The tag must point at the commit being built - # The same script release.yml uses, for the same guarantee: a published artifact names - # bytes a commit describes. Here it also catches the failure that produced the first - # preview -- a tag on a side branch, built and published as though it were the release - # line -- because the commit it names has to be the one this run checked out. + # release.yml's script, for the same guarantee: what is published is what a commit says. env: GH_TOKEN: ${{ github.token }} run: | @@ -112,9 +99,7 @@ jobs: | tee -a "$GITHUB_STEP_SUMMARY" - name: The tagged commit must be on main - # A preview is installed by users; it is not a scratch build. `build` running green on - # a branch says the tree works, not that it is the tree main has -- the first preview - # was green on its own branch while missing thirteen merged fixes. + # Green on a branch says the tree works, not that it is the tree main has. env: GH_TOKEN: ${{ github.token }} run: | @@ -146,7 +131,7 @@ jobs: ENGINE=$(sed -n 's/^engine.version=//p' scripts/engine.properties | head -1) echo "engine $ENGINE, pinned by SHA-256 in scripts/engine.properties" >> "$GITHUB_STEP_SUMMARY" - # One job per platform, on its own runner, because build-native.sh refuses to cross-build. + # One job per platform: build-native.sh refuses to cross-build. bundle: name: bundle ${{ matrix.platform }} needs: preflight @@ -171,8 +156,7 @@ jobs: - uses: actions/setup-java@v4 with: - # The floor, deliberately: the shim is built against Java 11's jni.h and the driver - # compiled by Java 11's javac, which is what makes maven.compiler.release=11 a fact. + # The floor, deliberately: it is what makes maven.compiler.release=11 a fact. distribution: temurin java-version: "11" cache: maven @@ -196,7 +180,10 @@ jobs: - name: Integration tests against the staged package # The bytes about to be zipped are new bytes, whatever `build` concluded for the commit. run: | - set -euo pipefail + set -eu + # stdin closed: chDB reads a non-TTY stdin with bytes on it as external data for an + # INSERT. See the same step in build.yml. + exec > "$GITHUB_STEP_SUMMARY" - name: Publish the release - # --prerelease, always: a preview must not become the repository's "Latest release". - # --verify-tag so a deleted or moved tag between preflight and here fails the upload - # rather than creating a tag of its own. env: GH_TOKEN: ${{ github.token }} TAG: ${{ needs.preflight.outputs.tag }} + EXPECTED_SHA: ${{ needs.preflight.outputs.sha }} run: | set -euo pipefail + + # Re-resolved here, not just in preflight: staging takes tens of minutes and a tag + # can be moved or deleted inside that window, which would upload these bundles under + # a tag naming a different commit. --verify-tag only checks that the tag exists. + REMOTE_SHA=$(gh api "repos/${{ github.repository }}/commits/$TAG" --jq '.sha') + if [ "$REMOTE_SHA" != "$EXPECTED_SHA" ]; then + echo "::error::$TAG now points at $REMOTE_SHA, not the $EXPECTED_SHA these bundles were built from" + exit 1 + fi + + # --prerelease so a preview never becomes "Latest release"; --draft=false because an + # existing draft would otherwise take the uploads and stay invisible. if gh release view "$TAG" --repo "${{ github.repository }}" >/dev/null 2>&1; then - gh release edit "$TAG" --repo "${{ github.repository }}" --prerelease + gh release edit "$TAG" --repo "${{ github.repository }}" --prerelease --draft=false else gh release create "$TAG" --repo "${{ github.repository }}" \ --title "chdb-java $TAG" --prerelease --verify-tag --generate-notes diff --git a/scripts/install-preview.sh b/scripts/install-preview.sh index 2f4d91f..9305de7 100755 --- a/scripts/install-preview.sh +++ b/scripts/install-preview.sh @@ -1,23 +1,11 @@ #!/usr/bin/env bash # -# Installs a published preview into a local Maven repository. +# Installs a published preview into a local Maven repository: downloads the asset for this +# platform, verifies it against the release's SHA256SUMS, and installs the jars with the POMs +# the build produced. Needs curl, unzip and mvn. # -# Until `org.chdb` exists on Maven Central there is nowhere for `mvn` to resolve the driver -# from, so a preview ships as one GitHub Release asset per platform: a zip holding the driver -# jar, the native package for that platform, and the four POMs the build produced. This script -# downloads the asset for the host it runs on, verifies it against the release's SHA256SUMS, -# and installs the contents with `install-file` so an ordinary `` resolves. -# -# It is deliberately the same bytes a Central release would carry, installed by hand rather -# than rebuilt: the POMs come out of the bundle rather than being generated here, so a preview -# a user installs is the artifact CI packaged, not an approximation of it. -# -# Usage: # install-preview.sh v1.0.0-preview.1 [--repo OWNER/REPO] [--maven-repo PATH] # -# Needs curl, unzip and mvn. Writes nothing outside the local Maven repository and one -# temporary directory. - set -euo pipefail usage() { @@ -36,7 +24,7 @@ die() { TAG='' REPO='chdb-io/chdb-java' -MAVEN_REPO=${MAVEN_REPO:-"$HOME/.m2/repository"} +MAVEN_REPO=${MAVEN_REPO:-"${HOME:-$PWD}/.m2/repository"} while [[ $# -gt 0 ]]; do case "$1" in @@ -77,6 +65,8 @@ while [[ $# -gt 0 ]]; do done [[ -n "$TAG" ]] || { usage; exit 2; } +# Before any cd: install_file runs Maven from $WORK, which the EXIT trap deletes. +[[ "$MAVEN_REPO" = /* ]] || MAVEN_REPO="$PWD/$MAVEN_REPO" [[ "$REPO" =~ ^[^/]+/[^/]+$ ]] || die "repository must look like OWNER/REPO: $REPO" case "$TAG" in v*-preview.*) ;; @@ -111,9 +101,7 @@ BASE_URL="https://github.com/${REPO}/releases/download/${TAG}" WORK=$(mktemp -d "${TMPDIR:-/tmp}/chdb-java-preview.XXXXXX") trap 'rm -rf "$WORK"' EXIT -# curl's own "error: 22" says nothing about which of the two plausible mistakes was made -- -# a tag that does not exist, or a release that never carried this platform -- and both are -# ordinary enough to name. +# curl's "error: 22" does not say whether the tag or the platform asset is the missing one. download() { local name=$1 curl --fail --location --retry 3 --silent --show-error \ @@ -159,8 +147,7 @@ read_property() { GROUP_ID=$(read_property chdb.java.preview.groupId) [[ -n "$GROUP_ID" ]] || die "bundle does not declare a Maven groupId" -# The asset name is chosen by this script from the tag, so a release that uploaded a bundle -# built at a different version would be installed under a version it does not carry. +# The asset name comes from the tag, so a mismatched bundle would install under the wrong version. BUNDLE_VERSION=$(read_property chdb.java.preview.version) [[ "$BUNDLE_VERSION" == "$VERSION" ]] || \ die "bundle declares version $BUNDLE_VERSION but $TAG names $VERSION; the release asset does not match its tag" @@ -171,8 +158,7 @@ install_file() { local packaging=${3:-jar} ( - # Do not let Maven inspect the caller's project POM. The preview POMs are the only metadata - # this operation needs, and a caller may be installing from an unrelated Maven project. + # From $WORK so Maven does not read the caller's own project POM. cd "$WORK" mvn -q -B -Dmaven.repo.local="$MAVEN_REPO" \ org.apache.maven.plugins:maven-install-plugin:3.1.2:install-file \ @@ -183,7 +169,7 @@ install_file() { ) } -# Install the parent first because the module POMs retain their normal parent relationship. +# Parent first: the module POMs keep their parent relationship. install_file "$PARENT_POM" "$PARENT_POM" pom install_file "$DRIVER_JAR" "$DRIVER_POM" install_file "$NATIVE_JAR" "$NATIVE_POM" diff --git a/scripts/package-preview.sh b/scripts/package-preview.sh index a8f9f95..91152d9 100755 --- a/scripts/package-preview.sh +++ b/scripts/package-preview.sh @@ -1,26 +1,19 @@ #!/usr/bin/env bash # -# Packages one platform's preview bundle: the zip that .github/workflows/preview-release.yml -# uploads as a GitHub Release asset and scripts/install-preview.sh installs from. +# Packages one platform's preview bundle: the jars Maven just built plus the POMs that +# describe them, zipped for a GitHub Release. The POMs are copied, not generated, so +# install-file keeps the parent relationship and the driver dependency. # -# A preview bundle is not a second build of anything. It is the jars Maven just produced plus -# the POMs that describe them, zipped, so that the bytes a user installs are the bytes CI -# tested. That is also why the POMs are copied rather than generated: `install-file` with a -# generated POM would drop the parent relationship and the dependency the native package -# declares on the driver. -# -# The layout below is the installer's contract; changing it means changing both. +# The layout is scripts/install-preview.sh's contract: # # chdb-java--/ -# lib/chdb-jdbc-.jar -# lib/chdb-native--.jar -# maven-poms/{chdb-java-parent,chdb-jdbc,chdb-native-,chdb-bom}.pom -# preview.properties groupId, version and platform, for the installer to read back +# lib/ chdb-jdbc and chdb-native- jars +# maven-poms/ parent, driver, native, BOM +# preview.properties groupId, version, platform # LICENSE, README.md # -# Usage: # scripts/package-preview.sh - +# set -euo pipefail usage() { diff --git a/scripts/verify-preview-bundle.sh b/scripts/verify-preview-bundle.sh index d282050..b555b89 100755 --- a/scripts/verify-preview-bundle.sh +++ b/scripts/verify-preview-bundle.sh @@ -1,30 +1,15 @@ #!/usr/bin/env bash # # Consumes a preview bundle the way a user will: installs it into an empty local Maven -# repository and builds a project that is not part of this reactor against it. +# repository, then builds a project outside this checkout against it. # -# The distinction matters, and an earlier version of this check missed it. Installing the -# bundle and then running a class off a hand-built `-cp` proves the JARs execute; it says -# nothing about whether Maven can resolve them. Everything a consumer depends on that a -# classpath does not exercise lives in the POMs -- the parent relationship, the BOM's -# dependencyManagement, the native package's dependency on the driver -- and all three are -# things `install-file` can get wrong. So the consumer here is a real project, outside this -# checkout, resolving from nothing but the repository the bundle was installed into: +# A classpath smoke test would not do. What install-file can break lives in the POMs -- the +# parent relationship, the BOM, the native package's dependency on the driver -- so the +# consumer here declares the native package with no version and lets resolution supply the +# rest, then runs a query in memory and one that has to survive a reopen. # -# - it declares the native package and nothing else, so `chdb-jdbc` has to arrive -# transitively through that POM; -# - it imports chdb-bom and omits every version, so the BOM has to be readable and complete; -# - it runs a query, on-disk and in-memory, so the engine inside the JAR has to unpack and -# load from a local repository path rather than from a build directory; -# - and every org.chdb artifact on the resulting classpath is checked to have come from that -# repository, because `org.chdb` not being on Central is a fact about today, not a -# guarantee. -# -# Usage: # scripts/verify-preview-bundle.sh # -# Needs unzip, mvn and a JDK. Writes only into a temporary directory, kept on failure. - set -euo pipefail die() { printf 'verify-preview-bundle: %s\n' "$*" >&2; exit 1; } @@ -58,8 +43,7 @@ mkdir -p "$M2" "$CONSUMER/src/main/java" step "Unpacking the bundle" unzip -q "$BUNDLE_ZIP" -d "$WORK/extracted" -# By the directory that holds preview.properties, not by taking the first one: `jar --create` -# writes a META-INF/MANIFEST.MF of its own beside the bundle directory. +# By preview.properties, not the first directory: `jar --create` writes its own META-INF. PROPERTIES=$(find "$WORK/extracted" -mindepth 2 -maxdepth 2 -name preview.properties -print -quit) [[ -n "$PROPERTIES" ]] || die "the bundle has no preview.properties" BUNDLE_ROOT=$(dirname "$PROPERTIES") @@ -96,9 +80,7 @@ install_file "$POMS/chdb-bom.pom" "$POMS/chdb-bom.pom" pom ok "parent, driver, native package and BOM installed" step "Building a project outside this checkout against it" -# One dependency, no version: the BOM has to supply the version and the native package's POM -# has to bring in the driver. A project that named both, with versions, would pass while the -# metadata this check is about was broken. +# One dependency, no version: the BOM supplies it and the native POM brings in the driver. cat > "$CONSUMER/pom.xml" < 4.0.0 @@ -136,7 +118,7 @@ import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.Statement; -/** A user's first five minutes: no Class.forName, one in-memory query, one that persists. */ +/** No Class.forName, one in-memory query, one that persists. */ public final class PreviewConsumer { public static void main(String[] args) throws Exception { try (Connection connection = DriverManager.getConnection("jdbc:chdb::memory:"); @@ -169,7 +151,7 @@ public final class PreviewConsumer { } JAVA -( cd "$CONSUMER" && mvn "${MVN_FLAGS[@]}" -q -Dmaven.repo.local="$M2" package ) \ +( cd "$CONSUMER" && mvn "${MVN_FLAGS[@]}" -q -Dmaven.repo.local="$M2" package "$WORK/build.log" 2>&1 || { tail -40 "$WORK/build.log"; die "the consumer project did not build; see $WORK/build.log"; } ok "resolved and compiled with only the BOM and the native package declared" @@ -177,13 +159,12 @@ ok "resolved and compiled with only the BOM and the native package declared" dependency:build-classpath -Dmdep.outputFile="$WORK/cp.txt" ) \ > "$WORK/resolve.log" 2>&1 || { tail -40 "$WORK/resolve.log"; die "could not resolve the consumer's classpath"; } -# The driver has to be here through the native package's POM, not because it was asked for. +# Transitive, not asked for. grep -q "chdb-jdbc-${VERSION}.jar" "$WORK/cp.txt" \ || die "chdb-jdbc is not on the classpath: the native package's POM did not bring it in" ok "chdb-jdbc arrived transitively" -# Everything under the group has to have come from the repository the bundle was installed -# into. A copy elsewhere serving these coordinates would make the whole run meaningless. +# Nothing may have come from anywhere but the bundle's own repository. while IFS= read -r entry; do case "$entry" in *"/${GROUP_PATH}/"*) @@ -194,8 +175,10 @@ done < <(tr ':' '\n' < "$WORK/cp.txt") ok "every $GROUP_ID artifact resolved from the bundle's repository" step "Running a query through it" +# stdin closed: chDB reads a non-TTY stdin with bytes on it as external data for an INSERT, +# and this consumer inserts. Same reason as the integration-test step in build.yml. OUTPUT=$( cd "$CONSUMER" && java -cp "target/classes:$(cat "$WORK/cp.txt")" \ - -Dchdb.cache.dir="$WORK/native-cache" PreviewConsumer "$WORK/storage" ) + -Dchdb.cache.dir="$WORK/native-cache" PreviewConsumer "$WORK/storage" Date: Mon, 21 Sep 2026 00:39:22 +0000 Subject: [PATCH 7/7] Say what Maven actually does with a preview qualifier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `1.0.0-preview.1` does not sort below `1.0.0`. ComparableVersion orders unknown qualifiers after the final release, and `preview` is not one of the qualifiers it knows, so the preview compares *newer* than the release it previews. Measured against maven-artifact 3.9.9: 1.0.0-preview.1 > 1.0.0 1.0.0-rc.1 < 1.0.0 The README and §4.3 claimed the opposite. Both now say what it does, and why it costs nothing as things stand -- previews are installed by hand into a local repository, never published beside a GA, and named exactly rather than matched by a range -- with `-rc.` as the answer if one ever has to live in a shared repository. The prose around all of this is cut back too. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/release.yml | 8 ++--- CHDB_JAVA_V1_WORK_PLAN.md | 63 ++++++++++++++--------------------- README.md | 51 ++++++++++++---------------- docs/publishing.md | 41 ++++++++++------------- docs/release-readiness.md | 6 ++-- 5 files changed, 69 insertions(+), 100 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f33a184..19eaddb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -64,11 +64,9 @@ on: # succeeded for this exact commit rather than assuming a tag implies a tested tree. tags: - "v*" - # Never a preview. `v1.0.0-preview.1` matches `v*`, and without this exclusion a - # preview tag would start the Central path as well: preflight would accept it -- the - # POMs carry that exact version and the tag names the commit -- and the run would go - # on to stage, sign and offer a preview as a release bundle in the portal, where a - # release cannot be unpublished. Previews are published by preview-release.yml. + # Never a preview: `v1.0.0-preview.1` matches `v*` and preflight would accept it, so + # the Central path would stage, sign and offer a preview in the portal, where a release + # cannot be unpublished. preview-release.yml publishes those. - "!v*-preview.*" workflow_dispatch: inputs: diff --git a/CHDB_JAVA_V1_WORK_PLAN.md b/CHDB_JAVA_V1_WORK_PLAN.md index fdb5c42..d0aaede 100644 --- a/CHDB_JAVA_V1_WORK_PLAN.md +++ b/CHDB_JAVA_V1_WORK_PLAN.md @@ -201,47 +201,34 @@ META-INF/sbom/ ### 4.3 Versioning rules -The Java binding carries its own SemVer version, independent of the engine it embeds: - -```text -MAJOR.MINOR.PATCH -``` - -The first release is `1.0.0`. What the number describes is the Java API — a `MAJOR` bump is a -breaking change to it, `MINOR` adds, `PATCH` fixes — which is the question a consumer asks a -version, and the only one it can answer. - -The engine version is not encoded in it. It is recorded where a machine can read it and a -person can look it up: `engine.version` in each native package's `manifest.properties`, the -pinned baseline plus SHA-256 in `scripts/engine.properties`, and the release notes. The -driver still refuses to load an engine that is not the one it was built against, so the -pairing is enforced by the ABI check rather than by the artifact's name. - -This replaces an engine-aligned scheme, `.` — `26.7.3.1` -for the first binding on engine 26.7.3. It read the wrong way round in both directions: the -move from `26.7.2-rc.2.1` to `26.7.3.1` looked like a major change and was none of the -binding's doing, while an actual break in the Java API could ship as a trailing `.2` that -nothing about the number marked as breaking. - -Examples: - -| Case | Maven version | Meaning | -|---|---|---| -| First release | `1.0.0` | the API this document specifies | -| Fix in the driver, loader or JNI shim | `1.0.1` | no API change | -| New engine baseline, no Java API change | `1.1.0` | engine recorded in the manifest, not here | -| Breaking change to the Java API | `2.0.0` | the only thing a major bump means | -| Preview of `1.0.0` | `1.0.0-preview.1` | sorts below `1.0.0`; see [docs/publishing.md](docs/publishing.md) section 8 | -| Release candidate for `1.0.0` | `1.0.0-rc.1` | sorts below `1.0.0` and above nothing else | +The binding is versioned on its own, in SemVer — `MAJOR.MINOR.PATCH`, first release `1.0.0`. +The number describes the Java API, which is the question a consumer asks it. + +The engine version is not in it. It is recorded in each native package's +`manifest.properties`, pinned with its SHA-256 in `scripts/engine.properties`, and named in +the release notes; the ABI check refuses any other engine build, so the pairing is enforced +rather than spelled. + +This replaces `.`, which read the wrong way round in both +directions: `26.7.2-rc.2.1` → `26.7.3.1` looked major and was not the binding's doing, while +a break in the Java API could ship as a trailing `.2`. + +| Case | Version | +|---|---| +| First release | `1.0.0` | +| Fix in the driver, loader or JNI shim | `1.0.1` | +| New engine baseline, no Java API change | `1.1.0` | +| Breaking change to the Java API | `2.0.0` | +| Preview of `1.0.0` | `1.0.0-preview.1` | Release rules: -- A release artifact in a Maven repository is immutable. Never overwrite `1.0.0`. Any addon, JNI, Java, POM, loader, checksum or single-platform fix ships as `1.0.1`. -- Within one binding release, `chdb-jdbc`, the four platform packages and `chdb-bom` carry exactly the same version. They ship together even when some platform content did not change, so the BOM and the platform packages never end up on mixed versions. -- Any engine change still produces a new binding release and a full four-platform test run. It is a `MINOR` bump when it changes what the driver can do and a `PATCH` when it does not; what it is never is invisible, because `manifest.properties` and the release notes name the engine. -- A Java artifact built on an engine RC is itself a preview: it ships as `-preview.` and cannot be V1 GA. V1 GA has to bind a stable chDB Core release. -- Development builds use `-SNAPSHOT`, which never enters a Maven Central release. -- A pre-release qualifier is `-preview.` or `-rc.`, both of which Maven orders below the release they qualify. A candidate and the GA never reuse the same immutable artifact. +- A published artifact is immutable. Never overwrite `1.0.0`; any fix ships as `1.0.1`. +- `chdb-jdbc`, the four platform packages and `chdb-bom` always carry the same version. +- An engine change is a `MINOR` bump when it changes what the driver can do and a `PATCH` when it does not. It is never invisible: the manifest and the release notes name the engine. +- A binding built on an engine RC is a preview and cannot be V1 GA. +- `-SNAPSHOT` is for development and never reaches Maven Central. +- **`-preview.` sorts *above* the release it previews.** Maven's `ComparableVersion` orders unknown qualifiers after the final release, so `1.0.0-preview.1` compares newer than `1.0.0` — measured, not assumed. It costs nothing here because previews are installed by hand into a local repository and never published beside a GA, and consumers name exact versions. If a preview ever has to live in a shared repository, use `-rc.`, which Maven does order below the release. To remove string-parsing ambiguity, every artifact manifest records these separately: diff --git a/README.md b/README.md index 7cfe07a..6903a21 100644 --- a/README.md +++ b/README.md @@ -5,10 +5,10 @@ ClickHouse. It runs the engine in your JVM's process — no server, no network streaming, forward-only result sets over ClickHouse SQL. > **Status: pre-release.** V1 is under construction against the work plan in -> [`CHDB_JAVA_V1_WORK_PLAN.md`](CHDB_JAVA_V1_WORK_PLAN.md). Nothing is published to Maven -> Central yet, and the public API is not frozen. Until the namespace exists, releases are -> published as [preview bundles](#a-preview-from-a-github-release) you install into your local -> Maven repository. See [What works today](#what-works-today). +> [`CHDB_JAVA_V1_WORK_PLAN.md`](CHDB_JAVA_V1_WORK_PLAN.md). Nothing is on Maven Central yet +> and the public API is not frozen; releases ship as +> [preview bundles](#a-preview-from-a-github-release). See +> [What works today](#what-works-today). ```java try (Connection connection = DriverManager.getConnection("jdbc:chdb::memory:"); @@ -64,18 +64,13 @@ on. The native package pulls in the driver, so declaring it alone is enough. ``` -**These coordinates do not resolve yet, and `org.chdb` is provisional.** Nothing is on Maven -Central: the namespace needs a DNS record and a licence decision first, tracked in -[docs/release-readiness.md](docs/release-readiness.md), and the published group id may end up -being `com.clickhouse` instead. A preview does not depend on that being settled — the bundle -carries the POMs it was built with and the installer reads the group id out of them — so a -change of namespace changes what you declare, not whether an installed preview keeps working. +**These coordinates do not resolve yet, and `org.chdb` is provisional** — nothing is on Maven +Central, and the published group id may end up being `com.clickhouse`. See +[docs/release-readiness.md](docs/release-readiness.md). Until then, install a preview. ### A preview from a GitHub Release -A preview is the same four artifacts a Central release would carry — the driver, one native -package, the parent POM and the BOM — published as one zip per platform and installed into -your local Maven repository: +Previews ship as one zip per platform, holding the same artifacts a Central release would: ```bash curl -fsSL -o /tmp/install-chdb-java-preview.sh \ @@ -84,15 +79,13 @@ chmod +x /tmp/install-chdb-java-preview.sh /tmp/install-chdb-java-preview.sh v1.0.0-preview.1 ``` -It downloads the bundle for the platform it runs on, checks it against the release's -`SHA256SUMS`, installs it with `mvn install-file`, and prints the coordinates. Nothing is -fetched at runtime afterwards: the engine is inside the native package. Pass -`--maven-repo /path/to/repository` to install somewhere other than `~/.m2/repository`, and -`--repo OWNER/REPO` to install from a fork's releases. +It downloads the bundle for this platform, verifies it against the release's `SHA256SUMS`, +installs it with `mvn install-file`, and prints the coordinates. Use `--maven-repo PATH` for a +repository other than `~/.m2/repository`. Then declare the dependency as above with +`1.0.0-preview.1`; because the bundle carries its own POMs, a later change of group id does +not strand an installed preview. -Then declare the dependency at the preview's version — the same XML as above with -`1.0.0-preview.1`. [Releases](https://github.com/chdb-io/chdb-java/releases) lists the -preview tags; each one is a commit on `main`, built and tested on all four platforms by +Each preview tag is a commit on `main`, built and tested on all four platforms by [`preview-release.yml`](.github/workflows/preview-release.yml). Building for several platforms — a CI matrix, or a distribution your users install on either @@ -135,15 +128,15 @@ unpacked. There is no all-platforms package, on purpose: it would be the sum of ### Versioning -The binding is versioned on its own, in SemVer: `1.0.0` is the first release, a major bump -means a breaking change to the Java API, and the engine version is not part of it. Which -engine a package embeds is recorded in its `manifest.properties`, pinned by SHA-256 in -[`scripts/engine.properties`](scripts/engine.properties), and named in the release notes — -`26.7.3` today. The driver refuses to load any other engine build, so that pairing is checked -rather than implied by a version string. See [work plan §4.3](CHDB_JAVA_V1_WORK_PLAN.md). +SemVer, on the binding alone: `1.0.0` is the first release and a major bump means a breaking +change to the Java API. The engine version is not part of it — it is in each package's +`manifest.properties`, pinned in [`scripts/engine.properties`](scripts/engine.properties) and +named in the release notes (`26.7.3` today), and the driver refuses to load any other build. +See [work plan §4.3](CHDB_JAVA_V1_WORK_PLAN.md). -A preview is that version with a `-preview.` qualifier: `1.0.0-preview.1` sorts below -`1.0.0` in Maven's ordering. Same coordinates, same scheme, published somewhere else. +A preview is that version with a `-preview.` qualifier. Maven orders an unknown qualifier +*after* the release, so `1.0.0-preview.1` compares newer than `1.0.0`: name the version you +want rather than a range, and note that previews are never published beside a GA. ## Connecting diff --git a/docs/publishing.md b/docs/publishing.md index c144978..03cb52e 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -5,7 +5,7 @@ marked by who can do it. The short version: the mechanics are done and tested, t needs one DNS record, and one licence question needs somebody with authority to answer it. Until those land there is nothing for `mvn` to resolve, so releases go out as preview bundles -on GitHub instead — section 8. +on GitHub — section 8. --- @@ -393,18 +393,14 @@ Each step can invalidate the next, so: ## 8. Previews, until Central exists -A preview is this same release, published somewhere a user can reach today. Same four -runners, same `build-native.sh`, same integration tests against the staged package; the -difference is only where the bytes go, and it is one workflow apart: -`.github/workflows/preview-release.yml` uploads one zip per platform to a GitHub Release, -and `scripts/install-preview.sh` installs a zip into a local Maven repository with -`install-file`. The POMs inside the bundle are the POMs the build produced, so what a user -installs is what CI packaged rather than something reconstructed from a template. +A preview is this release published somewhere reachable today. Same runners, same +`build-native.sh`, same integration tests; the difference is one workflow: +`.github/workflows/preview-release.yml` uploads one zip per platform to a GitHub Release, and +`scripts/install-preview.sh` installs a zip into a local Maven repository. The POMs in the +bundle are the ones the build produced. A GitHub Release rather than files in the repository, +because a native package is 112–167 MB. -Why a GitHub Release and not files in the repository: a native package is 112–167 MB, past -what a repository file may be. - -Cutting one is the same shape as cutting a release, because it *is* one: +Cutting one is cutting a release, because it is one: ```bash mvn versions:set -DnewVersion=1.0.0-preview.1 -DgenerateBackupPoms=false @@ -414,16 +410,13 @@ git tag -a v1.0.0-preview.1 -m "chdb-java 1.0.0-preview.1" git push origin v1.0.0-preview.1 ``` -`preview-release.yml` refuses the tag unless the POMs at that commit carry exactly that -version, the tag points at that commit, the commit is an ancestor of `main`, and `build` -concluded successfully for it. Those four checks are not theatre: the first preview, -`v1.0.0-preview.1`, was tagged on a side branch 67 commits behind `main` and published -binaries missing thirteen merged fixes, under a README whose install command pointed at a -script that only existed on that branch. Nothing had downloaded it, so it was deleted and the -number reused rather than left standing as the repository's newest release. Afterwards, -the ordinary "back to development" commit returns the POMs to a `-SNAPSHOT`. - -A preview tag is excluded from `release.yml`'s trigger, so it can never start the Central -path, and the release it creates is always marked pre-release, so it never becomes the -repository's "Latest release". +The workflow refuses the tag unless the POMs at that commit carry that version, the tag points +at that commit, the commit is an ancestor of `main`, and `build` was green for it. The first +attempt at `v1.0.0-preview.1` failed all four: tagged on a side branch 67 commits behind +`main`, it published binaries missing thirteen merged fixes. Nothing had downloaded it, so it +was deleted and the number reused. Afterwards the usual back-to-development commit returns the +POMs to a `-SNAPSHOT`. + +A preview tag cannot start the Central path — `release.yml` excludes it — and the release is +always marked pre-release. diff --git a/docs/release-readiness.md b/docs/release-readiness.md index c6250a2..ece860a 100644 --- a/docs/release-readiness.md +++ b/docs/release-readiness.md @@ -137,10 +137,8 @@ in CI. Afterwards, the ordinary "back to development" commit returns the POMs to a `-SNAPSHOT`. **Before any of this, a preview can go out.** Step 1 gates Central, not shipping: a preview is -the same commit, the same four-runner build and the same integration tests, published as -GitHub Release assets and installed with `scripts/install-preview.sh`. It needs no namespace, -no GPG key and no portal account, so it is the only thing on this page that can happen today. -The version is the release's with a `-preview.` qualifier — `1.0.0-preview.1` — and the +the same commit and the same four-runner build, published as GitHub Release assets and +installed with `scripts/install-preview.sh`. No namespace, no GPG key, no portal account. The runbook is [docs/publishing.md](publishing.md) section 8. ---