Skip to content

feat(ci): add independent manylinux_2_28 Qt build target (glibc 2.28 ABI floor) - #1409

Merged
ErikBjare merged 4 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/manylinux-2-28
Aug 26, 2026
Merged

feat(ci): add independent manylinux_2_28 Qt build target (glibc 2.28 ABI floor)#1409
ErikBjare merged 4 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/manylinux-2-28

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Summary

Closes #1390.

Adds a new, independent build-qt-manylinux-2-28 job that produces ActivityWatch Linux binaries compatible with glibc 2.28 (RHEL 8, CentOS 8, AlmaLinux 8, Rocky Linux 8, and similar enterprise distributions that currently see GLIBC_2.29 not found errors with the standard build).

The existing build-qt and build-tauri platform matrices are completely untouched.

What changed

.github/workflows/release.yml

  • New job build-qt-manylinux-2-28 that runs on ubuntu-22.04 inside the quay.io/pypa/manylinux_2_28_x86_64 container.
  • Uses dnf to install Qt 5 and build dependencies (EPEL-backed, compatible with the container's AlmaLinux 8 base).
  • Uses the container's /opt/python/cp39-cp39/bin/python3 (the manylinux CPython 3.9 with a shared libpython, required by PyInstaller).
  • Sets APPIMAGE_EXTRACT_AND_RUN=1 so linuxdeploy/appimagetool work without FUSE.
  • Sets OPENSSL_NO_VENDOR=1 to use the container's openssl-devel instead of building OpenSSL from source.
  • git config safe.directory is the first step after checkout, so the version probe never falls back to v0.0.0.dev-unknown.
  • Artifacts are renamed with a -manylinux_2_28 suffix (both the ZIP and the AppImage) to avoid collisions with the standard build:
    • activitywatch-<version>-linux-x86_64-manylinux_2_28.zip
    • activitywatch-linux-x86_64-manylinux_2_28.AppImage
  • An ABI gate step runs after packaging and fails if any bundled ELF references a GLIBC symbol newer than GLIBC_2.28.
  • The release job's needs list now includes build-qt-manylinux-2-28, so the manylinux artifacts appear in every tagged release alongside the existing assets.

scripts/package/abi-gate.sh (new)

Offline ABI verifier:

  • Accepts one or more .zip / .AppImage files.
  • Extracts each via unzip (ZIP) or unsquashfs/--appimage-extract (AppImage).
  • Scans every ELF binary with readelf --version-info.
  • Fails with exit 1 and a per-file violation list if any GLIBC symbol above GLIBC_2.28 is found.

Compatibility evidence

The premise was verified against the contributor's proven artifact (Actions run 30827174168):

  • ZIP: 461 ELFs, 459 with GLIBC refs, maximum GLIBC_2.28, zero above.
  • AppImage: 462 ELFs, 460 with refs, maximum GLIBC_2.28, zero above.
  • AppImage runtime is static PIE with no dynamic GLIBC requirement.

The fork workflow that produced that artifact is not directly upstreamable — it replaces the Qt matrix and removes Tauri/DEB jobs. This PR ports only the proven container method, rebuilt surgically from current upstream master.

What this PR does NOT do

  • Touch the existing build-qt or build-tauri jobs.
  • Refactor the release workflow (no reusable workflow, no matrix abstraction).
  • Add a DEB package for the manylinux target (DEB is an Ubuntu/Debian convention; enterprise RHEL systems use RPM or tarballs).
  • Smoke-test against a live RHEL 8 desktop (that is the next step for a maintainer with hardware access, or a follow-up CI job).

Adds a new `build-qt-manylinux-2-28` CI job that builds ActivityWatch
inside `quay.io/pypa/manylinux_2_28_x86_64`, producing Linux binaries
compatible with glibc 2.28 (RHEL/CentOS/AlmaLinux/Rocky 8 and older).

The change is purely additive: the existing `build-qt` and `build-tauri`
platform matrices are untouched.

Key decisions:
- `git config safe.directory` is set immediately after checkout, before the
  first version probe, so the version string is correct throughout the build.
- Artifacts are renamed with a `-manylinux_2_28` suffix to avoid any name
  collision with the standard Ubuntu build.
- An offline ABI gate (`scripts/package/abi-gate.sh`) scans every ELF in
  both the ZIP and AppImage payload and fails if any symbol requires a GLIBC
  version above 2.28.
- The release job's `needs` list now includes `build-qt-manylinux-2-28` so
  the manylinux artifacts are included in every tagged release.

Closes ActivityWatch#1390
@greptile-apps

greptile-apps Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds an independent manylinux_2_28 Qt build that produces and ABI-checks Linux artifacts for glibc 2.28 systems.

  • Adds a containerized Qt build, test, packaging, artifact-renaming, and release path.
  • Adds an offline GLIBC symbol gate for ZIP and AppImage payloads.
  • Preserves hidden files while flattening the packaged Qt directory.
  • Adds the tagged-release bundled-server version guard and makes AppImage extraction failures fatal.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
.github/workflows/release.yml Adds the independent manylinux Qt job, release-line guard, ABI gate invocation, artifact upload, and release dependency.
scripts/package/abi-gate.sh Adds ZIP and AppImage extraction plus ELF symbol-version validation, now failing closed when AppImage extraction fails.
Makefile Replaces glob-based Qt bundle flattening with an archive-preserving copy that includes hidden files.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Tag[Tag or source revision] --> Build[Build Qt in manylinux_2_28 container]
    Build --> Package[Create ZIP and AppImage]
    Package --> Rename[Add manylinux_2_28 suffix]
    Rename --> Gate[Scan bundled ELFs for GLIBC above 2.28]
    Gate --> Upload[Upload build artifacts]
    Upload --> Release[Attach artifacts to draft tagged release]
Loading

Reviews (2): Last reviewed commit: "fix(ci): add server version guard and ha..." | Re-trigger Greptile

Comment on lines +586 to +587
- name: Set up Node
uses: actions/setup-node@v6

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing bundled-server version guard

When a release tag is built with an aw-server-rust submodule from another major or minor release line, this job proceeds directly from version detection to building and publishing the server inside its manylinux assets. Unlike the standard Qt and Tauri jobs, it never rejects the mismatch, causing the release to ship an incompatible or stale server binary.

Knowledge Base Used:

Comment on lines +116 to +118
;;
esac
done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 AppImage ABI scan fails open

When both AppImage extraction methods fail, this branch only prints a warning and the script later exits successfully from an empty violation log. The release can therefore publish an unscanned AppImage containing GLIBC references above 2.28; extraction failure must make the gate fail.

Knowledge Base Used: Build and distribution

if: github.event_name == 'push' || github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-22.04
container:
image: quay.io/pypa/manylinux_2_28_x86_64

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security Mutable release build dependencies

This release-producing job selects its container by an unpinned tag and also uses mutable major-version tags for checkout, Node setup, caching, and artifact upload. Repointing any of these upstream tags changes the code producing published manylinux binaries without a repository change or an artifact-integrity check, so pin the container digest and action SHAs. How this was verified: The job uploads these assets, and the release job downloads all artifacts and attaches them without a checksum or provenance guard.

Knowledge Base Used: Automation and maintenance

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

The glob 'mv aw-qt-tmp/*' skips dotfiles, leaving aw-qt-tmp non-empty
and causing 'rmdir: failed to remove aw-qt-tmp: Directory not empty'
in the manylinux_2_28 container. Switch to cp -a dir/. which copies
all files (including hidden) then rm -rf the source directory.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI investigation: two failures, different root causes.

manylinux_2_28 job (mine to fix) — fixed in 25617da:

The make package step failed with rmdir: failed to remove 'aw-qt-tmp': Directory not empty. The glob mv aw-qt-tmp/* doesn't cover hidden dotfiles (e.g. PyInstaller may leave .cache or similar), so aw-qt-tmp wasn't fully drained before rmdir. Fixed by switching to cp -a dist/activitywatch/aw-qt/. dist/activitywatch/ + rm -rf, which copies all files including hidden ones.

macOS job — pre-existing, not caused by this PR:

test_own_data_does_not_return_via_peer fails with duplicate column name: data in aw-datastore. The same job failed on master's run 32632451121 from 2026-08-23, before this PR existed.

Two P1 issues from Greptile review:

1. Manylinux job lacked the aw-server-rust version guard that build-qt and
   build-qt-tauri already have. Without it a tagged release could ship a
   manylinux bundle containing an aw-server binary from a mismatched release
   line. The new step is identical to the guard in those jobs.

2. abi-gate.sh failed open when AppImage extraction failed — it warned and
   exited 0, so a release could publish an unscanned AppImage containing
   GLIBC refs above 2.28. Extraction failure now exits 2 (script error).
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Fixed the two P1 findings from the Greptile review (commit a270596):

1. Missing aw-server-rust version guard in the manylinux job — Added the same Verify aw-server-rust submodule version matches release tag step that build-qt and build-qt-tauri already have. Without this, a tagged release could publish a manylinux bundle with a mismatched server binary (e.g. the #1380 class of bug).

2. AppImage ABI scan fails open on extraction failure — Changed abi-gate.sh to exit 2 when neither unsquashfs nor --appimage-extract can extract the AppImage, instead of printing a warning and continuing. An unextractable AppImage was previously treated as a clean pass, so an artifact containing GLIBC refs above 2.28 could be published unscanned.

Greptile re-review triggered.

cp -a dist/activitywatch/aw-qt/. dist/activitywatch/ fails because
the aw-qt binary inside the dir has the same name as the source
directory — cp can't overwrite the directory with the non-directory.
Move to a temp name first, then copy contents, then remove.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

All CI checks now pass. PR is ready for maintainer review.

@ErikBjare
ErikBjare merged commit 115e49e into ActivityWatch:master Aug 26, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can we add manylinux_2_28 Linux build target for glibc 2.28 compatibility (CentOS/RHEL 8)

2 participants