Skip to content

Release workflow generation - #50

Merged
David Elner (delner) merged 3 commits into
mainfrom
release_workflow_generation
Jul 14, 2026
Merged

David Elner (delner) merged 3 commits into
mainfrom
release_workflow_generation

Conversation

@delner

@delner David Elner (delner) commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Adds bin/workflow: a CLI that generates a release workflow into a repo from a parameterized template, and keeps it in sync with upstream over time.

Why

Today, setting up releases for a new SDK means copying one of our in-repo release-*.yml files and hand-stripping the test-bed scaffolding (the version-bump job, local ./actions refs, version-apply step, PR triggers, Slack fallbacks.) It's easy to get wrong, and with no clean way to track which version of sdk-actions you're on or pull in later improvements.

bin/workflow replaces that with a real bootstrap-and-maintain flow:

  • Bootstrap in one command. You get a clean workflow (without scaffolding and placeholders) plus a printed checklist of the setup steps (environments, trusted publishing, optional Slack).
  • Stay in sync. The generated file carries a provenance header recording the template, the sdk-actions ref, and your params. That makes updates mechanical: re-render the old and new baselines and merge only the upstream delta in, preserving any local edits.
  • The templates become the canonical reference. Repos generate their workflow instead of copying ours; our in-repo workflows go back to being just test harnesses.

How you use it

Bootstrap a workflow (each template exposes its own flags; --help lists them):

bin/workflow generate release/ruby/turnkey \
  --gem-name braintrust --version-module Braintrust \
  --dest ../braintrust-ruby/.github/workflows/release.yml

Templates are release/<lang>/<shape><lang> is ruby / py / js, and <shape> is:

  • turnkey — we build and publish in one gated job.
  • custom — you build the artifact yourself; a gated, publish-only job verifies its attestation and ships it (same trusted-publishing + provenance guarantees).

Then, later, pull in upstream improvements:

bin/workflow compare release.yml    # preview the diff vs a freshly-rendered baseline
bin/workflow update  release.yml    # merge the upstream delta in, keeping your edits

update bumps the pinned sdk-actions ref and re-stamps the provenance header, so each update sets up the next one. Local customizations survive the merge; only genuine conflicts are flagged. bin/workflow validate <file> schema-checks a workflow before you commit it.

Design (high level)

  • Flat, self-contained templates. One template per language × shape, each reading top-to-bottom like the workflow it produces — so it doubles as the canonical, diffable reference. There are only six, and consumer workflows are thin wiring over the (already-DRY) actions, so composition would cost readability for little gain.
  • Provenance header = the update anchor. A machine-readable # sdk-actions: {…} comment records template + ref + params, which is all compare/update need to reproduce a baseline and 3-way-merge upstream changes in.
  • SHA-pinned, no new dependencies. The CLI is stdlib-only and reuses the existing ERB engine; generated workflows pin actions by SHA (--ref, default origin/main).
  • Golden fixtures guard drift. Each test/release/<lang>/ mirrors a real consumer repo; its rendered workflows are committed and diff-guarded in CI, so a template edit can't land without its output being reviewed — the same discipline already used for generated actions/.

Bonus: SBOM correctness fix

While validating the release path, this branch also corrects the generated CycloneDX SBOMs so they match the authoritative package-SBOM shape per ecosystem: a proper metadata.component subject (name + version + ecosystem-correct purl), runtime-only components, and a dependencies graph rooted at the subject. A small scripts/sbom_verify.rb check now runs in the release harnesses' dry-run, guarding the SBOM generators against silent content regressions (dev tooling leaking in, a dropped runtime dep, a missing subject/purl).

@delner
David Elner (delner) requested a review from a team June 17, 2026 18:21
@delner David Elner (delner) self-assigned this Jun 17, 2026
@delner David Elner (delner) added the enhancement New feature or request label Jun 17, 2026
@delner David Elner (delner) changed the title Release workflow for Javascript Release workflow generation Jun 17, 2026
@delner
David Elner (delner) force-pushed the release_workflow_generation branch 2 times, most recently from d195f1e to 97be4fa Compare July 9, 2026 18:30
@delner

Copy link
Copy Markdown
Collaborator Author

Note: This PR is based on #61 . We should wait for that to merge first.

@delner
David Elner (delner) marked this pull request as ready for review July 9, 2026 18:39
@delner
David Elner (delner) force-pushed the release_workflow_generation branch from 97be4fa to 22f561e Compare July 9, 2026 18:41
Comment thread bin/workflow

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about making these python scripts instead? I would expect everyone has that installed and set up properly to get braintrust itself working. Feels like it'll make contribution easier in the future as well.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is worthwhile, not because I didn't examine alternatives before I chose something, but for a few other reasons: (1) Ruby is common for dev-ops tooling (e.g. Chef) and does come pre-installed/available in many environments (Linux/Mac), (2) ERB is a popular, stable templating engine that serves the purpose well (I couldn't find better ones), (3) Ruby is a requirement for ERB.

Moreover, I don't think the value proposition to rewrite everything is strong enough when the product that's consumed doesn't actually contain or require any Ruby (its just GHA YAML), the parts written in Ruby is merely support tooling (doesn't get modified often), the Ruby toolset is trivial to install, and when it does need to be modified development is likely happening in managed/templated environments anyways (e.g. Starfolk).

@delner
David Elner (delner) force-pushed the release_workflow_generation branch from 22f561e to 8e41fdc Compare July 13, 2026 18:02
@delner
David Elner (delner) merged commit 2ecea1e into main Jul 14, 2026
44 checks passed
@delner
David Elner (delner) deleted the release_workflow_generation branch July 14, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants