Skip to content

feat: support AWS ECR image pushes as a second Dependency source - #32

Open
caseycs wants to merge 1 commit into
mainfrom
feat/ecr-push-events
Open

caseycs wants to merge 1 commit into
mainfrom
feat/ecr-push-events

Conversation

@caseycs

@caseycs caseycs commented Aug 24, 2026

Copy link
Copy Markdown
Owner

An ECR repository is a Dependency: when a new image lands in it, the repos whose manifests pin that image are out of date. ECR has no webhooks, so an EventBridge rule on the default bus delivers ECR Image Action / PUSH events to an SQS queue the pod long-polls. An ECR repository declares its Dependents in a renovate-trigger/dependents resource tag on the repository itself — the registry analogue of a renovate.trigger.json Trigger declaration, decentralized for the same reason as ADR-0004.

Off unless RT_ECR_QUEUE_URL is set. No AWS client is constructed, and a GitHub-only deployment renders and behaves exactly as before.

The core change

Collector.Add and Resolver.Resolve were keyed on a GitHub owner/name string. The Batch element is now a typed graph.Dependency{Kind, Name} and the Resolver dispatches to one DependentsFetcher per kind. Batching, the tumbling window, mutual exclusion, and job creation are untouched — a third source would be a fetcher plus a kind, not a new pipeline.

For ECR the name is the repository ARN: ListTagsForResource takes an ARN, the ARN is globally unique so two repositories named api in different accounts stay distinct, and the event's resources array is empty for this detail-type so it is built from account + region + repository-name (with the partition derived from the region, so GovCloud and China work).

Suggested review order

The diff is large but separable. It reads most easily in this order:

  1. internal/graph/ — the new value type and the shared parse/validate helpers.
  2. internal/resolve/resolver.go, internal/batch/collector.go, internal/webhook/handler.go, internal/ghapp/client.go — the seam widening. No behaviour change; every pre-existing test passes with only mechanical updates. Kept in one commit with the rest because main.go cannot compile between the two halves.
  3. internal/awsecr/Consumer (the SQS analogue of webhook) and TagReader (the analogue of ghapp).
  4. cmd/renovate-trigger/main.go, internal/config/ — wiring and the two new env vars.
  5. terraform/, chart/ — infrastructure and the chart knobs.
  6. docs/adr/0005-*.md — the reasoning, including the options rejected.

Decisions worth your attention

A customer-managed KMS key is required, not preferred. Verified against the EventBridge docs: it "does not support using Amazon SQS queues that are encrypted with an AWS owned key. This includes targets, as well as Amazon SQS queues specified as dead-letter queues for targets." That rules out SQS-managed SSE. The key is essentially the whole running cost — about $1/month; default-bus rules on AWS-service events are free and one consumer long-polling at 20s stays inside the SQS free tier.

An unparseable message body is deliberately not deleted. It will never become parseable, so the redrive policy surfaces it in the dead-letter queue rather than the app hiding it. Well-formed events we simply do not act on are deleted immediately. This is what gives the DLQ a purpose, given we otherwise delete on receive.

SQS is ingress transport only. This is the one real tension with ADR-0002, so it is written down rather than left for a reader to trip over: SQS is a durable queue with redelivery, close to the retry queue that ADR said this service would not build. We put one in front of the pipeline and then discard its durability — the message is deleted as soon as the Dependency joins the Batch, and retention defaults to one hour rather than fourteen days. The Batch is still in-memory and single-replica, so ADR-0001 is reinforced, not superseded.

Credentials via EKS Pod Identity only. AWS_EC2_METADATA_DISABLED=true is set whenever ECR is enabled, so a misconfigured association crash-loops at the boot-time queue check instead of silently running with the node instance profile.

A new go job in CI. There was none — go test never ran in CI, only the kind e2e. This change adds a concurrent poll loop sharing the collector with the HTTP server, so its tests would have been decorative. -race is on for that reason.

"Tag" now means four things — git tag, the tags key, image tag, AWS resource tag. CONTEXT.md is prescriptive, so it now disambiguates all four and forbids the bare word.

Security note for reviewers

On GitHub, opting a Dependency in requires write access to its default branch. On ECR it requires ecr:TagResource, which many CI roles already hold via AmazonEC2ContainerRegistryPowerUser — so anyone with it can nominate arbitrary GitHub repositories as Dependents. The ceiling is bounded (Renovate runs with the CronJob's own credentials and only opens PRs), but it is a real asymmetry. terraform/README.md carries a ready-to-use SCP denying the tag key outside a platform role.

Verification

Check Result
gofmt -l cmd internal clean
go vet ./... clean
go test -race ./... all packages pass
go mod tidy no diff
helm lint chart passes
helm unittest chart 27 tests pass
helm template with and without ECR enabled renders; schema rejects enabled: true without a queue URL and region
terraform fmt -check -recursive clean
terraform validate (module and example) valid
hack/e2e.sh not run — needs kind and Docker; the unchanged GitHub path is covered by the existing integration test

New tests include cross-kind integration coverage in collector_flow_test.go: a GitHub tag event and an ECR image push land in one Batch, resolve through different fetchers, and produce a single Renovate run whose RENOVATE_REPOSITORIES is the deduplicated union — with one Dependent declared by both, so dedup across kinds is proven. Plus the ECR-disabled path and multi-tag push dedup.

Known limitations, deliberately not addressed

  • The GitHub App credentials stay required even for an ECR-only deployment. Relaxing that touches config validation, the chart's existingSecret contract, the webhook route, and the install story. Recorded in ADR-0005 as a follow-up.
  • flushTimeout is still 30s against serial resolution. Pre-existing, but ECR adds a second network hop per Dependency, so a large Batch is likelier to exhaust the flush context. Worth a follow-up.
  • AWS caps a tag value at 256 characters, bounding an ECR repository to roughly a dozen Dependents. No GitHub-side equivalent.
  • Single region — ECR emits to the default bus in the pushing region and EventBridge targets must be same-region, so a multi-region estate needs one module instance per region.

Note this is a feat:, so release-please will cut a minor release of both the image and the chart.

🤖 Generated with Claude Code


AI session - caseycs
cd ~/github/renovate-trigger; claude -r 45790ce7-d141-4339-8a00-5520e8ab178a

An ECR repository is a Dependency: when a new image lands in it, the repos whose
manifests pin that image are out of date. ECR has no webhooks, so an EventBridge
rule on the default bus delivers ECR Image Action / PUSH events to an SQS queue
the pod long-polls. An ECR repository declares its Dependents in a
renovate-trigger/dependents resource tag on the repository itself — the registry
analogue of a renovate.trigger.json Trigger declaration, decentralized for the
same reason (ADR-0004).

The Batch element becomes a typed graph.Dependency{Kind, Name} and the Resolver
dispatches to one DependentsFetcher per kind. Batching, the tumbling window,
mutual exclusion, and job creation are unchanged, so a third source would be a
fetcher plus a kind rather than a new pipeline. For ECR the name is the repository
ARN: ListTagsForResource takes an ARN, it is globally unique so two repositories
named api in different accounts stay distinct, and the event's resources array is
empty for this detail-type so it is constructed from account + region + name.

The feature is off unless RT_ECR_QUEUE_URL is set — no AWS client is constructed
and a GitHub-only deployment renders and behaves exactly as before.

Notable decisions, all recorded in ADR-0005:

- A customer-managed KMS key is required, not preferred. EventBridge documents
  that it does not support SQS queues encrypted with an AWS owned key, for targets
  or for target dead-letter queues, which rules out SQS-managed SSE.
- An unparseable message body is deliberately not deleted, so the redrive policy
  surfaces it in the dead-letter queue instead of the app hiding it. Well-formed
  events we do not act on are deleted immediately.
- SQS is ingress transport only. Messages are deleted as soon as the Dependency
  joins the Batch and retention defaults to one hour, so ADR-0002's lossy-by-design
  stance holds end to end and ADR-0001 is reinforced rather than superseded.
- Credentials come from EKS Pod Identity only, with AWS_EC2_METADATA_DISABLED set
  when ECR is enabled so a misconfigured association fails at the boot-time queue
  check instead of silently using the node instance profile.

Also adds a go job to CI. There was none: go test never ran there, only the kind
e2e. This change introduces a concurrent poll loop sharing the collector with the
HTTP server, so its tests would otherwise be decorative.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@caseycs
caseycs force-pushed the feat/ecr-push-events branch from 576f2a6 to 5e83361 Compare August 24, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant