Skip to content

Reuse SesClient in AwsMailProvider#1044

Open
pditommaso wants to merge 2 commits into
masterfrom
reuse-aws-sdk-clients
Open

Reuse SesClient in AwsMailProvider#1044
pditommaso wants to merge 2 commits into
masterfrom
reuse-aws-sdk-clients

Conversation

@pditommaso
Copy link
Copy Markdown
Collaborator

Summary

  • AwsMailProvider.send() builds a fresh SesClient per invocation and never closes it — every send leaks the underlying Apache HTTP client, connection pool and SSL context. Hoisted to a singleton instance initialized in @PostConstruct and closed in @PreDestroy.
  • AwsEcrService.stsClient(region) builds a new DefaultCredentialsProvider on every call. Closing the STS client does not close the provider, so its IMDS/container credential refresh HTTP clients and background executors stay alive. Replaced with a single provider initialized in @PostConstruct and closed in @PreDestroy; stsClient(String region) is now non-static so it can use the shared field.

The other stsClient(String region, Credentials) overload uses a StaticCredentialsProvider, which is a stateless POJO — left as-is.

Background

While investigating a suspected memory leak in prod (which turned out to be JVM warmup transient — slope reverses after pods stabilize), the code review surfaced these two real leaks. Volume is low (Mail message sent ~110/day across pods, Assuming AWS role ~1/day), so neither materially contributes to RSS in current prod traffic — but both are strict resource leaks that compound over long-lived pods.

Test plan

  • ./gradlew :compileGroovy — passes
  • ./gradlew :test --tests '*AwsEcrServiceTest*' — 48/48 pass, 0 failures
  • CI green
  • Verify in dev: SES email send still works after rollout (uses the shared client across many sends)
  • Verify in dev: ECR jump-role assume still succeeds (shared DefaultCredentialsProvider across STS clients)

🤖 Generated with Claude Code

pditommaso and others added 2 commits May 15, 2026 21:19
`AwsMailProvider.send()` was building a fresh `SesClient` per invocation
and never closing it, leaving the underlying Apache HTTP client,
connection pool and SSL context alive. `AwsEcrService.stsClient(region)`
likewise built a new `DefaultCredentialsProvider` on every call; closing
the STS client does not close the provider, so its IMDS/container
credential refresh HTTP clients and background executors leaked.

Both clients are now created once per process (via `@PostConstruct`) and
released on shutdown (`@PreDestroy`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…plied credentials provider

Empirically verified with a tracking wrapper: StsClient.close() invokes
close() twice on the user-supplied AwsCredentialsProvider via the
internal AttributeMap.close() path. Sharing a single
DefaultCredentialsProvider across STS clients wrapped in
withCloseable { ... } would therefore tear down its underlying
ContainerCredentialsProvider / InstanceProfileCredentialsProvider
(HTTP client + refresh executor) after the first call, breaking
subsequent credential refreshes in prod where EKS pod identity or EC2
IMDS resolves.

Restoring the per-call DefaultCredentialsProvider.builder().build()
ephemeral instance — wasteful but correct. The leak rate (~1 STS call
per day in prod per investigation) does not justify a more invasive
restructuring. The AwsMailProvider singleton fix is kept since that
path has no withCloseable around the client.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pditommaso pditommaso changed the title Reuse AWS SDK clients in AwsMailProvider and AwsEcrService Reuse SesClient in AwsMailProvider May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant