[AMORO-4359][AMS] Support AWS Secrets Manager as a ConfigShade - #4360
[AMORO-4359][AMS] Support AWS Secrets Manager as a ConfigShade#4360wangxianghu wants to merge 1 commit into
Conversation
3d81103 to
98de4ee
Compare
e22116b to
fa2ba56
Compare
|
hi @xxubai could you please help merge this one, this pr has been verified in our prod cluster
|
xxubai
left a comment
There was a problem hiding this comment.
Please keep sensitive values out of exception chains. validateArn appends rawContent, while extractJsonField retains the Jackson parse exception; ConfigShadeUtils logs the full throwable. A plaintext or malformed secret can therefore reach startup logs
xxubai
left a comment
There was a problem hiding this comment.
Could this provider live in amoro-ams or an optional ConfigShade plugin instead? AWS resolution is only used during AMS startup, while putting the SDK in amoro-common makes unrelated format, optimizer, and client artifacts inherit the vendor dependency graph.
xxubai
left a comment
There was a problem hiding this comment.
Please complete the operator/release surface: add software.amazon.awssdk:secretsmanager to LICENSE-binary
6875fe3 to
da519fa
Compare
makes sense, moved to amoro-ams |
|
@xxubai done, thanks for your comments |
32f2e1a to
05fe2e6
Compare

Why are the changes needed?
Close #4359.
AMS keeps sensitive values such as
admin-passwordanddatabase.passwordinconfig.yaml. Today the only shade providers arebase64and the built-indefault, so operators still have to store the real secret (obfuscated at best) on disk. When AMS runs on AWS, the natural place for these secrets is AWS Secrets
Manager, with credentials resolved from the environment (EKS IRSA, ECS/EC2
instance profile, etc.) rather than checked into a file.
This PR adds an
aws-smConfigShadeimplementation so that a sensitive configvalue can be an AWS Secrets Manager ARN, and AMS resolves the real value at
startup:
omit it when the secret value is the plaintext itself.
starts with a wrong secret.
Brief change log
ARN into its value, extracting a JSON field when # is given.
extracting the region from the ARN.
testable wrapper over the AWS SDK using the URL-connection HTTP client (no
Netty), the default credentials provider chain, and per-process caching.
expected to be in one region, and a cross-region reference fails fast.
url-connection-client dependencies.
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Added TestSecretReference (ARN/region/#field parsing, whitespace, and
every malformed-input rejection) and TestAwsSecretsManagerConfigShade
(plaintext vs JSON-field decryption, dotted field names taken literally rather
than as a nested path, client memoization, cross-region fail-fast, and the
invalid-JSON / missing-field / malformed-ARN failure paths) — 23 tests, all
passing.
Add screenshots for manual tests if appropriate
Run test locally before making a pull request
Documentation