Skip to content

feat(aws): storage slice prereqs — SecretRotation (SAR) unblock, storage.rds scaffolding, CaCertificate - #142

Open
so0k wants to merge 5 commits into
mainfrom
storage-prereqs
Open

feat(aws): storage slice prereqs — SecretRotation (SAR) unblock, storage.rds scaffolding, CaCertificate#142
so0k wants to merge 5 commits into
mainfrom
storage-prereqs

Conversation

@so0k

@so0k so0k commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Storage slice PR 1/N — prerequisites: SecretRotation (SAR) unblock, storage.rds scaffolding, CaCertificate

First PR of the storage-services stack (aws-rds → docdb → elasticache → neptune → redshift, reference tag v2.263.0). This PR lands the shared prerequisites so the RDS PRs can stack cleanly on top.

1. encryption.SecretRotation re-enabled (SAR-backed rotation)

The previous "HARD BLOCKER" (construct always threw, claiming Terraform has no serverlessrepo resource) was incorrect: @cdktn/provider-aws ships aws_serverlessapplicationrepository_cloudformation_stack. SecretRotation now mirrors upstream aws-secretsmanager/lib/secret-rotation.ts (v2.263.0) with the SAR app deployed through that resource (CAPABILITY_IAM + CAPABILITY_RESOURCE_POLICY, rotation Lambda ARN from the stack's RotationLambdaARN output). Documented deviations: explicit CFN stack name (required by the Terraform resource; reuses the deterministic rotation function name), 'aws'-partition semantic-version pin, ImportedRotationLambda ARN-only wrapper.

Hardening that fell out of live testing (all drift-oracle-verified, see below):

  • escapeTerraformTemplateLiteral (src/aws/util.ts): excludeCharacters is free text that legitimately contains ${/%{ — escaped to $${/%%{ in every Terraform string-template context (SecretRotation SAR parameter + Secret.generateSecretString).
  • CFN parameter whitespace trim: CloudFormation trims edge whitespace from stack parameter values on read-back, so a leading space in excludeCharacters (upstream's default RDS exclude set starts with one) drifts forever. Since the value is a character set, edge whitespace is deterministically moved inward (moveEdgeWhitespaceInward).
  • Rotation ownership of the secret value: once a rotation schedule attaches, the SAR Lambda replaces AWSCURRENT out-of-band — without lifecycle.ignore_changes on the aws_secretsmanager_secret_version, the next terraform apply would clobber the rotated credentials with the stale initial value (and every plan reports drift). RotationSchedule now marks the owning Secret (duck-typed through SecretTargetAttachment), which emits ignore_changes = [secret_string, version_stages]. Upstream CloudFormation never reads the value back, so it has no equivalent problem. Unit tests cover both the rotated (ignore_changes present) and non-rotated (absent) paths.

2. storage.rds sub-namespace + CaCertificate

  • src/aws/storage/rds/ created (export * as rds from "./rds", same idiom as compute.batch), starting with ca-certificate.ts ported byte-close from upstream (including the deprecated RDS_CA_RDS2048_G1 alias quirks). This decouples the upcoming DocDB port (whose only rds dependency is CaCertificate) from the RDS PRs.

3. Live integration test (integ/aws/encryption, make secret-rotation)

End-to-end fixture: real Postgres db.t3.micro in an isolated VPC (no NAT) + Secrets Manager interface endpoint (private DNS), Secret.attach() folding connection fields, and SecretRotation with the AWS-published SecretsManagerRDSPostgreSQLRotationSingleUser@1.1.618 SAR app, rotateImmediatelyOnUpdate left at its default (true) so a real rotation executes. The Go test:

  1. asserts the rotation schedule + Lambda attach,
  2. polls DescribeSecret.LastRotatedDate until the four-step rotation completes against the live database,
  3. asserts the rotated password honors every excluded character — the exclude set deliberately contains ${ and %{, proving the escaping end-to-end through both template contexts,
  4. runs the post-apply drift oracle (tofu plan -detailed-exitcode == 0),
  5. destroys (SAR CFN stack deletion incl. Lambda VPC ENIs).

Run 1 receipt: rotation completed live; drift oracle then caught all three defects fixed above (secret-version replacement, SAR parameter whitespace trim, plus a fixture-level data-source password regeneration now under ignore_changes = [password]). Run 2 (fixes applied): apply → real rotation → zero-drift → destroy, receipts in thread.

Notes for reviewers

  • test/aws/encryption/secret-rotation.test.ts is the ported upstream suite (v2.263.0) + repo snapshot + the escaping/trim/ignore_changes regressions.
  • SecretRotationApplication catalog retains the v2.233.0-shaped global partition fallbacks; the per-application additionalSemanticVersions table v2.263.0 introduced is TODO-linked, and isMultiUser inference matches v2.263.0.
  • HostedRotation remains unavailable (CFN-transform-backed; no Terraform equivalent) — unchanged in this PR.

…rage.rds scaffolding + CaCertificate

Storage slice PR 1 (reference tag v2.263.0):

- encryption.SecretRotation: the 'no serverlessrepo resource' HARD BLOCKER
  was incorrect -- implement on aws_serverlessapplicationrepository_cloudformation_stack,
  mirroring upstream aws-secretsmanager/lib/secret-rotation.ts.
- escapeTerraformTemplateLiteral (src/aws/util.ts): escape ${ / %{ in free-text
  values (SAR excludeCharacters parameter, Secret.generateSecretString).
- moveEdgeWhitespaceInward: CloudFormation trims edge whitespace from stack
  parameter values (perpetual drift; upstream RDS default exclude set starts
  with a space) -- relocate edge whitespace inward (character set, order-free).
- RotationSchedule marks the owning Secret so its initial
  aws_secretsmanager_secret_version gets lifecycle.ignore_changes
  [secret_string, version_stages]: rotation replaces AWSCURRENT out-of-band;
  without this the next apply clobbers rotated credentials and every plan drifts.
- storage.rds sub-namespace (compute.batch idiom) with ca-certificate.ts ported
  byte-close from upstream -- decouples the DocDB port from the RDS PRs.
- Live integ (make secret-rotation): real Postgres db.t3.micro in an isolated
  VPC + Secrets Manager interface endpoint; a REAL rotation via
  SecretsManagerRDSPostgreSQLRotationSingleUser@1.1.618 completes, the rotated
  password honors all 15 excluded characters (incl. ${ / %{ payload), zero-drift
  plan, full destroy. PASS 1583s.
@so0k

so0k commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live apply/verify/destroy receipts — make secret-rotation (aws-vault → tcons-vincent, us-east-1)

Run 2 (final, all fixes applied) — PASS 1583.49s:

TestSecretRotation 15:46:25 Apply complete! Resources: 19 added, 0 changed, 0 destroyed.
secret_rotation_test.go:39: SAR rotation lambda attached: arn:aws:lambda:us-east-1:694710432912:function:secretrotationRotation409C9840
secret_rotation_test.go:53: rotation completed at 2026-08-06 08:46:24.701 +0000 UTC   <-- REAL rotation against the live Postgres instance
secret_rotation_test.go:72: rotated password honors all 15 excluded characters        <-- ${ / %{ payload arrived intact
TestSecretRotation 15:46:36 No changes. Your infrastructure matches the configuration. <-- drift oracle (tofu plan -detailed-exitcode == 0)
TestSecretRotation 16:07:12 Destroy complete! Resources: 19 destroyed.
--- PASS: TestSecretRotation (1583.49s)

Run 1 (drift-oracle catch — why the hardening in this PR exists): rotation completed live, but the post-apply plan showed exit code 2 with three drifts:

  1. aws_secretsmanager_secret_version forced replacement (rotation replaced AWSCURRENT out-of-band; a re-apply would have clobbered the rotated credentials with the stale initial password) → construct-level lifecycle.ignore_changes = [secret_string, version_stages] when a rotation schedule attaches.
  2. SAR stack parameter excludeCharacters lost its leading space on read-back (CloudFormation trims edge whitespace in parameter values) → moveEdgeWhitespaceInward (the value is a character set; order-free). Note upstream RDS's default password-exclude set starts with a space, so this would have bitten every RDS PR.
  3. Fixture-level: data.aws_secretsmanager_random_password regenerates every plan → ignore_changes = [password] on the test DB (also the correct real-world pattern for a rotated instance).

Destroy note: SAR CFN stack deletion takes ~7 min (rotation Lambda VPC ENI cleanup) — the 45m make timeout accounts for it.

so0k added 4 commits August 6, 2026 16:12
The default local backend embeds a machine-dependent absolute tfstate path
in synth output (conventions.md rule from PR #118) -- CI test shard 4/5
failed on the path mismatch.
Repo pattern: snapshotted test stacks pass gridBackendConfig to the
AwsStack constructor (secret.test.ts et al.); direct HttpBackend
attachment is the tech-debt variant.
…tion

encryption.Secret's generateSecretString path uses the
aws_secretsmanager_random_password DATA source, which generates a new
password on every plan/refresh -- every plan drifted and every apply
REPLACED the live password. CloudFormation generates once server-side at
create; parity requires lifecycle.ignore_changes [secret_string] on the
version when the value is construct-generated (rotation additionally
ignores version_stages, as before). Caught by the RDS foundations
integ drift oracle (TestRdsGroups).
Comment on lines +85 to +92
new dataAwsSecretsmanagerRandomPassword.DataAwsSecretsmanagerRandomPassword(
stack,
"MasterPassword",
{
passwordLength: 20,
excludePunctuation: true,
},
);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should use aws.encryption to generate secrets? or wait for CDKTN 0.24 and updated terraform-provider-aws bindings with function and ephemeral resources support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant