Skip to content

feat(aws): storage.s3tables — @aws-cdk/aws-s3tables-alpha port - #155

Open
so0k wants to merge 3 commits into
convert-aws-backupfrom
convert-aws-s3tables
Open

feat(aws): storage.s3tables — @aws-cdk/aws-s3tables-alpha port#155
so0k wants to merge 3 commits into
convert-aws-backupfrom
convert-aws-s3tables

Conversation

@so0k

@so0k so0k commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Storage slice PR 12 — storage.s3tables: the @aws-cdk/aws-s3tables-alpha port (v2.263.0-alpha.0)

Stacked on #154. The final module of the storage slice. TableBucket, TableBucketPolicy, Table, TablePolicy, Namespace, permissions, util — 188 unit tests across all 8 upstream suites. Fourth alpha port, same disciplines: alpha-churn tracker, provenance headers, deviation notes.

The maintenance_configuration find (workflow blocker → live finding, 11th live-only defect class)

aws_s3tables_table.maintenance_configuration is an object-typed provider attribute (not a block): Terraform object types require ALL members. FOUR wrong shapes were eliminated in sequence — the last one only catchable live:

  • omitting the absent member (compaction-only / snapshot-only) → terraform validate fails: attribute "iceberg_snapshot_management" is required;
  • a literal-null member → the generated L1's nested OutputReference setter crashes (Object.keys(null));
  • null-filled scalar leaves → passes terraform validate AND synth, but fails live apply: S3 Tables auto-populates server-side defaults for every unset maintenance value and returns them on read (Provider produced inconsistent result after apply: was null, but now cty.NumberIntVal(120) — receipts in thread);
  • the correct shape: pin every unset leaf to AWS's documented server-side default (compaction enabled/512MB; snapshot management enabled/120h/1 snapshot) so plan matches read-back.

Regression tests cover both one-sided shapes; the live fixture deploys BOTH one-sided tables so both default-filled sides are proven against AWS.

Mapping notes

  • encryptionencryption_configuration (sse_algorithm AES256/aws:kms + kms_key_arn — id-vs-ARN audit pre-applied); the upstream auto-created-KMS-key path mirrored with encryption.Key.
  • unreferencedFileRemovalmaintenance_configuration.iceberg_unreferenced_file_removal (Durations → day numbers, upstream validation verbatim).
  • account/region (newly exposed via AwsConstructProps) are stripped in every constructor — forwarding them defeats the same-account short-circuit in iam.Grant, spuriously attaching resource policies on every grant. Pinned by a regression test.
  • Policies render iam.PolicyDocument as resource_policy JSON.

Flagged for review

tableBucketName / tableName / namespaceName are kept required (upstream requires them; documented deviation note in-source). This diverges from the grid uniqueResourceName default used elsewhere in the slice — making them optional with grid defaults is an additive change we can make later if preferred.

Live integ (make s3tables.table) — receipts in thread

Real TableBucket + Namespace + ICEBERG Table: GetTableBucket/GetNamespace/GetTable read-backs, GetTableMaintenanceConfiguration reading back compaction enabled @128mb from the one-sided (null-filled) maintenance shape, drift oracle, clean destroy (force_destroy).

so0k added 3 commits August 7, 2026 11:10
…263.0-alpha.0

Full port of the s3tables alpha module into the storage.s3tables sub-namespace:
TableBucket, TableBucketPolicy, Table, TablePolicy, Namespace, permissions,
util — 188 unit tests across all 8 upstream suites.

- maintenance_configuration is an OBJECT-typed provider attribute requiring ALL
  members: absent compaction/snapshotManagement renders as a null-filled member
  (null leaves are valid; omitting the member fails terraform validate, and a
  literal-null MEMBER crashes the generated L1 setter) — regression tests for
  both one-sided shapes, independently proven via synth + tofu validate
- encryption → encryption_configuration (sse_algorithm AES256/aws:kms +
  kms_key_arn — id-vs-ARN audit); upstream auto-created-KMS-key path mirrored
  with encryption.Key
- unreferencedFileRemoval → maintenance_configuration
  iceberg_unreferenced_file_removal (Duration → day numbers, validation verbatim)
- account/region stripped in every constructor (grant same-account
  short-circuit footgun — pinned by regression test)
- tableBucketName/tableName/namespaceName kept REQUIRED per upstream fidelity
  (documented deviation from the grid-default house pattern — flagged for review)
- alpha provenance headers + churn tracker; policies render iam.PolicyDocument
  as resource_policy JSON
…table

Real S3 Tables stack: bucket/namespace/table read-backs (GetTableBucket /
GetNamespace / GetTable), GetTableMaintenanceConfiguration asserting the
compaction-only maintenance shape (enabled, 128MB target) — the null-filled
absent-member design proven live — and the post-apply drift oracle.

Adds aws-sdk-go-v2/service/s3tables v1.18.4.
…fter apply' finding

Live run 1 failed: S3 Tables AUTO-POPULATES server-side defaults for every
unset maintenance value and returns them on read (snapshot_management came
back enabled/1/120 against planned nulls), so null-filled leaves violate the
provider apply contract. Absent maintenance members/leaves are now pinned to
AWS's documented defaults (compaction enabled/512MB; snapshot management
enabled/120h/1 snapshot). Regression tests updated for both one-sided shapes;
integ fixture extended with a snapshot-only table so BOTH default-filled
sides are live-proven; orphaned run-1 resources destroyed cleanly (3).
@so0k

so0k commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live integ receipts — make s3tables.table (aws-vault → tcons-vincent, us-east-1) — PASS 41.01s (run 4; runs 1-3 documented below)

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
s3tables: table bucket tcons-s3tables-integ exists (arn:aws:s3tables:us-east-1:694710432912:bucket/tcons-s3tables-integ)
s3tables: namespace integ_ns exists
s3tables: table integ_table exists (ICEBERG, arn:aws:s3tables:...:bucket/tcons-s3tables-integ/table/2cf7b8de-...)
s3tables: compaction enabled at 128MB target file size
s3tables: snapshot-only table integ_snapshot_table reads back 48h/3 snapshots
No changes. Your infrastructure matches the configuration.   <-- post-apply drift oracle (tofu plan -detailed-exitcode == 0)
Destroy complete! Resources: 4 destroyed.
--- PASS: TestS3TablesTable (41.01s)

Run history (honesty log)

  • Run 1 FAILED — the PR's headline finding: Provider produced inconsistent result after apply ×3 on aws_s3tables_table — planned nulls for the absent snapshot-management member came back as AWS server-side defaults (was null, but now cty.NumberIntVal(120) / (1) / StringVal("enabled")). The null-filled shape had passed terraform validate AND unit synth — only live apply catches server-side-default read-backs. Fixed by pinning unset leaves to AWS's documented defaults; orphaned resources destroyed cleanly (3).
  • Runs 2-3 FAILED (transient, unrelated): 409 ConflictException: The bucket is in a transitional state because of a previous deletion attempt — S3 Tables blocks name reuse for ~15-25 min after deletion. No fix needed; retried after the propagation window.
  • Run 4 PASS: both one-sided maintenance tables (compaction-only AND snapshot-only) applied without provider errors, read back correctly, and the zero-drift re-plan proves the defaults pin holds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant