Skip to content

fix(sqs): per-(account, region) state infra; scope SQS queues to the request region - #2675

Merged
vieiralucas merged 11 commits into
mainfrom
fix-region-scoped-state
Oct 4, 2026
Merged

vieiralucas merged 11 commits into
mainfrom
fix-region-scoped-state

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Summary

Bug audit 2026-10-01 Tier 0.1 (and the shared infra for 0.2-0.6). MultiAccountState keys state by account only, so regional services that key resources by name collide across regions and their lists leak other regions. This PR adds the reusable per-(account, region) mechanism and converts SQS as the first service; the other services follow in stacked PRs.

Shared infra (fakecloud_core::multi_account)

  • RegionalState<T>: one account's state split per region (plain JSON object keyed by region, no tuple keys). region() never creates, region_mut() creates on first use.
  • MultiRegionState<T> = MultiAccountState<RegionalState<T>> with regional / regional_mut / regional_get_mut, by_arn / by_arn_mut (ARN's account + region, never creates), iter_regional, default_regional[_mut].
  • Snapshot migration: SplitByRegion trait + MultiAccountState::into_regional() / RegionalState::from_legacy() move each resource of a legacy account-wide state into the region its ARN names (records without one go to the server's default region). MultiAccountState::map_into for type-changing migrations.

SQS

  • Queues live per (account, region): the same name coexists in two regions; ListQueues, GetQueueUrl and every queue-addressed call see only the request region. A queue ARN of another region (or account) no longer resolves to a same-name local queue.
  • QueueUrl stays <endpoint>/<account>/<name> (byte-identical, tfacc's QueueNameFromURL needs exactly 3 path segments; one fakecloud endpoint serves every region). The request region picks which region's queue a URL addresses; the ARN always names its own region.
  • Every ARN-addressed path uses the ARN's region + account instead of the frozen server region: cross-service delivery (SNS/EventBridge/S3/Scheduler fanout), Lambda SQS ESM poller, Pipes SQS source, SQS resource-policy provider, IAM resource-tag lookup.
  • SqsDelivery::queue_arn_for_url now takes the region; Step Functions resolves a QueueUrl in the execution's region.
  • CloudFormation provisioner creates/updates/deletes queues in the stack's region; drift detection (resource_exists) takes the stack region.
  • Snapshot schema v3; v1/v2 snapshots are migrated on load (parse_sqs_snapshot), message move tasks follow their source queue.
  • Introspection (/_fakecloud/sqs/messages, expiration tick, force-dlq) cover every account and region; wire shapes unchanged, so no SDK change.

Test plan

  • cargo test -p fakecloud-core --lib (new RegionalState tests: isolation, by_arn, JSON round-trip, legacy split)
  • cargo test -p fakecloud-sqs (205 pass; new: two-region coexistence/list/GetQueueUrl/delete, cn-partition queue unreachable from us-east-1, delivery by ARN region, v1/v2 snapshot migration)
  • E2E sqs_regional (new: two SDK clients in different regions, SNS fanout lands in the ARN region, legacy v2 snapshot loads into ARN regions, restart keeps regions) plus sqs, sqs_persistence, sqs_message_move, sqs_kms: all pass
  • cargo nextest run -p fakecloud-conformance -E 'test(sqs)': 28/28
  • CFN/SFN/server targeted unit tests; cargo clippy --workspace --all-targets -- -D warnings; cargo fmt

Docs: website/content/docs/services/sqs.md (Regions section), website/content/docs/reference/persistence.md (new Regional state migration section; also lists the already-merged CloudFormation split). SDKs unaffected (no wire-shape change).


Summary by cubic

Adds a reusable per-(account, region) state container and scopes SQS queues to the request region, so the same queue name can exist independently in different regions without state leaking across them. Previously queues were keyed by account only, which caused regional services to collide.

Under the new behavior ListQueues, GetQueueUrl, and every queue-addressed call see only the request region, and all ARN-based lookups and deliveries (SNS/EventBridge/S3 fanout, Lambda ESM poller, Pipes, resource policies, IAM tag lookups) resolve the queue in its ARN's region and account. QueueUrl stays <endpoint>/<account>/<name>: the request region picks which queue a URL addresses, falling back to the queue of that account and name in the one other region that has it (None when ambiguous), while the ARN always names its own region. CloudFormation creates, updates, and drift-checks queues in the stack's region (stack sets check each instance's account and region); Step Functions resolves a QueueUrl in the execution's region.

Introspection now carries regions: /_fakecloud/sqs/messages entries include region and queueArn, and force-dlq targets one account and region via accountId/region query parameters instead of every same-named queue. A new account's first region inherits shared resources from the default account.

Migration

  • Snapshot schema is now v3; v1/v2 snapshots migrate on load through a generic legacy-split parse (MultiAccountState::map_into / RegionalState::map), placing each queue into the region its ARN names.
  • QueueUrl behavior is unchanged on the wire, so AWS SDK clients need no changes. The fakecloud SDKs were updated to expose the new introspection fields and to scope force_dlq by account and region.

Written for commit 893200c. Summary will update on new commits.

Review in cubic

Add RegionalState<T> / MultiRegionState<T> so a regional service keeps an
independent state per account and region, with ARN-addressed lookups and a
SplitByRegion migration that moves legacy account-wide snapshots into the
region each resource's ARN names.
Queues live per (account, region): the same queue name coexists in two
regions, ListQueues/GetQueueUrl/queue-addressed calls see only the request
region, and every delivery or lookup by queue ARN (SNS/EventBridge/S3 fanout,
Lambda ESM poller, Pipes, resource policies, IAM resource tags) uses the
ARN's region and account instead of the server region. Step Functions
resolves a QueueUrl in the execution's region. v2 snapshots migrate each
queue into the region its ARN names.
- /_fakecloud/sqs/messages entries carry the queue's region and ARN (queue
  URLs carry no region), in the server and all seven SDKs.
- force-dlq acts on one account and region (accountId/region query
  parameters, default the server's) instead of every same-named queue.
- A new account's first region inherits shared resources from the default
  account, as a new account did before the region split.
@vieiralucas
vieiralucas force-pushed the fix-region-scoped-state branch from 4f8497b to b3b7f3e Compare October 4, 2026 13:32
Queues are regional: each region's stack instance queue is listed by an
SQS client of that region (one per account and region), instead of
expecting one us-east-1 client to see both.
@vieiralucas
vieiralucas merged commit 6c56446 into main Oct 4, 2026
158 checks passed
@vieiralucas
vieiralucas deleted the fix-region-scoped-state branch October 4, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant