fix: preserve all cloud attributions

README.md

@@ -113,8 +113,8 @@ Each run creates a timestamped subdirectory under the output directory containin
 | `resolution_results_*.txt` | Successfully resolved domains and their IPv4/IPv6 addresses, pipe-delimited (`domain\|ip1\|ip2`). Prefixed `WILDCARD\|` (confirmed catch-all) or `WILDCARD_ZONE\|` (zone answers for anything, unverifiable) — see [Wildcard DNS detection](#wildcard-dns-detection) |
 | `unresolved_results_*.txt` | Domains that could not be resolved after all retries |
 | `takeover_candidates_*.txt` | `DANGLING\|origin\|target\|category\|recommendation\|evidence\|hops\|chain` — the chain records the full CNAME path (`a -> b -> c`), so the claimable hop is visible without re-resolving. Plus `NS_TAKEOVER\|` lines for unresolvable nameservers |
-| `csp_matches_*.txt` | One handoff record per matched address: `domain\|ip\|provider\|region\|service\|prefix\|border_group`. Prefixed `WILDCARD\|` when the resolution was a catch-all. See [Cloud IP attribution](#cloud-ip-attribution) |
-| `allocator-targets-v1.json` | Versioned provider-aware allocator handoff. Groups service, prefix, and border-group metadata per provider/hostname/address/region and excludes wildcard observations. |
+| `csp_matches_*.txt` | One handoff record per provider-published attribution: `domain\|ip\|provider\|region\|service\|prefix\|border_group`. One address can produce several records. Prefixed `WILDCARD\|` when the resolution was a catch-all. See [Cloud IP attribution](#cloud-ip-attribution) |
+| `allocator-targets-v1.json` | Versioned provider-aware allocator handoff. Groups service, prefix, and border-group metadata per provider/hostname/address/region and excludes wildcard observations; one address may produce multiple records when a provider publishes multiple regions. |
 | `environment_results_*.json` | Run metadata (command, external IP, Docker status) |
 | `provider_catalogues.json` | AWS, GCP, and Azure catalogue status, source, retrieval time, snapshot identifier, and any failure reason |
 | `{provider}_ip_ranges.json` | Validated provider ranges and provenance used by this run, from either a live source or a fresh cache |
@@ -161,8 +161,9 @@ little: of roughly 10,500 published AWS prefixes, over half carry the generic `A
 and the ones that matter operationally — `EC2` in a named region — look identical unless the
 provider's own metadata is kept.
 
-Each match is therefore written as a record carrying the provider's published region and
-service:
+Each attribution is therefore written as a record carrying the provider's published region and
+service. Providers can publish the same prefix under multiple services or regions, and an address
+can fall within overlapping prefixes, so one address may produce several records:
 
 ```
 domain|ip|provider|region|service|prefix|border_group

ROADMAP.md

@@ -2,6 +2,80 @@
 
 Single source of truth for planned work.
 
+## Goal-aligned roadmap — agreed August 2026
+
+This section supersedes the priority claims in the historical roadmap below. The
+older material remains as delivery history, but it was largely derived from a
+review performed against the wrong product goal.
+
+### Confirmed goal
+
+DNSResolver is the DNS-only front end for a separate cloud-address reclaim
+allocator. Cloud IP candidates are the primary product; dangling CNAME and NS
+findings are secondary observations. AWS, GCP, and Azure are required providers.
+The tool retains complete DNS observations, excludes wildcard, uncertain, and
+incomplete results from actionable output, and fails closed. It must work
+correctly before it is optimised. Lambda expansion is shelved.
+
+### Delivery sequence
+
+Each item is one independently verifiable PR. Branch from current `main`, run the
+exact local CI workflow, validate resolver changes against real DNS/provider data,
+then confirm GitHub jobs `test`, `gitleaks`, and `trufflehog` before squash merge.
+
+| # | Work item | State | Done when |
+|---|---|---|---|
+| 1 | Scope secret scans to changed commits | Merged, PR 160 | New-branch push and PR scans use a valid merge-base range; all actions are SHA-pinned; both event paths pass. |
+| 2 | Define the resolver-to-allocator contract | Merged, PR 159 | Schemas, examples, fail-closed invariants, current AWS consumer compatibility, and documentation drift tests pass. |
+| 3 | Make DNS resolution reliable | Merged, PR 161 | Timeouts/nameservers reach both resolver libraries; A and AAAA fallback results survive; final retry state is correct; public dual-stack, IPv6-only, and negative controls behave correctly. |
+| 4 | Fail closed on incomplete provider catalogues | Merged, PR 163 | AWS/GCP/Azure use bounded retries, validation, provenance, freshness-limited integrity-checked snapshots, and explicit states; no DNS processing occurs with an unusable provider. |
+| 5 | Publish the versioned allocator target document | Merged, PR 164 | A successful run atomically publishes schema-valid provider-aware JSON, groups repeated metadata, excludes wildcard observations, and retains the legacy pipe output. |
+| 6 | Pin Lambda input to the triggering object version | Merged, PR 165 | The reference handler reads the exact S3 object version from the event and rejects incomplete version information. This does not reopen Lambda expansion. |
+| 7 | Preserve every provider attribution | Implemented on `fix/preserve-cloud-attribution`; PR pending | Duplicate services, overlapping prefixes, identical CIDRs across providers, and multiple provider-published regions survive catalogue parsing, matching, pipe output, and JSON publication without changing the schema. Legacy scalar cache snapshots remain readable. |
+| 8 | Replace linear cloud-range matching with an indexed matcher | Deferred until measured need | A deterministic benchmark is defined before implementation; outputs are byte-for-byte equivalent to item 7; measured runtime and memory are reported at representative scale. |
+| 9 | Make output publication atomic and observable | Deferred; not an initial-workability gate | Required writes cannot be swallowed; partial runs cannot leave a stale actionable document; injected open/write/replace failures produce a nonzero, explicit failure with regression tests. |
+| 10 | Complete observation/run-manifest publication | Deferred; not an initial-workability gate | The checked observation and manifest contracts are emitted by real runs; manifest state reflects provider completeness and publication outcome; actionable output is null for incomplete/failed runs. |
+| 11 | Validate the real allocator consumer end to end | Planned, cross-repository | The AWS consumer ingests a current DNSResolver document unchanged; GCP/Azure route only to provider-aware implementations or are explicitly rejected; a synthetic authorized fixture proves no provider is misrouted. |
+| 12 | Measure and calibrate large-run behavior | Planned last | A repeatable representative benchmark replaces the unmeasured README scalability claim; resource limits and operational guidance reflect measured results. |
+
+### Explicitly not doing
+
+- No cloud allocation or reclaim API calls in DNSResolver.
+- No HTTP probing, TLS inspection, screenshots, or active takeover confirmation.
+- No Lambda expansion, deployment machinery, release, tag, or version bump in an
+  individual work-item PR.
+- No matcher-performance refactor inside attribution item 7; correctness is frozen
+  first so optimisation has a trustworthy equivalence oracle.
+- No separate security-hardening phase. Items 8-10 are optional reliability and
+  performance follow-ups, deferred until measured operational need.
+- No claim that unit coverage proves resolver behavior; real DNS, real provider
+  catalogues, consumer validation, and actual merge CI remain separate gates.
+
+### Delivery state — 2026-08-14
+
+Item 7 is implemented on `fix/preserve-cloud-attribution` and ready for its
+focused pull request.
+
+- Focused suite: 91 tests passed.
+- Full local CI: Ruff and format passed; baseline 5 passed; full suite 310 passed
+  at 95% coverage for `classes` and `imports`.
+- Live catalogues: 4,826 AWS prefixes and 44,225 Azure prefixes have multiple
+  attributions; Azure has 44,101 prefixes with multiple published regions.
+- Live end-to-end synthetic-address acceptance used current provider catalogues:
+  AWS preserved `AMAZON`, `EC2`, and `S3`; Azure preserved `global` and
+  `southeastasia` as separate schema-valid targets.
+- A production run through the host system resolver processed `s3.amazonaws.com`
+  and `aws.amazon.com` using current AWS, GCP, and Azure catalogues. It emitted 43
+  pipe attributions and 20 schema-valid allocator targets; one current S3 address
+  retained `AMAZON`, `EC2`, and `S3`. Direct public-resolver transport and the real
+  allocator consumer remain external acceptance gates.
+- Before delivery: review the diff, run TruffleHog v3.96.0 on the intended commit
+  range, commit, push, open one focused PR, and verify push, PR, and post-merge
+  `main` checks.
+- The live gate may be delegated using `docs/EXTERNAL-ACCEPTANCE.md`; its report
+  must keep automated, catalogue, system-resolver, public-resolver, and allocator
+  evidence separate.
+
 > **On its derivation.** The plan below came from [`REVIEW.md`](REVIEW.md) (2026-07-15), which
 > assessed the tool against a misread goal — cloud attribution treated as a supporting attribute
 > rather than as one of two co-equal products. That review now carries a correction, and the work

classes/allocator_contract.py

@@ -23,14 +23,11 @@ def publish_allocator_targets(csp_path, output_dir):
             target["provider"],
             target["hostname"],
             target["ip"],
+            target["region"],
         )
         if identity not in grouped:
             grouped[identity] = target
             continue
-        if grouped[identity]["region"] != target["region"]:
-            raise ValueError(
-                f"Conflicting regions for one allocator target at line {line_number}"
-            )
         for field in ("services", "prefixes", "network_border_groups"):
             grouped[identity][field].update(target[field])
 
@@ -44,7 +41,14 @@ def publish_allocator_targets(csp_path, output_dir):
                 "network_border_groups": sorted(target["network_border_groups"]),
             }
         )
-    targets.sort(key=lambda item: (item["provider"], item["hostname"], item["ip"]))
+    targets.sort(
+        key=lambda item: (
+            item["provider"],
+            item["hostname"],
+            item["ip"],
+            item["region"],
+        )
+    )
 
     temporary = destination.with_suffix(".json.tmp")
     try:

classes/csp_ip_addresses.py

@@ -7,27 +7,28 @@ def __init__(
         aws_ipv6,
         azure_ipv4,
         azure_ipv6,
-        metadata=None,
+        metadata_by_provider=None,
     ):
         self.gcp_ipv4 = gcp_ipv4
         self.gcp_ipv6 = gcp_ipv6
         self.aws_ipv4 = aws_ipv4
         self.aws_ipv6 = aws_ipv6
         self.azure_ipv4 = azure_ipv4
         self.azure_ipv6 = azure_ipv6
-        # CIDR -> (region, service). A match is only actionable downstream if the
-        # consumer knows where the address is allocated from and what it serves,
-        # so the publishers' own metadata is carried through rather than dropped.
-        self.metadata = metadata or {}
+        # Provider -> CIDR -> all published (region, service, border-group)
+        # attributions. Providers can publish the same CIDR, and one provider can
+        # publish a CIDR under multiple services, so neither dimension is scalar.
+        self.metadata_by_provider = metadata_by_provider or {}
 
-    def describe(self, cidr):
+    def describe(self, provider, cidr):
         """
-        Region, service and network border group for a matched prefix.
+        Every region, service and network border group for a provider prefix.
 
         Unpublished fields read 'unknown' rather than being inferred — only AWS
         publishes a border group, and not every prefix carries a region.
         """
-        return self.metadata.get(cidr, ("unknown", "unknown", "unknown"))
+        entries = self.metadata_by_provider.get(provider, {}).get(cidr)
+        return entries or [("unknown", "unknown", "unknown")]
 
     def get_gcp_ipv4(self):
         return self.gcp_ipv4

docs/ALLOCATOR-CONTRACT.md

@@ -22,7 +22,8 @@ breaking schema change when those allocators arrive.
 
 The shared resolver pipeline publishes `allocator-targets-v1.json` after a
 successful run while retaining the pipe-delimited `csp_matches_*.txt` interface.
-The JSON publisher groups provider metadata for identical targets and excludes
+The JSON publisher groups provider metadata by provider, hostname, IP, and
+provider-published region, and excludes
 `WILDCARD` and `WILDCARD_ZONE` observations. If any provider catalogue is
 unusable, or a CSP record is malformed, no allocator-target document is
 published.
@@ -54,7 +55,10 @@ published.
 
 The array form and the `hostname`, `ip`, and `region` names are the compatibility
 surface used by the current AWS consumer. Consumers should deduplicate by the
-combination of provider, hostname, and IP rather than assuming IP alone is unique.
+combination of provider, hostname, IP, and region rather than assuming IP alone
+is unique. One address may have multiple provider-published regions, particularly
+when Azure publishes both a global service tag and a regional tag for the same
+prefix; each region remains a separate target record.
 
 See the checked [AWS compatibility example](../contracts/examples/allocator-targets-v1.aws.json)
 and [multi-cloud example](../contracts/examples/allocator-targets-v1.multicloud.json).

docs/EXTERNAL-ACCEPTANCE.md

@@ -0,0 +1,77 @@
+# External acceptance for cloud attribution
+
+This is the handoff for an agent running on a host that can reach provider
+catalogue endpoints over HTTPS and public DNS resolvers over UDP/TCP port 53.
+It complements the mocked automated suite; it is deliberately not a pytest
+test because repository tests must never make real DNS or HTTP requests.
+
+## Copy/paste prompt
+
+```text
+Take over the live acceptance of DNSResolver cloud attribution. Work from the
+current pull-request branch; do not modify code unless a failed acceptance check
+identifies a reproducible defect. Do not push to main, tag a release, bump
+VERSION, or use real client/employer names. Use only public example/test domains
+and provider-owned service names.
+
+Read HANDOVER.md and AGENTS.md before running anything. Then:
+
+1. Record `git status --short --branch` and the exact commit SHA. Stop if the
+   checkout contains unrelated changes.
+2. Install both requirements files in Python 3.12 and run the repository's local
+   gates exactly:
+   `.venv/bin/ruff check .`
+   `.venv/bin/ruff format --check .`
+   `.venv/bin/pytest tests/baseline/ -v`
+   `.venv/bin/pytest --cov=classes --cov=imports --cov-report=term-missing`
+3. Run the focused attribution tests:
+   `.venv/bin/pytest -q tests/test_allocator_contract.py tests/test_allocator_publisher.py tests/test_cloud_ip_ranges.py tests/test_cloud_service_provider_checks.py tests/test_csp_ip_addresses.py tests/test_resolver.py`
+4. Run DNSResolver through its production CLI against a small file containing
+   public, provider-owned names. Include `s3.amazonaws.com`, which previously
+   resolved through the host's system resolver. Use a temporary config with no
+   `nameservers` key for the system-resolver run; the checked-in config supplies
+   explicit public resolvers and is not a system-resolver control.
+5. Repeat the run with `--nameservers 1.1.1.1,8.8.8.8`. This explicit-resolver
+   check is mandatory on your host because the originating agent's execution
+   environment blocked direct public DNS. Capture whether UDP and TCP port 53
+   work, rather than treating a timeout as a DNSResolver defect.
+6. For every successful production run, verify:
+   - `provider_catalogues.json` marks aws, gcp, and azure usable and records a
+     source, retrieval time, and snapshot identifier;
+   - `csp_matches_*.txt` retains every provider-published attribution for each
+     matched address, including repeated services, overlapping prefixes, or
+     multiple regions when present;
+   - `allocator-targets-v1.json` validates against
+     `contracts/allocator-targets-v1.schema.json`;
+   - every JSON target has `actionability: actionable`, a provider in aws/gcp/
+     azure, a non-empty provider-published region, and prefixes containing its IP;
+   - no `WILDCARD` or `WILDCARD_ZONE` observation enters the allocator document.
+7. If the allocator checkout is available, feed the produced JSON to the real
+   consumer. Confirm AWS is accepted unchanged. Confirm GCP and Azure are routed
+   only to provider-aware implementations or are explicitly rejected; they must
+   never be sent through the AWS allocation path.
+8. Report results as four separate evidence classes: automated tests, live
+   provider catalogue retrieval, system-resolver DNS, and explicit-public-
+   resolver DNS. Include commands, exit codes, output paths, redacted excerpts,
+   and the commit SHA. Do not describe a blocked or skipped check as passing.
+
+Success means all automated checks pass, all three catalogues are usable, at
+least one real DNS answer traverses the production pipeline into a schema-valid
+allocator document, and the explicit public-resolver result is evidenced. Real
+allocator compatibility is a separate result: passed, failed, or unavailable.
+```
+
+## Evidence expected back
+
+- Commit SHA and clean/dirty status.
+- Python and dependency versions.
+- Test counts and coverage.
+- Catalogue status for AWS, GCP, and Azure.
+- System-resolver and explicit-resolver commands and exit codes.
+- The relevant pipe records and corresponding allocator JSON records.
+- JSON Schema validation result.
+- Allocator-consumer result, or a precise statement that its checkout was not
+  available.
+- Any failure separated into product defect, network/environment limitation, or
+  unavailable external dependency.
+