feat(observability): solution attribution via native AWS_SDK_UA_APP_ID (#319, alt to #338)

AGENTS.md

@@ -67,6 +67,7 @@ Branch names: `(feat|fix|chore|docs)/<issue-number>-short-description` (e.g. `do
 - **Branch without issue number** — Unauthorized work.
 - **`MISE_EXPERIMENTAL=1`** — Required for `mise //cdk:build` and other namespaced tasks ([CONTRIBUTING.md](./CONTRIBUTING.md)).
 - **`prek install` fails** — Another hook manager owns `core.hooksPath`; see [CONTRIBUTING.md](./CONTRIBUTING.md).
+- **Dropping solution UA on a new AWS client (#319)** — every outbound AWS call carries `md/uksb-wt64nei4u6#{component}` (the `app/` segment is SDK-native via `AWS_SDK_UA_APP_ID`, set by `SolutionUaAspect`). Construct clients through the attributed factory: `cdk/src/` and `cli/src/` via `makeClient(Ctor, cfg)` / `makeDocClient(cfg)` (`cdk/src/handlers/shared/ua.ts`, `cli/src/ua.ts`); `agent/src/` via `aws_session.tenant_client`/`tenant_resource` (tenant-scoped) or `aws_session.platform_client` (unscoped, still attributed). A naked `new XxxClient({})` / `boto3.client(...)` silently loses solution attribution. Keep the three `ua` modules (`agent/src/ua.py`, `cdk/src/handlers/shared/ua.ts`, `cli/src/ua.ts`) identical in id/wire-format/sanitization.
 - **Transitive npm pin applied in only one place** — `integrations/jira-forge-app` is a standalone npm project *outside* the yarn workspaces, so a root `package.json` `resolutions` bump does **not** reach it. Mirror the pin into `integrations/jira-forge-app/package.json` `overrides` and re-lock (`npm install --package-lock-only`). `mise run drift-prevention` (`check:transitive-pin-sync`) fails locally if they drift. See [#712](https://github.com/aws-samples/sample-autonomous-cloud-coding-agents/issues/712).
 - **Package-specific pitfalls** — API type drift, CDK test bundling, Cedar parity, generated docs: see package `AGENTS.md` files.
 

agent/AGENTS.md

@@ -91,3 +91,4 @@ def test_a():
 - **Cedar parity** — `cedarpy==4.8.4` (agent) and `@cedar-policy/cedar-wasm` 4.8.2 (cdk) must move together. See [cdk/AGENTS.md](../cdk/AGENTS.md) and `docs/design/CEDAR_HITL_GATES.md` §15.6.
 - **Forgotten consumer** — Progress event schema changes need `cli/src/commands/watch.ts` and `test_progress_writer.py` updates.
 - **Image bundle** — CDK deploys this tree; root `mise run build` always runs agent quality.
+- **Un-attributed AWS SDK client (#319)** — build clients via `aws_session.tenant_client`/`tenant_resource` (tenant-scoped) or `aws_session.platform_client` (unscoped, still attributed); a naked `boto3.client(...)` silently drops solution attribution.

agent/src/aws_session.py

@@ -146,6 +146,8 @@ def _build_scoped_session(role_arn: str) -> Any:
     )
     from botocore.session import get_session as get_botocore_session
 
+    import ua
+
     region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
     task_id = _tags.get("task_id", "")
     # Role session name must be <=64 chars and match [\w+=,.@-]. task_id is a
@@ -156,8 +158,9 @@ def _build_scoped_session(role_arn: str) -> Any:
 
     # A dedicated STS client built from the *ambient* (compute-role) chain.
     # This is the role-chaining caller; the assumed SessionRole credentials it
-    # returns must NOT be used to build it, or refresh would recurse.
-    sts_client = boto3.client("sts", region_name=region)
+    # returns must NOT be used to build it, or refresh would recurse. Carries
+    # the static md/ UA segment so the assume-role call is attributed too.
+    sts_client = boto3.client("sts", region_name=region, config=ua.client_config())
 
     def _refresh() -> dict[str, str]:
         resp = sts_client.assume_role(
@@ -176,6 +179,10 @@ def _refresh() -> dict[str, str]:
         }
 
     botocore_session = get_botocore_session()
+    # Static md/ solution-attribution segment at the session level: it
+    # propagates to every client AND resource derived from this session, so
+    # all tenant-data calls carry it. (#319)
+    botocore_session.user_agent_extra = ua.static_user_agent_extra()
     # Deferred: the first assume_role happens on first credential use, not now,
     # so a transient STS hiccup at startup doesn't crash the agent before it
     # has even begun.
@@ -227,10 +234,19 @@ def get_session() -> Any:
                 ) from exc
         else:
             # Scoping not requested (local/dev/tests, or pre-provisioning):
-            # plain ambient session, behaviorally identical to pre-feature code.
-            _session = boto3.Session(
-                region_name=os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-            )
+            # plain ambient session. Built from an explicit botocore session so
+            # the static md/ solution-attribution segment rides every derived
+            # client/resource (propagation requires the botocore session). (#319)
+            from botocore.session import get_session as get_botocore_session
+
+            import ua
+
+            botocore_session = get_botocore_session()
+            botocore_session.user_agent_extra = ua.static_user_agent_extra()
+            region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
+            if region:
+                botocore_session.set_config_variable("region", region)
+            _session = boto3.Session(botocore_session=botocore_session)
             _scoped = False
         return _session
 
@@ -242,20 +258,56 @@ def is_scoped() -> bool:
     return bool(_scoped)
 
 
+def _merge_ua_config(kwargs: dict[str, Any]) -> dict[str, Any]:
+    """Return ``kwargs`` with the static md/ UA merged into any ``config``.
+
+    Preserves a caller-supplied ``botocore.config.Config``. Non-colliding keys
+    (``read_timeout`` etc.) survive via ``Config.merge``. ``user_agent_extra``
+    is the one key that *does* collide: botocore's ``merge`` gives precedence to
+    the argument, so a naive merge would silently drop the caller's extra. We
+    therefore *concatenate* both extras (caller first, then ours) so neither is
+    lost — matching the scoped-session path, which keeps both segments. (#319)
+    """
+    from botocore.config import Config
+
+    import ua
+
+    ua_config = ua.client_config()
+    existing = kwargs.get("config")
+    if existing is None:
+        kwargs["config"] = ua_config
+        return kwargs
+
+    caller_extra = getattr(existing, "user_agent_extra", None)
+    if caller_extra:
+        # merge() would let our user_agent_extra win outright; instead keep both
+        # by combining them into one extra. Overlay that combined UA onto
+        # ua_config (Config.merge lets the argument win only on the keys it
+        # sets — here just user_agent_extra), so every OTHER key ua_config
+        # carries survives. Falling through to the shared existing.merge below
+        # then preserves every caller key too, keeping this branch consistent
+        # with the no-collision path. (#319 review)
+        combined = f"{caller_extra} {ua.static_user_agent_extra()}"
+        ua_config = ua_config.merge(Config(user_agent_extra=combined))
+    kwargs["config"] = existing.merge(ua_config)
+    return kwargs
+
+
 def tenant_client(service_name: str, **kwargs: Any) -> Any:
     """boto3 client for tenant data.
 
     When the per-task SessionRole is configured, the client is built from the
-    tag-scoped, refreshable session. Otherwise it delegates directly to
-    ``boto3.client`` — behaviorally identical to the pre-feature code path
-    (and transparent to callers/tests that mock ``boto3.client``).
+    tag-scoped, refreshable session (which already carries the static md/ UA at
+    the session level). Otherwise it delegates directly to ``boto3.client`` —
+    behaviorally identical to the pre-feature code path (transparent to
+    callers/tests that mock ``boto3.client``) but with the md/ UA merged in.
     """
     session = get_session()
     if is_scoped():
         return session.client(service_name, **kwargs)
     import boto3
 
-    return boto3.client(service_name, **kwargs)
+    return boto3.client(service_name, **_merge_ua_config(kwargs))
 
 
 def tenant_resource(service_name: str, **kwargs: Any) -> Any:
@@ -265,4 +317,18 @@ def tenant_resource(service_name: str, **kwargs: Any) -> Any:
         return session.resource(service_name, **kwargs)
     import boto3
 
-    return boto3.resource(service_name, **kwargs)
+    return boto3.resource(service_name, **_merge_ua_config(kwargs))
+
+
+def platform_client(service_name: str, **kwargs: Any) -> Any:
+    """boto3 client for **platform** (non-tenant) calls on the ambient chain.
+
+    For the direct ``boto3.client(...)`` sites that deliberately bypass the
+    scoped session (CloudWatch Logs, Secrets Manager, bedrock-agentcore): they
+    talk to platform resources, not tenant data, so they use the compute role's
+    ambient credentials — but should still carry the static md/ solution
+    attribution. Merges the UA into any caller ``config``. (#319)
+    """
+    import boto3
+
+    return boto3.client(service_name, **_merge_ua_config(kwargs))

agent/src/bedrock_creds_helper.py

@@ -145,8 +145,13 @@ def resolve_credentials() -> dict[str, str]:
         return _ambient_credentials()
 
     try:
-        import boto3
+        # boto3 is imported here (not just via platform_client, which imports it
+        # lazily at call time) so a missing SDK still fails open to ambient creds
+        # — instead of raising an uncaught ImportError. (#319)
+        import boto3  # noqa: F401  -- availability probe for the fail-open below
         from botocore.exceptions import BotoCoreError, ClientError
+
+        from aws_session import platform_client
     except ImportError as exc:
         # boto3 missing/broken in the image is a packaging defect, not the
         # expected assume-role failure — name it explicitly so it can't hide.
@@ -157,7 +162,7 @@ def resolve_credentials() -> dict[str, str]:
     task_id = next((t["Value"] for t in tags if t.get("Key") == "task_id"), "")
     session_name = f"abca-bedrock-{task_id}"[:64] or "abca-bedrock"
     try:
-        resp = boto3.client("sts", region_name=region).assume_role(
+        resp = platform_client("sts", region_name=region).assume_role(
             RoleArn=role_arn,
             RoleSessionName=session_name,
             DurationSeconds=_CHAINED_SESSION_DURATION_S,

agent/src/config.py

@@ -54,10 +54,10 @@ def resolve_github_token() -> str:
         return cached
     secret_arn = os.environ.get("GITHUB_TOKEN_SECRET_ARN")
     if secret_arn:
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        client = boto3.client("secretsmanager", region_name=region)
+        client = platform_client("secretsmanager", region_name=region)
         resp = client.get_secret_value(SecretId=secret_arn)
         token = resp["SecretString"]
         # Cache in env so downstream tools (git, gh CLI) work unchanged
@@ -114,14 +114,19 @@ def resolve_linear_api_token(channel_metadata: dict[str, str] | None = None) ->
         import json
         from datetime import datetime, timedelta
 
-        import boto3
+        # boto3 is imported here (not just via platform_client, which imports it
+        # lazily at call time) so a missing SDK still degrades gracefully — skip
+        # Linear MCP — instead of raising an uncaught ImportError. (#319)
+        import boto3  # noqa: F401  -- availability probe for the graceful skip below
         from botocore.exceptions import BotoCoreError, ClientError
+
+        from aws_session import platform_client
     except ImportError as e:
         log("WARN", f"resolve_linear_api_token: boto3 unavailable ({e}); skipping")
         # nosemgrep: py-silent-success-masking -- optional Linear reactions token; boto3 unavailable
         return ""
 
-    sm = boto3.client("secretsmanager", region_name=region)
+    sm = platform_client("secretsmanager", region_name=region)
 
     def _fetch_token() -> dict | None:
         """Fetch + parse the per-workspace OAuth secret.
@@ -415,13 +420,18 @@ def resolve_jira_oauth_token(channel_metadata: dict[str, str] | None = None) ->
         import json
         from datetime import datetime
 
-        import boto3
+        # boto3 is imported here (not just via platform_client, which imports it
+        # lazily at call time) so a missing SDK still degrades gracefully — skip
+        # Jira feedback — instead of raising an uncaught ImportError. (#319)
+        import boto3  # noqa: F401  -- availability probe for the graceful skip below
         from botocore.exceptions import BotoCoreError, ClientError
+
+        from aws_session import platform_client
     except ImportError as e:
         log("WARN", f"resolve_jira_oauth_token: boto3 unavailable ({e}); skipping")
         return ""  # nosemgrep: py-silent-success-masking -- Jira feedback is advisory
 
-    sm = boto3.client("secretsmanager", region_name=region)
+    sm = platform_client("secretsmanager", region_name=region)
 
     def _fetch_token() -> dict | None:
         resp = sm.get_secret_value(SecretId=secret_arn)

agent/src/memory.py

@@ -35,12 +35,12 @@ def _get_client():
     global _client
     if _client is not None:
         return _client
-    import boto3
+    from aws_session import platform_client
 
     region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
     if not region:
         raise ValueError("AWS_REGION or AWS_DEFAULT_REGION must be set for memory operations")
-    _client = boto3.client("bedrock-agentcore", region_name=region)
+    _client = platform_client("bedrock-agentcore", region_name=region)
     return _client
 
 

agent/src/server.py

@@ -171,10 +171,10 @@ def _warn_cw_write_blocking(log_group: str, task_id: str | None, stamped: str) -
     covers both writers.
     """
     try:
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        client = boto3.client("logs", region_name=region)
+        client = platform_client("logs", region_name=region)
 
         stream = f"server_warn/{task_id or 'server'}"
         with _ctx_for_debug.suppress(client.exceptions.ResourceAlreadyExistsException):
@@ -198,10 +198,10 @@ def _warn_cw_write_blocking(log_group: str, task_id: str | None, stamped: str) -
 def _debug_cw_write_blocking(log_group: str, task_id: str | None, stamped: str) -> None:
     """Blocking CloudWatch write — only called from a background thread."""
     try:
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        client = boto3.client("logs", region_name=region)
+        client = platform_client("logs", region_name=region)
 
         stream = f"server_debug/{task_id or 'server'}"
         with _ctx_for_debug.suppress(client.exceptions.ResourceAlreadyExistsException):

agent/src/shell.py

@@ -75,10 +75,10 @@ def _log_error_cw_blocking(log_group: str, task_id: str | None, stamped: str) ->
     fire on the absence of the expected stream, not on this helper).
     """
     try:
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        client = boto3.client("logs", region_name=region)
+        client = platform_client("logs", region_name=region)
         stream = f"agent_error/{task_id or 'unknown'}"
         with contextlib.suppress(client.exceptions.ResourceAlreadyExistsException):
             client.create_log_stream(logGroupName=log_group, logStreamName=stream)

agent/src/telemetry.py

@@ -56,10 +56,10 @@ def _emit_metrics_to_cloudwatch(json_payload: dict) -> None:
     try:
         import contextlib
 
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        client = boto3.client("logs", region_name=region)
+        client = platform_client("logs", region_name=region)
 
         task_id = json_payload.get("task_id", "unknown")
         log_stream = f"metrics/{task_id}"
@@ -164,10 +164,10 @@ def _ensure_client(self):
 
         import contextlib
 
-        import boto3
+        from aws_session import platform_client
 
         region = os.environ.get("AWS_REGION") or os.environ.get("AWS_DEFAULT_REGION")
-        self._client = boto3.client("logs", region_name=region)
+        self._client = platform_client("logs", region_name=region)
 
         log_stream = f"trajectory/{self._task_id}"
         with contextlib.suppress(self._client.exceptions.ResourceAlreadyExistsException):

agent/src/ua.py

@@ -0,0 +1,76 @@
+"""Outbound AWS SDK User-Agent solution attribution (#319).
+
+Every AWS API call made by the agent carries two ABCA solution-attribution
+segments in the ``User-Agent`` header:
+
+    app/uksb-wt64nei4u6#{STACKNAME}      <- native AWS_SDK_UA_APP_ID env (no code here)
+    md/uksb-wt64nei4u6#agent             <- static, baked once at construction
+
+**The ``app/`` segment is emitted by the SDK itself.** Both botocore and the
+JS v3 SDK read the ``AWS_SDK_UA_APP_ID`` environment variable natively and
+render it as ``app/{value}`` (botocore ``configprovider.py`` maps it to the
+``user_agent_appid`` config; the value charset *includes* ``#``, so the
+``uksb-wt64nei4u6#{stack}`` form survives verbatim). CDK sets that env var on
+every Lambda / AgentCore runtime / ECS container, so this module contributes
+**nothing** to ``app/`` — and a customer can suppress it by setting the env
+var to the empty string. (This is the key simplification over the original
+``/``-separated design, which had to bypass the native field because ``/`` is
+not a legal app-id character. Using ``#`` keeps it native.)
+
+This module owns only the **static ``md/`` segment** — a stable
+per-component label baked once via ``user_agent_extra`` at session/client
+construction. There is intentionally no per-request trace handle and no
+event/middleware machinery: connection pools are never re-pinned, and
+request correlation is owned by X-Ray / structured-log request ids (#245),
+not the User-Agent.
+
+The TypeScript counterparts are ``cdk/src/handlers/shared/ua.ts`` and
+``cli/src/ua.ts`` — the solution id, wire format, and sanitization rules
+must stay identical across all three.
+"""
+
+from __future__ import annotations
+
+import string
+from typing import Any
+
+# AWS solution-attribution id for ABCA. Also appears (deploy-time
+# counterpart, #292) in the CloudFormation stack description in
+# ``cdk/src/main.ts`` and in the TS mirrors of this module. Per-surface
+# literal by design.
+SOLUTION_ID = "uksb-wt64nei4u6"
+
+# Stable per-component label: this surface IS the Python agent runtime.
+COMPONENT = "agent"
+
+# RFC 7230 token charset (the UA product-token alphabet). '#' is the
+# scheme's structural separator and is deliberately NOT here, so a hostile
+# component/label value cannot inject extra segments.
+_ALLOWED = frozenset(string.ascii_letters + string.digits + "!$%&'*+-.^_`|~")
+
+
+def sanitize_ua_value(raw: str) -> str:
+    """Replace every non-UA-token char (incl. non-ASCII) with ``-``."""
+    return "".join(c if c in _ALLOWED else "-" for c in raw)
+
+
+def static_user_agent_extra() -> str:
+    """The static ``md/`` segment baked at client/session construction.
+
+    Always ``md/{SOLUTION_ID}#{COMPONENT}`` — the ``app/`` segment is
+    contributed separately by the SDK from ``AWS_SDK_UA_APP_ID`` and is not
+    this module's concern.
+    """
+    return f"md/{SOLUTION_ID}#{sanitize_ua_value(COMPONENT)}"
+
+
+def client_config() -> Any:
+    """``botocore.config.Config`` carrying the static ``md/`` segment.
+
+    For direct ``boto3.client(...)`` call sites that don't go through a
+    shared session (see ``aws_session.platform_client``). Merge-friendly:
+    callers that already pass a ``Config`` should use ``.merge(...)``.
+    """
+    from botocore.config import Config
+
+    return Config(user_agent_extra=static_user_agent_extra())