feat(auth): add DISABLE_BOUND_ID_TOKEN option to opt out of certificate-bound ID tokens

[macastelaz]

Important

This must merge before the agentic-identities-bound-token feature branch is merged to main. Docs PR #14557 documents this option.

Why

When an agent identity certificate is available, ComputeEngineCredentials requests certificate-bound ID tokens by default. A bound ID token is only accepted by targets that authenticate the caller over mTLS with the same certificate. Targets reached over standard HTTPS, such as a Cloud Run service's *.run.app URL or a custom domain, reject it with 401 Unauthorized.

Per the ID token binding 1-pager (Sep 30 decision), ID tokens stay bound by default, and callers get a per-target bind_id_token setting to turn binding off where needed. The process-wide opt-out (GOOGLE_API_ENABLE_RUNTIME_BOUND_TOKEN=false) is unchanged.

What changes

• New IdTokenProvider.Option.DISABLE_BOUND_ID_TOKEN. Callers pass it per target, the same way as FORMAT_FULL and LICENSES_TRUE:

  IdTokenCredentials.newBuilder()
      .setIdTokenProvider((IdTokenProvider) credentials)
      .setTargetAudience("https://my-service-12345.us-central1.run.app")
      .setOptions(Arrays.asList(IdTokenProvider.Option.DISABLE_BOUND_ID_TOKEN))
      .build();

• ComputeEngineCredentials.idTokenWithAudience(): with the option, the certificate lookup is skipped and the identity endpoint is called with a plain GET. Without it, behavior is unchanged (bound whenever a certificate is available and binding isn't turned off by the env var).

• Other credential types (ServiceAccountCredentials, ImpersonatedCredentials, UserCredentials) already ignore options they don't use, so the same code works outside agent identity environments.

• Access tokens are unaffected.

API shape

Option is a set of on/off flags rather than a value, so this adds only the "off" flag (DISABLE_BOUND_ID_TOKEN). Leaving it unset means "auto", which currently means bound. This matches the 1-pager's tri-state (unset = auto) without adding an "enable" flag that would do nothing today; one can be added later without breaking callers if "auto" starts deciding per target. Python exposes the same control as bind_id_token: Optional[bool] (googleapis/google-cloud-python#18559).

Tests

Unit tests

New ComputeEngineCredentialsTest cases, all with an agent identity certificate present and binding enabled:

• idTokenWithAudience_disableBoundIdToken_requestsUnboundToken: GET, no request body, no forced format=full.
• idTokenWithAudience_disableBoundIdTokenWithFormatFull_requestsUnboundFullToken: still GET; format=full is kept when the caller asks for it.
• idTokenWithAudience_disableBoundIdToken_skipsCertificateLookup: with a certificate config pointing to a missing file, the default request fails, but the opted-out request succeeds, which shows the certificate is never read.
• idTokenCredentials_withDisableBoundIdTokenOption_requestsUnboundToken: the option set on IdTokenCredentials reaches ComputeEngineCredentials.

The existing bound-ID-token tests are unchanged and still pass. Full oauth2_http suite: 1064 tests, 0 failures. fmt is clean.

Live test on GKE

Run on a GKE cluster with agent identity enabled, using a test-only local merge of this PR with #14595 (GKE support) and #14556 (already on the base branch). A single process fetches ID tokens through IdTokenCredentials from ComputeEngineCredentials (ADC) and sends them to an agent-to-agent receiver pod. The receiver accepts only bound tokens over mTLS (whose cnf must match the client certificate), and only unbound tokens over plain HTTP.

Check	Defaults	GOOGLE_API_ENABLE_RUNTIME_BOUND_TOKEN=false
ID token with DISABLE_BOUND_ID_TOKEN	unbound (no cnf); receiver: plain HTTP 200, mTLS 401	unbound
ID token for another target without the option, same credentials, fetched afterwards	still bound (cnf = certificate leaf); receiver: mTLS 200	unbound
Access token after the ID-token opt-out	still bound (401 on a plain, non-mTLS endpoint)	unbound

The option unbinds only the ID token it's set on. Other targets and access tokens are unaffected, and it's harmless when binding is already off.

Internal links (Googlers only): results · harness · cluster setup and manifests (shared with #14595)

[gemini-code-assist]

Code Review

This pull request introduces AgentIdentityUtils to support Agent Identity runtime certificate discovery and token binding, enabling ComputeEngineCredentials to request certificate-bound access tokens from the GCE metadata server. Feedback on the changes highlights several locations in ComputeEngineCredentials where HttpResponse connections are not disconnected or closed upon receiving a 404 status code, potentially leading to HTTP connection leaks.

[lqiu96]

LGTM. FYi, I think you can update the PR and the CI should pass now