Skip to content

dependency: account for the HTTP Age header when timing lease re-renders - #2160

Draft
peteski22 wants to merge 4 commits into
hashicorp:mainfrom
peteski22:fix-lease-check-wait-cache-age
Draft

dependency: account for the HTTP Age header when timing lease re-renders#2160
peteski22 wants to merge 4 commits into
hashicorp:mainfrom
peteski22:fix-lease-check-wait-cache-age

Conversation

@peteski22

@peteski22 peteski22 commented Jul 22, 2026

Copy link
Copy Markdown

Description

leaseCheckWait decides how long Consul Template sleeps before re-rendering a non-renewable Vault secret, using lease_duration measured from now. Behind a caching proxy in front of Vault, the response has already been held for some time — reported in the HTTP Age header — so the lease has lease_duration - Age left, not lease_duration. Consul Template ignored Age, so it slept for most of the original lease again and re-rendered only after the credential had already expired. This shows up most as a lease approaches its max_ttl, where the template keeps rendering a secret that is already dead.

This change:

It is a no-op for responses served directly by Vault (Age is zero), so behaviour without a cache in front is unchanged.

The first two commits are a failing test and its fix; the third wires the value through from the api response.

Warning

This is a draft and must not be merged yet. It depends on hashicorp/vault#32040, which adds api.Secret.Age. Until that merges and a new github.com/hashicorp/vault/api is released, go.mod carries a temporary replace to an unreleased branch (marked // NOTE: TEMPORARY, DO NOT MERGE!). Before this can merge, that replace must be dropped and the api dependency bumped to the real release.

The dependency package's tests need consul and nomad on $PATH (its TestMain starts them), so the new test runs in CI.

Rationale

This is the client-side counterpart to hashicorp/vault#32040, which fixes api.LifetimeWatcher — the mechanism Consul Template uses for renewable secrets. This change covers the non-renewable secret path, which has its own timing logic in leaseCheckWait and is not touched by that fix.

Canonical issue: hashicorp/vault#19684 (Vault Agent lease cache interferes with template refresh for dynamic secrets). Background: hashicorp/vault#19227, hashicorp/vault#16439.

PCI review checklist

  • I have documented a clear reason for, and description of, the change I am making.

  • If applicable, I've documented a plan to revert these changes if they require more than reverting the pull request.

  • If applicable, I've documented the impact of any changes to security controls.

leaseCheckWait decides how long to sleep before re-rendering a
non-renewable secret, using lease_duration alone. Behind a caching proxy
the response reports the duration measured from issuance, and the HTTP
Age header says how stale it is, so the lease has lease_duration - Age
left, not lease_duration.

Add an inert Age field to Secret so the behaviour can be pinned, and a
failing test: a 100s lease returned with Age 90s has 10s left, but
leaseCheckWait sleeps ~90s and wakes long after it has expired.

The dependency package test binary needs consul and nomad on PATH
(TestMain starts them), so this runs in CI; the arithmetic is
deterministic because the package init pins VaultLeaseRenewalThreshold.

Ref: hashicorp/vault#19684
leaseCheckWait sleeps for a fraction of the lease duration before
re-rendering a non-renewable secret, measured from now. Behind a caching
proxy the response has already been held for its Age, so that much of the
wait has elapsed and the client would otherwise wake after the lease had
expired.

Subtract the Age from the computed sleep for lease-bearing responses,
clamped at zero. This is a no-op when Age is zero, so responses served
directly by Vault are unaffected.
Populate Secret.Age from the api response so leaseCheckWait can discount
the time a response spent in a cache. The api gained this field in
hashicorp/vault#32040.

The replace directive is a temporary bridge to that change on an
unreleased branch and must be dropped for a real api release before this
can merge.

Ref: hashicorp/vault#32040
@peteski22
peteski22 force-pushed the fix-lease-check-wait-cache-age branch from cecd28b to 8d15c86 Compare July 22, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant