Skip to content

Flaky gateway watcher tests on a two-file identity write #281

Description

@hexfusion

Two gateway watcher tests in gateway/ai-grid-filters/src/control.rs fail intermittently in CI:

  • control::tests::a_renewed_identity_restarts_the_pollers_with_no_config_change (line 1101): a settled identity restarts nothing more, left 2, right 1
  • control::tests::a_refused_file_neither_blocks_renewal_nor_repeats_its_rejection (line 631)

Seen on #278 (job 111475474259 and earlier runs). They pass locally and on main.

Likely cause: the tests write tls.crt and then tls.key as two separate writes while the watcher polls every 20ms. A poll between the two writes sees two changes and applies the identity twice. Kubernetes updates a Secret volume atomically through the ..data symlink, so production sees one change.

Proposed fix: have the tests swap both files atomically, the way the kubelet does, and keep the assertions as they are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions