Skip to content

branch-4.1: [fix](regression) Deflake the sql cache regression suites #67734 - #67735

Open
morningman wants to merge 1 commit into
apache:branch-4.1from
morningman:deflake-sql-cache-suites-4.1
Open

branch-4.1: [fix](regression) Deflake the sql cache regression suites #67734#67735
morningman wants to merge 1 commit into
apache:branch-4.1from
morningman:deflake-sql-cache-suites-4.1

Conversation

@morningman

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #67734, #67529

Problem Summary:

Two flaky sql cache suites on branch-4.1. Both fail the same way — a sql cache entry
that was primed a moment earlier is gone by the time it is asserted — and both were red
together in p0 build 124525 (5657 passed / 2 failed; no crash, no OOM, unrelated to
the PR that build was testing).

Root cause. The FE sql cache is a single Caffeine map shared by every session
(NereidsSqlCacheManager.sqlCaches), bounded by Config.sql_cache_manage_num, which
defaults to 100. Caffeine admits a newcomer through a window sized at 1% of that
bound
, so at the default the admission window holds a single entry and a just cached
statement is dropped as soon as any other session caches anything.
SessionVariable.enableSqlCache defaults to true, so the rest of the p0 suite running
concurrently against the same FE is enough. Ordinary LRU pressure is ruled out: only 15
distinct selects ran cluster wide during the 299ms window in which the flight suite lost
its entry. Verified against caffeine 3.2.4:

maximumSize=100      1 other insert after mine -> survived  7/20
maximumSize=10000    1 other insert after mine -> survived 20/20

This is not an FE bug — the cache is best effort and gives no retention guarantee.

Changes

1. arrow_flight_sql_p0/test_sql_cache_over_arrow_flight — cherry-picked from
#67734. Raise sql_cache_manage_num while the suite runs, as the other five sql cache
suites (mv_with_sql_cache, mtmv_with_sql_cache, parse_sql_from_sql_cache,
union_all_compensate, union_rewrite_grace_big) already do, and restore it afterwards
so the rest of the run does not keep 10000 cached plans alive in the FE heap. This suite
is identical on master and branch-4.1 and has failed on both.

2. query_p0/cache/sql_cache_object_type — picks up the hunk that the #67556
backport missed. Master fixed this in #67529 on 2026-09-04 by replacing the closing
assertTrue(hasSqlCache(objectSql)) with a re-prime, but #67556 backported the original
#67381 version of the file, so branch-4.1 kept the fragile assertion. It has failed 18
times on branch-4.1 since 2026-09-02
and is currently muted in TeamCity; the three
most recent failures were all branch-4.1 PRs. After this change the file is byte
identical to master again.

The TeamCity mute on query_p0.cache.sql_cache_object_type.sql_cache_object_type
should be removed once this lands.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

🤖 Generated with Claude Code

https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt

…pache#67734

Two regression-test-only changes.

1. test_sql_cache_over_arrow_flight, cherry-picked from apache#67734. The suite primes
   four sql cache entries on the MySQL control session, runs the same statements
   over Arrow Flight, and asserts the entries are still there. The FE sql cache
   is a single Caffeine map shared by every session and bounded by
   Config.sql_cache_manage_num (default 100); Caffeine admits a newcomer through
   a window sized at 1% of that bound, so at the default a just cached statement
   is dropped as soon as any other session caches anything, and enable_sql_cache
   defaults to true for the whole concurrently running p0 suite. Raise the bound
   while the suite runs and restore it afterwards, as the other five sql cache
   suites do. In p0 build 124525 the entry was gone 299ms after being primed.

2. sql_cache_object_type picks up the hunk that apache#67556 missed. The master fix in
   apache#67529 replaced the closing `assertTrue(hasSqlCache(objectSql))` with a
   re-prime for the same reason, but apache#67556 backported the original apache#67381
   version of the file, so branch-4.1 kept the fragile assertion. It has failed
   18 times here since 2026-09-02 and is currently muted; the three most recent
   failures were all on branch-4.1 PRs. With this the file is byte identical to
   master again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt
@morningman
morningman requested a review from yiguolei as a code owner September 9, 2026 10:24
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman

Copy link
Copy Markdown
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants