branch-4.1: [fix](regression) Deflake the sql cache regression suites #67734 - #67735
Open
morningman wants to merge 1 commit into
Open
branch-4.1: [fix](regression) Deflake the sql cache regression suites #67734#67735morningman wants to merge 1 commit into
morningman wants to merge 1 commit into
Conversation
…pache#67734 Two regression-test-only changes. 1. test_sql_cache_over_arrow_flight, cherry-picked from apache#67734. The suite primes four sql cache entries on the MySQL control session, runs the same statements over Arrow Flight, and asserts the entries are still there. The FE sql cache is a single Caffeine map shared by every session and bounded by Config.sql_cache_manage_num (default 100); Caffeine admits a newcomer through a window sized at 1% of that bound, so at the default a just cached statement is dropped as soon as any other session caches anything, and enable_sql_cache defaults to true for the whole concurrently running p0 suite. Raise the bound while the suite runs and restore it afterwards, as the other five sql cache suites do. In p0 build 124525 the entry was gone 299ms after being primed. 2. sql_cache_object_type picks up the hunk that apache#67556 missed. The master fix in apache#67529 replaced the closing `assertTrue(hasSqlCache(objectSql))` with a re-prime for the same reason, but apache#67556 backported the original apache#67381 version of the file, so branch-4.1 kept the fragile assertion. It has failed 18 times here since 2026-09-02 and is currently muted; the three most recent failures were all on branch-4.1 PRs. With this the file is byte identical to master again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #67734, #67529
Problem Summary:
Two flaky sql cache suites on branch-4.1. Both fail the same way — a sql cache entry
that was primed a moment earlier is gone by the time it is asserted — and both were red
together in p0 build 124525 (
5657 passed / 2 failed; no crash, no OOM, unrelated tothe PR that build was testing).
Root cause. The FE sql cache is a single Caffeine map shared by every session
(
NereidsSqlCacheManager.sqlCaches), bounded byConfig.sql_cache_manage_num, whichdefaults to 100. Caffeine admits a newcomer through a window sized at 1% of that
bound, so at the default the admission window holds a single entry and a just cached
statement is dropped as soon as any other session caches anything.
SessionVariable.enableSqlCachedefaults totrue, so the rest of the p0 suite runningconcurrently against the same FE is enough. Ordinary LRU pressure is ruled out: only 15
distinct selects ran cluster wide during the 299ms window in which the flight suite lost
its entry. Verified against caffeine 3.2.4:
This is not an FE bug — the cache is best effort and gives no retention guarantee.
Changes
1.
arrow_flight_sql_p0/test_sql_cache_over_arrow_flight— cherry-picked from#67734. Raise
sql_cache_manage_numwhile the suite runs, as the other five sql cachesuites (
mv_with_sql_cache,mtmv_with_sql_cache,parse_sql_from_sql_cache,union_all_compensate,union_rewrite_grace_big) already do, and restore it afterwardsso the rest of the run does not keep 10000 cached plans alive in the FE heap. This suite
is identical on master and branch-4.1 and has failed on both.
2.
query_p0/cache/sql_cache_object_type— picks up the hunk that the #67556backport missed. Master fixed this in #67529 on 2026-09-04 by replacing the closing
assertTrue(hasSqlCache(objectSql))with a re-prime, but #67556 backported the original#67381 version of the file, so branch-4.1 kept the fragile assertion. It has failed 18
times on branch-4.1 since 2026-09-02 and is currently muted in TeamCity; the three
most recent failures were all branch-4.1 PRs. After this change the file is byte
identical to master again.
The TeamCity mute on
query_p0.cache.sql_cache_object_type.sql_cache_object_typeshould be removed once this lands.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)
🤖 Generated with Claude Code
https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt