[fix](regression) Deflake test_sql_cache_over_arrow_flight - #67734
Open
morningman wants to merge 1 commit into
Open
[fix](regression) Deflake test_sql_cache_over_arrow_flight#67734morningman wants to merge 1 commit into
morningman wants to merge 1 commit into
Conversation
The suite primes four sql cache entries through the MySQL control session, runs the same statements over Arrow Flight, and finally asserts the entries are still there, to prove that a flight query never consumes the cache. That last block fails intermittently: in p0 build 124525 the `select 1 as c, 'x' as s` entry was gone 299ms after it had been primed, and the suite has failed six times on master and branch-4.1 since it was added. The FE sql cache is a single Caffeine map shared by every session, bounded by Config.sql_cache_manage_num, which defaults to 100. Caffeine admits a newcomer through a window sized at 1% of that bound, so at the default the window holds a single entry and a just cached statement is dropped again as soon as any other session caches anything. enable_sql_cache defaults to true, so the rest of the p0 suite running concurrently against the same FE is already enough: only 15 distinct selects ran cluster wide during that 299ms window, far fewer than the 100 an LRU would have needed. The other five sql cache suites in the repo (mv_with_sql_cache, mtmv_with_sql_cache, parse_sql_from_sql_cache, union_all_compensate, union_rewrite_grace_big) all raise the bound for this reason; this one was missing it. Raise it here as well while the suite runs, and put it back afterwards so the rest of the run does not keep 10000 cached plans and their result rows alive in the FE heap. The suites that raise it without restoring it left 97k live SqlCacheContext instances behind in the same build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
16 tasks
Contributor
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #67381
Problem Summary:
arrow_flight_sql_p0/test_sql_cache_over_arrow_flightis flaky. It primes four sqlcache entries through the MySQL control session, runs the same statements over Arrow
Flight, and then asserts the entries are still there, to prove that a flight query
never consumes the cache. That closing block fails intermittently on both master and
branch-4.1 — six times since the suite was added on 2026-09-02, most recently in p0
build 124525, where the
select 1 as c, 'x' as sentry was gone 299ms after it hadbeen primed:
Root cause. The FE sql cache is a single Caffeine map shared by every session
(
NereidsSqlCacheManager.sqlCaches), bounded byConfig.sql_cache_manage_num, whichdefaults to 100. Caffeine admits a newcomer through a window sized at 1% of that
bound, so at the default the admission window holds a single entry: a just cached
statement has frequency ~1 and loses the admission contest to an established victim as
soon as any other session caches anything.
SessionVariable.enableSqlCachedefaults totrue, so the rest of the p0 suite running concurrently against the same FE is alreadyenough to evict it.
The audit log rules out ordinary LRU pressure: only 15 distinct selects ran cluster
wide during that 299ms window, far fewer than the 100 an LRU would have needed. A local
run against caffeine 3.2.4 reproduces the admission behaviour directly:
This is not an FE bug — the sql cache is best effort and gives no retention guarantee.
It is the suite asserting a property the cache does not provide. The other five sql
cache suites in the repo (
mv_with_sql_cache,mtmv_with_sql_cache,parse_sql_from_sql_cache,union_all_compensate,union_rewrite_grace_big) alreadyraise the bound at their start for exactly this reason; this one was missing it. In p0
build 124525 the failing suites ran at 21:54 and 22:01, before any of those suites
raised the bound at 22:12.
Fix. Raise
sql_cache_manage_numto 10000 while the suite runs, and restore theprevious value afterwards. The restore is deliberate: this suite runs about half an
hour earlier in the p0 run than the five existing ones, and the ones that raise the
bound without restoring it left 97k live
SqlCacheContextinstances in the FE heapin the same build (post-GC live heap peaked at 4G of 8G in that window).
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)
🤖 Generated with Claude Code
https://claude.ai/code/session_01Njd8iDxdqc19QbLdNtZ7Pt