Repository navigation
Conversation
ClockCache is now an open-addressing table of immutable entries, published atomically and read with acquire loads, so hits take no lock and write no shared cache line (hits and misses are counted per thread). Misses and all other mutations still take the lock. Shared hits stay at ~15-25 ns at 8 threads (was ~330-620 ns) and are 2x faster single-threaded. Entries keep their key type, so concretely typed caches store keys unboxed. Task-local caches keep the previous locked implementation, now the internal LocalClockCache (a SlotCache), to avoid the per-thread counters in every task's cache. Adds concurrency stress tests and benchmark/contention.jl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Slots are read and written through `@atomic` indexing of an `AtomicMemory` on 1.12+. On 1.10 and 1.11 the pointer load and `setindex!` stay, with a test that the store compiles to a release store with a GC write barrier. - Fewer single-use helpers and descriptive names without leading underscores; the fields of `Entry` read on a hit come first. - The hand no longer errors when, after two sweeps of bits re-set by readers, it lands on a free position: it moves on to the next entry. - Remove benchmark/contention.jl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
lkdvos
commented
Oct 15, 2026
| mutable struct ClockCache{K, V} <: AbstractCache{K, V} | ||
| @atomic table::SlotTable # entries, read without the lock | ||
| occupied::Int # slots of `table` holding an entry or a tombstone | ||
| const ring::Vector{Union{Nothing, Entry{K, V}}} # the clock, `nothing` for a free position |
Member
Author
There was a problem hiding this comment.
Should we consider power of two sizes to have fast ring wrapping?
Member
Author
There was a problem hiding this comment.
The ring only wraps in evict!, under the lock on a miss, so a power-of-two ring would gain nothing measurable there, and it would leave empty positions for the hand to sweep. It now wraps with a comparison instead of %.
The modulo that mattered was in countlookup!, on every hit: mod(threadid() - 1, nthreads). The counter array now holds a power of two of cache lines, so it is a mask (cb8ecb7). Single-threaded Int hits: 9.1 ns (BenchmarkTools).
| # would bounce between cores on every hit. Two tasks on one thread cannot interleave between the | ||
| # load and the store, since neither yields; threads adopted after the cache was created share | ||
| # counters, and may then lose a few counts. | ||
| const HIT, MISS = 1, 2 |
Member
Author
There was a problem hiding this comment.
Done: @enum LookupOutcome HIT = 1 MISS = 2, and countlookup! takes a LookupOutcome.
The default constructor had unbound type parameters, which Aqua flags on Julia 1.10, since K and V only occur in the Union element type of ring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The counter array holds a power of two of cache lines, so a hit finds its thread's line with a mask instead of mod. Hits and misses are an @enum, and the hand wraps with a comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Revives #9, rebased onto the current main (the
SlotCacherefactor and the rename).Motivation: QuantumKitHub/SUNRepresentations.jl#57, where TensorKit reads the F- and R-symbol caches from many threads at once. The
LRUthere locks on every hit, so that PR adds a snapshot cache and a 64-shardLRU. With lock-free hits, a single cache per rank does the job without sharding.Changes
ClockCache(src/containers/clock.jl) is now a hash table of immutable entries, replaced as a whole when it fills up and read with acquire loads.AtomicMemory{Any}, read and written with@atomic :acquire/:release. Julia 1.10 and 1.11 lack atomic indexing, so they read aVector{Any}through a pointer and write it withsetindex!. A test checks that this store compiles to a release store with a GC write barrier there.ClockCache{NTuple{6, I}, Array{Float64, 4}}) store keys unboxed. Their hits allocate nothing, like those of{Any, Any}caches.LocalClockCache(aSlotCache), since a single task gains nothing from lock-free hits and the per-thread counters would cost memory in every task.test/test_concurrency.jlstress-tests every operation from many tasks on all three containers and checks that@cachedhits are inferred and allocate nothing. The container suite also runs onLocalClockCache.Benchmarks
One task per thread, each doing 200k lookups on one shared cache (task-local: one cache per task), Julia 1.13.1, shared 32-core machine (load average about 14), ns per lookup. Measured with a contention benchmark that was not kept in the repository (see the first commit):
ClockCachehitsLRUhits@cachedhits,ClockCache@cachedhits,LRUThe ranges span
Intkeys, 4-tuple keys and one hot key.Limits
setindex!into aVector{Any}compiling to a release store, which is codegen behaviour rather than documented API. Those versions no longer change, and the test above guards it; 1.12+ use the documentedAtomicMemoryAPI.Tested on Julia 1.10.12, 1.11.9, 1.12.7 and 1.13.1 with 8 threads.
🤖 Generated with Claude Code