Stampede Protection

Stampede Protection

A cache stampede (also called a thundering herd) occurs when many concurrent requests all miss the cache at the same time — for example, immediately after a key expires — and all proceed to hit the database simultaneously.

CacheWeave includes built-in stampede protection via ICacheStampedeProtector.

Default: In-Process Lock

The default implementation, InProcessStampedeProtector, uses a per-key SemaphoreSlim. When multiple concurrent requests miss the same key:

1. The first request acquires the lock and executes the factory (database call).
2. All other requests wait on the semaphore.
3. When the first request writes the result to the cache and releases the lock, the waiting requests re-check the cache and return the now-cached value — without re-executing the factory.

// Default — no configuration needed
builder.Services.AddCacheWeave(options => { ... });

This is effective for single-instance deployments or when all instances share the same in-process lock (i.e. sticky sessions). For multi-instance deployments, use a distributed lock.

Distributed Lock

For multi-instance deployments (e.g. Kubernetes, Azure App Service with multiple replicas), implement ICacheStampedeProtector using a distributed lock backed by Redis or another coordination primitive.

public interface ICacheStampedeProtector
{
    Task<T?> GetOrSetAsync<T>(
        string key,
        Func<Task<T?>> factory,
        TimeSpan? expiry,
        CancellationToken ct = default);
}

Example — Redis Distributed Lock (RedLock.net)

using RedLockNet;
using RedLockNet.SERedis;
using RedLockNet.SERedis.Configuration;

public class RedisStampedeProtector : ICacheStampedeProtector
{
    private readonly IDistributedLockFactory _lockFactory;
    private readonly ICacheProvider _cache;

    public RedisStampedeProtector(
        IDistributedLockFactory lockFactory,
        ICacheProvider cache)
    {
        _lockFactory = lockFactory;
        _cache = cache;
    }

    public async Task<T?> GetOrSetAsync<T>(
        string key,
        Func<Task<T?>> factory,
        TimeSpan? expiry,
        CancellationToken ct = default)
    {
        // Check cache first (fast path — no lock needed)
        var cached = await _cache.GetAsync<T>(key, ct);
        if (cached is not null) return cached;

        var lockKey = $"lock:{key}";
        var lockExpiry = TimeSpan.FromSeconds(30);
        var waitTime = TimeSpan.FromSeconds(10);
        var retryTime = TimeSpan.FromMilliseconds(200);

        await using var redLock = await _lockFactory.CreateLockAsync(
            lockKey, lockExpiry, waitTime, retryTime, ct);

        if (!redLock.IsAcquired)
            // Could not acquire lock — fall through to factory (safe degradation)
            return await factory();

        // Re-check cache after acquiring lock (another instance may have populated it)
        cached = await _cache.GetAsync<T>(key, ct);
        if (cached is not null) return cached;

        var value = await factory();
        if (value is not null)
            await _cache.SetAsync(key, value, expiry, ct);

        return value;
    }
}

Register before AddCacheWeave:

// Register RedLock factory
var multiplexers = new List<RedLockMultiplexer>
{
    new(ConnectionMultiplexer.Connect("localhost:6379"))
};
builder.Services.AddSingleton<IDistributedLockFactory>(
    RedLockFactory.Create(multiplexers));

// Register custom stampede protector (overrides InProcessStampedeProtector)
builder.Services.AddSingleton<ICacheStampedeProtector, RedisStampedeProtector>();

builder.Services.AddCacheWeave(options => { ... });

Disabling Stampede Protection

If your workload is read-heavy and the cost of a stampede is acceptable (e.g. cheap database queries), you can bypass the protector entirely by implementing a no-op:

public class NoOpStampedeProtector : ICacheStampedeProtector
{
    public Task<T?> GetOrSetAsync<T>(
        string key,
        Func<Task<T?>> factory,
        TimeSpan? expiry,
        CancellationToken ct = default)
        => factory();
}

builder.Services.AddSingleton<ICacheStampedeProtector, NoOpStampedeProtector>();
builder.Services.AddCacheWeave(options => { ... });

Behaviour Summary

Scenario	Behaviour
Single instance, default	In-process SemaphoreSlim per key — only one factory call per key at a time
Multi-instance, default	Each instance has its own lock — stampede possible across instances
Multi-instance, Redis lock	Distributed lock — only one factory call across all instances
No-op protector	All concurrent misses call the factory — no protection