Package-provided default cache settings per function #19

Packages currently cannot ship a default cache size for one of their own functions. The settings of a function come from the user's preferences (per function, then per package, then global) or else from the built-in defaults, so a package can only choose the default by either

• writing to the user's LocalPreferences.toml (set_cache_preferences!), which is not something a package should do on load, or
• calling set_cache_size! from __init__, which overrides whatever the user configured, on every load.

Use case

TensorKit.jl caches tree transformers for GPU storage in separate functions (device_treebraider, device_treetransposer, see QuantumKitHub/TensorKit.jl#568). These hold device memory, which is scarcer than host memory, so they should default to a smaller limit than the host transformers (and ideally to a byte budget, once these values have a cachesize method measuring their device buffers). Users should still be able to override this through the preferences.

Proposal

An overloadable hook providing code-level defaults, consulted after the preferences and before the built-in settings:

MemoizationKit.default_cache_settings(f) = (;)  # default: no package-provided settings

# in a package or extension:
MemoizationKit.default_cache_settings(::typeof(device_treebraider)) = (; maxsize = 1000)
# or
MemoizationKit.default_cache_settings(::typeof(device_treebraider)) = (; measure = "bytes", maxsize = 2^28)

so that the resolution order becomes

[<Package>.MemoizationKit.<f>] > [<Package>.MemoizationKit] > [MemoizationKit] > default_cache_settings(f) > built-in defaults

Open points:

• Precedence relative to the user's global or package-wide preferences: a function-specific package default is arguably more specific than a user's global maxsize, so it might be placed above [MemoizationKit] (and [<Package>.MemoizationKit]) instead, with only per-function user preferences taking precedence.
• Since the settings are read when the function's caches are first created, a method defined in an extension that is loaded after the first call would not apply until the caches are recreated (e.g. via set_cache_size!). This seems acceptable when the hook is defined in the same package or extension as the cached methods.
• Whether package-wide defaults are needed as well (e.g. default_cache_settings(::Module)), or whether per-function defaults suffice.