Discussion: Scoped/Implicit allocators as an alternative to explicit generic parameters to solve the library adoption bottleneck

[Ez-FlawLess]

Problem Description

The current design of the Allocator API relies heavily on explicit parameterization (e.g., Vec<T, A = Global>). While this works well for isolated or top-level applications, it creates a massive ecosystem bottleneck due to viral propagation.

For example, suppose I want to write a secure application where cryptographic tokens must be stored on a memory page configured with mlock and MADV_DONTDUMP. Under the current paradigm:

1. I create a custom SecureAllocator.
2. To use high-level libraries like jsonwebtoken, that crate must be explicitly rewritten to accept custom allocators for its internal data structures.
3. Furthermore, jsonwebtoken's cryptographic backends (and their dependencies) must also explicitly support and forward the allocator parameter.

If a single crate in this dependency chain does not support the Allocator API or fails to forward the generic allocator parameter down the line, the secure memory guarantee is broken, as allocations revert to the Global allocator. Expecting the entire Rust ecosystem to update their signatures to accommodate generic allocators is impractical and likely to fail.

Proposed Alternative: Scoped/Thread-Local Allocator Contexts

Instead of threading an allocator explicitly through every struct and function signature down the call stack, I want to propose a solution handled by the compiler/language. This would provide a mechanism to temporarily override or wrap the allocator for a specific execution scope, dictating that everything down the line uses it.

Conceptually, this would behave exactly as if we have changed the #[global_allocator] exclusively for that specific section of code.

This could look like a scoped context or wrapper function:

// Concept: Overriding the allocator down the entire call-stack line
rust::alloc::with_allocator(SecureAllocator, || {
    // Everything executed inside this closure—including third-party crates
    // that know nothing about SecureAllocator—defaults to using it instead of Global.
    let token = jsonwebtoken::decode::<Claims>(...); 
});

[zakarumych]

How are you going to deallocate it?
Global is ZST so it can't help you track which scope the allocation belongs to.
Thus you will need a way to figure it out from the pointer alone.
And if you have the ability then you don't need anything from Rust to do that already - just substitute default global allocator with scoped one.

[Ez-FlawLess]

@zakarumych Thanks for the clarification, that makes total sense. The scoped proposal was more of a conceptual 'wouldn't it be cool' idea without digging into the deallocation mechanics. You're completely right about the ZST and pointer tracking limitations.

Moving back to the root problem—the viral propagation bottleneck where every intermediate crate in a dependency chain has to explicitly support and forward generic allocators—are there other language-level or ecosystem design patterns being considered to solve this? How can we achieve targeted allocations (like secure memory) in deeply nested dependencies without forcing the entire ecosystem to rewrite their signatures?

[zakarumych]

Currently you can implement it yourself.
Just make your own scoped allocator.
Allocate with prefix to put necessary info to deallocate even if allocation scope ended.
Put it on crates.io and that would be ecosystem fix you are after ^^

[Ez-FlawLess]

@zakarumych That is an amazing idea, thanks. How would I know from the allocator that the allocation call has been made from a piece of code inside that alloc::with_allocator scope?