feat(driver, rt): multi buffer pool#954
Conversation
|
How do you want to use with this new API? There's no API to specify which buffer pool to use in the IO types. |
currently user can use driver op to submit a operation with the extra buffer pool after this PR, we can modify or extend the managed io trait to allow user use the extra buffer pool directly |
d34156a to
1eedc89
Compare
|
Our discussion focuses on two major points:
Here's my design to solve the problems. If we don't want to play with reference cycles, we have to keep the buffer pools inside the driver / runtime. The exposed API should be a handle to the buffer pool: impl Runtime {
pub fn buffer_manager(&self) -> BufferManager;
}
struct BufferManager { /* ... */ }
impl BufferManager {
pub fn create_pool(&self) -> BufferPoolHandle;
pub fn get_pool(&self, index: usize) -> Option<BufferPoolHandle>;
pub fn release_pool(&self, index: usize);
}
struct BufferPoolHandle(usize);
impl BufferPoolHandle {
pub fn index(&self) -> usize;
pub async fn with<F: Future>(&self, f: impl FnOnce() -> F) -> F::Output;
}The managed APIs are not modified, but all managed methods should be called inside one handle.with(|| async {
file.read_managed_at(0, 0).await.unwrap();
}).await;Would like advice from @Sherlock-Holo and @George-Miao . |
|
@Berrysoft I don't think it's necessary to force all managed API to be called inside with. Maybe add this to FutureExt and if user doesn't say which pool to use, provide a default one like what we have right now. |
|
Oh, fine. Then it might be like file.read_managed_at(0, 0).with_buffer_pool(&handle).await; |
|
@Berrysoft Yeah LGTM |
|
@Sherlock-Holo Could you remove the changes to And I wonder if it's possible to avoid such a release queue with |
this PR allow user create multi extra buffer pools, not just use the pre-created buffer pool