-
Notifications
You must be signed in to change notification settings - Fork 216
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
File open call in preload #1627
base: feature/xload
Are you sure you want to change the base?
Conversation
…e-fuse into feature/xload
…to feature/xload
…to sourav/xload
…to sourav/xload
…to sourav/xload
…to sourav/xload
…-storage-fuse into sourav/prefetchOpen
…e-fuse into sourav/prefetch
…e-fuse into sourav/prefetch
…age-fuse into sourav/statsPrefetch
…age-fuse into sourav/statsPrefetch
…-storage-fuse into sourav/prefetchOpen
case b = <-pool.blocksCh: | ||
break | ||
|
||
default: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we allocate a new block here then release shall also check if queue is full or not, and if so it shall release the memory otherwise this one block will linger for the lifespan of blobfuse.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Already doing that, https://github.com/Azure/azure-storage-fuse/blob/sourav/prefetchOpen/component/xload/blockpool.go#L144-L151
The release of block back to the pool is non-blocking and if the pool is full, the block will be deleted, and its memory will be released.
func (pool *BlockPool) Release(b *Block) {
select {
case pool.blocksCh <- b:
break
default:
_ = b.Delete()
}
}
✅ What
Preload operation downloads the blobs in parallel. If open call comes for a file and it has not been downloaded yet, then it will be scheduled for download on priority.
🤔 Why
The open call for a file downloads it on priority, if not yet scheduled for download. This is done so that the users will not have to wait till the scheduler picks it up for download.
NOTE
The UT will come in the next PR