Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File open call in preload #1627

Open
wants to merge 82 commits into
base: feature/xload
Choose a base branch
from
Open

Conversation

souravgupta-msft
Copy link
Member

✅ What

Preload operation downloads the blobs in parallel. If open call comes for a file and it has not been downloaded yet, then it will be scheduled for download on priority.

🤔 Why

The open call for a file downloads it on priority, if not yet scheduled for download. This is done so that the users will not have to wait till the scheduler picks it up for download.

NOTE

The UT will come in the next PR

case b = <-pool.blocksCh:
break

default:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we allocate a new block here then release shall also check if queue is full or not, and if so it shall release the memory otherwise this one block will linger for the lifespan of blobfuse.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already doing that, https://github.com/Azure/azure-storage-fuse/blob/sourav/prefetchOpen/component/xload/blockpool.go#L144-L151
The release of block back to the pool is non-blocking and if the pool is full, the block will be deleted, and its memory will be released.

func (pool *BlockPool) Release(b *Block) {
	select {
	case pool.blocksCh <- b:
		break
	default:
		_ = b.Delete()
	}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants