Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add support for multiple blockstores #3257

Closed
wants to merge 7 commits into from
Closed

WIP: Add support for multiple blockstores #3257

wants to merge 7 commits into from

Commits on Nov 3, 2016

  1. Add basic support for multiple blockstores as outlined in #3119. 

    Each datastore is mounted under a different mount point and a
multi-blockstore is used to check each mount point for the block.

The first mount checked of the multi-blockstore is considered the
"cache", all others are considered read-only.  This implies that the
garbage collector only removes block from the first mount.

This change also factors out the pinlock from the blockstore into its
own structure.  Only the multi-datastore now implements the
GCBlockstore interface.  In the future this could be separated out
from the blockstore completely.

For now caching is only done on the first mount, in the future this
could be reworked.  The bloom filter is the most problematic as the
read-only mounts are not necessary immutable and can be changed by
methods outside of the blockstore.

Right now there is only one mount, but that will soon change once
support for the filestore is added.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    3b752eb View commit details
    Browse the repository at this point in the history

  2. Implement multiblockstore.AllKeysChan(), GC now uses the first mount. 

    License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    a93b152 View commit details
    Browse the repository at this point in the history

  3. Add "block locate" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    d31cb4b View commit details
    Browse the repository at this point in the history

  4. "block/filestore rm": Allow removal of pinned but duplicate blocks. 

    License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    e32702e View commit details
    Browse the repository at this point in the history

  5. "add": add "--allow-dup" option. 

    This option adds a files the the primary blockstore even if the block
is in another blockstore such as the filestore.

License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    9c8159b View commit details
    Browse the repository at this point in the history

  6. In NewBlockstoreWPrefix don't assume "" is the default value. 

    License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    58cc20e View commit details
    Browse the repository at this point in the history

  7. Minor refactor of "block rm" command. 

    License: MIT
Signed-off-by: Kevin Atkinson <k@kevina.org>
    @kevina
    kevina committed Nov 3, 2016
    Configuration menu
    Copy the full SHA
    008de00 View commit details
    Browse the repository at this point in the history