[WIP] Loading revindex from zipstorage #1943
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## latest #1943 +/- ##
==========================================
+ Coverage 84.34% 91.04% +6.69%
==========================================
Files 130 99 -31
Lines 15320 11101 -4219
Branches 2176 2174 -2
==========================================
- Hits 12922 10107 -2815
+ Misses 2095 677 -1418
- Partials 303 317 +14
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. |
luizirber
force-pushed
the
lirber/greyhound_zipstorage
branch
from
65b1e9b
to
4db4f2d
Compare
luizirber
force-pushed
the
lirber/greyhound_zipstorage
branch
from
4db4f2d
to
1a1026f
Compare
luizirber
force-pushed
the
lirber/greyhound_zipstorage
branch
from
3428e54
to
81fb358
Compare
luizirber
force-pushed
the
lirber/greyhound_zipstorage
branch
from
81fb358
to
6500ed9
Compare
This was referenced Aug 21, 2022
|
Merged into #2230 |
luizirber
added a commit
that referenced
this pull request
Oct 17, 2022
luizirber
added a commit
that referenced
this pull request
Oct 22, 2022
luizirber
added a commit
that referenced
this pull request
Feb 18, 2023
luizirber
added a commit
that referenced
this pull request
Mar 20, 2023
luizirber
added a commit
that referenced
this pull request
Mar 31, 2023
luizirber
added a commit
that referenced
this pull request
May 20, 2023
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 19, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 20, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 21, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 22, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 22, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Aug 27, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Sep 12, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now
luizirber
added a commit
that referenced
this pull request
Sep 16, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now rename mastiff to branchwater add revindex test wip collection clippy fixes start using camino for paths cargo fmt Implement MemStorage and from_paths and from_sigs cleanup Replace DatasetID with Idx Use 32-bits Idx, instead of 64 use enum_dispatch to avoid repeating code implement selection more widely working with protein in tests... add semver-checks save/load collection from rocksdb
luizirber
added a commit
that referenced
this pull request
Sep 16, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now rename mastiff to branchwater add revindex test wip collection clippy fixes start using camino for paths cargo fmt Implement MemStorage and from_paths and from_sigs cleanup Replace DatasetID with Idx Use 32-bits Idx, instead of 64 use enum_dispatch to avoid repeating code implement selection more widely working with protein in tests... add semver-checks save/load collection from rocksdb
luizirber
added a commit
that referenced
this pull request
Sep 16, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now rename mastiff to branchwater add revindex test wip collection clippy fixes start using camino for paths cargo fmt Implement MemStorage and from_paths and from_sigs cleanup Replace DatasetID with Idx Use 32-bits Idx, instead of 64 use enum_dispatch to avoid repeating code implement selection more widely working with protein in tests... add semver-checks save/load collection from rocksdb
luizirber
added a commit
that referenced
this pull request
Sep 16, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now rename mastiff to branchwater add revindex test wip collection clippy fixes start using camino for paths cargo fmt Implement MemStorage and from_paths and from_sigs cleanup Replace DatasetID with Idx Use 32-bits Idx, instead of 64 use enum_dispatch to avoid repeating code implement selection more widely working with protein in tests... add semver-checks save/load collection from rocksdb
luizirber
added a commit
that referenced
this pull request
Sep 20, 2023
try to skip md5sum derive typed builder for GatherResult expose md5 start moving mastiff remove unused bigsi and sbt indices Expose iterator methods for Signature remove unused function preparing for MinHashOps Bump version, there will be many breaking changes... more splits default to large minhash, expose frozen use enum for MinHash thru FFI bug fixes and cleanup add c++ stdlib package for mamba fix finch feature try out roaring Add back check command initial update impl add semver check for rust bump once_cell fix rust ci start bringing #1943 more selection more picklist use dashmap in mem_revindex Revert "use dashmap in mem_revindex" This reverts commit 22727b7091dee2dbafd10ce707578c5085aa6cfc. bump rocksdb to 0.19 bump rocksdb to 0.20, bump MSRV to 1.60 update deps add cargo deny config use cibuildwheel configs in pyproject.toml flake cleanup fix cargo.lock tox updates don't worry with in-mem sigs for now rename mastiff to branchwater add revindex test wip collection clippy fixes start using camino for paths cargo fmt Implement MemStorage and from_paths and from_sigs cleanup Replace DatasetID with Idx Use 32-bits Idx, instead of 64 use enum_dispatch to avoid repeating code implement selection more widely working with protein in tests... add semver-checks save/load collection from rocksdb
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Very early manifest support in Rust