This repository has been archived by the owner on Mar 9, 2019. It is now read-only.
Add NoSync field to Options #721
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Read txns would lock pages allocated after the txn, keeping those pages off the free list until closing the read txn. Instead, track allocating txid to compute page lifetime, freeing pages if all txns between page allocation and page free are closed.
*: use coreos/bbolt as import path
When the database has a lot of freepages, the cost to sync all freepages down to disk is high. If the total database size is small (<10GB), and the application can tolerate ~10 seconds recovery time, then it is reasonable to simply not sync freelist and rescan the db to rebuild freelist on recovery.
add option to skip freelist sync
Garbage collect pages allocated after minimum txid
Writes pgidNoFreelist to the meta freelist page to detect when freelists haven't been synced down. Fixes boltdb#5
rebuild freelist when opening with FreelistSync after NoFreelistSync
gofmt, change git path
Confirm that the number of freed pages exceeds the overflow count, then check that reopening gives the same number of free pages.
count is not shifted up by start index when taking subslice of free list, dropping the last entry in the list.
Default/nil quick.Config uses 1000 rounds, causing TestBucker_Put_Single to run for over 3 minutes in CI. The default count (via qconfig()) for boltdb is 5, so use that.
* unconditionally free freelist, if any, when committing txn * only treat freelist pages as reachable if set to valid pgid Fixes boltdb#9
test: use qconfig() instead of defaults in TestBucket_Put_Single
fix NoSyncFreelist reachability checking
freelist: read all free pages on count overflow
Configure the db page size at runtime. Makes cross-arch debugging a bit easier.
Added a test to check that bucket is actually non-nil before getting a cursor on it. This happens when querying non-bucket entries, the Bucket() method does not return explicit errors, so we can guess it's probably because it's a key of the wrong type (ie: not leaf). No clear idea wether the results in the case described '-read-mode=seq -write-mode=rnd-nest' really do make sense, it looks like read are zero whatsoever.
By default, pages are split when they reach half full. For 'bolt compact' we want to fill the entire page for maximum compaction.
As of now the test fails and tries to delete the next key returned by the cursor, which happens to be a nested bucket. That's why Delete is fails. Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
Doc for Bucket.Delete says that a Delete() on non-existing key is a no-op. Right now it tries to delete the next key returned by the cursor. Fix this by checking for key equivalence before deletion. Fixes boltdb#349 Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
fix overflow breaking 32-bit test builds
Add support for mips, mips64
more fmt passes
Set FillPercent=1.0 in 'bolt compact'.
do not read freelist if database opened readonly
Options.PageSize
Fix deletion of non-existing keys
Fix issue boltdb#543 'bolt bench crashes'
Signed-off-by: Lewis Marshall <lewis@lmars.net>
These were previously removed but I find them quite useful so have reintroduced them. Signed-off-by: Lewis Marshall <lewis@lmars.net>
Add "buckets", "keys" and "get" CLI commands
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
README: update links to 'coreos/bbolt'
Avoid allocate a hole page, when the node size equals to pageSize
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
goreport + minor fixes
This allows enabling NoSync when you only have access to the Options passed into Open() and not the returned DB (as is frequently the case with libraries).
|
@tylertreat is switching to a fork feasible for your use case? if so this patch could go into the fork at http://github.com/coreos/bbolt which includes some extra enhancements/bugfixes |
|
@heyitsanthony is there more information on the fork somewhere? It looks like the README doesn't detail the purpose/goal of the fork. |
|
@tylertreat the purpose is to actively maintain boltdb; this project is being ignored. |
|
@heyitsanthony I mirrored the PR here: etcd-io/bbolt#30 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change to expose
NoSyncinOptions. Hopefully this is a reasonable change. The justification for this is I use a number of libraries which simply expose theOptionspassed intoOpen()but not the underlyingDBthey create. To enableNoSyncin these cases, I either have to fork the library or fork bolt.