add option to skip freelist sync #1
Conversation
xiang90
force-pushed
the
no_sync
branch
3 times, most recently
from
236381f to
dacec63
Compare
bucket_test.go
Outdated
| @@ -13,7 +13,7 @@ import ( | |||
| "testing" | |||
| "testing/quick" | |||
|
|
|||
| "github.com/boltdb/bolt" | |||
There was a problem hiding this comment.
can this renaming be split into a separate patch to merge first?
|
@heyitsanthony fixed. PTAL. |
db.go
Outdated
| return fids | ||
| } | ||
|
|
||
| func checkReachable(tx *Tx, b *Bucket, reachable map[pgid]*page) { |
There was a problem hiding this comment.
this is very similar to Tx.checkBucket; would it be possible to have this so both share some code? Maybe func (tx *Tx) freepages() []pgid?
There was a problem hiding this comment.
will give it a try.
db_test.go
Outdated
| } | ||
|
|
||
| // MustOpenDB returns a new, open DB at a temporary location with given options. | ||
| func MustOpenWitOption(o *bolt.Options) *DB { |
db_test.go
Outdated
| } | ||
| } | ||
|
|
||
| // MustOpenDB returns a new, open DB at a temporary location with given options. |
There was a problem hiding this comment.
// MustOpenWithOpen ...
db_test.go
Outdated
| @@ -1399,6 +1419,15 @@ func (db *DB) MustClose() { | |||
| } | |||
| } | |||
|
|
|||
| // MustClose reopen the database. Panic on error. | |||
There was a problem hiding this comment.
// MustReopen reopens...
freelist.go
Outdated
| @@ -22,6 +22,14 @@ func newFreelist() *freelist { | |||
| } | |||
| } | |||
|
|
|||
| func newFreelistWithIDs(ids []pgid) *freelist { | |||
There was a problem hiding this comment.
some indexing assumptions might be violated by passing in a raw pgid slice, how about:
func (f *freelist) readIds(ids []pgid) {
f.ids = ids
...
}where freelist.read()'s tail calls into readIds()
tx.go
Outdated
| if tx.meta.pgid > opgid { | ||
| if err := tx.db.grow(int(tx.meta.pgid+1) * tx.db.pageSize); err != nil { | ||
| if !tx.db.NoFreelistSync { | ||
| opgid := tx.meta.pgid |
There was a problem hiding this comment.
split into a separate method like tx.commitFreelist(...)?
it is safe from free list sync to no sync, but not the other way around. |
|
@heyitsanthony fixed. PTAL. |
|
lgtm but the test/second patch seems to have code that belongs in the implementation/first patch. Also it'd be nice to have some way to protect against no sync -> sync (commit with no-sync writes an invalid page for the freelist?), but that can be deferred; open an issue? |
yea... i made a mistake when rebasing... I guess i will just squash the two commits into one. I will open an issue on the other thing you mentioned. |
When the database has a lot of freepages, the cost to sync all freepages down to disk is high. If the total database size is small (<10GB), and the application can tolerate ~10 seconds recovery time, then it is reasonable to simply not sync freelist and rescan the db to rebuild freelist on recovery.
When the database has a lot of freepages, the cost to sync all
freepages down to disk is high. If the total database size is
small (<10GB), and the application can tolerate ~10 seconds
recovery time, then it is reasonable to simply not sync freelist
and rescan the db to rebuild freelist on recovery.