Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve test coverage for releaseRange rollbacks. #37

Merged
merged 1 commit into from
Sep 11, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 105 additions & 0 deletions tx_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -602,6 +602,111 @@ func TestTx_CopyFile_Error_Normal(t *testing.T) {
}
}

// TestTx_releaseRange ensures db.freePages handles page releases
// correctly when there are transaction that are no longer reachable
// via any read/write transactions and are "between" ongoing read
// transactions, which requires they must be freed by
// freelist.releaseRange.
func TestTx_releaseRange(t *testing.T) {
// Set initial mmap size well beyond the limit we will hit in this
// test, since we are testing with long running read transactions
// and will deadlock if db.grow is triggered.
db := MustOpenWithOption(&bolt.Options{InitialMmapSize: os.Getpagesize() * 100})
defer db.MustClose()

bucket := "bucket"

put := func(key, value string) {
if err := db.Update(func(tx *bolt.Tx) error {
b, err := tx.CreateBucketIfNotExists([]byte(bucket))
if err != nil {
t.Fatal(err)
}
return b.Put([]byte(key), []byte(value))
}); err != nil {
t.Fatal(err)
}
}

del := func(key string) {
if err := db.Update(func(tx *bolt.Tx) error {
b, err := tx.CreateBucketIfNotExists([]byte(bucket))
if err != nil {
t.Fatal(err)
}
return b.Delete([]byte(key))
}); err != nil {
t.Fatal(err)
}
}

getWithTxn := func(txn *bolt.Tx, key string) []byte {
return txn.Bucket([]byte(bucket)).Get([]byte(key))
}

openReadTxn := func() *bolt.Tx {
readTx, err := db.Begin(false)
if err != nil {
t.Fatal(err)
}
return readTx
}

checkWithReadTxn := func(txn *bolt.Tx, key string, wantValue []byte) {
value := getWithTxn(txn, key)
if !bytes.Equal(value, wantValue) {
t.Errorf("Wanted value to be %s for key %s, but got %s", wantValue, key, string(value))
}
}

rollback := func(txn *bolt.Tx) {
if err := txn.Rollback(); err != nil {
t.Fatal(err)
}
}

put("k1", "v1")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test sequence is deterministic. we could randomize the test sequence to increase our confidence. but i think it is OK for now. better than nothing.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, definitely better than nothing. Could leave behind a TODO comment about that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions. Yes, the goal of this test is fairly modest-- Guarantee that we have coverage of a releaseRange being applied to the txid ranges between long running transactions and that we check that the read transactions can read values from the txids they are pinned to after the release.

I'll flesh out the documentation of this test to make this more clear.

I'm working on a separate quick (randomized) test that might be better used for soak testing. I need to compare it with the existing simulation_test and then will send out an issue summarizing how it would work for review, and if there is interest in a PR, I'll submit one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpbetz thanks! make sense to me.

rtx1 := openReadTxn()
put("k2", "v2")
hold1 := openReadTxn()
put("k3", "v3")
hold2 := openReadTxn()
del("k3")
rtx2 := openReadTxn()
del("k1")
hold3 := openReadTxn()
del("k2")
hold4 := openReadTxn()
put("k4", "v4")
hold5 := openReadTxn()

// Close the read transactions we established to hold a portion of the pages in pending state.
rollback(hold1)
rollback(hold2)
rollback(hold3)
rollback(hold4)
rollback(hold5)

// Execute a write transaction to trigger a releaseRange operation in the db
// that will free multiple ranges between the remaining open read transactions, now that the
// holds have been rolled back.
put("k4", "v4")

// Check that all long running reads still read correct values.
checkWithReadTxn(rtx1, "k1", []byte("v1"))
checkWithReadTxn(rtx2, "k2", []byte("v2"))
rollback(rtx1)
rollback(rtx2)

// Check that the final state is correct.
rtx7 := openReadTxn()
checkWithReadTxn(rtx7, "k1", nil)
checkWithReadTxn(rtx7, "k2", nil)
checkWithReadTxn(rtx7, "k3", nil)
checkWithReadTxn(rtx7, "k4", []byte("v4"))
rollback(rtx7)
}

func ExampleTx_Rollback() {
// Open the database.
db, err := bolt.Open(tempfile(), 0666, nil)
Expand Down