Important information: the pruning functionality introduced with IAVL 0.13.0 via the options
KeepEvery
and KeepRecent
has multiple problems with data corruption, performance, and
memory usage. For these reasons, this functionality has now been removed. All 0.13 users are
urged to upgrade once the final 0.14.0 release is out, and to not change their pruning settings
while on 0.13.
If you are using 0.13 with a KeepEvery
value other than 1
(the default), then when upgrading
to 0.14 (or newer) it is important to follow these instructions:
-
If possible, upgrade after saving a multiple of
KeepEvery
- for example, withKeepEvery: 1000
then stop 1.13 immediately after saving e.g. version7000
to disk. Upgrading to 1.14 is then safe. -
Otherwise, after upgrading to 0.14, do not delete the last version saved to disk - this contains incorrect data that may cause data corruption, making the database unusable. For example, with
KeepEvery: 1000
then stopping 1.13 at version7364
(saving7000
to disk) and upgrading to 1.14 means version7000
must never be deleted.It may be possible to delete it if the same sequence of changes have been written to the newer versions as before the upgrade, and all versions between
7000
and7364
are deleted, but thorough testing and backups are recommended if attempting this.
Users wishing to prune historical versions can do so via MutableTree.DeleteVersion()
.
Special thanks to external contributors on this release: @ridenaio
- #274 Remove pruning options
KeepEvery
andKeepRecent
(see warning above) and therecentDB
parameter toNewMutableTreeWithOpts()
.
-
#271 Add
MutableTree.DeleteVersions()
for deleting multiple versions -
#235 Reduce
ImmutableTree.Export()
buffer size from 64 to 32 nodes
- #275 Fix data corruption with
LoadVersionForOverwriting
- [import] #230 Set correct version when committing an empty import.
- [#213] Added
ImmutableTree.Export()
andMutableTree.Import()
to export tree contents at a specific version and import it to recreate an identical tree.
- [dep] #220 Update tm-db to 0.5.0, which includes a new B-tree based MemDB used by IAVL for non-persisted versions.
- [nodedb] #219 Fix a concurrent database access issue when deleting orphans.
Special thanks to external contributors on this release: @rickyyangz, @mattkanwisher
- [pruning] #158 NodeDB constructor must provide
keepRecent
andkeepEvery
fields to define PruningStrategy. All Save functionality must specify whether they should flushToDisk as well usingflushToDisk
boolean argument. All Delete functionality must specify whether object should be deleted from memory only using thememOnly
boolean argument. - [dep] #194 Update tm-db to 0.4.0 this includes interface breaking to return errors.
- [orphans] #177 Collect all orphans after remove (@rickyyangz)
- #46 Removed all instances of cmn (tendermint/tendermint/libs/common)
Special thanks to external contributors on this release: @ethanfrey
IMPROVEMENTS
- Implement LazyLoadVersion (@alexanderbez) LazyLoadVersion attempts to lazy load only the specified target version without loading previous roots/versions. - see goDoc
- Move to go.mod (@Liamsi)
iaviewer
command to visualize IAVL database from leveldb (@ethanfrey)
IMPROVEMENTS
- Use Tendermint v0.30.2 and close batch after write (related pull request in Tendermint: tendermint/tendermint#3397)
IMPROVEMENTS
- Use Tendermint v0.30
BREAKING CHANGES
- Uses new Tendermint ReverseIterator API. See tendermint/tendermint#2913
IMPROVEMENTS
- Uses GoAmino v0.14
BREAKING CHANGES
- Changed internal database key format to store int64 key components in a full 8-byte fixed width (#107)
- Removed some architecture dependent methods (e.g., use
Get
instead ofGet64
etc) (#96)
IMPROVEMENTS
- Database key format avoids use of fmt.Sprintf fmt.Sscanf leading to ~10% speedup in benchmark BenchmarkTreeLoadAndDelete (#107, thanks to @silasdavis)
BREAKING CHANGES
- refactored API for clean separation of mutable and immutable tree (#92, #88);
with possibility to:
- load read-only snapshots at previous versions on demand
- load mutable trees at the most recently saved tree
BUG FIXES
- remove memory leaks (#92)
IMPROVEMENTS
- Change tendermint dep to ^v0.22.0 (#91)
BREAKING CHANGES
- getRangeProof and Get[Versioned][Range]WithProof return nil proof/error if tree is empty.
IMPROVEMENTS
- some minor changes: mainly lints, updated parts of documentation, unexported some helpers (#80)
IMPROVEMENTS
- RangeProof.ComputeRootHash() to compute root rather than provide as in Verify(hash)
- RangeProof.Verify*() first require .Verify(root), which memoizes
BREAKING CHANGES
- RangeProof.VerifyItem doesn't require an index.
- Only return values in range when getting proof.
- Return keys as well.
BUG FIXES
- traversal bugs in traverseRange.
- Swap
tmlibs
fortendermint/libs
- Remove
sha256truncated
in favour oftendermint/crypto/tmhash
- same hash function but technically a breaking change to the API, though unlikely to effect anyone.
NOTE this means IAVL is now dependent on Tendermint Core for the libs (since it
makes heavy use of the db
package). Ideally, that dependency would be
abstracted away, and/or this repo will be merged into the Cosmos-SDK, which is
currently is primary consumer. Once it achieves greater stability, we could
consider breaking it out into it's own repo again.
July 1st, 2018
BUG FIXES
- fix bug in iterator going outside its range
BREAKING CHANGES
- Nodes are encoded using proto3/amino style integers and byte slices (ie. varints and varint prefixed byte slices)
- Unified RangeProof
- Proofs are encoded using Amino
- Hash function changed from RIPEMD160 to the first 20 bytes of SHA256 output
BREAKING CHANGES
- LoadVersion and Load return the loaded version number
- NOTE: this behaviour was lost previously and we failed to document in changelog, but now it's back :)
IMPROVEMENT
- Remove spurious print statement from LoadVersion
BREAKING CHANGES
- NewTree order of arguments swapped
- int -> int64, uint64 -> int64
- NewNode takes a version
- Node serialization format changed so version is written right after size
- SaveVersion takes no args (auto increments)
- tree.Get -> tree.Get64
- nodeDB.SaveBranch does not take a callback
- orphaningTree.SaveVersion -> SaveAs
- proofInnerNode includes Version
- ReadKeyXxxProof consolidated into ReadKeyProof
- KeyAbsentProof doesn't include Version
- KeyRangeProof.Version -> Versions
FEATURES
- Implement chunking algorithm to serialize entire tree
First versioned release! (Originally accidentally released as v0.2.0)