forked from peter-lawrey/Java-Chronicle
-
Notifications
You must be signed in to change notification settings - Fork 0
craigday/Java-Chronicle
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
This library is an ultra low latency, high throughput, persisted, messaging and event driven in memory database. The typical latency is as low at 80 nano-seconds and supports throughputs of 5-20 million messages/record updates per second. It uses almost no heap, trivial GC impact, can be much larger than your physical memory size (only limited by the size of your disk) and can be shared *between processes* with better than 1/10th latency of using Sockets over loopback. It can change the way you design your system because it allows you to have independent processes which can be running or not at the same time (as no messages are lost) This is useful for restarting services and testing your services from canned data. e.g. like sub-microsecond durable messaging. You can attach any number of readers, including tools to see the exact state of the data externally. e.g. I use; od -t cx1 {file} to see the current state. =================================== Support Group =================================== https://groups.google.com/forum/?fromgroups#!forum/java-chronicle =================================== Version History =================================== Version 1.1 - Add support for OutputStream and InputStream required by ObjectOutputStream and ObjectInputStream. Using Java Serialization is not suggested as its relatively slow, but sometimes its your only option. ;) Version 1.0 - First formal release available in https://github.com/peter-lawrey/Java-Chronicle/tree/master/repository Version 0.5.1 - Fix code to compile with Java 6 update 31. (Previously only Java 7 was used) Version 0.5 - Add support for replication of a Chronicle over TCP to any number of listeners, either as a component or stand alone/independent application. Uses ChronicleSource and ChronicleSink. Add ChronicleReader to read records as they are added as text. (like less) Version 0.4 - Add support for writing text to the log file without creating garbage via vanilla.java.chronicle.ByteStringAppender interface. Useful for text logs. Version 0.3.1 - Add support for 24-bit int and 48-bit long values. Version 0.3 - Add support for unsigned byte, short and int. Add support for compacted short, unsigned short, int, unsigned int, long and double types. (Type will use half the size for small values otherwise 50% more) Version 0.2 - Add support for a 32-bit unsigned index. IntIndexedChronicle. This is slightly slower on a 64-bit JVM, but more compact. Useful if you don't need more than 4 GB of data. Version 0.1 - Can read/write all basic data types. 26 M/second (max) multi-threaded. It uses memory mapped file to store "excerpts" of a "chronicle" Initially it only supports an indexed array of data. =================================== Throughput Test - FileLoggingMain =================================== https://github.com/peter-lawrey/Java-Chronicle/blob/master/testing/src/main/java/vanilla/java/chronicle/impl/FileLoggingMain.java This test logs one million lines of text using Chronicle compared with Logger. To log 1,000,000 messages took 0.234 seconds using Chronicle and 7.347 seconds using Logger =================================== Throughput Test - IndexedChronicleThroughputMain =================================== Note: These timings include Serialization. This is important because many performance tests don't include Serialization even though it can be many times slower than the data store they are testing. https://github.com/peter-lawrey/Java-Chronicle/blob/master/testing/src/main/java/vanilla/java/chronicle/impl/IndexedChronicleLatencyMain.java On a 4.6 GHz, i7-2600, 16 GB of memory, Fast SSD drive. Centos 5.7. The average RTT latency was 175 ns. The 50/99 / 99.9/99.99%tile latencies were 160/190 / 2,870/3,610 - ByteBuffer (tmpfs) The average RTT latency was 172 ns. The 50/99 / 99.9/99.99%tile latencies were 160/190 / 2,780/3,520 - Using Unsafe (tmpfs) The average RTT latency was 180 ns. The 50/99 / 99.9/99.99%tile latencies were 160/190 / 3,110/19,110 - ByteBuffer (ext4) The average RTT latency was 178 ns. The 50/99 / 99.9/99.99%tile latencies were 160/190 / 3,100/19,090- Using Unsafe (ext4) https://github.com/peter-lawrey/Java-Chronicle/blob/master/testing/src/main/java/vanilla/java/chronicle/impl/IndexedChronicleThroughputMain.java On a 4.6 GHz, i7-2600, 16 GB of memory, Fast SSD drive. Centos 5.7. Took 12.416 seconds to write/read 200,000,000 entries, rate was 16.1 M entries/sec - ByteBuffer (tmpfs) Took 9.185 seconds to write/read 200,000,000 entries, rate was 21.8 M entries/sec - Using Unsafe (tmpfs) Took 25.693 seconds to write/read 400,000,000 entries, rate was 15.6 M entries/sec - ByteBuffer (ext4) Took 19.522 seconds to write/read 400,000,000 entries, rate was 20.5 M entries/sec - Using Unsafe (ext4) Took 71.458 seconds to write/read 1,000,000,000 entries, rate was 14.0 M entries/sec - Using Unsafe (ext4) Took 141.424 seconds to write/read 2,000,000,000 entries, rate was 14.1 M entries/sec - Using Unsafe (ext4) Note: in the last test, it is using 112 GB! of dense virtual memory in Java without showing a dramatic slow down or performance hit. The 14.1 M entries/sec is close to the maximum write speed of the SSD as each entry is an average of 28 bytes (with the index) => ~ 400 MB/s =================================== More compact Index for less than 4 GB of data =================================== https://github.com/peter-lawrey/Java-Chronicle/blob/master/testing/src/main/java/vanilla/java/chronicle/impl/IntIndexedChronicleThroughputMain.java on a 4.6 GHz, i7-2600 Took 6.325 seconds to write/read 100,000,000 entries, rate was 15.8 M entries/sec - ByteBuffer (tmpfs) Took 4.590 seconds to write/read 100,000,000 entries, rate was 21.8 M entries/sec - Using Unsafe (tmpfs) Took 7.352 seconds to write/read 100,000,000 entries, rate was 13.6 M entries/sec - ByteBuffer (ext4) Took 5.283 seconds to write/read 100,000,000 entries, rate was 18.9 M entries/sec - Using Unsafe (ext4)
About
Java Indexed Record Chronicle
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Java 100.0%