| Badge | Status |
|---|---|
| Current Version |  |
Redis is often referred as a data structures server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a server-client model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
Data structures implemented into Redis have a few special properties:
- Redis cares to store them on disk, even if they are always served and modified into the server memory. This means that Redis is fast, but that is also non-volatile.
- Implementation of data structures stress on memory efficiency, so data structures inside Redis will likely use less memory compared to the same data structure modeled using an high level programming language.
- Redis offers a number of features that are natural to find in a database, like replication, tunable levels of durability, cluster, high availability.
Another good example is to think of Redis as a more complex version of memcached, where the operations are not just SETs and GETs, but operations to work with complex data types like Lists, Sets, ordered data structures, and so forth.
If you want to know more, this is a list of selected starting points:
- Introduction to Redis data types. http://redis.io/topics/data-types-intro
- Try Redis directly inside your browser. http://try.redis.io
- The full list of Redis commands. http://redis.io/commands
- There is much more inside the Redis official documentation. http://redis.io/documentation
Pmem-Redis is one redis version that support Intel DCPMM(Data Center Persistent Memory) based on open source redis-4.0.0. It benefits the redis's performance by taking advantage of DCPMM competitive performance and persistency.
Basically Pmem-Redis covers many aspects that related to DCPMM usage:
- Five typical data structures optimization including: String, List, Hash, Set, Zset.
- DCPMM copy-on-write
- Redis LRU for DCPMM
- Redis defragmentation support for DCPMM
- Pointer-based redis AOF
- Persistent ring buffer
- clone the code:
git clone https://github.com/pmem/pmem-redis
- PMDK is one submodule of this repo, so you have to init this submodule in order to get code
git submodule init
git submodule update
| Switches | Value | Descriptions |
|---|---|---|
| USE_NVM | yes/no | DCPMM enable Switch. W/O this option, will compile opensource redis that does not support DCPMM. |
| AEP_COW | yes/no | DCPMM Copy-On-Write Switch. W/O this option, the BG save and replication will not support |
| SUPPORT_PBA | yes/no | Pointer Based Aof support Switch. W/O this option, PBA is not support, Same AOF mechanism with open source redis. |
| USE_AOFGUARD | yes/no | Write Turbo with DCPMM option switch. W/O this option, the AOF log write to the SSD by the page cache directly. |
Prerequisite
- Install
autoconf,automake,libtool,numactl-develandtclon system
If you want to build the Pmem-Redis for DCPMM, please install following packages as well:
libndctl-dev, libdaxctl-dev and libnuma-dev
Compile example:
If you want to build the original opensource redis-4.0.0, run command:
make
If you want to build the Pmem-Redis for DCPMM, run command:
make USE_NVM=yes
If you need enable AEP_COW, SUPPORT_PBA or USE_AOFGUARD, you need to enable USE_NVM compile option.
Pmem-Redis has some dependencies which are included into the deps directory.
make does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
When you update the source code with git pull or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
make distclean
This will clean: jemalloc, lua, hiredis, linenoise, pmdk, memkind, jemallocat, aofguard.
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
those options are cached indefinitely until you issue a make distclean
command.
| server command | argument | describe |
|---|---|---|
| --nvm-maxcapacity | Integer (G) | Set the maximum DCPMM capacity of the redis instance |
| --nvm-dir | Like: /mnt/pmem0 | Path of DCPMM DAX file system |
| --nvm-threshold | Integer (default 64) | Threshold of value size, if greater than threshold, the value may be stored on the DCPMM |
| --pointer-based-aof | yes/no | Enable or disable pointer based AOF |
| --use-aofguard | yes/no | Enable or disable persistent ring buffer |
Create namespace:
ndctl create-namespace -m fsdax -r <regionid>
Then you may see /dev/pmem<id> under /dev folder, for example, we have /dev/pmem0 in our machine.
Format the pmem device:
mkfs.ext4 /dev/pmem0
Mount as dax:
mkdir /mnt/pmem0
mount -o dax /dev/pmem0 /mnt/pmem0
src/redis-server --nvm-maxcapacity 1 --nvm-dir /mnt/pmem0 --nvm-threshold 64
This command starts a redis server that uses DCPMM device(/mnt/pmem0) with max capacity 1GB, and it moves the values whose length are greater than 64 into DCPMM.
-
In our Unit Test, we default read
/mnt/pmem0as pmem device and run test, please modify your pmem device folder to/mnt/pmem0to meet the requirement. -
If you enable --pointer-based-aof in redis server, than pmem-redis will generate one
.agfile under/mnt/pmem0device which link to the AOF file generated on disk. If you need to delete the AOF file, please make sure to delete the.agfile at the same time to avoid AOF load error.
- Please fork this repositry to your github account and start deveopment, please refer to redis code style
- Pull requests let you tell others about changes you've pushed to a repository on GitHub.
- Once a pull request is opened, you can discuss and review the potential changes with collaborators and add follow-up commits before the changes are merged into the repository.
Note: When working with pull requests, keep the following in mind:
If you're working in the shared repository model, we recommend that you use a topic branch for your pull request. While you can send pull requests from any branch or commit, with a topic branch you can push follow-up commits if you need to update your proposed changes.
When pushing commits to a pull request, don't force push. Force pushing can corrupt your pull request.
After initializing a pull request, you'll see a review page that shows a high-level overview of the changes between your branch (the compare branch) and the repository's base branch. You can add a summary of the proposed changes, review the changes made by commits, add labels, milestones, and assignees, and @mention individual contributors or teams. For more information, see "Creating a pull request."
Once you've created a pull request, you can push commits from your topic branch to add them to your existing pull request. These commits will appear in chronological order within your pull request and the changes will be visible in the "Files changed" tab.
Other contributors can review your proposed changes, add review comments, contribute to the pull request discussion, and even add commits to the pull request.
Enjoy!