Skip to content
View nemesisdb's full-sized avatar
  • UK

Block or report nemesisdb

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
nemesisdb/README.md

Warning

This repo is no longer maintained. I've started a new data cache which uses FlatBuffers: fcache.


NemesisDB

NemesisDB is an in-memory database using JSON over websockets. There are APIs for:

  • Key values
  • Arrays
  • Lists

KV

  • A JSON key is mapped to its value
  • Keys cannot expire, they must be deleted by command
  • A future update will add key expiry

Arrays

These are fixed sized arrays, implemented in contigious memory, with versions for:

  • Unsorted arrays for integers, strings and JSON objects
  • Sorted arrays for integers and strings

Lists

A node based linked list for JSON objects:

  • Common features plus splicing
  • More features will be added


Important

The Nemesis API docs have fallen behind after many recent changes, but the Python docs are current.



Python API

There is an early version of a Python API with docs here.

KV:

from ndb.client import NdbClient
from ndb.kv import KV

client = NdbClient()
await client.open('ws://127.0.0.1:1987/')

kv = KV(client)
await kv.set({'username':'billy', 'password':'billy_password'})

username = await kv.get(key='username')
print (username) # billy

values = await kv.get(keys=('username','password'))
print (values) # {'password':'billy_password', 'username':'billy'}

Sorted integer array:

from ndb.client import NdbClient
from ndb.arrays import SortedIntArrays


client = NdbClient()
await client.open('ws://127.0.0.1:1987/')

sortedInts = SortedIntArrays(client)

await sortedInts.create('array1', capacity=5)
await sortedInts.create('array2', capacity=6)

await sortedInts.set_rng('array1', [25,10,50,100,80])
await sortedInts.set_rng('array2', [10,25,100,120,200,5])

intersected = await sortedInts.intersect('array1', 'array2')
print(intersected) # [10,25,100]

Object list:

from ndb.client import NdbClient
from ndb.lists import ObjLists

client = NdbClient()
await client.open('ws://127.0.0.1:1987/')

lists = ObjLists(client)

await lists.create('names')

# add 3 items
await lists.add('names', [{'name':'James'}, {'name':'Jane'}, {'name':'John'}])
# add new head (Jack, James, Jane, John)
await lists.add_head('names', {'name':'Jack'}) 
# overwrite Jane and John (Jack, James, Brian, Bryan)
await lists.set('names', [{'name':'Brian'},{'name':'Bryan'}], start=2)
# splice Brian and Bryan to a newly created list
await lists.splice(destName='other_names', srcName='names', srcStart=2, srcEnd=4)

print(await lists.get_rng('names', start=0))        # [{'name': 'Jack'}, {'name': 'James'}]
print(await lists.get_rng('other_names', start=0))  #[{'name': 'Brian'}, {'name': 'Bryan'}]


Install

NemesisDB is available as a Debian package and Docker image:

You can compile for Linux, instructions below.



Persist and Restore

Key values can be saved to file and restored at either runtime or at startup in the command line.



Build - Linux Only

Important

C++20 required.

  1. Clone via SSH with submodules: git clone --recursive git@github.com:nemesisdb/nemesisdb.git
  2. Prepare and grab vcpkg libs: cd nemesisdb && ./prepare_vcpkg.sh
  3. With VS Code (assuming you have C/C++ and CMake extensions):
    • code .
    • Select kit (tested with GCC 12.3 and GCC 13.1)
    • Select Release variant
    • Select target as nemesisdb
    • Build
  4. Binary is in server/Release/bin

Run

After installing via dpkg:

  • cd /usr/local/bin/nemesisdb
  • Then cd <version>

After build:

  • cd <clone_location>/server/Release/bin

In both cases, start server listening on 127.0.0.1:1987 and persistence disabled:

./nemesisdb --config=default.jsonc

ctrl+c to exit.



External Libraries

Externals are either GitHub submodules or managed by vcpkg.

Server:

  • uWebsockets : WebSocket server
  • jsoncons : json
  • ankerl::unordered_dense for map and segmented map
  • plog : logging
  • uuid_v4 : create UUIDs with SIMD
  • Boost Program Options : argv options

Tests Client (will be replaced with Python):

  • nlohmann json
  • Boost Beast (WebSocket client)
  • Google test

Popular repositories Loading

  1. nemesisdb nemesisdb Public

    C++ powered cache with key-values, lists and arrays. Python API.

    C++ 3

  2. docs-deploy docs-deploy Public

    Docs for NemesisDB

    HTML