asyncio-foundationdb

FoundationDB drivers for asyncio tested with CPython, and PyPy 3.8+.

Marketing

In the fast-paced digital landscape, FoundationDB is the unsung hero of data management. That Key-Value Store serve as the backbone of countless applications, providing lightning-fast access to essential information. With its simple yet powerful structure, FoundationDB effortlessly organizes and retrieve data, making it the go-to choice for developers seeking speed and efficiency. Whether you're building a dynamic web application, a datalake knowledge base, a robust caching system, or a real-time analytics platform, FoundationDB is your trusty ally, ensuring seamless data access and enabling innovation at the speed of thought. Discover the key to data-driven success with FoundationDB – where simplicity meets scalability, and speed meets reliability

• Getting started
• ChangeLog
  • v0.12.x
• import found
• from found import bstore
• from found import nstore
• from found import eavstore
• from found import pstore
• from found import vnstore

Enter the world of data organization and retrieval with FoundationDB, the Ordered Key-Value Store. FoundationDB is your solution for maintaining structured data in a way that preserves both order and simplicity. With the power to efficiently sort and access data, FoundationDB is a versatile tool for a wide range of applications. From managing time-series data in financial systems, or hierarchies, to optimizing search functionality in e-commerce platforms, FoundationDB offers an elegant and reliable solution. Take control of your data, embrace order, and unlock a new level of efficiency with FoundationDB - where data is not just stored, but intelligently organized for streamlined success.

FoundationDB, the bedrock of modern data infrastructure, is the groundbreaking distributed database system that unlocks new frontiers in reliability, scalability, and performance. With a unique combination of ACID transactions, distributed architecture, and a highly versatile data model, FoundationDB seamlessly handles complex workloads while ensuring data integrity. It's the go-to choice for organizations seeking a solid foundation for mission-critical applications, from e-commerce platforms to cutting-edge IoT ecosystems. Harness the power of FoundationDB and experience a world where your data is always available, always consistent, and always ready to fuel your boldest innovations.

Build on a solid foundation with FoundationDB.

Installation

In a minute, install foundationdb, getting the latest stable release from the official release page: https://github.com/apple/foundationdb/releases

Then install asyncio drivers asyncio-foundationdb:

pip install asyncio-foundationdb

Example

async def readme():

    async def get(tx, key):
        out = await found.get(tx, key)
        return out

    async def set(tx, key, value):
        await found.set(tx, key, value)

    db = await found.open()
    out = await found.transactional(db, get, b'hello')
    assert out is None

    await found.transactional(db, set, b'hello', b'world')
    out = await found.transactional(db, get, b'hello')
    assert out == b'world'

    await found.transactional(db, set, b'azul', b'world')
    out = await found.transactional(db, get, b'azul')
    assert out == b'world'

    async def query(tx, key, other):
        out = await found.all(found.query(tx, key, other))
        return out

    out = await found.transactional(db, query, b'', b'\xFF')
    assert [(b'azul', b'world'), (b'hello', b'world')]

asyncio.run(readme())

ChangeLog

v0.12.0

• Move back to GitHub;
• Add versioned generic tuple store (code name vnstore)

v0.10.x

• Almost full rewrite
• Remove hooks for the time being
• Port Generic Tuple Store aka. nstore
• Add blob store aka. bstore
• Add Entity-Attribute-Value store aka. eavstore
• Add inverted index store aka. pstore

import found

found.BaseFoundException

All found exceptions inherit that class.

found.FoundException

Exception raised when there is an error foundationdb client driver, or foundationdb server side.

await found.open(cluster_file=None)

Open database.

Coroutine that will open a connection with the cluster specified in the file cluster_file. If cluster_file is not provided the default is /etc/foundationdb/fdb.cluster. Returns a database object.

await found.transactional(db, func, *args, snapshot=False, **kwargs)

Operate a transaction for func.

Coroutine that will operate a transaction against db for func. If snapshot=True then the transaction is read-only. func will receive an appropriate transaction object as first argument, then args, then kwargs. Because of errors transactional might run func several times, hence func should be idempotent.

The function func receive transaction object that should be passed to other database functions. It has a property vars that is a dictionary that can be used to cache objects for the extent of the transaction.

await found.get(tx, key)

Get the value associated with key.

Coroutine that will fetch the value associated with key inside the database associated with tx. key must be bytes. In case of success, returns bytes. Otherwise, if there is no value associated with key, returns the object None.

await found.set(tx, key, value)

Set key to value.

In the database associated with tx, associate key with value. Both key and value must be bytes.

found.pack(tuple)

Serialize python objects tuple into bytes.

found.pack_with_versionstamp(tuple)

Serialize python objects tuple into bytes. tuple may contain found.Versionstamp objects.

found.unpack(bytes)

Deserialize bytes into python objects.

found.Versionstamp(...)

FIXME.

await found.clear(tx, key, other=None)

Remove key or keys.

In the database associated with tx, clear the specified key or range of keys.

key and other if provided must be bytes.

If other=None, then clear the association that might exists with key. Otherwise, if other is provided, found.clear will remove any association between key and other but not the association with other if any (that is other is excluded from the range).

await found.query(tx, key, other, *, limit=0, mode=STREAMING_MODE_ITERATOR)

Fetch key-value pairs.

In the database associated with tx, generate at most limit key-value pairs inside the specified range, with the specified order.

If key < other then found.query generates key-value pairs in lexicographic order. Otherwise, if key > other then found.query generates key-value pairs in reverse lexicographic order, that is starting at other until key.

If limit=0, then found.query generates all key-value pairs in the specified bounds. Otherwise if limit > 0 then, it generates at most limit pairs.

The keyword mode can be one the following constant:

• found.STREAMING_MODE_WANT_ALL
• found.STREAMING_MODE_ITERATOR
• found.STREAMING_MODE_EXACT
• found.STREAMING_MODE_SMALL
• found.STREAMING_MODE_MEDIUM
• found.STREAMING_MODE_LARGE
• found.STREAMING_MODE_SERIAL

found.next_prefix(key)

Returns the immediatly next byte sequence that is not prefix of key.

found.lt(key, offset=0)

found.lte(key, offset=0)

found.gt(key, offset=0)

found.gte(key, offset=0)

await found.read_version(tx)

found.set_read_version(tx, version)

found.add(tx, key, param)

found.bit_and(tx, key, param)

found.bit_or(tx, key, param)

found.bit_xor(tx, key, param)

found.max(tx, key, param)

found.byte_max(tx, key, param)

found.min(tx, key, param)

found.byte_min(tx, key, param)

found.set_versionstamped_key(tx, key, param)

found.set_versionstamped_value(tx, key, param)

from found import bstore

bstore.BStoreException

Exception specific to bstore.

bstore.make(name, prefix)

Handle over a bstore called name with prefix.

await bstore.get_or_create(tx, bstore, blob)

await bstore.get(tx, bstore, uid)

from found import nstore

nstore.NStoreException

Exception specific to nstore.

nstore.make(name, prefix, n)

Create a handle over a nstore called name with prefix and n columns.

The argument name should be a string, it is really meant to ease debugging. prefix should be a tuple that can be packed with found.pack. Last but not least, n is the number of columns in the returned tuple store (or, if you prefer, the number of tuple items).

It is preferable to store the returned value.

await nstore.add(tx, nstore, *items, *, value=b'')

In the database associated with tx, as part of nstore, add items associated with value.

await nstore.remove(tx, nstore, *items)

In the database associated with tx, as part of nstore, remove items and the associated value.

await nstore.get(tx, nstore, *items)

In the database associated with tx, as part of nstore, get the value associated with items. If there is no such items in nstore, returns None.

nstore.var(name)

Create a variable called name for use with nstore.query.

await nstore.query(tx, nstore, pattern, *patterns)

In the database associated with tx, as part of nstore, generate mappings that match pattern and patterns. Both pattern and patterns may contain nstore.var that will be replaced with matching values in the generic tuple store.

from found import eavstore

eavstore.make(name, prefix)

Create a handle over an eavstore called name with prefix.

The argument name should be a string, it is really meant to ease debugging. prefix should be a tuple that can be packed with found.pack.

await eavstore.create(tx, eavstore, dict)

Store a dictionary.

In the database associated with tx, as part of eavstore, save dict and returns its unique identifier.

await eavstore.get(tx, eavstore, uid)

Fetch a dictionary.

In the database associated with tx, as part of eavstore, retrieve the dictionary associated with uid. If there is no such dictionary, returns an empty dictionary.

await eavstore.remove(tx, eavstore, uid)

Clear a dictionary.

In the database associated with tx, as part of eavstore, remove the dictionary associated with uid.

await eavstore.update(tx, eavstore, uid, dict)

Update a dictionary.

In the database associated with tx, as part of eavstore, replace the dictionary associated with uid with dict.

await eavstore.query(tx, eavstore, key, value)

Lookup dictionaries according to sppecification.

In the database associated with tx, as part of eavstore, generates unique identifier for dictionaries that have key equal to value.

from found import pstore

pstore.PStoreException

Exception specific to pstore.

pstore.make(name, prefix, pool)

A handle over a pstore called name with prefix, that will use pool.

await pstore.index(tx, store, docuid, counter)

Associates docuid with counter.

Coroutine that associates the identifier docuid with the dict-like counter inside the database associated with tx at store for later retriaval with pstore.search.

counter must be a dict-like mapping string to integers bigger than zero.

await pstore.search(tx, store, keywords, limit)

Return a sorted list of at most limit documents matching keywords.

from found import vnstore

vnstore.make(name, prefix, items)

Create a handle over a vnstore called name with the prefix tuple prefix, and items as column names.

The argument name should be a string, it is really meant to ease debugging. prefix should be a tuple that can be packed with found.pack. Last but not least, items is the columns in the returned tuple store (or, if you prefer, the name of tuple items).

It is preferable to store the returned value.

await vnstore.change_create(tr, vnstore)

Return the unique idenifier of a new change in database. Its initial signifiance is None which means it is invisible to other transactions, and its message None.

vnstore.change_continue(tr, vsntore, changeid)

Against transaction tr, and vnstore, continue a change changeid.

await vnstore.change_message(tr, vnstore, changeid, message)

Replace the exisiting message of changeid with message

await vnstore.change_apply(tr, vnstore, changeid)

Apply the change changeid against vnstore, setting the next uuid7 as significance.

Known issue: Weak serializability

The use of uuid7 instead of versionstamps can break things when changes happen over overlapping versioned triples. Strict ordering, serializability is not guaranteed, hence one transaction may write, a value based on a value that was overwritten by another change that appears to be commited after according to its uuid7 significance. Even if changes are commited in the correct order uuid7 does not guarantee serializability.

In other words, as long as we rely uuid7 we can't consider changes commited with vnstore_change_apply happen as if all changes were commited after the other, that is, there is no serializability guarantee.

Contact me for workarounds

Known issue: consistency

Since changes may be constructed with several transactions, it is possible that two changes introduce consistency bugs.

Contact me for workarounds

await vnstore.ask(tr, vnstore, *items)

Return True if items is alive in the space vnstore.

await vnstore.remove(tr, vnstore, *items)

Remove items from vnstore.

await vnstore.query(tr, vnstore, pattern, *pattern)

Return immutable mappings where vnstore.var from pattern, and patterns are replaced with objects from vnstore.