v0.5.0

Breaking Changes

• The Minimum Supported Rust Version (MSRV) has been changed to 1.70.

• All features in the bonsaidb crate have been updated to support the new
  features in Rust 1.60. Instead of needing separate flags for
  client-websockets and server-websockets, a single feature flag
  websockets is now able to work in conjunction with the client/server
  features to ensure everything works correctly.

  The net change is fewer feature flags. If you find that a feature flag is
  missing on upgrade, try removing "local-", "client-", or "server-" from the
  beginning of the feature and recompiling.

• The Key implementation for Option<T> has changed. If you wish to preserve
  backwards compatibility, wrap the key type with OptionKeyV1.

  The old behavior was broken in two ways:

  • The length of the key was improperly always reported as a constant length.
    This only affects code that was using composite keys.
  • When using a type that can encode itself as 0 bytes, there was no way to
    distinguish between a 0-length contained value and None. Thus, Some("")
    would panic as a precaution. However, this type of lurking panic in an
    esoteric edge case is exactly the behavior a database should never exhibit.

• The Key implementation for tuples (e.g., (T1, T2)) has changed. If you
  wish to preserve backwards compatibility, wrap the key type with TupleKeyV1.

  The old behavior had an incorrect sort order for the generated bytes. Instead
  of sorting only by the values themselves, the lengths of the variable-length
  Key types were taking precedence over the values. For more information, see
  issue #240.

  This change also deprecates encode_composite_key()/decode_composite_key()
  in favor of two new types: CompositeKeyEncoder and CompositeKeyDecoder.

• Collection's get/list/list_with_prefix/insert/overwrite and View's
  with_key_range/with_key_prefix have all been updated to accept borrowed
  representations of keys. For example, when a Collection's PrimaryKey is
  String, &str can now be supplied to these functions.

  For most users, adding & in front of the argument will generally fix the
  compiler error upon upgrade.

• DocumentId is no longer Copy, and now can support up to 64kb of data.
  Previously, the type was limited to 64 total bytes of stack space.

• [Async]StorageConnection::schematic() have been moved to its own trait,
  HasSchema.

• [Async]StorageConnection::authenticate() no longer takes a username or user
  id directly. This has been moved to the Authentication type.

• AuthenticationMethod has moved from bonsaidb::core::permissions::bonsai to
  bonsaidb::core::connection

• ApiName has been moved from bonsaidb::core::schema to
  bonsaidb::core::api.

• Various error variants that were simply String representations have been
  consoldiated into bonsaidb::core::Error::Other.

• bonsaidb::server::Backend now takes a &self parameter in all functions
  except configure(). Implementing Default for your Backend implementor
  will allow all existing code to continue working.

• Client::effective_permissions() is no longer async.

• CustomServer::connected_clients() is no longer async.

• CustomServer::broadcast() is no longer async.

• CustomServer::listen_on now takes impl Into<BonsaiListenConfig> instead of
  just a u16 parameter specifying the port. BonsaiListenConfig implements
  From<u16> to minimize code breakage.

• The command-line Serve type has had its listen-on field changed to a
  SocketAddr type to reflect the support for BonsaiListenConfig. The full
  address and port must be specified when providing a listen-on argument now.

• Client has been renamed to AsyncClient, and a new type, BlockingClient
  has been added. This splits the Client into two separate parts: an async
  version and a blocking version. This makes it less likely to accidentally call
  a blocking method in an async context.

  Client::send_api_request_async has been renamed to send_api_request.

  This change has also been introduced to RemoteDatabase and
  RemoteSubscriber: both async and blocking versions are available.

• #254 Key::from_ord_bytes has had its &'k [u8] parameter changed to a new type
  with an additional lifetime: ByteSource<'k, 'e>. This new type allows
  from_ord_bytes to be called with an owned Vec<u8>, a Key-lifetime bound
  byte slice (&'k [u8]), or an ephemeral byte slice (&'e [u8]).

  This change allows code paths that can pass an owned Vec<u8> in for decoding
  to not require allocations in some cases -- for example, when using
  Vec<u8>::from_ord_bytes.

  This change also means that Key implementors should expect to be called with
  any of the three variants of ByteSource, because BonsaiDb's internal code
  only passes borrowed slices in some code paths.

  Thank you to @asonix for the request and help on implementation!

• KeyEncoding::describe is a new function that allows a key encoder to
  document the data contained within the encoded representation. The goal of
  this is to allow tools to generically operate on key types used within
  BonsaiDb. This function is automatically implemented when using the Key
  derive.

  The new KeyDescription type uses this function to create a nested
  representation of the contained information.

  Types that utilize a custom encoding format can use KeyDescription::Other to
  uniquely identify the key.

• Schematic has had several methods changed to impl Iterator of the original
  type being returned to avoid extra unnecessary allocations. These methods are:

  • Schematic::views_in_collection
  • Schematic::eager_views_in_collection
  • Schematic::collections

• StorageConnection::list_available_schemas/AsyncStorageConnection::list_available_schemas
  now return a Vec<SchemaSummary>. The SchemaSummary type contains
  additional information such as what collections and views each schema
  contains.

• bonsaidb::cli::Command now flattens the server command rather than
  in-lining the admin command. This introduces additional top-level commands
  that were previously hidden underneath the server command.

• ConnectedClient::all_sessions() is a new function that returns all of the
  active sessions for the given client.

• View schemas are now partially deriveble, and have had their map/reduce
  functionality split into their own traits. The following notes all apply to
  this refactoring:

  • ViewSchema::map() and ViewSchema::reduce() have been moved to a new
    trait: MapReduce.
  • CollectionViewSchema has been removed, and the map() and reduce()
    functions have been moved to a new trait: CollectionMapReduce.
  • ViewSchema::unique() and ViewSchema::lazy() have been replaced with
    ViewSchema::update_policy(), which returns a new enum: ViewUpdatePolicy.
    This enum contains three variants: Lazy, Eager, and Unique, allowing the same
    options that the previous two methods supported without any ambiguities.
  • ViewSchema is now implementable/derivable for all views, regardless of
    whether the map/reduce functionality utilizes SerializedCollections or
    not.

• ViewSchema::MappedKey<'doc> is a new associated type with a generic
  associated lifetime that enables map()/reduce() to utilize borrowed data
  for the view's key type. This has caused the map() and reduce() functions
  to have new lifetime annotations added. These changes are part of an effort to
  support more flows where borrowing data is possible to minimize allocations
  while indexing and querying views.

  For all existing users, setting this associated type to the view's Key type
  will work, or pasting this associated type definition in:

  type MappedKey<'doc> = <Self::View as View>::Key

• natural_id support in the Collection derive macro has been changed. The
  attribute now expects an expression rather than a closure. The expression can
  reference self. This was done to avoid the required type annotations that
  the closure approach required.

  Alternatively, #[natural_id] can be annotated directly on a field to have it
  become the natural id automatically.

• bonsaidb::server::api::Handler has had its generic arguments order reversed,
  which allows the type to specify a default Backend of NoBackend.

• View query APIs now return CollectionMap types instead of Map types. The
  change allows for the Collection::PrimaryKey type to be used instead of
  DocumentId. The affected APIs are:

  • bonsaidb::core::connection::View::query()
  • bonsaidb::core::connection::View::query_with_docs()
  • bonsaidb::core::connection::AsyncView::query()
  • bonsaidb::core::connection::AsyncView::query_with_docs()
  • bonsaidb::core::connection::LowLevelConnection::query()
  • bonsaidb::core::connection::LowLevelConnection::query_with_docs()
  • bonsaidb::core::connection::AsyncLowLevelConnection::query()
  • bonsaidb::core::connection::AsyncLowLevelConnection::query_with_docs()
  • MappedDocuments: Both mappings and documents have had their types
    updated.
  • MappedSerialiedSocuments::deserialized()

Deprecated

• bonsaidb::core::connection::ViewMappings as been moved to
  bonsaidb::core::schema::view::ViewMappings. A deprecated re-export has been
  provided to minimize code breakage when upgrading.

Added

• [#239][239] Key can now be derived on enums and structs, allowing an easier way
  to use composite keys. The primary-keys example has been updated to use t...

v0.4.1

All Commits since last release: v0.4.0...v0.4.1

This release is a minor release with an important bug fix for the view indexing system.

Fixed

• The View indexing system had a bug when deleting the last view entries for a
  key while also inserting new entries for that key in the same mapping update
  operation. This prevented the recording of new entries being made during that
  mapping operation. This bug was introduced during the optimizations in v0.3.0.

  All views will be reindexed automatically on upgrade.

• insert_bytes/push_bytes no longer require SerializedCollection to be
  implemented.

Known Issues

• #240: Tuples used as keys do not sort correctly. The format for tuples will be
  changing in v0.5.0. Compatibility types will be provided to make it easy to
  preserve existing functionality. The linked issue describes the scenario
  that does not work correctly currently.

v0.4.0

Blog Post: https://bonsaidb.io/blog/bonsaidb-v0-4-0/
All commits since last release: v0.3.0...v0.4.0

Breaking Changes

The blog post has a section discussing updating existing projects. If you have any issues upgrading, don't hesitate to open an issue, a discussion, or reach out on Discord.

• BonsaiDb now has both an async interface as well as a blocking interface. This
  has caused significant changes, but they can be summarized simply:

  • Connection-related async-compatible traits have had the Async prefix added
    to them.

    Blocking	Async
    Connection	AsyncConnection
    StorageConnection	AsyncStorageConnection
    PubSub	AsyncPubSub
    Subscriber	AsyncSubscriber
    KeyValue	AsyncKeyValue
    LowLevelConnection	AsyncLowLevelConnection

  • Functions that take parameters of the above traits now are offered in pairs:
    a blocking function and an async function with "_async" at the end of the
    name. For example, SerializedCollection::get() is the blocking version of
    SerializedCollection::get_async().

  • For bonsaidb-local, the Database and Storage types are now blocking
    implementations. Under the hood, BonsaiDb previously used
    tokio::task::spawn_blocking() to wrap calls to the database in an async
    API. New types, AsyncDatabase and AsyncStorage have been added that
    provide the previous behavior. The types can be converted between each other
    using helpers as/into/to_blocking/async available on each type.

    These changes allow bonsaidb-local to be compiled without Tokio. To enable
    tokio, enable feature async if using bonsaidb-local directly, or enable
    feature local-async when using the bonsaidb crate.

  • For bonsaidb-server, it still uses networking driven by Tokio.
    Server/CustomServer implement AsyncStorageConnection, and Server can
    convert to Storage via the From trait for synchronous database access.

  • For bonsaidb-client, Client implements both AsyncStorageConnection and
    StorageConnection and is safe for use in both synchronous and asynchronous
    contexts. In WASM, Client only implements AsyncStorageConnection. For
    all other platforms, the Client builder supports specifying the Tokio
    runtime handle if needed. Otherwise, the current runtime will be used or a
    default runtime will be created automatically if unavailable.

• Connection::query_with_docs/Connection::query_with_connection_docs now
  verify the user has permission to DocumentAction::Get. This allows schema
  authors to allow querying views without allowing the documents themselves to
  be fetched.

• ViewAction::DeleteDocs has been removed. Delete docs is now composed of two
  permissions checks: ViewAction::Query to retrieve the list of documents to
  delete, and DocumentAction::Delete for each document retrieved. This ensures
  if permission is denied to delete a specific document, it still cannot be
  deleted through delete_docs().

• All APIs have had their limit parameters changed from usize to u32.
  Since usize is platform-dependent, picking a fixed-width type is more
  appropriate.

• CustomApi has been renamed to Api and changed significantly.

  On the Server, Apis are registered on the ServerConfiguration. The Api
  implementor is treated as the "request" type and is what Clients send to the
  Server. The Api::Response type is what the Server sends to the Client.
  Out-of-band responses can still be delivered.

  On the Client, Apis can simply be used without any extra steps. If you
  expect out-of-band responses, callbacks can be registered when building the
  client.

  Internally, all BonsaiDb APIs have been refactored to use this -- there is no
  distinction.

• The multiuser feature flag has been removed. In the end this caused a lot of
  needless conditional compilation for removing a single lightweight dependency.

• User::assume_identity and Role::assume_identity allow assuming the
  identity of a user or role by their unique ID or name. The connection must be
  permitted with the newly added ServerAction::AssumeIdentity for the
  appropriate resource name (user_resource_name or role_resource_name).

• StorageConnection::authenticate and StorageConnection::assume_identity
  both return a new instance with the new authentication. This enables
  authenticating as multiple roles with the same underlying storage connection.

  StorageConnection::session() is a new function that returns the current
  Session, if one exists. This new type contains information about any
  currently authenticated identity, the unique id of the session, and the
  current effective permissions.

  This release note applies equally to AsyncStorageConnection.

• LowLevelConnection and AsyncLowLevelConnection have been added to group
  functionality that is not generally meant for the average user to utilize. The
  methods that were documented as low-level in Connection/AsyncConnection
  have been moved to this trait. Additionally, new methods that allow performing
  operations without the generic types have been added to this trait. This
  functionality is what will be useful in providing applications that can
  interact with BonsaiDb without having the Schema definitions available.

• PubSub/AsyncPubSub now allows any Serialize implementation to be used as
  the topic parameter. New methods publish_bytes and publish_bytes_to_all
  have been added enabling publishing raw payloads.

• CollectionName/SchemaName have had common methods extracted to a trait,
  Qualified. This was part of a refactoring to share code between these two
  types and the newly introduced ApiName type.

• BackupLocation and VaultKeyStorage have been changed to blocking traits.
  bonsaidb-keystorage-s3 wraps a tokio Runtime handle as the AWS SDK requires
  Tokio.

• ServerConfiguration now takes a Backend generic parameter, which must
  match the CustomServer being created. In general the Rust compiler should be
  able to infer this type based on usage, and therefore shouldn't be a breaking
  change to many people.

• The Backend trait now has an associated Error type, which allows for
  custom error types to be used. When an error occurs during initialization, it
  is returned. Currently, errors that are returned during client connection
  handling are printed using log::error and ignored.

• Key has had its encoding functionality moved to a new trait, KeyEncoding.
  KeyEncoding has been implemented for borrowed representations of Key
  types.

  This change allows all view query and collection access to utilize borrowed
  versions of their key types. For example, if a View's Key type is String,
  it is now possible to query the view using an &str parameter.

Added

• Range::default() now returns an unbounded range, and Bound::default()
  returns Bound::Unbounded.

• Range now has several builder-pattern style methods to help construct
  ranges. In general, users should simply use the built-in range operators
  (.., start.., start..end, start..=end), as they are able to represent
  nearly every range pattern. The built-in range operators do not support
  specifying an excluded start bound, while the new method Range::after allows
  setting an excluded start bound.

• [#215][215]: StorageConnection::delete_user() is a new function that allows
  deleting a user by name or id. Deleting a user is permitted with the
  ServerAction::DeleteUser action.

• bonsaidb_core::key::encode_composite_field and
  bonsaidb_core::key::decode_composite_field have been added which allow
  building more complex Key implementations that are composed of multiple
  fields. These functions are what the Key implementation for tuples is
  powered by.

• Key is now implemented for [u8; N].

• [#221][221]: headers() has been as a function to all collection list
  builders, enabling querying just the headers of a document.

• Transaction now has apply() and apply_async(), which the higher-level
  API to LowLevelConnection::apply_transaction.

• ArgonConfiguration can now be specified when building
  StorageConfiguration/ServerConfiguration using Builder::argon.

• SystemTime and Duration now have Key implementations.

• bonsaidb_core::key::time has been added which contains a wide array of types
  that enable storing timestamps and durations with limited resolution, powered
  by variable integer encoding to reduce the number of bytes needed to encode
  the values.

  These types are powered by two traits: TimeResolution and TimeEpoch. Using
  these traits, the LimitedResolutionDuration and LimitedResolutionTimestamp
  types can be used for completely custom resolutions (e.g., 15 minutes) and
  epochs (the base moment in time to store the limited resolution relative to).

  By constraining the resolution and using an epoch that is closer to the
  average timestamp being stored, we can reduce the number of bytes required to
  represent the values from 12 bytes to significantly fewer.

  These type aliases have been added in these three categories:

  • Durations: Weeks, Days, Hours, Minutes, Seconds, Milliseconds,
    Microseconds, and Nanoseconds.
  • Timestamps relative to the Unix Epoch (Jan 1, 1970 00:00:00 UTC):
    WeeksSinceUnixEpoch, DaysSinceUnixEpoch, ...
  • Timestamps relative to th...

v0.3.0

Blog Post: https://bonsaidb.io/blog/bonsaidb-v0-3-0

This release welcomes @vbmade2000 as a contributor to BonsaiDb. Thank you!

Breaking Changes

• bonsaidb::local::jobs is now private. It used to be a separate, public crate
  in the PliantDb days. After thinking about the job scheduler more, this
  initial implementation is better suited for the internal task management than
  the higher-level jobs system. As such, it has been internalized.

• bonsaidb::core::transaction::Changes::Documents has been changed to store
  the CollectionNames separately from the ChangedDocuments. This makes the
  transaction log entries smaller, as collection names aren't copied for each
  document.

  The storage layer is fully backwards compatible and will automatically convert
  existing transactions to the new format.

Fixed

• Listing executed transactions that were written in v0.1 was broken in
  v0.2. Backwards compatibility is now automatically tested to help ensure
  this sort of issue won't happen in the future again.

Added

• SerializedCollection::list_with_prefix,
  connection::Collection::list_with_prefix, and
  connection::View::with_key_prefix have been added as an easy way to filter
  results based on whether the key starts with the given prefix.

  This is supported by a new trait, IntoPrefixRange. This trait has been
  implemented for all byte-based key implementations as well as for String.

• Operation::push_serialized has been added, which calls natural_id before
  creating an Operation::Insert variant.

• Tasks::parallelization and Builder::workers_parallelization have been
  added as a way to control how many threads can be used by any given
  task/worker. This is automatically configured to be the number of cpu cores
  detected.

• count() is a new function on the list builders, available via:

  • SerializedCollection::all(db).count().await
  • SerializedCollection::list(42.., db).count().await
  • db.collection::<Collection>().all().count().await
  • db.collection::<Collection>().list(42..).count().await

  The performance of this call is not as good as it will eventually be, as it is
  currently doing more work than strictly necessary.

• #215: StorageConnection::delete_user has been added, enabling
  deletions of users without directly interacting with the admin database.

Changed

• The view map/reduce system has been optimized to take advantage of some
  parallelism. The view system is still not hightly optimized, but this change
  makes a significant improvement on performance.

v0.2.0

Blog post: https://bonsaidb.io/blog/bonsaidb-v0-2-0

Breaking Changes

• bonsaidb::core::Error::DocumentConflict now contains a Header instead of
  just the document's ID. This allows an application to re-submit an update with
  the updated header without another request to the database.

• StorageConfiguratation::vault_key_storage now uses an Arc instead of a
  Box. This change allows StorageConfiguration and ServerConfiguration to
  implement Clone.

• Document::create_new_revision has been removed. It was meant to be an
  internal function.

• Document now requires AsRef<Header> and AsMut<Header> instead of
  Deref<Header>/DerefMut. The publicly visible change is that the shortcut
  of accessing document.header.emit_* through deref by using document.emit_*
  will no longer work. This impacts CollectionDocument, OwnedDocument, and
  BorrowedDocument.

  This removes a little magic, but in some code flows, it was impossible to use
  Deref anyways due to Deref borrowing the entire document, not just the header.

• Collection::PrimaryKey is a new associated type that allows customizing the
  type that uniquely identifies documents inside of a Collection. Users of the
  derive macro will be unaffected by this change. If you're upgrading existing
  collections and wish to maintain backwards compatibility, use u64 as the
  type.

  A natural_id() function can now be implemented on SerializedCollection or
  DefaultSerialization which allows extracting a primary key value from a new
  document being pushed

  A new example, primary-keys.rs, as been added showing basic usage of
  changing the primary key type. This change resulted in a sequnce of breaking
  changes that will be listed independently.

• Key has been moved from bonsaidb::core::schema::view to
  bonsaidb::core::key.

• Key::as/from_big_endian_bytes have been renamed to Key::as/from_ord_bytes.

• Key::first_value() and Key::next_value() are new provided functions. By
  default, these functions return NextValueError::Unsupported.

  Key::first_value() allows a Key type to define the first value in its
  sequence. For example, 0_u64 is the result of u64::first_value().

  Key::next_value() allows a Key type to find the next value in sequence
  from the current value. Implementors should never wrap, and should instead
  return NextValueError::WouldWrap.

  Sensible defaults have been implemented for all numeric types.

• Connection and its related types have had all previously hard-coded
  primary keys pf u64 changed to generic parameters that can accept either a
  DocumentId or Collection::PrimaryKey. The affected methods are:

  • Connection::insert
  • Connection::overwrite
  • Connection::get
  • Connection::get_multiple
  • Connection::list
  • connection::Collection::insert
  • connection::Collection::overwrite
  • connection::Collection::get
  • connection::Collection::get_multiple
  • connection::Collection::list
  • SerializedCollection::insert
  • SerializedCollection::insert_into
  • SerializedCollection::overwrite
  • SerializedCollection::overwrite_into
  • SerializedCollection::get
  • SerializedCollection::get_multiple
  • SerializedCollection::list
  • transaction::Operation::insert_serialized
  • transaction::Operation::overwrite_serialized

• Header::id has changed from u64 to DocumentId, and
  CollectionHeader<PrimaryKey> has been added which contains
  Collection::PrimaryKey deserialized.

  These previous usages of Header have been changed to CollectionHeader:

  • Connection::insert result type
  • Connection::overwrite result type
  • connection::Collection::insert result type
  • connection::Collection::insert_bytes result type
  • connection::Collection::push result type
  • connection::Collection::push_bytes result type
  • CollectionDocument::header

  The Header::emit* functions have been moved to a new trait, Emit. This
  trait is implemented by both Header and CollectionHeader. The functions moved are:

  • emit()
  • emit_key()
  • emit_value()
  • emit_key_and_value()

  These functions now return a Result, as encoding a primary key value can
  fail if it is larger than DocumentId::MAX_LENGTH.

• HasHeader is a new trait that allows accessing a Header generically from
  many types. This type is used in Connection::delete and
  connection::Collection::delete.

• Types and functions that used u64 as a document ID have been replaced with
  DocumentIds. The number of locations are too many to list. If you need to
  convert from a u64 to a DocumentId, you can use DocumentId::from_u64().

• Document::contents and Document::set_contents are now ore "painful" to
  access due to the generic parameter added to Document.
  SerializedCollection::document_contents(doc) and
  SerializedCollection::set_document_contents(doc, new_contents) have been
  provided as easier ways to invoke the same functionality. For example:

  let contents = doc.contents::<MyCollection>()?;

  Becomes:

  let contents = MyCollection::document_contents(&doc)?;

• Backups made prior to 0.2.0 will not be able to be restored with this
  updated version. The document IDs are encoded differently than in prior
  versions.

Added

• Optional compression is now available, using the LZ4 algorithm.
  StorageConfiguration::default_compression controls the setting. When using
  the bonsaidb crate, the feature can be made available using either
  local-compression or server-compression. When using bonsaidb-server or
  bonsaidb-local directly, the feature name is compression.

  This compression is currently applied on all chunks of data written to
  BonsaiDb that are larger than a hardcoded threshold. This includes the
  Key-Value store. The threshold may be configurable in the future.

  Some of the benchmark suite has been expanded to include comparisons between
  local storage with and without compression.

• Added ability to "overwrite" documents without checking the stored revision
  information. Because this removes a layer of safety, it has its own permissible
  action: DocumentAction::Overwrite. The functions that have been added are:

  • connection::Connection::overwrite
  • connection::Collection::overwrite
  • schema::SerializedCollection::overwrite
  • schema::SerializedCollection::overwrite_into
  • document::CollectionDocument::overwrite
  • transaction::Transaction::overwrite
  • transaction::Operation::overwrite

Changed

• Internal dependencies between crates are now pinned based on their needs. This
  means that bonsaidb-sever will require a matching verison of
  bonsaidb-local when compiling. A simple example of a change that is a
  breaking compilation change but is not breaking from a compatibility
  standpoint is a change to a structure where #[serde(rename)] is used to
  remap an old value to a new value.

  The only crate currently not pinning its dependencies is
  bonsaidb-keystorage-s3. This crate, and hopefully many crates to come, are
  only tying themselves to the public API of BonsaiDb.

  This may generate slightly more crate updates than absolutely necessary, but
  for a small team it seems like the most manageable approach.

Fixed

• The view system now tracks an internal version number in addition to the
  version specified in the view definiton. This allows internal structures to be
  upgraded transparently.
• Applying a transaction to a collection with a unique view now ensures a view
  mapping job has finished if the view's integrity check spawns one before
  allowing the transaction to begin.

v0.1.2

v0.1.2

Fixes

• #196: Fixed a panic a user reported relating to an undocumented panic
  within tokio::time::sleep().

v0.1.1: Derive Macros + Documentation Improvements

Thanks goes to @ModProg for his contributions on the new derive macros!

This release contains no breaking changes.

Added

• SchemaName:private and CollectionName::private are two new constructors
  that allow defining names without specifying an authority. Developers
  creating reusable collections and/or schemas should not use these methods as
  namespacing is meant to help prevent name collisions.

• connection::Collection::all() and SchemaCollection::all() have been
  implemented as simple wrappers around list(..).

• #146, #187: The Schema, Collection, and View traits can
  now be derived rather than manually implemented:

  #[derive(Debug, Schema)]
  #[schema(name = "my-schema", collections = [Shape])]
  struct MySchema;
  
  #[derive(Debug, Serialize, Deserialize, Collection)]
  #[collection(name = "shapes", views = [ShapesByNumberOfSides])]
  struct Shape {
      pub sides: u32,
  }
  
  #[derive(Debug, Clone, View)]
  #[view(collection = Shape, key = u32, value = usize, name = "by-number-of-sides")]
  struct ShapesByNumberOfSides;

• #169: Memory-only instances of Storage can be created now. This is primarily intended for testing purposes.

Changed

• Inline examples have been added for every connection::Collection and
  connection::View function.
• All examples have been updated to the new derive macro syntax. Additionally,
  documentation and examples for deriving Schema, Collection, and View have been
  added to the respective traits.

v0.1.0 - The first alpha release of BonsaiDb

v0.1.0

Announcement Blog Post: https://bonsaidb.io/blog/announcing-bonsaidb-alpha
Closed Issues: https://github.com/khonsulabs/bonsaidb/milestone/2?closed=1
All Commits: v0.1.0-dev.4...v0.1.0

Added

• bonsaidb::local::admin now exposes collections that are used to manage BonsaiDb.
• Ability to add users, set a user's password, and log in as a user.
• Each bonsaidb::local::Storage now has a unique ID. It will be randomly
  generated upon launch. If for some reason a random value isn't desired, it can
  be overridden in the Configuration.
• Centralized secrets vault: Enables limited at-rest encryption. See
  bonsaidb::core::vault for more information.
• For serializable types, Collection now defines easier methods for dealing
  with documents. NamedCollection allows collections that expose a unique name
  view to have easy ways to convert between IDs and names.
• Server Backend trait now defines connection lifecycle functions that can be
  overridden to customize behavior when clients connect, disconnect, or
  authenticate.
• Client now has a build() method to allow for customizing connections.
• CustomApi responses can now be sent by the server via
  ConnectedClient::send(). The client can now register a callback to receive
  out-of-band responses.
• Backend has a new associated type, CustomApiDispatcher which replaces
  Server::set_custom_api_dispatcher. This change allows custom api dispatchers
  to receive the server or client during initialization if needed. For
  example, if a custom API needs to know the caller's identity, you can store
  the client in your dispatcher and access it in your handlers.
• Backend has a new associated type: ClientData. This associated type can be
  used to associate data on a per-ConnectedClient basis.
• ServerConfiguration has a new setting:
  client_simultaneous_request_limit. It controls the amount of query
  pipelining a single connection can achieve. Submitting more queries on a
  single connection will block reading additional requests from the network
  connection until responses have been sent.
• ServerConfiguration now supports authenticated_permissions,
  allowing a set of permissions to be defined that are applied to any user that
  has successfully authenticated.
• CollectionView is a new trait that can be implemented instead of View. The
  map() function takes a CollectionDocument parameter that is already
  deserialized for you.
• bonsaidb_core now has two new macros to ease some tedium of writing simple
  views: define_basic_mapped_view and define_basic_unique_mapped_view.
• BonsaiDb now uses log internally to report
  errors that are being ignored or happened in a background task.
• Multiple crates now offer an "instrument" feature which enables
  instrumentation using the tracing ecosystem.
• Moved all database() functions to StorageConnection. This allows fully
  generic code to be written against a "server".
• Added listen_for_shutdown() which listens for SIGINT and SIGQUIT and attemps
  to shut the server down gracefully.
• Automatic TLS certificate acquisition can be performed using ACME on TCP port
  443. This is automatically performed if the feature flag is enabled.
• BonsaiDb server can now listen for generic TCP connections with and without
  TLS. This is primarily designed to allow extending the HTTP port to support
  additional HTTP endpoints than just websockets. However, because the TLS
  certificate aquired on port 443 can be used in other protocols such as SMTP,
  it can be useful to allow BonsaiDb to control the networking connections.
  listen_for_tcp_on and listen_for_secure_tcp_on accept a parameter that
  implements the TcpService trait. See the Axum example for how this
  integration can be used in conjunction with websockets.
• Added convenience methods to Transaction and Operation to make it easier
  to build multi-operation transactions.
• The Key trait is now automatically implemented for tuples of up to 8 entries
  in length.
• CollectionDocument::modify() enables updating a document using a flow that
  will automatically fetch the document if a conflict is detected, re-invoking
  the callback to redo the modifications. This is useful for situations where
  you may have cached a document locally and wish to update it in the future,
  but don't want to refresh it before saving. It also will help, in general,
  with saving documents that might have some contention.
• CustomServer::authenticate_client_as() allows setting a ConnectedClient's
  authenticated user, skipping BonsaiDb's authentication. This allows developers
  to still utilize the users and permissions within BonsaiDb while
  authenticating via some other mechanism.
• SerializedCollection::list() and SerializedCollection::get_multiple() have
  been added to make it easier to retrieve CollectionDocument<T>s.

Changed

• Configuration has been refactored to use a builder-style pattern.
  bonsaidb::local::config::Configuration has been renamed
  StorageConfiguration, and bonsaidb::server::ServerConfiguration has been
  renamed ServerConfiguration.

• Database::open_local and Storage::open_local have been renamed to open.

• Database::open, Storage::open, and Server::open no longer take a path
  argument. The path is provided from the configuration.

• Listing all schemas and databases will now include the built-in admin database.

• The underlying dependency on sled has been changed for an in-house storage
  implementation nebari.

• The command-line interface has received an overhaul.

  • A new trait, CommandLine can be implemented on a type that implements
    Backend to utilize the built-in, extensible command line interface. An
    example of this is located at
    ./examples/basic-server/examples/cli.rs.
  • The parameter types to execute() functions have changed.
  • This interface will receive further refinement as part of switching to clap
    3.0 once it is fully released.

• View::map now returns a Mappings instead of an Option, allowing for
  emitting of multiple keys.

• View mapping now stores the source document header, not just the ID.

• ServerConfiguration::default_permissions has been changed into a
  DefaultPermissions enum.

• Changed the default serialization format from CBOR to an in-house format,
  Pot.

• Key now has a new associated constant: LENGTH. If a value is provided, the
  result of converting the value should always produce the length specified.
  This new information is used to automatically implement the Key trait for
  tuples.

• The Key implementation for EnumKey has been updated to use
  ordered-varint to minimize the size of the indexes. Previously, each entry
  in the view was always 8 bytes.

• Connection and SerializedCollection have had their insert/insert_into
  functions modified to include an id parameter. New functions named push and
  push_into have been added that do not accept an id parameter. This is in an
  effort to keep naming consistent with common collection names in Rust.

• Operation::insert and Command::Insert now take an optional u64 parameter
  which can be used to insert a document with a specific ID rather than having
  one chosen. If an document exists already, a conflict error will be returned.

• bonsaidb::local::backup has been replaced with Storage::backup and
  Storage::restore. This backup format is incompatible with the old format,
  but is built with proper support for restoring at-rest encrypted collections.
  Backups are not encrypted, but the old implementation could not be updated
  to support restoring the documents into an encrypted state.

  This new functionality exposes BackupLocation, an async_trait that enables
  arbitrary backup locations.

• KeyValue storage has internally changed its format. Because this was
  pre-alpha, this data loss was premitted. If this is an issue for anyone, the
  data is still there, the format of the key has been changed. By editing any
  database files directly using Nebari, you can change the format from
  "namespace.key" to "namespace\0key", where \0 is a single null byte.

• ExecutedTransactions::changes is now a Changes enum, which can be a list
  of ChangedDocuments or ChangedKeys.

• The Key-Value store is now semi-transactional and more optimized. The behavior
  of persistence can be customized using the key_value_persistence
  option
  when opening a BonsaiDb instance. This can enable higher performace at the
  risk of data loss in the event of an unexpected hardware or power failure.

• A new trait, SerializedCollection, now controls serialization within
  CollectionDocument, CollectionView, and other helper methods that
  serialized document contents. This allows any serialization format that
  implements transmog::Format can be used within BonsaiDb by setting the
  Format associated type within your SerializedCollection implementation.

  For users who just want the default serialization, a convenience trait
  DefaultSerialization has been added. All types that implement this trait
  will automatically implement SerializedCollection using BonsaiDb's preferred
  settings.

• A new trait, SerializedView, now controls serialization of view values. This
  uses a similar approach to SerializedCollection. For users who just want the
  default serialization, a convenience trait `DefaultViewSe...

v0.1.0-dev.4: Custom Apis, Unique Views, WebAssembly

Welcome to the first release on GitHub. I wanted to take this moment to talk a little bit about the development of this project.

I've added a CHANGELOG.md. I may not do much to highlight breaking changes this early on in development, but once we have a 0.1.0 release, I'll start tracking breaking changes. At this point, PliantDb is purely experimental, and while I applaud anyone attempting to use it, using it this early will likely break as I continue to develop the API.

For the near future, I want the flexibility to change drastic things without worrying about how it impacts existing user's code. Until we have a successful decent-sized project using PliantDb, I think it's impossible to be confident everything is designed well.

Lastly, I wanted to mention that this year seems to just fly by. I hadn't realized it was April 26 when the last release happened. I have a goal to get a new version of Cosmic Verge running on PliantDb, but we are ambitious. In the process, we are redoing our presentation layer. As such, it may have looked like I lost interest in this project and moved on.

Fear not, this project is near and dear to my heart, which is why the first product of the Gooey project is this example which shows how to build a simple Gooey app and PliantDb server with a basic custom API. Because of the new support for WebAssembly, the example even works in a modern browser.

If you have any questions or feedback, feel free to join the discussion on our Discourse forums.

From the CHANGELOG

Added

• View::unique() has been added, allowing for a View to restrict saving documents when multiple documents would end up with the same key emitted for this view. For example, if you have a User collection and want to ensure each User has a unique email_address, you could create a View with the key of email_address and return true from unique(), and PliantDb will enforce that constraint.

• Permissions support has been added across the platform with granular access. The pliantdb::core::permissions module contains the data types involved. More documentation and examples are to-come -- users and roles haven't been added yet.

• The initial underpinnings of customizing the PliantDb server have been added. First, there's the Backend trait. Right now, its only purpose is to allow defining a CustomApi. This allows applications built with PliantDb to extend the network protocol with Request and Response types that just need to support serde. For a full example, check out this in-development Gooey example.

• An initial version of a WebAssembly client is now supported. It only supports WebSockets. While there has been news of QUIC support in the browser, it's a limited implementation that only exposes an HTTP protocol. As such, it is incompatible with the PliantDb protocol. Eventually, we hope to support WebRTC as an alternative to TCP in WebAssembly. The example linked in the previous bullet point can be built and loaded in a browser.