Skip to content

Commit

Permalink
Add delete/1 function, to delete an entire bucket.
Browse files Browse the repository at this point in the history
Upgrade shards dep to the latest release 0.3.1
Fix tests
  • Loading branch information
cabol committed Sep 9, 2016
1 parent f624be7 commit ed52d33
Show file tree
Hide file tree
Showing 8 changed files with 363 additions and 49 deletions.
196 changes: 193 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
# KVX

This is a simple/basic [Elixir](http://elixir-lang.org/) in-memory Key/Value Store
using [Shards](https://github.com/cabol/shards) – which is the default adapter.
This is a simple/basic in-memory Key/Value Store written in [**Elixir**](http://elixir-lang.org/)
and using [**Shards**](https://github.com/cabol/shards) as default adapter.

Again, **KVX** is a simple library, most of the work is done by **Shards**, and
its typical use case might be as a **Cache**.

## Usage

Add `kvx` to your Mix dependencies:

```elixir
defp deps do
[{:kvx, "~> 0.1.0"}]
[{:kvx, "~> 0.1.1"}]
end
```

Expand All @@ -21,6 +24,16 @@ defmodule MyTestMod do
end
```

## Getting Started!

Let's try it out, compile your project and start an interactive console:

```
$ mix deps.get
$ mix compile
$ iex -S mix
```

Now let's play with `kvx`:

```elixir
Expand Down Expand Up @@ -76,6 +89,23 @@ config :kvx,
shards_mod: :shards
```

Besides, you can define bucket options in the config:

```elixir
config :kvx,
adapter: KVX.Bucket.Shards,
ttl: 43200,
shards_mod: :shards,
buckets: [
mybucket1: [
n_shards: 4
],
mybucket2: [
n_shards: 8
]
]
```

In case of Shards adapter, run-time options when calling `new/2` function, are
the same as `shards:new/2`. E.g.:

Expand All @@ -84,3 +114,163 @@ MyModule.new(:mybucket, [n_shards: 4])
```

> **NOTE:** For more information check [KVX.Bucket.Shards](./lib/kvx/adapters/shards/bucket_shards.ex).
## Running Tests

```
$ mix test
```

### Coverage

```
$ mix coveralls
```

> **NOTE:** For more coverage options check [**excoveralls**](https://github.com/parroty/excoveralls).
## Example

As we mentioned before, one of the most typical use case might be
use **KVX** as a **Cache**. Now, let's suppose you're working with
[**Ecto**](https://github.com/elixir-ecto/ecto), and you want to be
able to cache data when you call `Ecto.Repo.get/3`, and on other hand,
be able to handle eviction, remove/update cached data when they
change or mutate – typically when you call `Ecto.Repo.insert/2`,
`Ecto.Repo.update/2`, etc.

To do so, let's implement our own `CacheableRepo` to encapsulate
data access and caching logic. First let's create our bucket and
the `Ecto.Repo` in two separated modules:

```elixir
defmodule MyApp.Cache do
use KVX.Bucket
end

defmodule MyApp.Repo do
use Ecto.Repo, otp_app: :myapp
end
```

Now, let's code our `CacheableRepo`, re-implementing some `Ecto.Repo`
functions but adding caching. It is as simple as this:

```elixir
defmodule MyApp.CacheableRepo do
alias MyApp.Repo
alias MyApp.Cache

require Logger

def get(queryable, id, opts \\ []) do
get(&Repo.get/3, queryable, id, opts)
end

def get!(queryable, id, opts \\ []) do
get(&Repo.get!/3, queryable, id, opts)
end

def get_by(queryable, clauses, opts \\ []) do
get(&Repo.get_by/3, queryable, clauses, opts)
end

def get_by!(queryable, clauses, opts \\ []) do
get(&Repo.get_by!/3, queryable, clauses, opts)
end

defp get(fun, queryable, key, opts) do
b = bucket(queryable)
case Cache.get(b, key) do
nil ->
value = fun.(queryable, key, opts)
if value != nil do
Logger.debug "CACHING <get>: #{inspect key} => #{inspect value}"
Cache.set(b, key, value)
end
value
value ->
Logger.debug "CACHED <get>: #{inspect key} => #{inspect value}"
value
end
end

def insert(struct, opts \\ []) do
case Repo.insert(struct, opts) do
{:ok, schema} = rs ->
schema
|> bucket
|> Cache.del(schema.id)
rs
error ->
error
end
end

def insert!(struct, opts \\ []) do
rs = Repo.insert!(struct, opts)
rs
|> bucket
|> Cache.del(rs.id)
rs
end

def update(struct, opts \\ []) do
case Repo.update(struct, opts) do
{:ok, schema} = rs ->
schema
|> bucket
|> Cache.set(schema.id, schema)
rs
error ->
error
end
end

def update!(struct, opts \\ []) do
rs = Repo.update!(struct, opts)
rs
|> bucket
|> Cache.set(rs.id, rs)
rs
end

def delete(struct, opts \\ []) do
case Repo.delete(struct, opts) do
{:ok, schema} = rs ->
schema
|> bucket
|> Cache.del(schema.id)
rs
error ->
error
end
end

def delete!(struct, opts \\ []) do
rs = Repo.delete!(struct, opts)
rs
|> bucket
|> Cache.del(rs.id)
rs
end

# function to resolve what bucket depending on the given schema
def bucket(MyApp.ModelA), do: :b1
def bucket(%MyApp.ModelA{}), do: :b1
def bucket(MyApp.ModelB), do: :b2
def bucket(%MyApp.ModelB{}), do: :b2
def bucket(_), do: :default
end
```

Now that we have our `CacheableRepo`, it can be used instead of `Ecto.Repo`
(since it is a wrapper on top of it, but it adds caching) for data you
consider can be cached, for example, you can use it from your
**Phoenix Controllers** – in case you're using [Phoenix](http://www.phoenixframework.org/).

## Copyright and License

Copyright (c) 2016 Carlos Andres Bolaños R.A.

**KVX** source code is licensed under the [**MIT License**](LICENSE.md).
7 changes: 6 additions & 1 deletion config/test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,9 @@ use Mix.Config
# KVX config
config :kvx,
adapter: KVX.Bucket.Shards,
ttl: 1
ttl: 1,
buckets: [
mybucket: [
n_shards: 2
]
]
42 changes: 42 additions & 0 deletions lib/kvx.ex
Original file line number Diff line number Diff line change
@@ -1,2 +1,44 @@
defmodule KVX do
@moduledoc """
This is a simple/basic in-memory Key/Value Store written in
[**Elixir**](http://elixir-lang.org/) and using
[**Shards**](https://github.com/cabol/shards)
as default adapter.
Again, **KVX** is a simple library, most of the work
is done by **Shards**, and its typical use case might
be as a **Cache**.
## Adapters
**KVX** was designed to be flexible and support multiple
backends. We currently ship with one backend:
* `KVX.Bucket.Shards` - uses [Shards](https://github.com/cabol/shards),
to implement the `KVX.Bucket` interface.
**KVX** adapters config might looks like:
config :kvx,
adapter: KVX.Bucket.Shards,
ttl: 43200,
shards_mod: :shards,
buckets: [
mybucket1: [
n_shards: 4
],
mybucket2: [
n_shards: 8
]
]
In case of Shards adapter, run-time options when calling `new/2`
function, are the same as `shards:new/2`. E.g.:
MyModule.new(:mybucket, [n_shards: 4])
## Example
Check the example [**HERE**](https://github.com/cabol/kvx#example).
"""
end
41 changes: 37 additions & 4 deletions lib/kvx/adapters/shards/bucket_shards.ex
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,29 @@ defmodule KVX.Bucket.Shards do
* `:shards_mod` - internal Shards module to use. By default, `:shards`
module is used, which is a wrapper on top of `:shards_local` and
`:shards_dist`.
* `:buckets` - this can be used to set bucket options in config,
so it can be loaded when the bucket is created. See example below.
Run-time options when calling `new/2` function, are the same as
`shards:new/2`. For example:
MyModule.new(:mybucket, [n_shards: 4])
## Example:
config :kvx,
adapter: KVX.Bucket.Shards,
ttl: 43200,
shards_mod: :shards,
buckets: [
mybucket1: [
n_shards: 4
],
mybucket2: [
n_shards: 8
]
]
For more information about `shards`:
* [GitHub](https://github.com/cabol/shards)
Expand All @@ -29,18 +46,25 @@ defmodule KVX.Bucket.Shards do

## Setup Commands

def new(bucket, opts \\ []) do
def new(bucket, opts \\ []) when is_atom(bucket) do
case Process.whereis(bucket) do
nil -> new_bucket(bucket, opts)
_ -> bucket
end
end

defp new_bucket(bucket, opts) do
{^bucket, _} = @shards.new(bucket, opts)
bucket
opts = maybe_get_bucket_opts(bucket, opts)
@shards.new(bucket, opts)
end

defp maybe_get_bucket_opts(bucket, []) do
:kvx
|> Application.get_env(:buckets, [])
|> Keyword.get(bucket, [])
end
defp maybe_get_bucket_opts(_, opts), do: opts

## Storage Commands

def add(bucket, key, value, ttl \\ @default_ttl) do
Expand Down Expand Up @@ -116,11 +140,20 @@ defmodule KVX.Bucket.Shards do
bucket
end

def flush!(bucket) do
def delete(bucket) do
true = @shards.delete(bucket)
bucket
end

def flush(bucket) do
true = @shards.delete_all_objects(bucket)
bucket
end

## Extended functions

def __shards_mod__, do: @shards

## Private functions

defp seconds_since_epoch(diff) do
Expand Down
Loading

0 comments on commit ed52d33

Please sign in to comment.