This is a fork of the awesome idb
library, which adds the ability to sync an IndexedDB database with a remote REST API.
The source code for the example above can be found here.
Bundle size: ~3.11 kB brotli'd
Table of content
Since it is a fork of the idb
library, synceddb
shares the same Promise-based API:
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const transaction = db.transaction('items', 'readwrite');
await transaction.store.add({ id: 1, label: 'Dagger' });
// short version
await db.add('items', { id: 1, label: 'Dagger' });
More information here.
Every change is tracked in a store. The SyncManager then sync these changes with the remote REST API when the connection is available, making it easier to build offline-first applications.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.start();
// will result in the following HTTP request: POST /items
await db.add('items', { id: 1, label: 'Dagger' });
// will result in the following HTTP request: DELETE /items/2
await db.delete('items', 2);
See also: Expectations for the REST API
The LiveQuery provides a way to run a query every time the underlying stores are updated:
import { openDB, LiveQuery } from 'synceddb';
const db = await openDB('my awesome database');
let result;
const query = new LiveQuery(['items'], async () => {
// result will be updated every time the 'items' store is modified
result = await db.getAll('items');
});
// trigger the liveQuery
await db.put('items', { id: 2, label: 'Long sword' });
// or manually run it
await query.run();
Inspired from Dexie.js liveQuery.
Entities without keyPath
are not currently supported.
- no version history
Only the last version of each entity is kept on the client side.
- basic conflict management
The last write wins (though you can customize the behavior in the onpusherror
handler).
npm install synceddb
Then:
import { openDB, SyncManager, LiveQuery } from 'synceddb';
async function doDatabaseStuff() {
const db = await openDB('my awesome database');
// sync your database with a remote server
const manager = new SyncManager(db, 'https://example.com');
manager.start();
// create an auto-reloading query
let result;
const query = new LiveQuery(['items'], async () => {
// result will be updated every time the 'items' store is modified
result = await db.getAll('items');
});
}
For database-related operations, please see the idb
documentation.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.start();
Additional options for all HTTP requests.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com', {
fetchOptions: {
headers: {
'accept': 'application/json'
},
credentials: 'include'
}
});
manager.start();
Reference: https://developer.mozilla.org/en-US/docs/Web/API/fetch
The number of ms between two fetch requests for a given store.
Default value: 30000
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com', {
fetchInterval: 10000
});
manager.start();
A function that allows to override the request path for a given request.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com', {
buildPath: (operation, storeName, key) => {
if (storeName === 'my-local-store') {
if (key) {
return `/the-remote-store/${key[1]}`;
} else {
return '/the-remote-store/';
}
}
// defaults to `/${storeName}/${key}`
}
});
manager.start();
A function that allows to override the query params of the fetch requests.
Defaults to ?sort=updated_at:asc&size=100&after=2000-01-01T00:00:00.000Z,123
.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com', {
buildFetchParams: (storeName, offset) => {
const searchParams = new URLSearchParams({
sort: '+updatedAt',
size: '10',
});
if (offset) {
searchParams.append('after', `${offset.updatedAt}+${offset.id}`);
}
return searchParams;
}
});
manager.start();
The name of the attribute that indicates the last updated date of the entity.
Default value: updatedAt
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com', {
updatedAtAttribute: 'lastUpdateDate'
});
manager.start();
Starts the sync process with the remote server.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.start();
Stops the sync process.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.stop();
Clears the local stores.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.clear();
Returns whether a given entity currently has local changes that are not synced yet.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
await db.put('items', { id: 1 });
const hasLocalChanges = await manager.hasLocalChanges('items', 1); // true
Called after some entities are successfully fetched from the remote server.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.onfetchsuccess = (storeName, entities, hasMore) => {
// ...
}
Called when something goes wrong when fetching the changes from the remote server.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.onfetcherror = (err) => {
// ...
}
Called after a change is successfully pushed to the remote server.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.onpushsuccess = ({ operation, storeName, key, value }) => {
// ...
}
Called when something goes wrong when pushing a change to the remote server.
import { openDB, SyncManager } from 'synceddb';
const db = await openDB('my-awesome-database');
const manager = new SyncManager(db, 'https://example.com');
manager.onpusherror = (change, response, retryAfter, discardLocalChange, overrideRemoteChange) => {
// this is the default implementation
switch (response.status) {
case 403:
case 404:
return discardLocalChange();
case 409:
// last write wins by default
response.json().then((content) => {
const version = content[VERSION_ATTRIBUTE];
change.value[VERSION_ATTRIBUTE] = version + 1;
overrideRemoteChange(change.value);
});
break;
default:
return retryAfter(DEFAULT_RETRY_DELAY);
}
}
The first argument is an array of stores. Every time one of these stores is updated, the function provided in the 2nd argument will be called.
import { openDB, LiveQuery } from 'synceddb';
const db = await openDB('my awesome database');
let result;
const query = new LiveQuery(['items'], async () => {
// result will be updated every time the 'items' store is modified
result = await db.getAll('items');
});
import { openDB, LiveQuery } from 'synceddb';
import { useEffect, useState } from 'react';
export default function MyComponent() {
const [items, setItems] = useState([]);
useEffect(() => {
let query;
openDB('test', 1, {
upgrade(db) {
db.createObjectStore('items', { keyPath: 'id' });
},
}).then(db => {
query = new LiveQuery(['items'], async () => {
setItems(await db.getAll('items'));
});
query.run();
});
return () => {
// !!! IMPORTANT !!! This ensures the query stops listening to the database updates and does not leak memory.
query?.close();
}
}, []);
return (
<div>
<!-- ... -->
</div>
);
}
<script setup>
import { openDB, LiveQuery } from 'synceddb';
import { ref, onBeforeUnmount } from 'vue';
const items = ref([]);
const db = await openDB('test', 1, {
upgrade(db) {
db.createObjectStore('items', { keyPath: 'id' });
},
})
const query = new LiveQuery(['items'], async () => {
items.value = await db.getAll('items');
});
await query.run();
onBeforeUnmount(() => {
// !!! IMPORTANT !!! This ensures the query stops listening to the database updates and does not leak memory.
query.close();
});
</script>
Changes are fetched from the REST API with GET
requests:
GET /<storeName>?sort=updated_at:asc&size=100&after=2000-01-01T00:00:00.000Z,123
Explanations:
sort=updated_at:asc
indicates that we want to sort the entities based on the date of last updatesize=100
indicates that we want 100 entities maxafter=2000-01-01T00:00:00.000Z,123
indicates the offset (with an update date above2000-01-01T00:00:00.000Z
, excluding the entity123
)
The query parameters can be customized with the buildFetchParams
option.
Expected response:
{
data: [
{
id: 1,
version: 1,
updatedAt: '2000-01-01T00:00:00.000Z',
label: 'Dagger'
},
{
id: 2,
version: 12,
updatedAt: '2000-01-02T00:00:00.000Z',
label: 'Long sword'
},
{
id: 3,
version: -1, // tombstone
updatedAt: '2000-01-03T00:00:00.000Z',
}
],
hasMore: true
}
A fetch request will be sent for each store of the database, every X seconds (see the fetchInterval option).
Each successful readwrite transaction will be translated into an HTTP request, when the connection is available:
Operation | HTTP request | Body |
---|---|---|
db.add('items', { id: 1, label: 'Dagger' }) |
POST /items |
{ id: 1, version: 1, label: 'Dagger' } |
db.put('items', { id: 2, version: 2, label: 'Long sword' }) |
PUT /items/2 |
{ id: 2, version: 3, label: 'Long sword' } |
db.delete('items', 3) |
DELETE /items/3 |
|
db.clear('items') |
one DELETE request per item |
Success must be indicated by an HTTP 2xx response. Any other response status means the change was not properly synced. You can customize the error handling behavior with the onpusherror
method.
Please see the Express server there for reference.
Here are some alternatives that you might find interesting:
- idb: https://github.com/jakearchibald/idb
- Dexie.js: https://dexie.org/ (and its ISyncProtocol part)
- pouchdb: https://pouchdb.com/
- Automerge: https://github.com/automerge/automerge
- Yjs: https://github.com/yjs/yjs