Joplin Synchronization API documentation.
git clone https://github.com/khuongduy354/joplin-sync-lib.git
npm install
// Options to run:
// 1. run in transpile mode (without build)
npm run dev
// 2. build and run
npm run build
npm run start
// src/index.ts is main driver code,
// sync target is in src/sample_app/Storage/fsSyncTarget, check here for any changes
see DEV.md for more explains
Before using Synchronizer methods, some setup steps is needed, make sure you run npm install
at the root of this repo.
There're some terms we'll be using: Synchronizer, Sync target, File API, drivers,... Checks the explanation here: https://joplinapp.org/help/dev/spec/sync/#vocabulary
Synchronizer is the main class that we'll focus on, as it contains all methods for syncing with the remote. To acquire a synchronizer instance, some setup steps are needed:
- Support Joplin's items
To provide supports for Joplin's items, especially serialization, some of the classes methods need to be overriden, 3 types of supported items are Note, Resource, and Folder, to setup run loadClasses() once before any of synchronizer code.
import { loadClasses } from "./helpers/item";
loadClasses() // place this at the topmost
- Pick a sync target
Currently it only supports FileSystem and Memory. In the future JoplinServer, WebDav, OneDrive,... may be added. It takes a database instance as argument, currently it's not necessary so we'll set it to null.
const syncTarget = new FileSystemSyncTarget(null)
- Init File API
File API provide methods to interact with storage system, our Synchronizer will need it in order to perform CRUD operations to files on remote.
Currently, FileSystemSyncTarget requires a path points to the directory that function as a remote storage, and MemorySyncTarget requires no arguments.
const syncPath = "src/sample_app/Storage/fsSyncTarget";
await syncTarget.initFileApi(syncPath);
- Initialize the synchronizer
At this step we can initialize a synchronizer, it should we able to perform operations on syncTarget above
const syncer = await syncTarget.synchronizer();
- Initialize remote sync info
It is recommended to create a remote from a Joplin client (Desktop for e.g) by completing the instructions when setup Syncing, and click Synchronise at least once, the remote will be initialized here.
If it's not initialized by any other clients, run the below code
await syncer.initSyncInfo();
At this step your synchronizer is setup properly and you can use the Sync API.
// The most basic things you can do is create items
const note = createNote({ //helper to create note
title: mailTitle,
body: mailBody,
parent_id: "asdas",
});
await syncer.createItems({items: [note]})
After running the above code, check your remote storage, if you use the syncPath from example, then check directory: src/sample_app/Storage/fsSyncTarget
, you should see a new markdown file created
When working with resources, consider to have a localResourceContentPath field, it should be the relative path to the blob.
let localResourceContentPath = "./src/sample_app/Storage/resource/image.png";
const resource = createResource({ localResourceContentPath });
await syncer.createItems({items: [resource]})
// create a resource will create 2 files: 1 metadata file (.md) and a blob
Main docs for Sync API users.
- To use this API, assuming that sync target is available (user setup and ran synchronize on joplin client at least once)
- Assuming that the library supports only sync version 3 (latest sync version).
// 1. Pick a sync target
// currently, database set to null as argument, in the future, we may inject a db instance
const syncTarget = new FileSystemSyncTarget(null);
// 2. Init File API
// depending on file api, it may be different, for e.g: filesystem file api need a path to a directory on the machine.
const syncPath = "src/sample_app/Storage/fsSyncTarget"; // filesystem sync target
await syncTarget.initFileApi(syncPath);
// or, a MemorySyncTarget which doesn't need to provide anything
// const syncTarget = new MemorySyncTarget(null);
await syncTarget.initFileApi();
// 3. Retrieve synchronizer
// with the synchronizer we can perform operations directly to sync target
const syncer = await syncTarget.synchronizer();
// 4. Initialize Sync target (creates info.json on remote)
await syncer.initSyncInfo();
After initializing Synchronizer from above steps, these methods are supported:
// GET a single item from remote,
// Provide either an id or a path of item
// unserializeItem option is default to false, if true will return item as an object, return as string otherwise
.getItem(getItemInput): getItemOutput
type getItemInput = {
path?: string;
id?: string;
unserializeItem?: boolean;
};
type getItemOutput = string | null | Item;
// GET multiple items from remote
// Similar to getItem, provide a list of ids (paths are not supported), unserializeAll === true will return items as an array of object, return an array of string otherwise
.getItems(getItemsInput): getItemsOutput
type getItemsInput = {
ids: string[];
unserializeAll?: boolean;
};
type getItemsOutput = Item[];
// GET items metadata from remote
// context.timetamp return items after specified timestamp (inclusive), this is useful for delta, or detecting new items.
// outputLimit is default to 50, will retrieve a maximum of X items, if there're more items needed retrieving, the output contains a hasMore flag and a timestamp to fetch more.
.getItemsMetadata(getItemsMetadataInput): getItemsMetadataOutput
type getItemsMetadataInput = {
context: {
timestamp?: number; //in unixMs, retrieve items with .updated_time field after timestamp (inclusive)
};
outputLimit?: number; // default to 50
};
type getItemsMetadataOutput = {
items: Item[];
hasMore: boolean;
context: {
timestamp: number; // use to fetch more, or keep track of next fetch for delta
};
};
// UPDATE an item
// To avoid conflict, this method only allow updating an item if its .updated_time field on remote is matched exactly with lastSync parameter.
.updateItem(updateItemInput): updateItemOutput
type updateItemInput = {
item: Item;
lastSync: number; // timestamp in unixMs
};
type updateItemOutput = {
// conflicted means the client timestamp is older than remote, which means another client has updated and this client hasn't pull the changes yet.
// inaccurate timestamp means the client timestamp is newer than remote, which shouldn't be possible, because lastSync timestamp should be updated whenever both sides sync, the client can't independent sync, and has newer timestamp than remote. This is a result of wrongly tracked timestamp on client.
// succeeded means the lastSync arguments exactly equal item.updated_time on remote, and the item will be updated, it will return a newSyncTime, which client should keep track and use as lastSync argument for next update.
status: "conflicted" | "inaccurate timestamp" | "succeeded";
message: string;
// return when conflicted, use this to resolve conflict
remoteItem?: Item;
// return when success
newItem?: Item;
oldItem?: Item;
newSyncTime?: number; // updated timestamp
};
// CREATE multiple items on remote
// The provided items should at least have .type_ field, if it's a resource (type_ == 4), then provide a path to the resource
// Items ids will be generated automatically during creation regardless of input contains id or not, this prevent the client to provide an already available id and cause conflict.
.createItems(createItemsInput): createItemsOutput
type createItemsInput = {
items: Item[];
};
type createItemsOutput = {
createdItems: CreateItem[];
failedItems: { item: any; error: any }[];
};
// DELETE multiple items
// The provided items should at least have .type_ field and id, if it's a resource (type_ == 4), then this operation will find the blob and metadata, and delete both (2 delete API calls) for each item.
.deleteItems(deleteItemsInput): deleteItemsOutput
type deleteItemsInput = {
deleteItems: Item[];
};
type deleteItemsOutput = {
status:
| "succeeded"
| "item not found" // item with provided id not available on remote
| "could not delete item" // unknown error
| "read-only item can't be deleted";
item?: Item;
error?: any;
};
// VERIFY sync info version and E2E settings on remote
// Run before every Sync operations, it will fetch remote sync info (info.json file) and make sure its sync version is 3
// Then it will looks for remote E2E settings, and compare to the input E2E settings and prompts approriate actions client has to do to resolve conflicts (if happens), see E2E docs.
.verifySyncInfo(verifySyncInfoInput): verifySyncInfoOutput
type verifySyncInfoInput = {
E2E: {
ppk?: PublicPrivateKeyPair;
e2ee: boolean;
};
};
type verifySynInfoOutput = {
status: "success" | "aborted";
message: string;
remoteSyncInfo?: any; // for debug
};
- Synchronizer.e2eInfo stores encryption setup data, this decides whether CREATE, UPDATE methods will encrypt before upload.
type e2eInfo = {
ppk?: PublicPrivateKeyPair;
e2ee: boolean; // encryption enable or not
activeMasterKeyId?: string;
};
type PublicPrivateKeyPair = {
id: string;
keySize: number;
publicKey: string;
privateKey: {
encryptionMethod: EncryptionMethod;
ciphertext: string;
};
createdTime: number;
}
- Users (clients) cant use Sync Library to enable/disable remote E2E because to do so, it requires proper re-encrypt and reupload data, this reduces data corruption.
- To enable E2E, the process is as follow:
- Enable E2E in 1 Joplin app, provide password and all necessary data, click Synchronise and make sure Synchronisation succeeds, instruction: Joplin enabling e2ee
- In Sync library, fetch the remote sync info and extract all necessary E2E data:
const remoteSyncInfo = fetchSyncInfo(synchronizer.api());
const localE2EInfo = extractE2EInfoFromSyncInfo(remoteSyncInfo)
- Run Synchronizer.setupE2E(localE2EInfo):
const res: setupE2EOutput = synchronizer.setupE2E(localE2EInfo)
type setupE2EOutput = {
status: "succeeded" | "aborted";
message: string;
remoteInfo?: SyncInfo;
e2eInfo?: e2eInfo; // returned when e2eInfo is set successfully
};
- If res.status is succeeded, then E2E is synced properly between client and remote. Most of the aborted cases are either users forgot to provide localE2EInfo (no E2E setup) or remote's e2e has changed since last time, the fix is to repeat from step 2 again.
- To disable E2E, use 1 Joplin app to disable E2E, from step 2 and on, do the same as process above
- When Synchronizer E2E is enabled, CREATE and UPDATE operations will encrypt items automatically for each call (no extra parameters required).
- Only 2 Synchronizer operations: CREATE and UPDATE can apply encryption, which uses the ItemUploader class, it will take the E2E input from Synchronizer and perform encryption accordingly.
- For READ methods, client may need extra code to decrypt content with master key, because results return from Sync API will be encrypted string.