go get github.com/lukechampine/jj
jj
implements a JSON transaction journal. It enables efficient ACID
transactions on JSON objects. If your disk operations require something faster
than rewriting a whole file on each save, but you aren't ready to design a
full database schema, jj
might be for you.
Each Journal represents a single JSON object. The object is serialized as an
"initial object" followed by a series of update sets, one per line. Each
update specifies a field and a modification. See the Update
section for a
full specification.
During operation, the object is first loaded by reading the file and applying each update to the initial object. It is subsequently modified by appending update sets to the file, one per line. At any time, a "checkpoint" may be created, which clears the Journal and starts over with a new initial object. This allows for compaction of the Journal file.
In the event of power failure or other serious disruption, the most recent
update set may be only partially written. Partially written update sets are
simply ignored when reading the Journal. Individual updates may also be
ignored if they are malformed, though other updates in the set may be applied.
See the Update
section for an explanation of malformed updates.
// create initial object
var obj struct {
Foo struct {
Bar struct {
Baz []int `json:"baz"`
} `json:"bar"`
Quux int `json:"quux"`
} `json:"foo"`
}
obj.Foo.Bar.Baz = []int{1,2,3}
obj.Foo.Quux = 6
// create journal with initial object
// (error handling omitted)
j, _ := jj.OpenJournal("myjournal", obj)
// record a set of updates to the object
_ = j.Update([]jj.Update{
jj.NewUpdate("foo.bar.baz.2", 7),
jj.NewUpdate("foo.quux", 7),
})
// close and reopen the journal, applying the updates
_ = j.Close()
j, _ = jj.OpenJournal("myjournal", &obj)
println(obj.Foo.Bar.Baz[2]) // -> 7
println(obj.Foo.Quux) // -> 7
An Update is a modification of a path in a JSON object. A "path" in this
context means an object or array element. Syntactically, a path is a set of
accessors joined by the .
character. An accessor is either an object key
or an array index. For example, given this object:
{
"foo": {
"bars": [
{"baz":3}
]
}
}
The following path accesses the value "3":
foo.bars.0.baz
The path is accompanied by a new object. Thus, to increment the value "3" in the above object, we would use the following Update:
{
"p": "foo.bars.0.baz",
"v": 4
}
All permutations of the Update object are legal. However, malformed updates are ignored during application. An Update is considered malformed in three circumstances:
- Its Path references an element that does not exist at application time. This includes out-of-bounds array indices.
- Its Path contains invalid characters (e.g.
"
). See the JSON spec. - Value contains invalid JSON or is empty.
Other special cases are handled as follows:
- If Path is
""
, the entire object is replaced. - If an object contains duplicate keys, the first key encountered is used.
Finally, to enable efficient array updates, the length of the array (at application time) may be used as a special array index. When this index is the last accessor in Path, Value will be appended to the end of the array. If the index is not the last accessor, the Update is considered malformed (and thus is ignored).
An important aspect of jj
is that you cannot "read" from the journal; you
can only apply updates. The only time you retrieve data from the journal is
when loading it from disk, usually during initialization. This means that you
must keep your in-memory copy of the data in sync with the journal.