Add UPGRADING.md

CHANGELOG.md

@@ -59,10 +59,6 @@
 * Remove testling from `package.json` (@vweevers)
 * Remove `level.js` logo (@vweevers)
 
-**Historical Note** As a result of removing `IDBWrapper`, only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE.
-
-**Historical Note** Though `level-js` now passes the full `abstract-leveldown` test suite, fulfulling the snapshot guarantee (reads not being affected by simultaneous writes) means a loss of backpressure as iterators have to keep reading from their IndexedDB cursors. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees.
-
 **Historical Note** This release introduced the boolean `binaryKeys` and `arrayKeys` properties on the constructor, indicating whether the environment supports binary and array keys respectively. These properties may become private.
 
 **Historical Note** The vendored `IndexedDBShim` is still included, but likely to be removed.

README.md

@@ -1,31 +1,47 @@
 # level-js
 
-> An [`abstract-leveldown`](https://github.com/Level/abstract-leveldown) compliant store on top of [IndexedDB](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API), which is in turn implemented on top of [LevelDB](https://github.com/google/leveldb) which brings this whole shebang full circle.
+> An [`abstract-leveldown`][abstract-leveldown] compliant store on top of [IndexedDB][indexeddb], which is in turn implemented on top of [LevelDB][leveldb] which brings this whole shebang full circle.
 
-[![level badge][level-badge]](https://github.com/level/awesome)
+[![level badge][level-badge]][awesome]
 [![npm](https://img.shields.io/npm/v/level-js.svg)](https://www.npmjs.com/package/level-js)
 [![npm next](https://img.shields.io/npm/v/level-js/next.svg)](https://www.npmjs.com/package/level-js)
 [![Travis](https://secure.travis-ci.org/Level/level-js.svg?branch=master)](http://travis-ci.org/Level/level-js)
 [![npm](https://img.shields.io/npm/dm/level-js.svg)](https://www.npmjs.com/package/level-js)
 [![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)
 
+## Table of Contents
+
+<details><summary>Click to expand</summary>
+
+-   [Background](#background)
+-   [Example](#example)
+-   [Browser Support](#browser-support)
+-   [Type Support](#type-support)
+-   [Install](#install)
+-   [API](#api)
+-   [Running Tests](#running-tests)
+-   [Big Thanks](#big-thanks)
+-   [License](#license)
+
+</details>
+
 ## Background
 
 Here are the goals of `level-js`:
 
 -   Store large amounts of data in modern browsers
--   Pass the full `abstract-leveldown` test suite
--   Support [Buffer](https://nodejs.org/api/buffer.html) values (in all target environments)
--   Support all key types of IndexedDB Second Edition, including binary keys (depends on environment)
--   Support all value types of the structured clone algorithm (depends on environment) except for `null` and `undefined`
+-   Pass the full [`abstract-leveldown`][abstract-leveldown] test suite
+-   Support [`Buffer`][buffer] keys and values
+-   Support all key types of IndexedDB Second Edition
+-   Support all value types of the [structured clone algorithm][structured-clone-algorithm] except for `null` and `undefined`
 -   Be as fast as possible
--   Sync with [multilevel](https://github.com/juliangruber/multilevel) over either ASCII or binary transports.
+-   Sync with [multilevel](https://github.com/juliangruber/multilevel) over ASCII or binary transports.
 
-Being `abstract-leveldown` compliant means you can use many of the [Level modules](https://github.com/Level/awesome/) on top of this library. For some demos of it working, see @brycebaril's presentation [Path of the NodeBases Jedi](http://brycebaril.github.io/nodebase_jedi/#/vanilla).
+Being `abstract-leveldown` compliant means you can use many of the [Level modules][awesome] on top of this library. For some demos of it working, see @brycebaril's presentation [Path of the NodeBases Jedi](http://brycebaril.github.io/nodebase_jedi/#/vanilla).
 
 ## Example
 
-**This assumes use of version `3.0.0-rc1`. The next release will have an upgrade guide. Until then, please see the [changelog](CHANGELOG.md).**
+**If you are upgrading:** please see [UPGRADING.md](UPGRADING.md).
 
 ```js
 var levelup = require('levelup')
@@ -43,10 +59,104 @@ db.put('hello', Buffer.from('world'), function (err) {
 })
 ```
 
+In ES6 browsers:
+
+```js
+const levelup = require('levelup')
+const leveljs = require('level-js')
+const db = levelup(leveljs('bigdata'))
+
+await db.put('hello', Buffer.from('world'))
+const value = await db.get('hello')
+```
+
 ## Browser Support
 
 [![Sauce Test Status](https://saucelabs.com/browser-matrix/level-js.svg)](https://saucelabs.com/u/level-js)
 
+## Type Support
+
+Unlike [`leveldown`][leveldown], `level-js` does not stringify keys or values. This means that in addition to strings and Buffers you can store almost any JavaScript type without the need for [`encoding-down`][encoding-down].
+
+### Values
+
+All value types of the [structured clone algorithm][structured-clone-algorithm] are supported except for `null` and `undefined`. Depending on the environment, this includes:
+
+-   Number, including `NaN`, `Infinity` and `-Infinity`
+-   String, Boolean, Date, RegExp, Array, Object
+-   ArrayBuffer or a view thereof (typed arrays);
+-   Map, Set, Blob, File, FileList, ImageData (limited support).
+
+In addition `level-js` stores [`Buffer`][buffer] values without transformation. This works in all target environments because `Buffer` is a subclass of `Uint8Array`, meaning such values can be passed to `IndexedDB` as-is.
+
+When getting or iterating binary values, regardless of whether they were stored as a `Buffer`, `ArrayBuffer` or a view thereof, values will return as a `Buffer`. This behavior can be disabled, in which case `ArrayBuffer` returns as `ArrayBuffer`, typed arrays return as typed arrays and `Buffer` returns as `Uint8Array`:
+
+```js
+db.get('key', { asBuffer: false })
+db.iterator({ valueAsBuffer: false })
+```
+
+If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `put` or `batch`. For example, IE does not support typed array values. At the time of writing, Chrome is the only browser that supports all types listed above.
+
+Due to the special meaning that `null` and `undefined` have in `abstract-leveldown` iterators and Node.js streams, values of this type are converted to empty strings prior to storage.
+
+### Keys
+
+All key types of IndexedDB Second Edition are supported. Depending on the environment, this includes:
+
+-   Number, including `Infinity` and `-Infinity`, but not `NaN`
+-   Date, except invalid (`NaN`)
+-   String
+-   ArrayBuffer or a view thereof (typed arrays);
+-   Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves.
+
+In addition you can use [`Buffer`][buffer] keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. When iterating binary keys, regardless of whether they were stored as `Buffer`, `ArrayBuffer` or a view thereof, keys will return as a `Buffer`. This behavior can be disabled, in which case binary keys will always return as `ArrayBuffer`:
+
+```js
+db.iterator({ keyAsBuffer: false })
+```
+
+Note that this behavior is slightly different from values due to the way that IndexedDB works. IndexedDB stores binary _values_ using the structured clone algorithm, which preserves views, but it stores binary _keys_ as an array of octets, so that it is able to compare and sort differently typed keys.
+
+If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `get`, `put`, `del`, `batch` or an iterator. Exceptions are:
+
+-   `null` and `undefined`: rejected early by `abstract-leveldown`
+-   Boolean and `NaN`: though invalid per the IndexedDB specification, they are converted to strings for `abstract-leveldown` compatibility;
+-   Binary and array keys: if not supported by the environment, `level-js` falls back to `String(key)`.
+
+### Normalization
+
+If you desire normalization for keys and values (e.g. to stringify numbers), wrap `level-js` with [`encoding-down`][encoding-down]. Alternatively install [`level-browserify`][level-browserify] which conveniently bundles [`levelup`][levelup], `level-js` and `encoding-down`. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior or to smooth over type differences between browsers. For example, you could have [`leveldown`][leveldown] in a backend and `level-js` in the frontend.
+
+Another reason you might want to use `encoding-down` is that the structured clone algorithm, while rich in types, can be slower than `JSON.stringify`.
+
+### Buffer vs ArrayBuffer
+
+For interoperability it is recommended to use `Buffer` as your binary type. While we recognize that Node.js core modules are moving towards supporting `ArrayBuffer` and views thereof, `Buffer` remains the primary binary type in the Level ecosystem.
+
+That said: if you want to `put()` an `ArrayBuffer` you can! Just know that it will come back as a `Buffer` by default. If you want to `get()` or iterate stored `ArrayBuffer` data as an `ArrayBuffer`, you have a few options. Without `encoding-down`:
+
+```js
+const db = levelup(leveljs('mydb'))
+
+// Yields an ArrayBuffer, Buffer and ArrayBuffer
+const value1 = await db.get('key', { asBuffer: false })
+const value2 = await db.get('key')
+const value3 = value2.buffer
+```
+
+With `encoding-down` (or `level-browserify`) you can use the `id` encoding to selectively bypass encodings:
+
+```js
+const encode = require('encoding-down')
+const db = levelup(encode(leveljs('mydb'), { valueEncoding: 'binary' }))
+
+// Yields an ArrayBuffer, Buffer and ArrayBuffer
+const value1 = await db.get('key', { valueEncoding: 'id' })
+const value2 = await db.get('key')
+const value3 = value2.buffer
+```
+
 ## Install
 
 ```bash
@@ -95,3 +205,23 @@ Cross-browser Testing Platform and Open Source ♥ Provided by [Sauce Labs](http
 [MIT](./LICENSE.md) © 2012-present [Max Ogden](https://github.com/maxogden) and [Contributors](./CONTRIBUTORS.md).
 
 [level-badge]: http://leveldb.org/img/badge.svg
+
+[indexeddb]: https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API
+
+[leveldb]: https://github.com/google/leveldb
+
+[buffer]: https://nodejs.org/api/buffer.html
+
+[awesome]: https://github.com/Level/awesome
+
+[abstract-leveldown]: https://github.com/Level/abstract-leveldown
+
+[levelup]: https://github.com/Level/levelup
+
+[leveldown]: https://github.com/Level/leveldown
+
+[level-browserify]: https://github.com/Level/level-browserify
+
+[encoding-down]: https://github.com/Level/encoding-down
+
+[structured-clone-algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm

UPGRADING.md

@@ -0,0 +1,93 @@
+# Upgrade Guide
+
+This document describes breaking changes and how to upgrade. For a complete list of changes including minor and patch releases, please refer to the [changelog][changelog].
+
+## V3
+
+This release brings `level-js` up to par with latest [`levelup`][levelup] (v2), [`abstract-leveldown`][abstract-leveldown] (v5) and IndexedDB Second Edition. It targets modern `browserify` preferring [`Buffer`][buffer] over `ArrayBuffer`. Lastly, [`IDBWrapper`][idbwrapper] has been replaced with straight IndexedDB code.
+
+### Usage with [`levelup`][levelup]
+
+Usage has changed to:
+
+```js
+const levelup = require('levelup')
+const leveljs = require('leveljs')
+
+const db = levelup(leveljs('mydb'))
+```
+
+From the old:
+
+```js
+const db = levelup('mydb', { db: leveljs })
+```
+
+Friendly reminder: encodings have moved from [`levelup`][levelup] to [`encoding-down`][encoding-down]. To get identical functionality to `levelup < 2` please use the [`level-browserify`][level-browserify] convenience package or wrap `level-js` with `encoding-down`:
+
+```js
+const encode = require('encoding-down')
+const db = levelup(encode(leveljs('mydb')))
+```
+
+### New database prefix
+
+The default prefix of the [`IDBDatabase`][idbdatabase] name has changed from `IDBWrapper-` to `level-js-`. To access databases created using `level-js < 3`, pass a custom prefix to the `level-js` constructor:
+
+```js
+const db = levelup(leveljs('mydb', { prefix: 'IDBWrapper-' }))
+```
+
+### Browser support
+
+As a result of removing [`IDBWrapper`][idbwrapper], only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE.
+
+:fire: Internet Explorer 10 is no longer supported.
+
+### Type support
+
+All value types of the [structured clone algorithm][structured-clone-algorithm] and all key types of IndexedDB Second Edition are now supported. This means you can store almost any JavaScript type without the need for [`encoding-down`][encoding-down]. In addition, you can use [`Buffer`][buffer] for both keys and values. For details and caveats please see the [readme][readme].
+
+### No backpressure
+
+In `level-js`, iterators are powered by IndexedDB cursors. To fulfill [`abstract-leveldown`][abstract-leveldown] snapshot guarantees (reads not being affected by simultaneous writes) cursors are started immediately and continuously read from, filling an in-memory cache.
+
+Though `level-js` now passes the full [`abstract-leveldown`][abstract-leveldown] test suite, fulfilling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees.
+
+### Removed `raw` option
+
+Because `level-js` no longer stringifies values, the `raw` option (which bypassed conversion) became unnecessary and has been removed. If you use [`level-browserify`][level-browserify] or [`levelup`][levelup] with [`encoding-down`][encoding-down], you can store and retrieve raw values (as returned by IndexedDB) using the `id` encoding. Please refer to the [readme][readme] for an example.
+
+### New `destroy()` function signature
+
+Previously, a `level-js` instance could be passed to `destroy()`:
+
+```js
+leveljs.destroy(db, callback)
+```
+
+This was useful to destroy a database that used a custom prefix. The new signature is `destroy(location[, prefix], callback)`.
+
+### Strict `.batch(array)`
+
+The upgrade to [`abstract-leveldown`][abstract-leveldown] comes with a [breaking change](https://github.com/Level/abstract-leveldown/commit/a2621ad70571f6ade9d2be42632ece042e068805) for the array version of `.batch()`. This change ensures all elements in the batch array are objects. If you previously passed arrays to `.batch()` that contained `undefined` or `null`, they would be silently ignored. Now this will produce an error.
+
+[readme]: README.md
+
+[changelog]: CHANGELOG.md
+
+[buffer]: https://nodejs.org/api/buffer.html
+
+[idbwrapper]: https://www.npmjs.com/package/idb-wrapper
+
+[abstract-leveldown]: https://github.com/Level/abstract-leveldown
+
+[levelup]: https://github.com/Level/levelup
+
+[encoding-down]: https://github.com/Level/encoding-down
+
+[level-browserify]: https://github.com/Level/level-browserify
+
+[idbdatabase]: https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase
+
+[structured-clone-algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm

package.json

@@ -8,14 +8,15 @@
   "scripts": {
     "test": "standard && airtap --local --no-coverage test/index.js",
     "test-browsers": "standard && airtap --sauce-connect --loopback airtap.local --no-coverage test/index.js",
-    "remark": "remark README.md CONTRIBUTORS.md -o"
+    "remark": "remark README.md CONTRIBUTORS.md UPGRADING.md -o"
   },
   "files": [
     "index.js",
     "iterator.js",
     "util",
     "CONTRIBUTORS.md",
     "CHANGELOG.md",
+    "UPGRADING.md",
     "sauce-labs.svg"
   ],
   "browser": {
@@ -38,13 +39,23 @@
     "buffer": "~5.1.0",
     "level-community": "~3.0.0",
     "remark-cli": "^5.0.0",
+    "remark-collapse": "~0.1.2",
     "remark-git-contributors": "~0.2.0",
+    "remark-toc": "~5.0.0",
     "standard": "^11.0.1",
     "tape": "^4.0.0"
   },
   "remarkConfig": {
     "plugins": {
-      "remark-git-contributors": "level-community"
+      "remark-git-contributors": "level-community",
+      "remark-toc": {
+        "maxDepth": 2,
+        "tight": true
+      },
+      "remark-collapse": {
+        "test": "Table of Contents",
+        "summary": "Click to expand"
+      }
     }
   },
   "repository": {