Skip to content

Commit

Permalink
- Enhancement: Make addition of .sqlite extension to __sysdb__ co…
Browse files Browse the repository at this point in the history 
…nditional

    on `CFG.addSQLiteExtension` setting
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
    repeat `IDBFactory.open` call to the same name and version (assuming
    no deletes or aborts causing rollbacks) will reuse the same SQLite
    `openDatabase` instance
- Enhancement: Support `autoName` config to interpret empty string name as a
    cue for creating a database name automatically (introspect on
    `IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
    and listing to be of SQLite in-memory databases (for Node); name supplied
    by user is still used (including to automatically build a cache since
    SQLite does not allow naming of in-memory databases); the name is also
    accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
    name/version tracking to also be within an in-memory database; part of
    indexeddbshim#278 (untested); builds on work by @ThomasGreiner)
  • Loading branch information
@brettz9
brettz9 committed Apr 9, 2017
1 parent 91b62aa commit a49ef5e
Show file tree
Hide file tree
Showing 18 changed files with 803 additions and 119 deletions.
16 changes: 16 additions & 0 deletions CHANGES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,22 @@ they were actually changes since a more recent version on `master`.
- Enhancement: Add `deleteDatabaseFiles` config to allow file itself to
be deleted in Node; will throw `UnknownError` if there are any errors
removing file (besides a missing file)
- Enhancement: Make addition of `.sqlite` extension to `__sysdb__` conditional
on `CFG.addSQLiteExtension` setting
- Enhancement: Support `autoName` config to interpret empty string name as a
cue for creating a database name automatically (introspect on
`IDBDatabase.name` to get the actual name used) (untested)
- Enhancement: Support `memoryDatabase` config to cause all opening, deleting,
and listing to be of SQLite in-memory databases (for Node); name supplied
by user is still used (including to automatically build a cache since
SQLite does not allow naming of in-memory databases); the name is also
accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
name/version tracking to also be within an in-memory database; part of
#278 (untested); builds on work by @ThomasGreiner)
- Enhancement: Support `cacheDatabaseInstances` config to ensure that any
repeat `IDBFactory.open` call to the same name and version (assuming
no deletes or aborts causing rollbacks) will reuse the same SQLite
`openDatabase` instance
- Add missing API: Add `IDBCursor.continuePrimaryKey`
- Add missing API: Implement `IDBObjectStore.getKey`
- Add missing APIs: Implement `IDBIndex.getAll/getAllKeys`
Expand Down
21 changes: 20 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,10 @@ shimIndexedDB.__setConfig(property, value);
The available properties relevant to browser or Node are:

- __DEBUG__ - Boolean (equivalent to calling `shimIndexedDB.__debug(val)`)
- __cacheDatabaseInstances__ - Config to ensure that any repeat
`IDBFactory.open` call to the same name and version (assuming
no deletes or aborts causing rollbacks) will reuse the same SQLite
`openDatabase` instance.
- __checkOrigin__ - Boolean on whether to perform origin checks in `IDBFactory`
methods (`open`, `deleteDatabase`, `webkitGetDatabaseNames`); effectively
defaults to true (must be set to `false` to cancel checks); for Node testing,
Expand Down Expand Up @@ -315,7 +319,22 @@ browser, particularly if one changes the defaults.
to escape NFD-escaping characters to avoid clashes on MacOS which
performs NFD on files
- __addSQLiteExtension__ - Boolean on whether to add the `.sqlite` extension
to file names; defaults to `true`
to database file names (including `__sysdb__` which tracks versions);
defaults to `true`
- __autoName__ - Boolean config to interpret empty string name as a
cue for creating a database name automatically (introspect on
`IDBDatabase.name` to get the actual name used); `false` by default
- __memoryDatabase__ - (Node-only) string config to cause all opening,
deleting, and listing to be of SQLite in-memory databases; name supplied
by user is still used (including to automatically build a cache since
SQLite does not allow naming of in-memory databases); the name is also
accessible to `IDBFactory.webkitGetDatabaseNames()`; causes database
name/version tracking to also be within an in-memory database; if
set in the browser, avoids normal database name escaping meant
for Node compatibility; allowable values include the empty string,
`":memory:"`, and `file::memory:[?optionalQueryString][#optionalHash]`.
See <https://sqlite.org/inmemorydb.html> and <https://sqlite.org/uri.html>
for more on the function and form of such values

The following config items are for Node only and are mostly for development
debugging.
Expand Down
136 changes: 121 additions & 15 deletions dist/indexeddbshim-UnicodeIdentifiers-node.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8581,6 +8581,11 @@ var CFG = {};
// Boolean for verbose reporting
'DEBUG', // Effectively defaults to false (ignored unless `true`)

'cacheDatabaseInstances', // Boolean (effectively defaults to true) on whether to cache WebSQL `openDatabase` instances
'autoName', // Boolean on whether to auto-name databases (based on an auto-increment) when
// the empty string is supplied; useful with `memoryDatabase`; defaults to `false`
// which means the empty string will be used as the (valid) database name

// Determines whether the slow-performing `Object.setPrototypeOf` calls required
// for full WebIDL compliance will be used. Probably only needed for testing
// or environments where full introspection on class relationships is required;
Expand Down Expand Up @@ -8642,7 +8647,12 @@ var CFG = {};
// characters to avoid clashes on MacOS which performs NFD on files
// Boolean on whether to add the `.sqlite` extension to file names;
// defaults to `true`
'addSQLiteExtension',
'addSQLiteExtension', ['memoryDatabase', function (val) {
// Various types of in-memory databases that can auto-delete
if (/^(?::memory:|file::memory:(\?[^#]*)?(#.*)?)?$/.test(val)) {
throw new TypeError('`memoryDatabase` must be the empty string, ":memory:", or a "file::memory:[?queryString][#hash] URL".');
}
}],

// NODE-SPECIFIC CONFIG
// Boolean on whether to delete the database file itself after `deleteDatabase`;
Expand All @@ -8654,11 +8664,19 @@ var CFG = {};
'sqlTrace', // Callback not used by default
'sqlProfile' // Callback not used by default
].forEach(function (prop) {
var validator = void 0;
if (Array.isArray(prop)) {
validator = prop[1];
prop = prop[0];
}
Object.defineProperty(CFG, prop, {
get: function get() {
return map[prop];
},
set: function set(val) {
if (validator) {
validator(val);
}
map[prop] = val;
}
});
Expand Down Expand Up @@ -10197,12 +10215,25 @@ function _interopRequireWildcard(obj) { if (obj && obj.__esModule) { return obj;

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }

var websqlDBCache = {};
var sysdb = void 0;
var nameCounter = 0;

function hasNullOrigin() {
return _CFG2.default.checkOrigin !== false && ((typeof location === 'undefined' ? 'undefined' : _typeof(location)) !== 'object' || !location || location.origin === 'null');
}

function getLatestCachedWebSQLVersion(name) {
return Object.keys(websqlDBCache[name]).map(Number).reduce(function (prev, curr) {
return curr > prev ? curr : prev;
}, 0);
}

function getLatestCachedWebSQLDB(name) {
return websqlDBCache[name] && websqlDBCache[name][// eslint-disable-line standard/computed-property-even-spacing
getLatestCachedWebSQLVersion()];
}

/**
* Craetes the sysDB to keep track of version numbers for databases
**/
Expand All @@ -10216,7 +10247,7 @@ function createSysDB(success, failure) {
if (sysdb) {
success();
} else {
sysdb = _CFG2.default.win.openDatabase('__sysdb__.sqlite', 1, 'System Database', _CFG2.default.DEFAULT_DB_SIZE);
sysdb = _CFG2.default.win.openDatabase(typeof _CFG2.default.memoryDatabase === 'string' ? _CFG2.default.memoryDatabase : '__sysdb__' + (_CFG2.default.addSQLiteExtension !== false ? '.sqlite' : ''), 1, 'System Database', _CFG2.default.DEFAULT_DB_SIZE);
sysdb.transaction(function (systx) {
systx.executeSql('CREATE TABLE IF NOT EXISTS dbVersions (name VARCHAR(255), version INT);', [], success, sysDbCreateError);
}, sysDbCreateError);
Expand Down Expand Up @@ -10278,8 +10309,16 @@ IDBFactory.prototype.open = function (name /* , version */) {
if (hasNullOrigin()) {
throw (0, _DOMException.createDOMException)('SecurityError', 'Cannot open an IndexedDB database from an opaque origin.');
}

if (_CFG2.default.autoName && name === '') {
name = 'autoNamedDatabase_' + nameCounter++;
}
name = String(name); // cast to a string
var sqlSafeName = util.escapeSQLiteStatement(name);

var useMemoryDatabase = typeof _CFG2.default.memoryDatabase === 'string';
var useDatabaseCache = _CFG2.default.cacheDatabaseInstances !== false || useMemoryDatabase;

var escapedDatabaseName = void 0;
try {
escapedDatabaseName = util.escapeDatabaseNameForSQLAndFiles(name);
Expand All @@ -10302,7 +10341,15 @@ IDBFactory.prototype.open = function (name /* , version */) {
}

function openDB(oldVersion) {
var db = _CFG2.default.win.openDatabase(escapedDatabaseName, 1, name, _CFG2.default.DEFAULT_DB_SIZE);
var db = void 0;
if ((useMemoryDatabase || useDatabaseCache) && name in websqlDBCache && websqlDBCache[name][version]) {
db = websqlDBCache[name][version];
} else {
db = _CFG2.default.win.openDatabase(useMemoryDatabase ? _CFG2.default.memoryDatabase : escapedDatabaseName, 1, name, _CFG2.default.DEFAULT_DB_SIZE);
if (useDatabaseCache) {
websqlDBCache[name][version] = db;
}
}
req.__readyState = 'done';
if (version === undefined) {
version = oldVersion || 1;
Expand Down Expand Up @@ -10332,6 +10379,7 @@ IDBFactory.prototype.open = function (name /* , version */) {
function reportError() {
throw new Error('Unable to roll back upgrade transaction!');
}

// Attempt to revert
if (oldVersion === 0) {
systx.executeSql('DELETE FROM dbVersions WHERE "name" = ?', [sqlSafeName], cb, reportError);
Expand All @@ -10356,11 +10404,24 @@ IDBFactory.prototype.open = function (name /* , version */) {
req.result.__versionTransaction = null;
sysdbFinishedCb(systx, false, function () {
req.transaction.__transFinishedCb(false, function () {
if (useDatabaseCache) {
if (name in websqlDBCache) {
delete websqlDBCache[name][version];
}
}
ev.complete();
req.__transaction = null;
});
});
};
req.transaction.on__preabort = function () {
// We ensure any cache is deleted before any request error events fire and try to reopen
if (useDatabaseCache) {
if (name in websqlDBCache) {
delete websqlDBCache[name][version];
}
}
};
req.transaction.on__abort = function () {
req.__transaction = null;
setTimeout(function () {
Expand Down Expand Up @@ -10421,18 +10482,34 @@ IDBFactory.prototype.open = function (name /* , version */) {
}, dbCreateError);
}

createSysDB(function () {
sysdb.readTransaction(function (sysReadTx) {
sysReadTx.executeSql('SELECT "version" FROM dbVersions WHERE "name" = ?', [sqlSafeName], function (sysReadTx, data) {
if (data.rows.length === 0) {
// Database with this name does not exist
openDB(0);
} else {
openDB(data.rows.item(0).version);
}
var latestCachedVersion = void 0;
if (useDatabaseCache) {
if (!(name in websqlDBCache)) {
websqlDBCache[name] = {};
}
if (version === undefined) {
latestCachedVersion = getLatestCachedWebSQLVersion(name);
} else if (websqlDBCache[name][version]) {
latestCachedVersion = version;
}
}

if (latestCachedVersion) {
openDB(latestCachedVersion);
} else {
createSysDB(function () {
sysdb.readTransaction(function (sysReadTx) {
sysReadTx.executeSql('SELECT "version" FROM dbVersions WHERE "name" = ?', [sqlSafeName], function (sysReadTx, data) {
if (data.rows.length === 0) {
// Database with this name does not exist
openDB(0);
} else {
openDB(data.rows.item(0).version);
}
}, dbCreateError);
}, dbCreateError);
}, dbCreateError);
}, dbCreateError);
}

return req;
};
Expand Down Expand Up @@ -10463,6 +10540,8 @@ IDBFactory.prototype.deleteDatabase = function (name) {
throw err; // throw new TypeError('You have supplied a database name which does not match the currently supported configuration, possibly due to a length limit enforced for Node compatibility.');
}

var useMemoryDatabase = typeof _CFG2.default.memoryDatabase === 'string';

var req = _IDBRequest.IDBOpenDBRequest.__createInstance();
var calledDBError = false;
var version = 0;
Expand Down Expand Up @@ -10517,6 +10596,32 @@ IDBFactory.prototype.deleteDatabase = function (name) {
// `dbVersions` change if they fail
sysdb.transaction(function (systx) {
systx.executeSql('DELETE FROM dbVersions WHERE "name" = ? ', [sqlSafeName], function () {
// Todo: We should also check whether `dbVersions` is empty and if so, delete upon
// `deleteDatabaseFiles` config. We also ought to do this when aborting (see
// above code with `DELETE FROM dbVersions`)
if (useMemoryDatabase) {
var latestSQLiteDBCached = websqlDBCache[name] ? getLatestCachedWebSQLDB(name) : null;
if (!latestSQLiteDBCached) {
console.warn('Could not find a memory database instance to delete.');
return;
}
var _sqliteDB = latestSQLiteDBCached._db && latestSQLiteDBCached._db._db;
if (!_sqliteDB || !_sqliteDB.close) {
console.warn('The `openDatabase` implementation does not have the expected `._db._db.close` method for closing the database');
return;
}
_sqliteDB.close(function (err) {
if (err) {
console.warn('Error closing (destroying) memory database');
return;
}
delete websqlDBCache[name]; // New calls will treat as though never existed
});
return;
}
if (_CFG2.default.cacheDatabaseInstances !== false && name in websqlDBCache) {
delete websqlDBCache[name];
}
if (_CFG2.default.deleteDatabaseFiles !== false) {
require('fs').unlink(require('path').resolve(escapedDatabaseName), function (err) {
if (err && err.code !== 'ENOENT') {
Expand All @@ -10529,8 +10634,8 @@ IDBFactory.prototype.deleteDatabase = function (name) {
return;
}

var db = _CFG2.default.win.openDatabase(escapedDatabaseName, 1, name, _CFG2.default.DEFAULT_DB_SIZE);
db.transaction(function (tx) {
var sqliteDB = _CFG2.default.win.openDatabase(escapedDatabaseName, 1, name, _CFG2.default.DEFAULT_DB_SIZE);
sqliteDB.transaction(function (tx) {
tx.executeSql('SELECT "name" FROM __sys__', [], function (tx, data) {
var tables = data.rows;
(function deleteTables(i) {
Expand Down Expand Up @@ -13073,6 +13178,7 @@ IDBTransaction.prototype.__abortTransaction = function (err) {
_CFG2.default.DEBUG && console.log('Rollback succeeded', me);
}

me.dispatchEvent((0, _Event.createEvent)('__preabort'));
me.__requests.filter(function (q) {
return q.req && q.req.__readyState !== 'done';
}).reduce(function (promises, q) {
Expand Down
8 changes: 4 additions & 4 deletions dist/indexeddbshim-UnicodeIdentifiers-node.min.js
View file

Large diffs are not rendered by default.

0 comments on commit a49ef5e

Please sign in to comment.