Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow parallel execution of node event loop and sqlite3 in Statement:Work_AfterAll #1514

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

alex3d
Copy link

@alex3d alex3d commented Oct 19, 2021

This PR allows the eager start of the next queued query execution in Statement::All - before the JS callback execution.
In some cases, it could double performance by allowing parallel execution of node event loop and sqlite3 query processing in the thread pool.

Here is a microbenchmark with 1.7 performance improvement:

$ node perf.js # before PR 
elapsed 1488 ms,  1.488 ms per SELECT, event loop utilization=0.5520233139076274

$ node perf.js # after PR
elapsed  856 ms,  0.856 ms per SELECT, event loop utilization=0.9129801163162077
const sqlite3 = require('./lib/sqlite3');
const {promisify} = require('util');
const {performance} = require('perf_hooks');

async function main() {
  let db = new sqlite3.Database(':memory:');
  const prepare = (sql) => new Promise((resolve, reject) => {
    const stmt = db.prepare(sql, err => {
      if (err===null)
        resolve(stmt);
      else
        reject(err);
    });
  });
  
  await promisify(db.run).call(db, "CREATE TABLE lorem (info TEXT)");
  let insert = await prepare("INSERT INTO lorem VALUES (?)");
  for (var i = 0; i < 2000; i++) {
      await promisify(insert.run).call(insert, "Ipsum " + i);
  }

  let select = await prepare("SELECT max(id), max(info) FROM (SELECT rowid AS id, info FROM lorem) GROUP BY id%500;");
  let all = promisify(select.all.bind(select));
  const QUERIES = 1000;
  const T0 = Date.now(), ELU0=performance.eventLoopUtilization();
  
  await Promise.all(new Array(QUERIES).fill(0).map(() => all().then(v=>JSON.stringify(v)) ));
  
  const T1 = Date.now(), ELU1=performance.eventLoopUtilization(ELU0);
  console.log(`elapsed ${T1-T0} ms,  ${(T1-T0)/QUERIES} ms per SELECT, event loop utilization=${ELU1.utilization}`);
}

main().catch(console.error);

@daniellockyer
Copy link
Member

Hey @alex3d! 👋🏻 Looks interesting! Would you be able to rebase the PR?

@alex3d
Copy link
Author

alex3d commented Jun 7, 2022

@daniellockyer I've rebased PR and checked microbenchmark again - performance improvement stayed roughly the same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants