fix: refactor transactions to use their own event loops

[chrisrossi]

Referring to issue #426, the problem ultimately turned out to be that
we could fall out of the transaction scope and trigger a commit while
there is still work left queued on the event loop, including, in this
case, the tasklet that would eventually schedule the call to delete,
causing the delete to never actually happen.

The fix is to go ahead and consume the event loop queues before
scheduling the call to COMMIT. However, if there are other tasks
happening in parallel, this can really mess with the natural sequence of
events in ways that can cause things to blow up. (All of the
parallel_transaction tests in tests/system/test_misc.py for
instance, will fail.) The fix for that is to give each transaction its
own event loop, so that when it calls _eventloop.run prior to commit,
it is only flushing tasks that pertain to it.

Fixes #426

[chrisrossi]

Following up on the main commit message, the move to separate event loops for transactions is not without its own complications. Sometimes flow of control will get passed back to the main loop before the transaction callback, if it is a tasklet, has finished. In this case, the main loop needs to be able to poke the inner loops to keep doing work until they've finished. This is handled by adding an idle handler to the main loop, run_inner_loop, that runs one item from the inner loop and reschedules itself, or if there is no work to do, doesn't reschedule itself.

There is also a minor change in the way we're handling deciding whether to call AllocateIds. We only need to call AllocateIds if some piece of code has inserted a new entity and has asked for the returned key value before our call to Commit. If no one ever asks for the key, we don't need to call AllocateIds and can let the id be allocated during the Commit call itself. We use an idle handler to Allocate the ids, since idle handlers generally only get called when someone is waiting on something we don't have yet. Previously, if we got to Commit before the idle handler got run, we'd just take the ids that needed to be allocated out of the idle handlers queue, so there wouldn't be anything for it to do if it called after Commit. Now, since we're flushing the event loop before calling Commit, the idle will always be called before the commit, but if we're flushing the queues to prepare for a commit, we don't need to allocate ids, because we're about to call Commit. So now we call _datastore_api.prepare_to_commit right before flushing the event loop, which sets a flag that signals the idle handler that we don't need to allocate ids.

[chrisrossi]

Wasn't used.

[chrisrossi]

Sanity check.

[cguardia]

Wow, this took time to parse. Looks great.

[chrisrossi]

Pretty much took all week to figure out! Thanks!