Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Robustify Process handling and extend a bit #127

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Commits on Oct 1, 2019

  1. Robustify subprocess handling

    Python from 3.2 on support timeouts on subprocesses directly
    subprocess32 backports that to Python 2
    krono committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    ed23f59 View commit details
    Browse the repository at this point in the history
  2. Code defaults via default parameters.

    Easier to read. Also, `timeout=None` because that's what modern
    subprocess expects
    krono committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    5ac081c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d46f79c View commit details
    Browse the repository at this point in the history
  4. Robustify process handling in presence of threads

    There are many caveats, but at least Ctrl-C now works as expected.
    krono committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    5ee64a0 View commit details
    Browse the repository at this point in the history
  5. Isolate persistence writing to file to one thread

    There were out-of-order writes to the persitence file when there were
    multiple threads. This confines file-writing to a dedicated thread that
    works a Queue.
    krono committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    44fe332 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2019

  1. Configuration menu
    Copy the full SHA
    cc6b5f2 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2019

  1. make sure we have a thread

    krono committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    f3012f1 View commit details
    Browse the repository at this point in the history
  2. Add queue-based pulling parallel scheduler

    The existing parallel scheduler behaves as is
    (With common behavior lifted to a new base clase).
    
    A new parallel scheulder maintains a work queue and the workers retrieve
    directly from there.
    
    a distinction is made for py2 vs py3, to make use of the built-in thread
    pool executor when possible.
    
    No new magic so far
    krono committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    92e9701 View commit details
    Browse the repository at this point in the history
  3. fix typo

    krono committed Oct 7, 2019
    Configuration menu
    Copy the full SHA
    d404b8e View commit details
    Browse the repository at this point in the history