Skip to content

Commit

Permalink
Merge pull request #245 from djarecka/satra-enh-dask
Browse files Browse the repository at this point in the history
adding comments on Dask worker [skip ci]
  • Loading branch information
djarecka authored May 5, 2020
2 parents 82c7886 + 939b422 commit acc9010
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 2 deletions.
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ Feature list:
2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
3. `splitter` and `combiner` provides many ways of compressing complex loop semantics
4. Cached execution with support for a global cache across dataflows and users
5. Distributed execution (presently via ConcurrentFutures and SLURM, more on their way)
5. Distributed execution, presently via ConcurrentFutures, SLURM,
and Dask (this is an experimental implementation with limited testing)

[[API Documentation](https://nipype.github.io/pydra/)]

Expand All @@ -40,6 +41,14 @@ cd pydra
pip install -e .[dev]
```

If you want to test execution with Dask:
```
git clone git@github.com:nipype/pydra.git
cd pydra
pip install -e .[dask]
```


It is also useful to install pre-commit:

```
Expand Down
4 changes: 3 additions & 1 deletion pydra/engine/workers.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,9 @@ async def _verify_exit_code(self, jobid):


class DaskWorker(Worker):
"""A worker to execute in parallel using Python's concurrent futures."""
""" A worker to execute in parallel using Dask.distributed.
This is an experimental implementation with limited testing.
"""

def __init__(self, **kwargs):
"""Initialize Worker."""
Expand Down

0 comments on commit acc9010

Please sign in to comment.