You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Although pickling simple minimal tasks sometimes works, this a) is untested and b) fails as soon as inputs or outputs involve dynamically created types (which are quite common in smttask). Thus we need
Support for pickling tasks with dynamic types. These may be introduced via
task generators (task_generators.py)
multi-outputs (see base.TaskOutput.__init_subclass__)
Comprehensive tests
Include all types of tasks (esp. all generated types) and both single- and multi-output tasks.
Test multiprocessing where:
Tasks are passed as arguments
Tasks are returned as results
Tasks are executed in the subprocess, and their result is returned.
The main use case I see for supporting pickle is to allow parallelizing task execution with the multiprocessing module.
Although pickling simple minimal tasks sometimes works, this a) is untested and b) fails as soon as inputs or outputs involve dynamically created types (which are quite common in smttask). Thus we need
The main use case I see for supporting pickle is to allow parallelizing task execution with the multiprocessing module.
Relevant SO question: Pickling dynamically generated classes?
The text was updated successfully, but these errors were encountered: