Share with workers a private copy of data structures set by main thread, stored on the namespace
(ns
) object
#252
Labels
enhancement
something nice to have but it is not neither critical nor urgent
Describe the feature you'd like
The
namespace
orns
object is used to communicate or share data structures between the main thread and the workers and between the workers.These data structures need to be created by the
sharer
in order to be, well, really shared and if they are mutable, they must also be protected with a kind of a lock (RLock
), also created bysharer
.This allows a communication back and forward between the main thread and the workers and between the workers.
But in some cases it is needed something simpler: a main thread share information with its worker in one way fashion and the workers don't need to communicate between themselves. This scenario makes a fully N-to-N shared data structure and the requirement of a lock totally unnecessary.
In this case
byexample
could allow to create non-shared data-structures like plainlist
,dict
orint
in the main thread and make them visible in the workers as private copies.To ensure that this will work in
threading
and inmultiprocessing
environments, the private objects must be pickle-able and the copy process should be(take main's object)->pickle->(get the bytes)->unpickle->(get the copy object)
Additional context (optional)
The documentation of
byexample
does not explain what happen ifns
is used to store objects not created bysharer
. In principle we could forbid them or we could make it private. In any case it may break some 3rd party plugin but the impact should be quite low.The text was updated successfully, but these errors were encountered: