Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloudpickle can produce incorrect pickles #129

Closed
pitrou opened this issue Nov 2, 2017 · 2 comments
Closed

cloudpickle can produce incorrect pickles #129

pitrou opened this issue Nov 2, 2017 · 2 comments
Labels

Comments

@pitrou
Copy link
Member

pitrou commented Nov 2, 2017

>>> import multiprocessing as mp, cloudpickle, pickletools
>>> v = mp.Value('i')
>>> cloudpickle.dumps(v)
b'\x80\x02'
>>> import pickletools
>>> pickletools.dis(cloudpickle.dumps(v))
    0: \x80 PROTO      2
Traceback (most recent call last):
  File "<ipython-input-28-9e637ba398b4>", line 1, in <module>
    pickletools.dis(cloudpickle.dumps(v))
  File "/home/antoine/miniconda3/envs/dask36/lib/python3.6/pickletools.py", line 2398, in dis
    for opcode, arg, pos in genops(pickle):
  File "/home/antoine/miniconda3/envs/dask36/lib/python3.6/pickletools.py", line 2238, in _genops
    raise ValueError("pickle exhausted before seeing STOP")
ValueError: pickle exhausted before seeing STOP
@rgbkrk rgbkrk added the bug label Nov 2, 2017
@adamklein
Copy link

I've run into this as well.

The problem is runtime exceptions can be swallowed here:
https://github.com/cloudpipe/cloudpickle/blob/master/cloudpickle/cloudpickle.py#L269
in my case, it turned out it was a changing dictionary while cloudpickle iterated through it; in @pitrou's example, it's RuntimeError: Synchronized objects should only be shared between processes through inheritance Should the except block end with a raise instead?

@pitrou
Copy link
Member Author

pitrou commented Dec 13, 2017

Thanks for diagnosing this. It should be pretty easy to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants