Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't export class with decorated method #13

Closed
v1lev opened this issue Apr 14, 2019 · 4 comments
Closed

Can't export class with decorated method #13

v1lev opened this issue Apr 14, 2019 · 4 comments

Comments

@v1lev
Copy link

v1lev commented Apr 14, 2019

I tried to save class definition with one decorated method using dill and got the following error:
NotImplementedError: object proxy must define __reduce_ex__()
Traceback
Sample code:

import dill
from wrapt_timeout_decorator import timeout


class Test:
    @staticmethod
    @timeout(5)
    def foo():
        pass


with open('test.pkl', 'wb') as f:
    dill.dump(Test, f)
@bitranox
Copy link
Owner

bitranox commented Apr 14, 2019

which os ?
which python version ?

and did You notice the Readme :

under Windows classes and functions in the main context can not be pickled, You need to put the decorated Classes and functions into another module. In general (especially for windows) , the main() program should not have anything but the main function, the real thing should happen in the modules. I am also used to put all settings or configurations in a different file - so all processes or threads can access them (and also to keep them in one place together, not to forget typing hints and name completion in Your favorite editor)

please check and let me know

@v1lev
Copy link
Author

v1lev commented Apr 14, 2019

OS Windows 10.0.17134.0
Python 3.7.3 (or 3.6.8)
I missed readme note for the first time. Yes, it works if class is defined in another module, but it also add a requirement to have that module for deserialization. The goal is to define a class somewhere, save it with dill and load in another machine.
I can successfully serialize the same example class using timeout_decorator module instead and load it in virtual machine.

@bitranox
Copy link
Owner

bitranox commented Apr 15, 2019

ok, I think thats not a big downside.
I personally would make a module like "remote_class_imports.py" and put all those pickled imported classes there.
Really no need to have it in the main context.

however, it is an issue of the "wrapt" package.

@bitranox
Copy link
Owner

it is indeed an issue for wrapt, see :

https://wrapt.readthedocs.io/en/latest/changes.html

If pickle.dump() or pickle.dumps() is used on an instance of the ObjectProxy class, a NotImplementedError exception is raised, with a message indicating that the object proxy must implement the reduce_ex() method. This is in place of the default TypeError exception with message indicating a pickle error.

I will check if I can monkey-patch it.

another note on Your code :
You need to put the decorator BEFORE @staticmethod

see : https://wrapt.readthedocs.io/en/latest/decorators.html#decorating-static-methods

so to wrap it up :

# main.py

import dill
import remote_classes

main():
    with open('test.pkl', 'wb') as f:
        dill.dump(remote_classes.Test, f)


if __name__ == '__main__':    # always shield the main context in Windows !!!
    main()

# remote_classes.py

from wrapt_timeout_decorator import timeout

class Test(object):       # avoid old style classes
    @timeout(5)           # decorator should be set BEFORE @staticmethod
    @staticmethod
    def foo():
        pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants