Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change from pickle to hdf5 #36

Open
pdh21 opened this issue Mar 15, 2017 · 3 comments
Open

Change from pickle to hdf5 #36

pdh21 opened this issue Mar 15, 2017 · 3 comments

Comments

@pdh21
Copy link
Member

pdh21 commented Mar 15, 2017

Saving xid+ classes as pickles is not a good idea. Would be better to move to something like hdf5. This is going to require some serious thought. Input from @yannick1974 would probably be a good idea

@ivvv
Copy link

ivvv commented Mar 15, 2017

I've read this blog on HDF5. I have also considered how to save python classes, avoiding pickles, but nothing obvious. JSON maybe, but not sure what are the limitations.

@pdh21
Copy link
Member Author

pdh21 commented Mar 15, 2017

Hi @ivvv I had read exactly the same blog but it seemed to raise more questions then answers. I will definitely take a look at JSON!

@yannick1974
Copy link
Member

You won't directly save the classes to JSON (nor HDF5) like you do with pickle (or dill). You can only serialise the data from your class to a file, an implement a way to rebuild you class (in fact, we are talking about instances) from the data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants