Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve graph io #182

Open
fguiotte opened this issue Feb 26, 2020 · 2 comments
Open

Improve graph io #182

fguiotte opened this issue Feb 26, 2020 · 2 comments
Labels
feature request Feature request

Comments

@fguiotte
Copy link
Contributor

Pink graph io is time consuming compared to tree io (for the same image) :

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   25.021   25.021   25.021   25.021 {built-in method higra.higram._read_graph_pink}
        1    1.480    1.480    1.480    1.480 {built-in method higra.higram._read_tree}

That would be nice to offer a binary save alternative.

@PerretB
Copy link
Member

PerretB commented Feb 26, 2020

Yes, pinkio is convenient (and easy to implement) but clearly very inefficient, both from a memory and time point of view: it was written essentially to offer a backward compatibility with the internal tools we were using before. I don't have any knowledge about standard binary graph format (if any, the one that I found where all text based) and any suggestion is welcome.

As a homemade solution, I think that we can just store the source and target arrays obtained from UndirectedGraph.edge_list and reconstruct it with UndirectedGraph.add_edges. Then, in Python, I would use a hdf5 file (with H5py) to store the arrays. The hdf5 format is very efficient and it would be easy to also store vertex/edges attributes in the same file.

@PerretB PerretB added the feature request Feature request label Feb 26, 2020
@PerretB
Copy link
Member

PerretB commented Oct 17, 2022

Note that graphs are now picklable which should be quite efficient but is not a long term storage solution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Feature request
Projects
None yet
Development

No branches or pull requests

2 participants