Adding more vertices to a huge graph is very slow

[luvicri]

Hi there.
II have a big problem. I need to create a big graph, with more than 1800000 vertices. It takes more than 5 minutes to create just 20k nodes(simple nodes with a simple string attribute). So i realized that the problem is the creation of nodes. When the block of code where nodes are created starts, the system slows down. Anybody can help me to solve this issue?

[clue]

Thanks for your interesting question @luvicri!

It takes more than 5 minutes to create just 20k nodes

I consider this to be a defect which should be fixed :-)

You've already mailed me some details which indicate that this is related to how we allocate the next free vertex ID (which happens to be a naive O(n) implementation). We should keep an eye on this, but this will probably be resolved once we address #131.

Your mail also suggested the workaround to explicitly pass vertext IDs to Graph::createVertex() and I've verified this indeed resolves the performance issue here.

more than 1800000 vertices

There's certainly room for improvement in this library and I'm all for addressing any performance issues. However, I'm not sure supporting millions of objects should be our main focus here.

This is a PHP library after all, which stores all objects in memory. This also means that there's currently no way to store more objects than what your system (RAM) permits. If you're looking into storing multi gigabytes worth of objects, you might be better off using some full-fledged graph database.

I hope this helps.

[clue]

You've already mailed me some details which indicate that this is related to how we allocate the next free vertex ID (which happens to be a naive O(n) implementation). We should keep an eye on this, but this will probably be resolved once we address #131.

I'll keep track of this in #131 and close this ticket for now.

[luvicri]

Yes, i switched on a graph db. But i would like to work with you to improve the library, because is really well done! I can’t work now on the library but maybe between a month…
The issues about performance are a trip for me! :)

Thank’s for you help!

Il giorno 12/ott/2015, alle ore 12:21, Christian Lück <notifications@github.com mailto:notifications@github.com> ha scritto:

Thanks for your interesting question @luvicri https://github.com/luvicri!

It takes more than 5 minutes to create just 20k nodes

I consider this to be a defect which should be fixed :-)

You've already mailed me some details which indicate that this related to how we allocate the next free vertex ID (which happens to be a naive O(n) implementation). We should keep an eye on this, but this will probably be resolved once we address #131 #131.

Your mail also suggested the workaround to explicitly pass vertext IDs to Graph::createVertex() and I've verified this indeed resolves the performance issue here.

more than 1800000 vertices

There's certainly room for improvement in this library and I'm all for addressing any performance issues. However, I'm not sure supporting millions of objects should be our main focus here.

This is a PHP library after all, which stores all objects in memory. This also means that there's currently no way to store more objects than what your system (RAM) permits. If you're looking into storing multi gigabytes worth of objects, you might be better off using some full-fledged graph database.

I hope this helps.

—
Reply to this email directly or view it on GitHub #136 (comment).