[FEA]: nx-cugraph should intercept built-in constructors like from_pandas_edgelist
if NETWORKX_BACKEND_PRIORITY=cugraph
#4456
Labels
feature request
New feature or request
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
High
Please provide a clear description of problem this feature solves
When we use nx-cugraph, we currently need to create the NetworkX graph on the CPU regardless of whether every algorithm we intend to use is supported by the cuGraph backend. As a result, we pay a non-trivial performance penalty converting between CPU and GPU graphs.
The new caching mechanism configurable via
CACHE_CONVERTED_GRAPH=True
was designed to address this problem, making it possible to only pay this cost once per graph if you're going to run multiple algorithms.But it would be great to avoid this cost in the first place by dispatching on the graph construction operators in addition to the algorithms. In the example below, we spend significant time in
from_pandas_edgelist
and_convert_graph
(the latter of which is only a one-time cost if we use caching).If I've already committed to using the cuGraph backend as the top priority backend, I'd ideally just create the graph on the GPU and only pay the CPU/GPU conversion cost if I need to fallback to the CPU.
But cuGraph supports
from_pandas_edgelist
and it's much faster (100ms vs 8s in this case):Describe your ideal solution
The following code should dispatch to the cuGraph backend for
from_pandas_edgelist
in addition topagerank
.Describe any alternatives you have considered
No response
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: