ImportError with NeighborSampler in PyG: requires 'pyg-lib' or 'torch-sparse' even when 'pyg-lib' is installed

[KamilZPWr]

Hello community,

I'm encountering an issue when running the LinkNeighborLoader in the context of the torch_geometric library. My code works seamlessly with torch-sparse, but when I transition to using pyg-lib, I am presented with the following error:

File python3.10/site-packages/torch_geometric/sampler/neighbor_sampler.py:276, in NeighborSampler._sample(self, seed, seed_time, **kwargs)
    273     num_sampled_nodes = num_sampled_edges = None
    275 else:
--> 276     raise ImportError(f"'{self.__class__.__name__}' requires "
    277                       f"either 'pyg-lib' or 'torch-sparse'")
    279 if num_sampled_edges is not None:
    280     num_sampled_edges = remap_keys(
    281         num_sampled_edges,
    282         self.to_edge_type,
    283     )

ImportError: 'NeighborSampler' requires either 'pyg-lib' or 'torch-sparse'

This is perplexing because I am certain that pyg-lib is installed on my virtual environment. To validate this, I've successfully imported it manually in the python console running on the same venv.

Has anyone encountered a similar issue or have any insights into what might be going wrong? Additionally, I tested with a GPU-based configuration and encountered the same problem.

Thanks in advance for any assistance or insights!

Environment details:

• torch_geometric 2.4.0,
• pyg-lib 0.2.0,
• 1.13.1+cpu,
• Ubuntu 22.04.2 LTS

[wsad1]

Sorry about this.
Could you try creating a new venv and installing from scratch via

pip install torch_geometric

# Optional dependencies:
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-1.13.0+cpu.html

More installation options here.

[KamilZPWr]

Nope, didn't help. Solution and the real issue below.

[rusty1s]

What does

import torch_geometric
print(torch_geometric.typing.WITH_PYG_LIB)

return?

[asmit203]

False

[rusty1s]

Mh, okay, but

import pyg_lib

works for you? Is there any warning shown when doing

import torch_geometric

[KamilZPWr]

So when I did that, I was getting True. I found the issue and I am gonna post it below.

[Muyiiiii]

I find the solution ！！！Just install pyg cluster sparse and so on before the torch_geometric !!!

[KamilZPWr]

I executed my code on a Linux-based virtual machine, while my local system runs on Windows. It slipped my attention that there was a slight version discrepancy in PyG between the two setups. The error I encountered on the virtual machine stemmed from the fact that the 'if' condition was expanded. Here's the code for reference:

# TODO Support induced subgraph sampling in `pyg-lib`.
if (torch_geometric.typing.WITH_PYG_LIB and self.subgraph_type != SubgraphType.induced):

Given that my graph was of the 'induced' type, this led to the error. My confusion was amplified because, in my local version of PyG on Windows, only the typing check was present in this line.

I appreciate your assistance in resolving this!

[WHU-LX]

Hi, I also had this problem but today I have solved it! Hope it can help u well!

1- Conda create a new environment, I create 'PyG' environment with Python 3.9.
2- You should install PYTORCH first, as reconmmended in page 'https://pytorch.org/get-started/previous-versions/', conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 cpuonly -c pytorch.
3- Then, install those packages IN ORDER: ①torch_scatter②torch_sparse③torch_cluster④torch_spline_conv(Step3)⑤torch-geometric(Step4)，You can download the wheel from website 'https://data.pyg.org/whl/', and the version should match your pytorch version, for me, it's pytorch==2.1.0.
4- Finally, pip install torch_geometric, as recommended in 'https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html', because I use CPU, it will be different if u use GPU, CUDA, maybe, so, double check it.
5- If u use jupyter lab, remenber pip install ipykernel to change your environment.

Finally, it works. Wish it will help you too! Good luck!

Wuhan University, Gou Zhonghua's Lab.

[Joe-Hall-Lee]

Thank you!

[meredith-zeng]

It works for me!! Thanks!

[hmam9090]

I followed the steps but still not working for me!

[CottePutman]

It worked, apperciated!

[ankursworld]

Hello KamilZPWr,
Thank you for your help. I am posting as I faced the same issue albeit with different versions - in case someone else has the same problems. Skip ahead to outcome if problem is not relevant.

Context: Environment: Windows, CPU, Python 3.12, using pip.
Using relbench[full] alongwith Torch and PyG. I followed the instructions on the websites (Torch, then PyG, then others like torch-sparse)
Problems:
I got the error "ImportError: 'NeighborSampler' requires either 'pyg-lib' or 'torch-sparse'"
I tried installing pyg-lib and ran into issue that it was not available on Windows (github table shows it is on Windows, but it is not)
My installation showed torch_sparse as installed but importing it failed. This is the key problem to solve.

My guess is that it was version and install sequence related (thanks to instructions above).

Solution

1. I re-installed Pytorch to 2.3.1+cpu (not 2.4 that was the latest)
2. I downloaded wheels the closest wheels from https://data.pyg.org/whl/torch-2.3.0%2Bcpu.html for pyg_lib, torch cluster/scatter/sparse/spline
3. I installed in the exact same sequence as listed by KamilZPWr (scatter etc. in point above) - using the wheels above.
4. The list didnt mention pyg. But I installed pyg after others using the wheel downloaded from link above.
5. I finally installed pyg using the direct command on website "pip install torch_geometric"
6. Standard stuff (Jupyter/ipykernel..)
7. I also installed pytorch frame. I did that without specifying version.

And it worked.
Hope it is helpful to others in same situation as me.

[devinbost]

I was hitting this issue, and installing all the dependencies as suggested in this thread didn't resolve the issue (despite it taking hours to run to build all the wheels).
I tried instead to use pyg-lib, and it's crashing my session without a strack trace. Reproduction case is here: https://colab.research.google.com/drive/1LRa5ifkzx8kiAu7neZGwDjbOXXQxw07E?usp=sharing

I noticed it supports Torch 2.2.0, not Torch 2.2.1 (which is what comes with Google Colab), but it's not clear to me if that should be an issue.
I've tried every other workaround at this point that I can find, so I don't know if I'll be able to use PyG in this project if I can't get this working.

[alarso16]

I had the same issue, but when I switched from the L4 GPU to the T4 GPU, it worked.

[rusty1s]

Thanks for sharing the Collab link. Note that sentence-transformers reinstalls PyTorch for whatever reason (with a whole new NVIDIA CUDA runtime on top). The installation of PyG is fast and works flawlessly on Collab the last time I checked.

There is indeed an issue with METIS partitioning. You can fix this for now via

!pip install torch-sparse -f https://data.pyg.org/whl/torch-2.2.1+cu121.html

I'll look into it.

[rusty1s]

Here is a short Collab that runs fine with PyG+pyg-lib+METIS: https://colab.research.google.com/drive/1XAjcjRHrSR_ypCk_feIWFbcBKyT4Lirs?usp=sharing&authuser=2&pli=1#scrollTo=wLxawwYlgjDb

[devinbost]

Much appreciated!

[Muyiiiii]

I find the solution ！！！Just install pyg cluster sparse and so on before the torch_geometric!!!

[nelsonaloysio]

FWIW, this issue was solved for me on a conda env using CUDA 12.2 by simply installing pyg-lib for cu121 from PyPI:

pip install pyg-lib -f https://data.pyg.org/whl/torch-${TORCH}+${CUDA}.html

Further details on the repository: pygteam/pyg-lib.

[lukalafaye]

Hello,

I had the same issue, using Google Collab here. Managed to make pyg work with Pytorch 2.4.0 and Cuda 12.4. Here's my code:

Install Cuda 12.4 for ubuntu 22.04 (look up instructions for your specific OS) on nvidia website

wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda-repo-ubuntu2204-12-4-local_12.4.0-550.54.14-1_amd64.deb
dpkg -i cuda-repo-ubuntu2204-12-4-local_12.4.0-550.54.14-1_amd64.deb
cp /var/cuda-repo-ubuntu2204-12-4-local/cuda-*-keyring.gpg /usr/share/keyrings/
apt-get update
apt-get install -y cuda-12-4 cuda-toolkit-12-4
export CUDA_PATH=/usr/local/cuda-12.4/
nvcc --version

Install pyg with:

pip install torch-geometric \
  torch-sparse \
  torch-scatter \
  torch-cluster \
  torch-cluster \
  pyg-lib \
  -f https://data.pyg.org/whl/torch-2.4.0+cu124.html

If you already have any of these librairies installed already, make sure you pip uninstall them before running my code...

[AKookani]

Thank you so much. After two hours of going through the internet and asking chat GPT to see how to fix the dependencies, your comment solved everything like a piece of cake :)

[matteotolloso]

This works! Thank you.

[ShenShan123]

I also encounter this problem. However, I got a True when using print(torch_geometric.typing.WITH_PYG_LIB). Then I jumped into the "torch_geometric/sampler/neighbor_sampler.py", line 452, in _sample, and found the if condition is determined by the subgraph_type in the LinkNeighborLoader. So make sure you use the correct "subgraph_type" argument to eliminate the error. Besides, in the older version of torch_geometric, there is no "subgraph_type" but a "directed" argument, which is deprecated.