Update installation to python 3.10, torch 2.1 (#119)

* chore: updated python and torch versions in installation instructions. Seems to work with python=3.10 torch=2.1.2+cu121

* chore: updated requirement files

* chore: updated python version and added wandb as requirement

* chore: updated README

* chore: added tox in optional dependencies

* fix: we still need to be specific about torch-geometric versions

* fix: attempt to fix that error: Run actions/setup-python@v3, Version 3.1 was not found in the local cache, Error: Version 3.1 with arch x64 not found

* chore: made dev-3.10.in recursive over main-3.10.in to take advantage of pip-compile-multi

* fix: attempted to fix cross-platform building issues (for Mac and Windows) by using pip-compile-cross-platform. NOTE: I had to remove typeguard from dev-3.10.in (but not from pyproject.toml)

* fix: removed erroneous outputs from requirements files

* fix: updated torch version in --find-links

* fix: kept pip-compile-cross-platform but removed the --hash entry (manually) to allow pip to use --find-links to find the wheels and avoid the torch-cluster installation error where it needs torch to be already be installed to build

* fix: removed misplaced --find-links

* fix: removes additional info to package specifications to avoid cross-platform crashes

* fix: removed nvidia requirements (only test on cpu)

* tox

* fix: another test

* tox

* chore: changing pip-compile-multi for pip-compile-cross-platform

* feat: added generate_requirements.sh

* fix: added shebang to generate_requirements.sh

* fix: adding --user option for pip installs inside build-and-test workflow

* Revert "fix: adding --user option for pip installs inside build-and-test workflow"

This reverts commit 1a5c37d.

* attempt: added python -m before pip install for tox tests

* fix: added tox in dev-3.10.in dependencies

* fix: adding quote to dev installation line

.github/workflows/build-and-test.yaml

@@ -12,9 +12,9 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - {name: Linux, python: '3.9', os: ubuntu-latest, tox: py39}
-          - {name: Windows, python: '3.9', os: windows-latest, tox: py39}
-          - {name: Mac, python: '3.9', os: macos-latest, tox: py39}
+          - {name: Linux, python: '3.10', os: ubuntu-latest, tox: py310}
+          - {name: Windows, python: '3.10', os: windows-latest, tox: py310}
+          - {name: Mac, python: '3.10', os: macos-latest, tox: py310}
     steps:
       - uses: actions/checkout@v3
       - uses: actions/setup-python@v3

.github/workflows/code-quality.yaml

@@ -12,9 +12,10 @@ jobs:
       - uses: actions/checkout@v3
       - uses: actions/setup-python@v3
         with:
-          python-version: 3.9
+          python-version: 3.10.13
           cache: 'pip'
           cache-dependency-path: 'requirements/*.txt'
+          architecture: 'x64'
       - name: update pip
         run: |
           pip install -U wheel

.pre-commit-config.yaml

@@ -2,7 +2,7 @@
 
 exclude: ^(docs/|.tox)
 default_language_version:
-  python: python3.9
+  python: python3.10
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.1.0

README.md

@@ -47,22 +47,29 @@ A good place to get started is with the [sEH fragment-based MOO task](src/gflown
 This package is installable as a PIP package, but since it depends on some torch-geometric package wheels, the `--find-links` arguments must be specified as well:
 
 ```bash
-pip install -e . --find-links https://data.pyg.org/whl/torch-1.13.1+cu117.html
+pip install -e . --find-links https://data.pyg.org/whl/torch-2.1.2+cu121.html
 ```
 Or for CPU use:
 
 ```bash
-pip install -e . --find-links https://data.pyg.org/whl/torch-1.13.1+cpu.html
+pip install -e . --find-links https://data.pyg.org/whl/torch-2.1.2+cpu.html
 ```
 
 To install or [depend on](https://matiascodesal.com/blog/how-use-git-repository-pip-dependency/) a specific tag, for example here `v0.0.10`, use the following scheme:
 ```bash
 pip install git+https://github.com/recursionpharma/gflownet.git@v0.0.10 --find-links ...
 ```
 
-If package dependencies seem not to work, you may need to install the exact frozen versions listed `requirements/`, i.e. `pip install -r requirements/main_3.9.txt`.
+If package dependencies seem not to work, you may need to install the exact frozen versions listed `requirements/`, i.e. `pip install -r requirements/main-3.10.txt`.
 
 ## Developing & Contributing
 
-External contributions are welcome. We use `tox` to run tests and linting, and `pre-commit` to run checks before committing.
+External contributions are welcome.
+
+To install the developers dependencies
+```
+pip install -e '.[dev]' --find-links https://data.pyg.org/whl/torch-2.1.2+cu121.html
+```
+
+We use `tox` to run tests and linting, and `pre-commit` to run checks before committing.
 To ensure that these checks pass, simply run `tox -e style` and `tox run` to run linters and tests, respectively.

generate_requirements.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Usage: ./generate_requirements.sh <ENV-NAME>  (e.g. ./generate_requirements.sh dev-3.10)
+
+# set env variable
+# to allow pip-compile-cross-platform to use pip with --find-links.
+# not entirely sure this is needed.
+export PIP_FIND_LINKS=https://data.pyg.org/whl/torch-2.1.2+cpu.html
+
+# compile the dependencies from .in files
+pip-compile-cross-platform requirements/$1.in --min-python-version 3.10 -o requirements/$1.txt
+
+# remove the hashes from the .txt files
+# this is slightly less safe in terms of reproducibility
+# (e.g. if a package was re-uploaded to PyPI with the same version)
+# but it is necessary to allow `pip install -r requirements` to use --find-links
+# in our case, without --find-links, torch-cluster often cannot find the
+# proper wheels and throws out an error `no torch module` when trying to build
+sed -i '/--hash=/d' requirements/$1.txt
+sed -i 's/\\//g' requirements/$1.txt
+
+# removes the nvidia requirements
+sed -i '/nvidia/d' requirements/$1.txt

pyproject.toml

@@ -46,21 +46,21 @@ line_length = 120
 
 [tool.black]
 line-length = 120
-target-version = ["py311"]
+target-version = ["py310"]
 
 [project]
 name = "gflownet"
 readme = "README.md"
 classifiers = ["Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3"]
 keywords = ["gflownet"]
-requires-python = ">=3.8,<3.10"
+requires-python = ">=3.10,<3.11"
 dynamic = ["version"]
 dependencies = [
-    "torch==1.13.1",
-    "torch-geometric==2.3.1",  # Pinning until we adapt the code to newer versions
-    "torch-scatter==2.1.1",
-    "torch-sparse==0.6.17",
-    "torch-cluster==1.6.1",
+    "torch==2.1.2",
+    "torch-geometric==2.4.0",
+    "torch-scatter==2.1.2",
+    "torch-sparse==0.6.18",
+    "torch-cluster==1.6.3",
     "rdkit",
     "tables",
     "scipy",
@@ -73,6 +73,7 @@ dependencies = [
     "pyro-ppl",
     "gpytorch",
     "omegaconf>=2.3",
+    "wandb",
     "pandas", # needed for QM9 and HDF5 support.
 ]
 
@@ -82,11 +83,12 @@ dev = [
     "black",
     "isort",
     "mypy",
-    "pip-compile-multi",
+    "pip-compile-cross-platform",
     "pre-commit",
     "pytest",
     "pytest-cov",
     "ruff",
+    "tox",
     "typeguard",
     "types-pkg_resources",
     # Security pin

requirements/dev-3.10.in

@@ -0,0 +1,13 @@
+-r main-3.10.in
+bandit[toml]
+black
+isort
+mypy
+pip-compile-multi
+pre-commit
+pytest
+pytest-cov
+ruff
+tox
+types-pkg_resources
+gitpython>=3.1.30

requirements/dev-3.10.txt

@@ -0,0 +1,111 @@
+#
+# This file is autogenerated by pip-compile-cross-platform
+# To update, run:
+#
+#    pip-compile-cross-platform requirements/dev-3.10.in --output-file requirements/dev-3.10.txt --min-python-version 3.10
+#
+absl-py==2.1.0 ; python_version >= "3.10" and python_version < "4.0"
+antlr4-python3-runtime==4.9.3 ; python_version >= "3.10" and python_version < "4.0"
+appdirs==1.4.4 ; python_version >= "3.10" and python_version < "4.0"
+bandit[toml]==1.7.7 ; python_version >= "3.10" and python_version < "4.0"
+black==24.2.0 ; python_version >= "3.10" and python_version < "4.0"
+blosc2==2.5.1 ; python_version >= "3.10" and python_version < "4"
+botorch==0.9.5 ; python_version >= "3.10" and python_version < "4.0"
+build==1.0.3 ; python_version >= "3.10" and python_version < "4.0"
+certifi==2024.2.2 ; python_version >= "3.10" and python_version < "4.0"
+cfgv==3.4.0 ; python_version >= "3.10" and python_version < "4.0"
+charset-normalizer==3.3.2 ; python_version >= "3.10" and python_version < "4.0"
+click==8.1.7 ; python_version >= "3.10" and python_version < "4.0"
+colorama==0.4.6 ; python_version >= "3.10" and python_version < "4.0" and (platform_system == "Windows" or sys_platform == "win32" or os_name == "nt")
+coverage[toml]==7.4.1 ; python_version >= "3.10" and python_version < "4.0"
+cvxopt==1.3.2 ; python_version >= "3.10" and python_version < "4.0"
+distlib==0.3.8 ; python_version >= "3.10" and python_version < "4.0"
+docker-pycreds==0.4.0 ; python_version >= "3.10" and python_version < "4.0"
+exceptiongroup==1.2.0 ; python_version >= "3.10" and python_version < "3.11"
+filelock==3.13.1 ; python_version >= "3.10" and python_version < "4.0"
+fsspec==2024.2.0 ; python_version >= "3.10" and python_version < "4.0"
+gitdb==4.0.11 ; python_version >= "3.10" and python_version < "4.0"
+gitpython==3.1.42 ; python_version >= "3.10" and python_version < "4.0"
+gpytorch==1.11 ; python_version >= "3.10" and python_version < "4.0"
+grpcio==1.60.1 ; python_version >= "3.10" and python_version < "4.0"
+identify==2.5.34 ; python_version >= "3.10" and python_version < "4.0"
+idna==3.6 ; python_version >= "3.10" and python_version < "4.0"
+iniconfig==2.0.0 ; python_version >= "3.10" and python_version < "4.0"
+isort==5.13.2 ; python_version >= "3.10" and python_version < "4.0"
+jaxtyping==0.2.25 ; python_version >= "3.10" and python_version < "4.0"
+jinja2==3.1.3 ; python_version >= "3.10" and python_version < "4.0"
+joblib==1.3.2 ; python_version >= "3.10" and python_version < "4.0"
+linear-operator==0.5.1 ; python_version >= "3.10" and python_version < "4.0"
+markdown-it-py==3.0.0 ; python_version >= "3.10" and python_version < "4.0"
+markdown==3.5.2 ; python_version >= "3.10" and python_version < "4.0"
+markupsafe==2.1.5 ; python_version >= "3.10" and python_version < "4.0"
+mdurl==0.1.2 ; python_version >= "3.10" and python_version < "4.0"
+mpmath==1.3.0 ; python_version >= "3.10" and python_version < "4.0"
+msgpack==1.0.7 ; python_version >= "3.10" and python_version < "4"
+multipledispatch==1.0.0 ; python_version >= "3.10" and python_version < "4.0"
+mypy-extensions==1.0.0 ; python_version >= "3.10" and python_version < "4.0"
+mypy==1.8.0 ; python_version >= "3.10" and python_version < "4.0"
+ndindex==1.8 ; python_version >= "3.10" and python_version < "4"
+networkx==3.2.1 ; python_version >= "3.10" and python_version < "4.0"
+nodeenv==1.8.0 ; python_version >= "3.10" and python_version < "4.0"
+numexpr==2.9.0 ; python_version >= "3.10" and python_version < "4.0"
+numpy==1.26.4 ; python_version >= "3.10" and python_version < "4.0"
+omegaconf==2.3.0 ; python_version >= "3.10" and python_version < "4.0"
+opt-einsum==3.3.0 ; python_version >= "3.10" and python_version < "4.0"
+packaging==23.2 ; python_version >= "3.10" and python_version < "4.0"
+pathspec==0.12.1 ; python_version >= "3.10" and python_version < "4.0"
+pbr==6.0.0 ; python_version >= "3.10" and python_version < "4.0"
+pillow==10.2.0 ; python_version >= "3.10" and python_version < "4.0"
+pip-compile-multi==2.6.3 ; python_version >= "3.10" and python_version < "4.0"
+pip-tools==7.3.0 ; python_version >= "3.10" and python_version < "4.0"
+pip==24.0 ; python_version >= "3.10" and python_version < "4.0"
+platformdirs==4.2.0 ; python_version >= "3.10" and python_version < "4.0"
+pluggy==1.4.0 ; python_version >= "3.10" and python_version < "4.0"
+pre-commit==3.6.1 ; python_version >= "3.10" and python_version < "4.0"
+protobuf==4.25.3 ; python_version >= "3.10" and python_version < "4.0"
+psutil==5.9.8 ; python_version >= "3.10" and python_version < "4.0"
+py-cpuinfo==9.0.0 ; python_version >= "3.10" and python_version < "4.0"
+pyarrow==15.0.0 ; python_version >= "3.10" and python_version < "4.0"
+pygments==2.17.2 ; python_version >= "3.10" and python_version < "4.0"
+pyparsing==3.1.1 ; python_version >= "3.10" and python_version < "4.0"
+pyproject-hooks==1.0.0 ; python_version >= "3.10" and python_version < "4.0"
+pyro-api==0.1.2 ; python_version >= "3.10" and python_version < "4.0"
+pyro-ppl==1.8.6 ; python_version >= "3.10" and python_version < "4.0"
+pytest-cov==4.1.0 ; python_version >= "3.10" and python_version < "4.0"
+pytest==8.0.0 ; python_version >= "3.10" and python_version < "4.0"
+pyyaml==6.0.1 ; python_version >= "3.10" and python_version < "4.0"
+rdkit==2023.9.5 ; python_version >= "3.10" and python_version < "4.0"
+requests==2.31.0 ; python_version >= "3.10" and python_version < "4.0"
+rich==13.7.0 ; python_version >= "3.10" and python_version < "4.0"
+ruff==0.2.1 ; python_version >= "3.10" and python_version < "4.0"
+scikit-learn==1.4.0 ; python_version >= "3.10" and python_version < "4.0"
+scipy==1.12.0 ; python_version >= "3.10" and python_version < "4.0"
+sentry-sdk==1.40.4 ; python_version >= "3.10" and python_version < "4.0"
+setproctitle==1.3.3 ; python_version >= "3.10" and python_version < "4.0"
+setuptools==69.1.0 ; python_version >= "3.10" and python_version < "4.0"
+six==1.16.0 ; python_version >= "3.10" and python_version < "4.0"
+smmap==5.0.1 ; python_version >= "3.10" and python_version < "4.0"
+stevedore==5.1.0 ; python_version >= "3.10" and python_version < "4.0"
+sympy==1.12 ; python_version >= "3.10" and python_version < "4.0"
+tables==3.9.2 ; python_version >= "3.10" and python_version < "4.0"
+tensorboard-data-server==0.7.2 ; python_version >= "3.10" and python_version < "4.0"
+tensorboard==2.16.1 ; python_version >= "3.10" and python_version < "4.0"
+tf-keras==2.15.0 ; python_version >= "3.10" and python_version < "4.0"
+threadpoolctl==3.3.0 ; python_version >= "3.10" and python_version < "4.0"
+tomli==2.0.1 ; python_full_version <= "3.11.0a6" and python_version >= "3.10"
+toposort==1.10 ; python_version >= "3.10" and python_version < "4.0"
+torch-cluster==1.6.3 ; python_version >= "3.10" and python_version < "4.0"
+torch-geometric==2.4.0 ; python_version >= "3.10" and python_version < "4.0"
+torch-scatter==2.1.2 ; python_version >= "3.10" and python_version < "4.0"
+torch-sparse==0.6.18 ; python_version >= "3.10" and python_version < "4.0"
+torch==2.1.2 ; python_version >= "3.10" and python_version < "4.0"
+tqdm==4.66.2 ; python_version >= "3.10" and python_version < "4.0"
+triton==2.1.0 ; platform_system == "Linux" and platform_machine == "x86_64" and python_version >= "3.10" and python_version < "4.0"
+typeguard==2.13.3 ; python_version >= "3.10" and python_version < "4.0"
+types-pkg-resources==0.1.3 ; python_version >= "3.10" and python_version < "4.0"
+typing-extensions==4.9.0 ; python_version >= "3.10" and python_version < "4.0"
+urllib3==2.2.0 ; python_version >= "3.10" and python_version < "4.0"
+virtualenv==20.25.0 ; python_version >= "3.10" and python_version < "4.0"
+wandb==0.16.3 ; python_version >= "3.10" and python_version < "4.0"
+werkzeug==3.0.1 ; python_version >= "3.10" and python_version < "4.0"
+wheel==0.42.0 ; python_version >= "3.10" and python_version < "4.0"