Incompatibilities with Python 3.10 for the aesara 2.6.6 and causalgraphicalmodels-0.0.4 packages.

[gekaklam]

Hello,

I've just started going through Statistical Rethinking v2, so I cloned the repo and tried to run the notebooks from chapter 2 to 6. I've encountered the following errors, for which I've identified the root causes and some possible solutions. However since I don't know how these would affect future chapters I didn't issue a pull request.

Installation info

I'm on Ubuntu 20.04 and I installed using the recommended command: conda env create -f environment_v4.yml then added the kernel with python -m ipykernel install --user --name stat-rethink2-pymc4 --display-name "Python 3.10 (stat-rethink2-pymc4)" (even though this doesn't exist in the README, I assume it should be as in pymc3 version).

Issue 01: Failure to import scipy.stats as stats

Already from chapter 02 when I tried to run the notebook I got the following error

---------------------------------------------------------------------------
NoSectionError                            Traceback (most recent call last)
File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/configparser.py:237, in AesaraConfigParser.fetch_val_for_key(self, key, delete_key)
    236 try:
--> 237     return self._aesara_cfg.get(section, option)
    238 except InterpolationError:

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/configparser.py:782, in RawConfigParser.get(self, section, option, raw, vars, fallback)
    781 try:
--> 782     d = self._unify_values(section, vars)
    783 except NoSectionError:

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/configparser.py:1153, in RawConfigParser._unify_values(self, section, vars)
   1152     if section != self.default_section:
-> 1153         raise NoSectionError(section) from None
   1154 # Update with the entry specific variables

NoSectionError: No section: 'blas'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/configparser.py:353, in ConfigParam.__get__(self, cls, type_, delete_key)
    352 try:
--> 353     val_str = cls.fetch_val_for_key(self.name, delete_key=delete_key)
    354     self.is_default = False

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/configparser.py:241, in AesaraConfigParser.fetch_val_for_key(self, key, delete_key)
    240 except (NoOptionError, NoSectionError):
--> 241     raise KeyError(key)

KeyError: 'blas__ldflags'

During handling of the above exception, another exception occurred:

ModuleNotFoundError                       Traceback (most recent call last)
Input In [1], in <cell line: 4>()
      2 import matplotlib.pyplot as plt
      3 import numpy as np
----> 4 import pymc as pm
      5 import scipy.stats as stats

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/pymc/__init__.py:50, in <module>
     45     augmented = f"{augmented} -fno-unwind-tables -fno-asynchronous-unwind-tables"
     47     aesara.config.gcc__cxxflags = augmented
---> 50 __set_compiler_flags()
     52 from pymc import gp, ode, sampling
     53 from pymc.aesaraf import *

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/pymc/__init__.py:33, in __set_compiler_flags()
     31 def __set_compiler_flags():
     32     # Workarounds for Aesara compiler problems on various platforms
---> 33     import aesara
     35     current = aesara.config.gcc__cxxflags
     36     augmented = f"{current} -Wno-c++11-narrowing"

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/__init__.py:126, in <module>
    122     return as_tensor_variable(x, **kwargs)
    125 # isort: off
--> 126 from aesara import scalar, tensor
    127 from aesara.compile import (
    128     In,
    129     Mode,
   (...)
    135     shared,
    136 )
    137 from aesara.compile.function import function, function_dump

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/tensor/__init__.py:98, in <module>
     96 # adds shared-variable constructors
     97 from aesara.tensor import sharedvar  # noqa
---> 98 from aesara.tensor import (  # noqa
     99     basic_opt,
    100     blas,
    101     blas_c,
    102     blas_scipy,
    103     nnet,
    104     opt_uncanonicalize,
    105     subtensor_opt,
    106     xlogx,
    107 )
    110 # isort: off
    111 from aesara.tensor import linalg  # noqa

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/tensor/blas.py:165, in <module>
    163 from aesara.tensor import basic as at
    164 from aesara.tensor.basic_opt import local_dimshuffle_lift
--> 165 from aesara.tensor.blas_headers import blas_header_text, blas_header_version
    166 from aesara.tensor.elemwise import DimShuffle, Elemwise
    167 from aesara.tensor.exceptions import NotScalarConstantError

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/tensor/blas_headers.py:1015, in <module>
    997             header += textwrap.dedent(
    998                 """\
    999                     static float sdot_(int* Nx, float* x, int* Sx, float* y, int* Sy)
   (...)
   1009                     """
   1010             )
   1012     return header + blas_code
-> 1015 if not config.blas__ldflags:
   1016     _logger.warning("Using NumPy C-API based implementation for BLAS functions.")
   1019 def mkl_threads_text():

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/configparser.py:357, in ConfigParam.__get__(self, cls, type_, delete_key)
    355 except KeyError:
    356     if callable(self.default):
--> 357         val_str = self.default()
    358     else:
    359         val_str = self.default

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/aesara/link/c/cmodule.py:2710, in default_blas_ldflags()
   2699 warn_record = []
   2700 try:
   2701     # We do this import only here, as in some setup, if we
   2702     # just import aesara and exit, with the import at global
   (...)
   2708     # This happen with Python 2.7.3 |EPD 7.3-1 and numpy 1.8.1
   2709     # isort: off
-> 2710     import numpy.distutils.system_info  # noqa
   2712     # We need to catch warnings as in some cases NumPy print
   2713     # stuff that we don't want the user to see.
   2714     # I'm not able to remove all printed stuff
   2715     with warnings.catch_warnings(record=True):

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/numpy/distutils/system_info.py:196, in <module>
    191 from numpy.distutils.exec_command import (
    192     find_executable, filepath_from_subprocess_output,
    193     )
    194 from numpy.distutils.misc_util import (is_sequence, is_string,
    195                                        get_shared_lib_extension)
--> 196 from numpy.distutils.command.config import config as cmd_config
    197 from numpy.distutils import customized_ccompiler as _customized_ccompiler
    198 from numpy.distutils import _shell_utils

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/numpy/distutils/command/config.py:19, in <module>
     17 import distutils
     18 from numpy.distutils.exec_command import filepath_from_subprocess_output
---> 19 from numpy.distutils.mingw32ccompiler import generate_manifest
     20 from numpy.distutils.command.autodist import (check_gcc_function_attribute,
     21                                               check_gcc_function_attribute_with_intrinsics,
     22                                               check_gcc_variable_attribute,
   (...)
     25                                               check_restrict,
     26                                               check_compiler_gcc)
     28 LANG_EXT['f77'] = '.f'

File ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/numpy/distutils/mingw32ccompiler.py:28, in <module>
     26 import distutils.cygwinccompiler
     27 from distutils.unixccompiler import UnixCCompiler
---> 28 from distutils.msvccompiler import get_build_version as get_build_msvc_version
     29 from distutils.errors import UnknownFileError
     30 from numpy.distutils.misc_util import (msvc_runtime_library,
     31                                        msvc_runtime_version,
     32                                        msvc_runtime_major,
     33                                        get_build_architecture)

ModuleNotFoundError: No module named 'distutils.msvccompiler'

Issue 01: Possible Solution.

I assume this is because Python 3.10 deprecated distutils. This has been fixed on later versions of the aesara library, but version 2.6.6 which is fixed at the env, still doesn't have the change. After removing the fixed package number of the environment file on all packages except PyMC, and recreating the environment the issue disappeared.. However, I don't know if this would break anything else on the other chapters, due to dependencies.

Issue 02: Failure when importing causalgraphicalmodels

In chapter 05, we have from causalgraphicalmodels import CausalGraphicalModel. When I try to execute this I'm getting ImportError: cannot import name 'Iterable' from 'collections'. As shown on the link the solution is to modify the import statement for the causalgraphicalmodels package.

In my case the file was ~/anaconda3/envs/stat-rethink2-pymc_v4/lib/python3.10/site-packages/causalgraphicalmodels/cgm.py and I had to change:

from collections import Iterable

to

try:
    # Python <= 3.9
    from collections import Iterable
except ImportError:
    # Python > 3.9
    from collections.abc import Iterable

I know that this is another packages issue, but I'm putting it as a ref in case anyone else has the same issues.