Cross-platform linux-64 solve failing on a Mac M2

[maximlt]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

I'm on a Mac M2 and running this command to solve on linux-64 fails at solving:

CONDA_SUBDIR=linux-64 conda create --override-channels --yes --dry-run --channel defaults -n testsubdir python=3.10 "scipy"

With:

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides __glibc >=2.17 needed by libgcc-ng-11.2.0-h1234567_0

Could not solve for environment specs
The following packages are incompatible
├─ python 3.10**  is installable with the potential options
│  ├─ python [2.7.18|3.10.10|...|3.9.18] would require
│  │  ├─ libgcc-ng >=11.2.0 , which requires
│  │  │  └─ __glibc >=2.17 , which is missing on the system;
│  │  └─ libstdcxx-ng >=11.2.0 , which requires
│  │     └─ __glibc >=2.17 , which is missing on the system;
│  └─ python [3.10.0|3.10.3|3.10.4], which can be installed;
└─ scipy is not installable because there are no viable options
   ├─ scipy [0.19.1|1.0.0|...|1.2.1] would require
   │  └─ python >=2.7,<2.8.0a0  but there are no viable options
   │     ├─ python [2.7.13|2.7.14|...|2.7.18] conflicts with any installable versions previously reported;
   │     └─ python [2.7.18|3.10.10|...|3.9.18], which cannot be installed (as previously explained);
   ├─ scipy [0.19.1|1.0.0|1.0.1|1.1.0] would require
   │  └─ python >=3.5,<3.6.0a0 , which conflicts with any installable versions previously reported;
   ├─ scipy [0.19.1|1.0.0|...|1.5.2] would require
   │  └─ python >=3.6,<3.7.0a0 , which conflicts with any installable versions previously reported;
   ├─ scipy [1.1.0|1.2.0|...|1.7.3] would require
   │  └─ python >=3.7,<3.8.0a0  but there are no viable options
   │     ├─ python [2.7.18|3.10.10|...|3.9.18], which cannot be installed (as previously explained);
   │     └─ python [3.7.0|3.7.1|...|3.7.9] conflicts with any installable versions previously reported;
   ├─ scipy [1.10.0|1.10.1|...|1.9.3] would require
   │  └─ libstdcxx-ng >=11.2.0 , which cannot be installed (as previously explained);
   ├─ scipy [1.3.1|1.3.2|...|1.7.3] would require
   │  └─ python >=3.8,<3.9.0a0  but there are no viable options
   │     ├─ python [2.7.18|3.10.10|...|3.9.18], which cannot be installed (as previously explained);
   │     └─ python [3.8.0|3.8.1|...|3.8.8] conflicts with any installable versions previously reported;
   ├─ scipy [1.5.2|1.6.0|...|1.7.3] would require
   │  └─ python >=3.9,<3.10.0a0  but there are no viable options
   │     ├─ python [2.7.18|3.10.10|...|3.9.18], which cannot be installed (as previously explained);
   │     └─ python [3.9.0|3.9.1|...|3.9.7] conflicts with any installable versions previously reported;
   └─ scipy 1.7.3 would require
      ├─ libgfortran-ng with the potential options
      │  ├─ libgfortran-ng [11.2.0|7.5.0|8.4.0|9.3.0] would require
      │  │  └─ __glibc >=2.17 , which is missing on the system;
      │  └─ libgfortran-ng [5.4.0|7.2.0|7.3.0|8.2.0], which can be installed;
      └─ libgfortran4 >=7.5.0 , which requires
         └─ libgfortran-ng 7.5.0 *_17, which conflicts with any installable versions previously reported.

cc @jezdez

Conda Info

active environment : testaproject
    active env location : /Users/mliquet/miniconda3/envs/testaproject
            shell level : 1
       user config file : /Users/mliquet/.condarc
 populated config files : /Users/mliquet/.condarc
          conda version : 23.9.0
    conda-build version : not installed
         python version : 3.10.13.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.5.2=0
                          __unix=0=0
       base environment : /Users/mliquet/miniconda3  (writable)
      conda av data dir : /Users/mliquet/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/mliquet/miniconda3/pkgs
                          /Users/mliquet/.conda/pkgs
       envs directories : /Users/mliquet/miniconda3/envs
                          /Users/mliquet/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.9.0 requests/2.31.0 CPython/3.10.13 Darwin/22.6.0 OSX/13.5.2 solver/libmamba conda-libmamba-solver/23.9.1 libmambapy/1.5.1
                UID:GID : 502:20
             netrc file : None
           offline mode : False

Conda Config

==> /Users/mliquet/.condarc <==
auto_activate_base: False
solver: libmamba

Conda list

No response

Additional Context

No response

[isuruf]

It does not work with classic solver as well right?

[maximlt]

Right.

[jaimergp]

Yes, it's a known limitation. To emulate a different OS you also have to inject the virtual packages. Try this:

CONDA_OVERRIDE_GLIBC=2.17 CONDA_SUBDIR=linux-64 conda create --override-channels --yes --dry-run --channel defaults -n testsubdir python=3.10 "scipy"

Output

Retrieving notices: ...working... done
Channels:
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/jrodriguez/.local/anaconda/envs/testsubdir

  added / updated specs:
    - python=3.10
    - scipy


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    _libgcc_mutex-0.1          |             main           3 KB  defaults
    _openmp_mutex-5.1          |            1_gnu          21 KB  defaults
    blas-1.0                   |              mkl           6 KB  defaults
    bzip2-1.0.8                |       h7b6447c_0          78 KB  defaults
    ca-certificates-2023.08.22 |       h06a4308_0         123 KB  defaults
    intel-openmp-2023.1.0      |   hdb19cb5_46305        17.1 MB  defaults
    ld_impl_linux-64-2.38      |       h1181459_1         654 KB  defaults
    libffi-3.4.4               |       h6a678d5_0         142 KB  defaults
    libgcc-ng-11.2.0           |       h1234567_1         5.3 MB  defaults
    libgfortran-ng-11.2.0      |       h00389a5_1          20 KB  defaults
    libgfortran5-11.2.0        |       h1234567_1         2.0 MB  defaults
    libgomp-11.2.0             |       h1234567_1         474 KB  defaults
    libstdcxx-ng-11.2.0        |       h1234567_1         4.7 MB  defaults
    libuuid-1.41.5             |       h5eee18b_0          27 KB  defaults
    mkl-2023.1.0               |   h213fc3f_46343       171.6 MB  defaults
    mkl-service-2.4.0          |  py310h5eee18b_1          54 KB  defaults
    mkl_fft-1.3.8              |  py310h5eee18b_0         216 KB  defaults
    mkl_random-1.2.4           |  py310hdb19cb5_0         312 KB  defaults
    ncurses-6.4                |       h6a678d5_0         914 KB  defaults
    numpy-1.26.0               |  py310h5f9d8c6_0          10 KB  defaults
    numpy-base-1.26.0          |  py310hb5e798b_0         7.1 MB  defaults
    openssl-3.0.12             |       h7f8727e_0         5.2 MB  defaults
    python-3.10.13             |       h955ad1f_0        26.8 MB  defaults
    readline-8.2               |       h5eee18b_0         357 KB  defaults
    scipy-1.11.3               |  py310h5f9d8c6_0        20.4 MB  defaults
    sqlite-3.41.2              |       h5eee18b_0         1.2 MB  defaults
    tbb-2021.8.0               |       hdb19cb5_0         1.6 MB  defaults
    tk-8.6.12                  |       h1ccaba5_0         3.0 MB  defaults
    tzdata-2023c               |       h04d1e81_0         116 KB  defaults
    xz-5.4.2                   |       h5eee18b_0         642 KB  defaults
    zlib-1.2.13                |       h5eee18b_0         103 KB  defaults
    ------------------------------------------------------------
                                           Total:       270.2 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  blas               pkgs/main/linux-64::blas-1.0-mkl 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2023.08.22-h06a4308_0 
  intel-openmp       pkgs/main/linux-64::intel-openmp-2023.1.0-hdb19cb5_46305 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_0 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgfortran-ng     pkgs/main/linux-64::libgfortran-ng-11.2.0-h00389a5_1 
  libgfortran5       pkgs/main/linux-64::libgfortran5-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0 
  mkl                pkgs/main/linux-64::mkl-2023.1.0-h213fc3f_46343 
  mkl-service        pkgs/main/linux-64::mkl-service-2.4.0-py310h5eee18b_1 
  mkl_fft            pkgs/main/linux-64::mkl_fft-1.3.8-py310h5eee18b_0 
  mkl_random         pkgs/main/linux-64::mkl_random-1.2.4-py310hdb19cb5_0 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  numpy              pkgs/main/linux-64::numpy-1.26.0-py310h5f9d8c6_0 
  numpy-base         pkgs/main/linux-64::numpy-base-1.26.0-py310hb5e798b_0 
  openssl            pkgs/main/linux-64::openssl-3.0.12-h7f8727e_0 
  python             pkgs/main/linux-64::python-3.10.13-h955ad1f_0 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  scipy              pkgs/main/linux-64::scipy-1.11.3-py310h5f9d8c6_0 
  sqlite             pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0 
  tbb                pkgs/main/linux-64::tbb-2021.8.0-hdb19cb5_0 
  tk                 pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 
  tzdata             pkgs/main/noarch::tzdata-2023c-h04d1e81_0 
  xz                 pkgs/main/linux-64::xz-5.4.2-h5eee18b_0 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0 



DryRunExit: Dry run. Exiting.

[maximlt]

Is there any way for conda/libmamba to provide a better error message in that case? That would help a lot.

[jaimergp]

We could detect whether the requested --platform is same OS as native (e.g. osx-arm64 vs osx-64) or not (osx-64 vs linux-64), and in that case print a warning before the solve. This part belongs on conda/conda.

We could also decorate the unsolvable error message with some tip if we detect that we are solving for non-native.

[maximlt]

Anything would help! Should I open an issue on conda/conda then?

[jaimergp]

Sorry I dropped the ball on this one.

Anything would help! Should I open an issue on conda/conda then?

For the first part yes, but we'll need to see whether the conda team finds that useful. I think it's more likely we tackle the second part here, if you ask me.