Help installing on Win64 with MSYS2 (mingw64 compilers) #244
Comments
|
If I am installing using setup.py, then I get a different problem. |
|
Hmm... Your first post was because the HDF5 library + headers were not available. Your second post is because the I don't have a windows box to test this with. So I can't be of much help. I guess the fortran problem could be googleable? |
|
Dear Nick, Thank you for your comments. After installing
the installation with pip install sisl finished reporting success. However, I still could not import sisl module Any hint is appreciated. PS. I am using a free unregistered windows in the virtualbox to do these trials. PPS. The Fortran compiler has been provided by MSYS2 tools. The C compiler has been provided by Visual Studio. It would be nice to convince |
|
Hmmm, could you try: where |
|
Oh, |
|
Thank you for your comments. I had tried some of these options. I guess a minimal sisl with just one small subroutine using a fortran function would help in debugging. Whenever I use --global-option, pip recompiles cython and numpy. This takes a load of time and produces the non-working installation. I used -v. It uncovers a lot indeed, but I am lacking a better control on the compilers and on what has to be compiled. I had more control when using python setup.py install, but still no luck (because my msys2 gfortran accepts posix slashes in paths, not windows ones I think). |
|
Yeah, pip is not always easy to work with. To better control the environment with pip you could do something like this (this command will require you have all dependencies pre-installed and reachable in What does this give of output? |
|
|
Ah, yes, download the sisl.tar.gz, extract, cd, and do |
|
I know how to solve that. I will solve. However, the problem is that it starts to ask about visual studio C/C++. cygwinc I don't have. |
|
Hmm, I don't know what this means, but this seems like a general case.. I didn't get if you know what to do, or if you still need help ;) It seems this is the correct direction? |
|
This particular trouble I can solve (by installing Visual C). I will see what comes later. I still use your experience on this. |
|
Below is the output. The installation is successful. However, I still experience the same trouble when importing sisl |
|
Hmm, I must admit I have no clue... It is also unclear to me what the importance of the compilers are. You seem to be using msys64 for fortran sources, but MSVC for c-code, so basically you are compiling with one compiler and linking with another. This could be the reason. Could you try with Or try something that changes the C-compiler. Play around with I.e. the module |
|
Thank you for your comments. The problem is that it is difficult to control the C compiler and there is no Fortran compiler provided by the visual studio. |
|
Hmm. the flag |
|
I already specifying --compiler=gcc, but the pip machinery still uses visual studio's C. |
|
I said you should try Nevermind that mingw32 refers to 32-bit, it is handled correctly internally (at least it should). |
|
I investigated that already. The option --compiler has no effect in my case. |
|
What would be a minimal example to reproduce this trouble? Isn't it using any Fortran source code from Python? |
|
hmmm. ok... Then I am clueless. Other people more experienced with the windows + fortran sources could be more helpful. Yeah, I guess a minimal example of a simple fortran source, compiled in the way I add the sources would be usable as a test case. I.e. something that uses |
|
Could you kindly provide such a "Hello world" example? |
|
I think the problem comes from the full You could try with a simple code:
subroutine hello_world()
print *,"tshetsh"
endAnd a #!/usr/bin/env python3
import sys
import subprocess
import multiprocessing
import os
import os.path as osp
import argparse
# pkg_resources are part of setuptools
import pkg_resources
# We should *always* import setuptools prior to Cython/distutils
import setuptools
min_version ={
"numpy": "1.13",
}
# Although setuptools is not shipped with the standard library, I think
# this is ok since it should get installed pretty easily.
from setuptools import Command, Extension
from setuptools import find_packages
# Patch to allow fortran sources in setup
# build_ext requires numpy setup
# Also for extending build schemes we require build_ext from numpy.distutils
from distutils.command.sdist import sdist
from numpy.distutils.command.build_ext import build_ext as numpy_build_ext
from numpy.distutils.core import Extension as FortranExtension
from numpy.distutils.core import setup
# Custom command classes
cmdclass = {}
suffix = ".c"
# Retrieve the compiler information
from numpy.distutils.system_info import get_info
# use flags defined in numpy
all_info = get_info('ALL')
cmdclass["sdist"] = sdist
extensions = []
# Specific Fortran extensions
ext_fortran = {
"package1._sources": {
"sources": [f"package1/_src/{f}" for f in
("hello_world.f90")
],
},
}
for name, data in ext_fortran.items():
ext = FortranExtension(name,
sources=data.get("sources"),
depends=data.get("depends", []),
include_dirs=data.get("include", None),
define_macros=macros + data.get("macros", []),
)
extensions.append(ext)
# Override build_ext command (typically called by setuptools)
cmdclass["build_ext"] = numpy_build_ext
# The install_requires should also be the
# requirements for the actual running of sisl
setuptools_kwargs = {
"install_requires": [
"setuptools",
"numpy >= " + min_version["numpy"],
],
"setup_requires": [
"numpy >= " + min_version["numpy"],
],
}
metadata = dict(
name="package1",
maintainer="foo",
description="bar",
long_description="snthaoe"
long_description_content_type="text/markdown",
packages=
# We need to add sisl.* since that recursively adds modules
find_packages(include=["package1", "package1.*"])
ext_modules=extensions,
cmdclass=cmdclass,
**setuptools_kwargs
)
if __name__ == "__main__":
metadata["version"] = "1.0"
# Freeze to support parallel compilation when using spawn instead of fork
multiprocessing.freeze_support()
setup(**metadata)Then create the small package, (here just named |
|
Thank you very much! |
|
I hope this guides you in the right direction. Let me advice you to do this on Unix first! ;) |
|
Right. It would be nice to have a demonstrably working example. |
|
I created this https://github.com/kovalp/sisl_approach_f2py It does not compile. pip install -vvv --no-cache-dir --no-deps --no-index --no-build-isolation --compile --global-option build --global-option --compiler=unix --global-option --fcompiler=gnu95 . This hello world example assumes a bit more knowledge than I have. |
|
A temporary solution would be add a [build] section to setup.cfg as following in this case, doing The Visual Studio is not necessary in a clean install I bet. |
|
Shouldn't we produce a binary version of SISL in this way? |
|
Good to know. Thank you very much! |
Thanks! |
By defining the pydistutils.cfg with the build section (and without this section in setup.cfg) I could build and install and bdist_wheel with python 3.8.5 Here is the wheel file I get. sisl-0.10.0-cp38-cp38-win_amd64.whl.zip The issue can be closed, I think. |
2 similar comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Great, thanks! |
|
I returned to compile the package and discovered some unintended effects. At the end of working on this issue last time, we came up with a recommendation to modify the $HOME/pydistutils.cfg This makes unnecessary to modify the setip.cfg in the root of sisl when one builds/installs the library with python setup.py install, i.e. without pip. This is indeed works still. However, it has a side effect of crashing the regular pip commands. Any pip operation is impossible with the pydistutils.cfg modifications we end up recommending. Pip complains about [build] fcompiler key: |
I would argue that this is not related to the sisl installation. Sadly it seems that pip is broken on windows since it doesn't catch the correct options for the local I'll close this again because this is not related to sisl, but an upstream issue. Thanks! |
|
python setup.py install stopped working (again the DLL not found problem). I am not sure whether this is due to Python/Numpy/Scipy or Sisl. |
|
But it worked before, so what did you do differently? |
|
I updated sisl. I didn't updated anything else. It should be in sisl. So, I will find the version of sisl from before and try again. |
|
Hmm, I haven't changed anything in the build of sisl since I last fixed stuff for windows. And do you still have the correct flags in the |
|
Yes |
then |
|
https://github.com/kovalp/sisl_approach_f2py This is also failing to import package1. I was not touching this repo since we finished debugging. So, something unprecedented is going on in my Windows installations. |
|
I was updating pip... Maybe this is the reason. |
|
Apparently, I was installing python 3.8 and the compilation of sisl stopped to work. I will try to compile using different pythons and report back here. Good news: using python 3.7.7 from https://www.python.org/downloads/release/python-377/ I am able to compile/import sisl. Sorry, I forgot about this python update sic(!) |
|
Great :) |
|
Hi Nick, Could you look at this? I am sure you would solve it faster than me. I would test your solution in a range of Python distributions. |
Thanks for looking into this. However, to debug this one really needs a windows box, ;) As I recall the windows machinery puts in the dll in the top package directory, but you'll have to confirm this for me (figure out all dll's created at the installation instance. Then try and add something like this in the from sys import platform as _platform
if _platform.startswith("win"):
import os as _os
if hasattr(_os.add_dll_directory):
import pathlib as _pl
_os.add_dll_directory(_pl.Path(__file__).parent)or something similar, I really have no idea what windows does for its dll, so you'll have to play with the |
|
Note: a compilation with mingw64 toolchain in cygwin has failed with a message about wrong dimensions in Python library. I think it would be more difficult to resolve. |
|
@kovalp the build system has been updated, and some details regarding numpy linking has been fixed, if you want to retry on your windows box it would be nice, I still don't have a windows box to test on... :( |
|
Thank you for the update. I will keep it in mind.
…On Tue, Dec 14, 2021 at 2:45 PM Nick Papior ***@***.***> wrote:
@kovalp <https://github.com/kovalp> the build system has been updated,
and some details regarding numpy linking has been fixed, if you want to
retry on your windows box it would be nice, I still don't have a windows
box to test on... :(
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#244 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAKSHC7FECAIFMQGYXT5RTUQ5DAFANCNFSM4PFSL5BQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Describe the issue
I am trying to get sisl installed on Windows, in plain Python installation (i.e. not Anaconda).
My best take is to install MSYS2, install gcc and gfortran packages there and then pip gets farther as seen in the output below
Version details
Run the below code and add to issue (if an issue is relevant for the issue):
$ ipython
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.12.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import sys
...: print(sys.version)
3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)]
import sislis not working yet.The text was updated successfully, but these errors were encountered: