"No suitable image found" on OS-X

[PeterJacob]

Hi, I'm running Mac OS-X 10.12.3 (16D32). During the installation via pip there were no errors, but when I try to run a simple script fitting the model I get the following error. Does anyone understand what's going wrong?

A colleague of mine has the library running without problems, but doesn't remember how exactly.

Peters-MacBook-Pro:glmnet_r_python petersmit$ python glmnet_r_python.py 
Traceback (most recent call last):
  File "glmnet_python.py", line 154, in <module>
    fit = glmnet(x=x, y=y, family='gaussian', alpha=0.25, thresh=0.001, maxit=1000, nlambda=100)
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/glmnet.py", line 450, in glmnet
    thresh, isd, intr, maxit, family)
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/elnet.py", line 17, in elnet
    glmlib = loadGlmLib() 
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/loadGlmLib.py", line 24, in loadGlmLib
    glmlib = ctypes.cdll.LoadLibrary(glmnet_so)
  File "/Users/petersmit/anaconda/lib/python3.6/ctypes/__init__.py", line 422, in LoadLibrary
    return self._dlltype(name)
  File "/Users/petersmit/anaconda/lib/python3.6/ctypes/__init__.py", line 344, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found.  Did find:
	/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00

The file mentions in the traceback (GLMnet.so) is the same as the one in the repo, but for some reason he has a file with different size. If I rename the version installed on my pc with his version I get the following traceback:

Traceback (most recent call last):
  File "glmnet_python.py", line 154, in <module>
    fit = glmnet(x=x, y=y, family='gaussian', alpha=0.25, thresh=0.001, maxit=1000, nlambda=100)
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/glmnet.py", line 450, in glmnet
    thresh, isd, intr, maxit, family)
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/elnet.py", line 17, in elnet
    glmlib = loadGlmLib() 
  File "/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/loadGlmLib.py", line 24, in loadGlmLib
    glmlib = ctypes.cdll.LoadLibrary(glmnet_so)
  File "/Users/petersmit/anaconda/lib/python3.6/ctypes/__init__.py", line 422, in LoadLibrary
    return self._dlltype(name)
  File "/Users/petersmit/anaconda/lib/python3.6/ctypes/__init__.py", line 344, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/GLMnet.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
  Referenced from: /Users/petersmit/anaconda/lib/python3.6/site-packages/glmnet_python/GLMnet.so
  Reason: image not found

I tried installing Xcode + glibfortran for OS-X, but that doesn't help. Traceback is the same.

[PeterJacob]

Ok, after more fiddling it seems like I got things to run. I had to copy libgfortran.3.dylib to a magic location.

mkdir -p /usr/local/opt/gcc/lib/gcc/6/
cp /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libgfortran.3.dylib /usr/local/opt/gcc/lib/gcc/6/

[bbalasub1]

Thanks. That is good to know.

[lmackey]

I'm encountering the same issue that @PeterJacob described. I'm running OS X version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`---------------------------------------------------------------------------
OSError Traceback (most recent call last)
in ()
1 from glmnet_python import glmnet
----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options)
448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm,
449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam,
--> 450 thresh, isd, intr, maxit, family)
451 elif (family == 'binomial') or (family == 'multinomial'):
452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family)
15
16 # load shared fortran library
---> 17 glmlib = loadGlmLib()
18
19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/loadGlmLib.py in loadGlmLib()
22 def loadGlmLib():
23 if os.name == 'posix':
---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so)
25 return(glmlib)
26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name)
424
425 def LoadLibrary(self, name):
--> 426 return self._dlltype(name)
427
428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error)
346
347 if handle is None:
--> 348 self._handle = _dlopen(self._name, mode)
349 else:
350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find:
/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
`

I've tried the libgfortran.3.dylib fix that @PeterJacob suggested, but my traceback is unchanged. @PeterJacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

[bbalasub1]

Peter, Are you still having luck with the glmnet? I am wondering if I should add your fix to the FAQ. Please let us know if the fix actually worked for you.

…

On Thu, Jun 1, 2017 at 5:53 AM, Peter Smit ***@***.***> wrote: Ok, after more fiddling it seems like I got things to run. I had to copy libgfortran.3.dylib to a magic location. mkdir -p /usr/local/opt/gcc/lib/gcc/6/ cp /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libgfortran.3.dylib /usr/local/opt/gcc/lib/gcc/6/ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKpDE0ukiz0pewagdw1Gsgg2AGk5oIAnks5r_ooTgaJpZM4Nsr2l> .

[mandycoston]

Is there a recommended fix for this? I'm getting the same issue on OSX 10.13.2

[marnixkoops]

I have fixed the problem by first placing the libgfortran.3.dylib file in the location suggested by PeterJacob. First I located the file with:

$ sudo find / -name libgfortran.3.dylib

Next move the file as suggested:

mkdir -p /usr/local/opt/gcc/lib/gcc/6/
cp /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libgfortran.3.dylib 
/usr/local/opt/gcc/lib/gcc/6/

After that I installed gfortran from here
Then it still did not work. The final step which fixed the problem for me was replacing my GLMnet.so file with the one from a computer where the error was not happening. See the attached zip:
GLMnet.so.zip

During this process I replaced the old file so unfortunately I can not check what exactly the difference is which may be causing this trouble.

@mandycoston @bbalasub1 @lmackey

[ljollans]

marnixkoops solution worked for me (using the path indicated in the error I was getting for moving the gfortran files)- thanks for uploading the file!

[obarnstedt]

For me, this worked:

mkdir -p /usr/local/opt/gcc/lib/gcc/6/
cp /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libgfortran.3.dylib 
/usr/local/opt/gcc/lib/gcc/6/

And replacing the GLMnet.so as @marnixkoops suggested.
Interestingly, macOS Big Sur tried to block access to the .so file, so I've had to grant access first; maybe this is part of the problem?
Thanks, in any case, for all these solutions!