ImportError: failed to find libmagic. Check your installation

[lucas-rudd]

I believe this may be an issue with the magic package used by thehive4py/api.py

After installing elastalert v 0.1.3.6 I get the following stacktrace when I try to run an elastalert test with a folder of rules

Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
RMTMACSIERRA01:infrastructure-cm lucas.rudd$ docker run --rm -it -e ES_HOST=search-pss-pf-prod-es-51-gcm4nbuhxjuo6telgbszkp3i7u.us-east-1.es.amazonaws.com -e AWS_PROFILE=asurion-soluto-nonprod.dev -v ~/.aws:/home/elastalert/.aws  -v ~/Desktop/rules/:/home/elastalert/rules --entrypoint elastalert asurionpss/elastalert:test --verbose --es_debug
Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation

After manually installing libmagic the issue persists.

Has anyone else experienced this issue?

[synhershko]

Here too

[synhershko]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

[lsoumille]

@synhershko Your solution doesn't work for me because python-magic requirement is >=0.4.15. Here is the error message:

pkg_resources.DistributionNotFound: The 'python-magic>=0.4.15' distribution was not found and is required by elastalert

[primeroz]

I just installed libmagic and it fixed that error. I am running an alpine container

[eliasbrange]

+1. Our elastalert suddenly started failing in our CI due to libmagic not being installed. Might want to add it to the docs that it is required. Even better, don't try to import it unless a Hive rule is configured.

[wuxinwei]

+1. I got the same problem when I was run elastalert at alpine container

[wuxinwei]

@primeroz , Can you tell me that how do you solve this problem, just install libmagic when build docker image?

[eliasbrange]

@primeroz , Can you tell me that how do you solve this problem, just install libmagic when build docker image?

We've just added apk add libmagic.

[primeroz]

@wuxinwei as @eliasbrange mentioned just apk add libmagic

[wuxinwei]

@eliasbrange ,thank you, the solution which you mentioned solved my problem

[Samuel-Gong]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

[skatakar]

Observed the same issue while running elastalert
1)D:\elastalert>elastalert-test-rule "D:\elastalert\example_rules\example_frequenc
y.yaml"
Traceback (most recent call last):
File "D:\Python27\Scripts\elastalert-test-rule-script.py", line 11, in <module

load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert-test-r

ule')()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 480, in l
oad_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2693, in
load_entry_point
return ep.load()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2324, in
load
return self.resolve()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2330, in
resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "D:\Python27\Lib\site-packages\elastalert\test_rule.py", line 20, in
import elastalert.config
File "D:\Python27\Lib\site-packages\elastalert\config.py", line 9, in

import alerts

File "D:\Python27\Lib\site-packages\elastalert\alerts.py", line 31, in <module

from thehive4py.api import TheHiveApi

File "D:\Python27\Lib\site-packages\thehive4py\api.py", line 7, in
import magic
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation

D:\elastalert>

C:\Users\u3xlbes>elastalert --start NOW --verbose
Traceback (most recent call last):
File "D:\Python27\Scripts\elastalert-script.py", line 11, in
load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 480, in l
oad_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2693, in
load_entry_point
return ep.load()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2324, in
load
return self.resolve()
File "D:\Python27\Lib\site-packages\pkg_resources_init_.py", line 2330, in
resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "D:\Python27\Lib\site-packages\elastalert\elastalert.py", line 21, in
from alerts import DebugAlerter
File "D:\Python27\Lib\site-packages\elastalert\alerts.py", line 31, in <module

from thehive4py.api import TheHiveApi

File "D:\Python27\Lib\site-packages\thehive4py\api.py", line 7, in
import magic
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation

C:\Users\u3xlbes>

we are using python 2.7.15 . any workaround for this issue?

[skatakar]

Do we have any workaround or solution for this issue?

[eliasbrange]

Do we have any workaround or solution for this issue?

Check out https://github.com/pidydx/libmagicwin64 for installing libmagic on windows.

[skatakar]

Do we have any workaround or solution for this issue?

Check out https://github.com/pidydx/libmagicwin64 for installing libmagic on windows.

I have downloaded all 4 files which are mentioned in the above link and placed under C:\Windows\System32.. But when I try to import magic or import liblogic, its giving same error.

import magic
Traceback (most recent call last):
File "", line 1, in
File "D:\Python27\Lib\site-packages\magic.py", line 181, in
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
import libmagic
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named libmagic

Do we need to run/configure any thing after download those 4 files?

we have python 32bit on windows 2012 64 bit

platform.architecture()
('32bit', 'WindowsPE')

[MickaelHUMPHREYS]

Hi,

I had the same problem.
I solved it by adding this requirement : python-magic
pip install python-magic==0.4.15

[Ogagnon]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

I love you.

[efinal]

This worked for me:
pip uninstall python-magic
pip install python-magic-bin==0.4.14

it works thank you

[andresjbotello]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

This has worked for me, thanks!!!

[yustiks]

I have the same issue (Linux mint), I tried to remove magic by this command:
pip uninstall python-magic
and then reinstall, but it didn't work.
What I did in the end: I cloned the github code: https://github.com/ahupp/python-magic
into the local folder to the directory where the actual project is, and then renamed the folder to 'magic'. After that it worked.

[lobinator]

This worked for me:
pip uninstall python-magic
pip install python-magic-bin==0.4.14

Worked for me too. Note that I uninstalled libmagic in addition to python-magic.

[roykingtree]

neither solution worked for me

[Lebostein]

What is this for a crap? I get this error on Windows and Mac OS.

[saginadir]

Solved it by running

brew install libmagic

[richardbalwane]

Use the method by @synhershko
i.e, 1. pip uninstall python-magic
2. pip install python-magic-bin
Except, don't include the python-magic-bin version!

[javadnikbakht]

I'm on MacOS m1 and I fix this issue by installing libmagic using Homebrew:

brew install libmagic

[ShuxunoO]

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

it really works!

[ZhengPeng7]

This worked for me:
pip uninstall python-magic
pip install python-magic-bin==0.4.14

This works for my windows 11.

[gelodefaultbrain]

On my Macbook with M1 chip. This worked for me.

• brew install libmagic
• pip install python-magic-bin==0.4.14

I tried this by only doing brew install libmagic and having the python-magic already works

Thanks!!

[ArmanChand]

brew install libmagic

this code works like magic . Easy and quick fix

[GreysonHYH]

When I run pip uninstall python-magic and then pip install python-magic-bin==0.4.14 I get the error message: ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none) ERROR: No matching distribution found for python-magic-bin==0.4.14. Has anyone else seen this error?

same,bro

[rtviii]

In my case (M1 MacBook), it was all about telling the system where to find the library files. So after obvious brew install libmagic (on M1s, homebrew installs libmagic to /opt/homebrew/Cellar/libmagic), it was necessary to create a symbolic link from the library files to the place where the system expects libmagic to be located, which is /usr/local/lib.

Therefore, this worked for me:

sudo ln -s /opt/homebrew/Cellar/libmagic/5.41/lib/libmagic.dylib /usr/local/lib/libmagic.dylib

FYI: my version of python-magic is 0.4.18 (and I wasn't willing to downgrade as suggested above)

savior

[spam0115]

None of these solutions are working for me in windows. Tried installing libmagic, tried 1. pip uninstall python-magic
2. pip install python-magic-bin, tried the windows libmagic dll prebuilts. No change to my error message "
libmagic is unavailable but assists in filetype detection on file-like objects.Please consider installing libmagic for better results."

[Viswahs]

This worked for me:

pip uninstall python-magic
pip install python-magic-bin==0.4.14

[ktaborowski]

pip install python-magic-bin==0.4.14
ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none)
ERROR: No matching distribution found for python-magic-bin==0.4.14

Is not for Linux, the package is only for Widnows and MacOS https://pypi.org/project/python-magic-bin/0.4.14/#files

For me it works (22.04 Ubuntu):

pip uninstall python-magic
apt install libmagic1
pip install python-magic

[danruggi]

pip uninstall python-magic
apt install libmagic1
pip install python-magic

Worked for me too.

[Vishnu-Dhanabalan]

In my case (M1 MacBook), it was all about telling the system where to find the library files. So after obvious brew install libmagic (on M1s, homebrew installs libmagic to /opt/homebrew/Cellar/libmagic), it was necessary to create a symbolic link from the library files to the place where the system expects libmagic to be located, which is /usr/local/lib.

Therefore, this worked for me:

sudo ln -s /opt/homebrew/Cellar/libmagic/5.41/lib/libmagic.dylib /usr/local/lib/libmagic.dylib

FYI: my version of python-magic is 0.4.18 (and I wasn't willing to downgrade as suggested above)

This worked for me (M2 pro). Just make sure you use the correct version inside the /opt/homebrew/Cellar/libmagic/.

[akshayvkt]

I was using an online IDE environment (Kaggle) and this was what worked for me as well:

!pip uninstall python-magic
!apt install libmagic1 -y
!pip install python-magic

[k1tano]

pip install python-magic-bin==0.4.14
ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none)
ERROR: No matching distribution found for python-magic-bin==0.4.14

Is not for Linux, the package is only for Widnows and MacOS https://pypi.org/project/python-magic-bin/0.4.14/#files

For me it works (22.04 Ubuntu):

pip uninstall python-magic
apt install libmagic1
pip install python-magic

work for ubuntu 18 too

[egsoftjam]

Hello everyone, after having a lot of issues with it, the solution that worked for me was to use the following commands (use python_magic_bin without version):
python-magic==0.4.27
python_magic_bin

It downloads python-magic-bin==0.4.14 but if I mention the version, it doesn't work. Strange behavior.

[tomtastic]

MacOS (maybe others too)

$ pip install pylibmagic
$ pip install python-magic

Then ensure you import pylibmagic before you import magic

[franko172000]

MacOS (maybe others too)

$ pip install pylibmagic
$ pip install python-magic

Then ensure you import pylibmagic before you import magic

Thank you! it worked

[AIchemizt]

Follow this article for Mac users:

https://nixloop.com/fix/fixing-libmagic-installation-issues-on-apple-silicon-macs-for-python/

[puneetkaursidhu]

This worked for me:

pip uninstall python-magic
pip install python-magic

I'm on Apple M2 Max.

[denivic]

Not specifying the version for python-magic-bin worked for me (Windows):

pip uninstall python-magic
pip install python-magic-bin

[mburaksayici]

MAC M1/M2 People,
This should work, and please note this solution because when you restart terminal, python related things in Mac somehow restarts, you probably faced that

https://stackoverflow.com/a/79042514/6806531

brew install libmagic

pip uninstall python-magic
pip install python-magic

# comma separated list for --find-links
export PIP_FIND_LINKS=https://github.com/ddelange/python-magic/releases/expanded_assets/0.4.28.post7
pip install --force-reinstall python-magic

Also within the codebase, use that:

`
mime = magic.Magic(mime=True, magic_file = "/usr/local/Cellar/libmagic/5.45/share/misc/magic.mgc") # For local test on mac : mime = magic.Magic(mime=True, magic_file = "/usr/local/Cellar/libmagic/5.45/share/misc/magic.mgc")
file_type = mime.from_buffer(file_bytes)

`

You can findyour magic.mgc file by

brew list libmagic
it's in share/misc/magic.mgc