(Django)(MAC Big Sur 11.0.1) ImportError: dlopen #1216
Comments
|
Second this. Unable to use natively under M1. I am forced to use under Intel Translation / Rosetta 2. Psycopg3 also does not compile natively... |
|
Is there an M1 to test with? |
|
I'm more than happy to volunteer my own time to test... I can either run whatever tests you need, or I can volunteer to allow you to remote in, etc. |
|
I would appreciate that, thank you. Most likely rebuilding packages using the newer pypa infrastructure would just solve the issue. Could you please checkout https://github.com/psycopg/psycopg2-wheels/ and check if the Thank you very much. Ref. pypa/pip#9138 |
|
As far as I can tell, with Rosetta 2, everything was fine (which seems correct, since I can install via PIP under Rosetta 2). But at the end, it dies looking for Cellar? I don't know what that's about? Trying next under Native M1. |
|
First attempt: But that may be due to previously running the build under Rosetta 2 in that folder structure. But a significantly faster built (Rosetta took about 2-3 minutes?, M1 about 20-30 seconds). And ended with the seller / sed messages above. |
|
|
|
I'm trying to make sense out of the build_macos script? And the more I look through the build script, it's heavily personalized, and is requiring virtual environments that I don't have configured, and home-brew which is not yet working reliably under the native M1. I couldn't get the scripts as supplied to work under either Intel Mode, or M1 Native mode. Instead I just grabbed the psycopg2 dist from GitHub, and ran: psycopg2 % python3 setup.py sdist bdist_wheel Which created the Intel and m1 native based wheels. I'm unclear on what all the futzing around in the other scripts is suppose to accomplish? I haven't had a chance to test these yet... |
|
The futzing is to create self-contained pacakges, containing the libpq. Without it, the wheels need some system libraries to run, and because on mac os there is no standard for installation, people complain that it's too difficult. I'll try and re-run the packages build, maybe you can help me testing if the packages work alright. |
|
@bschollnick Do packages available here work? https://upload.psycopg.org/psycopg2-2.8.7.dev0/ Thank you! |
|
Works fine under Intel Translation, but fails under M1 Native. python3 -c"from packaging import tags; print('\n'.join([str(t) for t in tags.sys_tags()]))" |head -5 |
|
Is there anything else that I can do to help? |
|
FYI, I meant to post this here. But I ended up reposting it back to the same Pillow Thread (too many windows open at once). Okay, obviously I'm missing something here. _
_ So I probably need to change the target build architecture, but I don't see where to do that... ? |
|
Hello @bschollnick I think you should hear from the people who contributed to pypa/pip#9138 I have no details about the M1 architecture and the specific details to address to build packages there. |
|
As a quick aside, thank you both very much for your efforts :^] If anyone with more know-how would like to remote in on my machine to further troubleshoot, let me know |
|
version 2.8.1 works fine for me |
|
|
I can confirm 2.8.6 (including This is on a Macbook Air M1 running arm64. |
|
@kevbradwick yes, it works for you, but you are a good soul who has a compiler and libraries on your machine. Which I appreciate, were everyone like you... This problem here is with people who expect a binary package to work. Every few years, OSX people come out with a different machine with all the cheese moved around. or missing keys. Why people buy this stuff is beyond my belief. So, this ticket is about making the binary package work. psycopg2 code works, but "Downloading psycopg2-2.8.6.tar.gz" says that you have installed the package from source, not from binary (although you indeed requested the Solving this problem requires the OSX build script to be fixed and the package to be generated by Travis. Happy to receive a MR for it. |
|
Okay, it's now installing via PIP... Which is great, but when Django attempts to access the database.. I am seeing an exception for Symbol not found (Symbol not found: _PQbackendPID) I have rebuilt the virtual environment, just in case there had been some left-overs from the trials... But no change. |
|
Wait a sec. Why is it calling something in python 3.9? I'm running stock MOSX 3.8. I'll have to check that home-brew hasn't messed my paths.. |
|
Okay, rebuild the virtual environment again, and now it's showing the right python version... But same issue. |
|
how did you install postgres? did you compile it manually? |
|
Note that you don't need to compile postgres in order to build psycopg2, but you may need to compile the libpq, the postgres client-side library. psycopg2 does it in the |
|
As I mentioned, I can install it (via pip install psycopg2), but when used to connect to a postgresql server on the same machine, I get the symbol error.
➜ psycopg2-test pip install psycopg2
That’s not binary? That’s the source. Isn’t psycopg2-binary is the binary version?
And our configurations are not the same… You are running Python39, whereas I am using stock 3.8.2.
I hate to ask, but have you used it in the test environment?
For example:
(quickbbs) benjamin@Nerv quickbbs % pip install psycopg2
Collecting psycopg2
Using cached psycopg2-2.8.6.tar.gz (383 kB)
Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) ... done
Created wheel for psycopg2: filename=psycopg2-2.8.6-cp38-cp38-macosx_10_14_arm64.whl size=225151 sha256=207fec8ca5550433e0d15746854c5910d5b3d017c1a4986e0f420446c7fae75d
Stored in directory: /Users/benjamin/Library/Caches/pip/wheels/70/5e/69/8a020d78c09043156a7df0b64529e460fbd922ca065c4b795c
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.8.6
WTH? Why are you getting the universal wheel, and I’m getting the Arm64?
That’s could be the issue….
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.8.6
(quickbbs) benjamin@Nerv quickbbs % server4
Running on nerv.local
Debug is True
Running on nerv.local
Debug is True
Watching for file changes with StatReloader
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/Users/benjamin/venvs/quickbbs/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 25, in <module>
import psycopg2 as Database
File "/Users/benjamin/venvs/quickbbs/lib/python3.8/site-packages/psycopg2/__init__.py", line 51, in <module>
from psycopg2._psycopg import ( # noqa
ImportError: dlopen(/Users/benjamin/venvs/quickbbs/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so, 2): Symbol not found: _PQbackendPID
Referenced from: /Users/benjamin/venvs/quickbbs/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so
Expected in: flat namespace
in /Users/benjamin/venvs/quickbbs/lib/python3.8/site-packages/psycopg2/_psycopg.cpython-38-darwin.so
During handling of the above exception, another exception occurred:
But I don’t know.
- Benjamin
… On Jan 15, 2021, at 1:06 PM, Kevin Bradwick ***@***.***> wrote:
I can confirm 2.8.6 (including psycopg2-binary) works for me.
➜ psycopg2-test pip install psycopg2
Collecting psycopg2
Downloading psycopg2-2.8.6.tar.gz (383 kB)
|████████████████████████████████| 383 kB 3.8 MB/s
Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) ... done
Created wheel for psycopg2: filename=psycopg2-2.8.6-cp39-cp39-macosx_10_9_universal2.whl size=226356 sha256=bf4bdc2b8d73783423eec6a5f094d5d18f8b178f83a67017db821fb1e7353e6a
Stored in directory: /Users/kevinbradwick/Library/Caches/pip/wheels/a2/07/10/a9a82e72d50feb8d646acde6a88000bbf2ca0f82e41aea438a
Successfully built psycopg2
Installing collected packages: psycopg2
Successfully installed psycopg2-2.8.6
This is on a Macbook Air M1 running arm64.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#1216 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHPEZXHUTSG6LB5ZZLJRZLS2B735ANCNFSM4VS5GJ4Q>.
|
|
Postgres is working fine. I’m using PostgresApp…
And Psycopg2 is working fine in Intel mode, as well.
- Benjamin
… On Jan 15, 2021, at 5:33 PM, Kevin Bradwick ***@***.***> wrote:
how did you install postgres? did you compile it manually?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#1216 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHPEZUNGB23RBYFHKOLWHDS2C7DTANCNFSM4VS5GJ4Q>.
|
|
@bschollnick Exact same problem here per #1208 I'm using python@3.9 from homebrew, though |
This ^ says that the package installed is the source one. The packages are found from this list: https://pypi.org/project/psycopg2-binary/#files Your It sounds silly that there is a source choice too in the binary package... Initially there wasn't, but that created more problems. I can't remember exactly, but they were of the tone that if someone has However, if you |
could it be that the binary needs some shared dynamic library (also arm64) but the one you have installed is via Postgress.app which is x86? |
|
Yes: It needs the libraries built by scripts/build_libpq_macos.sh |
|
I fixed the same problem in Mac BigSur 11.5.1 M1 following the tips of this thread in the next order:
Thanks to everyone who helped. |
|
I'm still having this issue: My setup: What I have tried: also: not sure if this is related since it's an import issue: UPDATE: |
|
|
|
If anyone is struggling to make it work, make sure your terminal application supports M1 natively(the inbuilt terminal or iterm2 support m1 without rosetta) |
|
|
|
@samrathchadha thank you so much for this: I'm using Alacritty and its ARM support is busted, so I had to install via x86 🤦. |
|
No problem @nicksergeant |
Thank you for this, I just discovered this issue yesterday with Django, I gave it ago and and restarted my terminal and sure enough, at last started working :) |
|
Hi there, I face the same issue as some of you guys. And after trying a lot of different solutions you proposed, I'm still faced with this error when trying to create an engine:
I'm on Big Sur 11.0 M1 and using Pycharm as my IDE. Any tip would be greatly appreciated! |
|
@EtienneLeconte-Stuart try following @bartfto post below as that did work for me and I was using Big Sur 11.0 M1 and got this fixed just 4 days ago :) / Also might I also add, What I also found with this is - There is another version of Python3 which is supported for 'universal' which is made for M1 (which i didn't know either) '' My setup: What I have tried: also: not sure if this is related since it's an import issue: UPDATE: |
|
@lyntree thanks a lot for your answer! I tried the proposed solution and unfortunately that did nothing for me... I'm still at the same point. Something must definitely be wrong with my configuration, but I struggle to get what. |
|
thank you @lyntree & everyone else above! after a longgggg time attempting most everything above, the key for me was "create new virtualenv" I am running MacOS Big Sur Version 11.3.1, M1 I deleted the old venv I was working with & did the following: the issue was magically solved good luck everyone else, this is definitely a frustrating error! |
|
Thats brilliant to hear, I don't always use the create a new env as I
don't find its useful to me much but then again, sometimes it works :)
Glad to hear you got it working in end :)
…On Mon, 17 Jan 2022 at 17:25, Lyndsey ***@***.***> wrote:
thank you @lyntree <https://github.com/lyntree> & everyone else above!
after a longgggg time attempting most everything above, the key for me was
"create new virtualenv"
I am running MacOS Big Sur Version 11.3.1, M1
I deleted the old venv I was working with & did the following:
in root folder ran brew install postgresql
created new virtualenv
in project folder ran pip3 install psycopg2-binary --no-cache-dir''
the issue was magically solved .. I also ran pip3 uninstall psycopg2
previously so this may have been key as well
good luck everyone else, this is definitely a frustrating error!
—
Reply to this email directly, view it on GitHub
<#1216 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQ7KPVNUCLGTXHBGRLP6J3DUWRGKDANCNFSM4VS5GJ4Q>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Bridget
Kindest Regards,
*Lyntree Web Studio *
Website Design | Website Hosting | Mobile Applications
*Full Stack App|Web|Mobile Developer*
*Website:* https://lyntree.co.uk | https://lyntree.io tbc
*Phone */ *WhatsApp*: 07761 308429
*Email:* ***@***.*** / ***@***.***
|
|
@nicksergeant @Jfeng3 @dvarrazzo @kevbradwick @leohakim I am having a similar issue connecting to a Postgres server through psycopg2 , please please help me have a look if anyone is free, thank you very much. I have tried to uninstall and reinstall, change the env installation method like the above solution but it doesn't work at all , the error is the same I'm running: Mac Big Sur 11.2 when I running "python manage.py makemigrations", the errors accur as follows: Traceback (most recent call last): During handling of the above exception, another exception occurred: _Traceback (most recent call last): |
You are the best man, you saved my life. |
|
@winny1212 -- were you able to resolve the issue you posted? I got the same error today, and there are so many different proposed 'solutions' I'm not sure where to start; my error output is identical to yours, and occurs for the same reason -- when I run python manage.py makemigrations. |
Maybe you can try with this as is solving the architecture issue with Macs M1 but maybe you already have tried. You have to run this commands with Rosseta terminal, just that's it, almost for me: env /usr/bin/arch -arm64 /bin/zsh --login |
|
None of the above solutions worked for me, so I tinkered around and got this to work: Cheers! |
|
An additional data point: I found that my m1 Mac was installing an x86_64 version of any extension packages (not pure Python), in my case I eventually discovered by trial and error how to overcome this, which was to uninstall with This tells Having discovered this I investigated why the standard process was producing x86 binaries. Examining the value of ARCHFLAGS in a brand new shell to my horror showed me this: This was a setting I tracked down to my .zshrc, but I have no memory of that line so I'm assuming it was an Apple precautionary measure I didn't notice when I migrated from Mea culpa for not checking more carefully when I migrated (usual story, did it in a rush while busy with other things).
|
|
Thanks for the info above, was encountering this after upgrading from python3.8 to python3.10 and installing psycopg2-binary. My machine is OSX Big Sur 12.3.1, M1 Pro. My project uses pipenv so just needed to convert these pip args: into the following environment variables: Reinstalled psycopg2-binary and everything was working again. |
Worked fine for me and I guess this is proper solution !!! |
|
@HardyNapses thank you, this fixed it for me! |
|
Assuming this was fixed with the release of M1 binary packages and can be fixed. |
Worked for me, thank you |
|
Closing the conversation because the ticket is solve and only attracting metoo replies. (on a nonsense answer, but hey, whatever floats your macOS). |
and others
Apologies in advance -- this is probably doubling up on #1200 and #1208, but I'm curious as to whether there's any workaround whatsoever (as I really would like to use postgresql with my project, but I'm too much of a newb to comprehend most of the issues I run into). I've tried pretty much every suggestion to be found through googling to no avail. The issue is as below:
Installed pyscopg2 to the venv without any issues using:
pip install psycopg2Meanwhile, Postgresql was successfully installed and the db I'd like to connect to is active (using pgadmin4). Django settings are configured accordingly.
However, when trying to access psycopg2 through the terminal, whether it be through:
python -c "import psycopg2"orpython manage.py migrate, etc.The below error occurs:
(venv) ...-MacBook-Air project % python -c "import psycopg2" Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/.../Documents/project/venv/lib/python3.9/site-packages/psycopg2/__init__.py", line 51, in <module> from psycopg2._psycopg import ( # noqa ImportError: dlopen(/Users/.../Documents/project/venv/lib/python3.9/site-packages/psycopg2/_psycopg.cpython-39-darwin.so, 2): no suitable image found. Did find: /Users/.../Documents/project/venv/lib/python3.9/site-packages/psycopg2/_psycopg.cpython-39-darwin.so: mach-o, but wrong architecture /Users/.../Documents/project/venv/lib/python3.9/site-packages/psycopg2/_psycopg.cpython-39-darwin.so: mach-o, but wrong architectureAny help/insight would be greatly appreciated. Or are all of us who jumped ship to M1 doomed until Apple stop breaking things? :)
The text was updated successfully, but these errors were encountered: