-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Publish PEP 517 sdist #8
Comments
Meanwhile, is there any other way to build orjson on alpine Linux? |
You can try the instructions in https://github.com/ijl/orjson#install. Building a pyo3 project against Python using musl libc may not succeed right now, I don't know. |
I would love to use orjson as well, but I like my small alpine containers. Sounds like pyo3 is the project that would know about musl / alpine linux things. |
The pyo3 project says to use (your instructions do not mention this flag)
I'll give it a try to see if this can be built from scratch on Alpine. |
2.0.7 includes an sdist using pyo3-pack. @konstin, thank you for adding that to pyo3-pack. |
I should probably create a different github issue, but here's my failed attempt at installing 2.0.7 on Alpine (in docker). I tried with and without the --user flag, as root and as a normal user. I have (latest ?) pip (19.2.3).
|
musl libc isn't supported. |
Got it. Thanks.
… On Aug 29, 2019, at 9:26 AM, ijl ***@***.***> wrote:
musl libc isn't supported.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AC2O6UIANDTUOD52UAGELWLQG72FFA5CNFSM4GSDE6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PB7VA#issuecomment-526262228>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UND4YFXE2HW4ZP7INTQG72FFANCNFSM4GSDE6LQ>.
|
FYI I just learned about the bitnami docker images which are glibc based, but very small like Alpine.
… On Aug 29, 2019, at 9:33 AM, Benjamin Sergeant ***@***.***> wrote:
Got it. Thanks.
> On Aug 29, 2019, at 9:26 AM, ijl ***@***.*** ***@***.***>> wrote:
>
> musl libc isn't supported.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AC2O6UIANDTUOD52UAGELWLQG72FFA5CNFSM4GSDE6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PB7VA#issuecomment-526262228>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UND4YFXE2HW4ZP7INTQG72FFANCNFSM4GSDE6LQ>.
>
|
I managed to get it working on Alpine Linux by hacking the dist into believing it is "manylinux" with a couple lines. |
Here is my Docker recipe to have orjson on Alpine Linux:
And yes it works. Update: versions > 2.0.9 fail with "_register_atfork: symbol not found" |
Thanks for sharing, I might give it a try.
… On Aug 30, 2019, at 7:24 AM, gatopeich ***@***.***> wrote:
Here is my Docker recipe to have orjson on Alpine Linux:
FROM python:3.7-alpine
RUN echo 'manylinux1_compatible = True' > /usr/local/lib/python3.7/_manylinux.py
RUN pip3 install orjson
RUN rm /usr/local/lib/python3.7/_manylinux.py
And yes it works.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#8?email_source=notifications&email_token=AC2O6UKZCVSU5E4WI6WGVLTQHEURHA5CNFSM4GSDE6L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5RZVJI#issuecomment-526621349>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UMSYXQI6ULFPZ46KFDQHEURHANCNFSM4GSDE6LQ>.
|
i had try this,success install ,,but import raise error
have any idear? |
This annoying symbol dependency was added in v2.0.10. |
Got the latest orjson working on on alpine3.11
|
How does this install orjson? |
This allows one to install a python requirements file that includes the latest orjson. Without these lines, one will run into a compilation error. I've added |
An update for anyone coming to this issue looking to run newest orjson on Alpine.
I have this running in two different performance-sensitive web applications right now and seems to be doing perfectly fine. |
Tested, the above recipe also works on Alpine 3.10, with Python 3.7 using the following recipe, but it doesn't work on Alpine 3.11, with Python 3.8.
When trying the same in Alpine 3.11 and Python 3.8, the python3 executable always crashes after being patched with gcompat, this includes running running normal python applications like PIP too. |
Working in easy steps with gcompat + patchelf, without explicit patching, on alpine:3.11:
|
That's great news. Perhaps it's a newer version of gcompat now on alpine 3.11? |
Must be a combination of gcompat and patchelf because if I don't install patchelf there is an import error. |
@gatopeich Any chance you know why it doesn't work on python3.8-alpine3.10? |
@NotoriousRebel there was some library symbol missing on 3.10, and no supported way to amend it. |
I know its been 6 months since I've commented here, but revisiting this now for a different project I'm working on, and I've investigated the issue with Python 3.8 and Alpine 3.11 a bit more. However, in the last 6 months there has been a lot of changes in orjson, particularly in relation to building and publishing. Eg, manylinux1 builds are no longer published since 3.4.0, so we need to use v3.3.1 or older. Also, orjson v2.6.5 and above requires So here is a basic docker recipe for using orjson in alpine v3.11 with Python 3.8
|
And finally, I just did a build on Alpine 3.12 with Python 3.8.5. Theres not much different here, except we need to install py3-pip to use
|
The above solution doesn't seem to work for the latest version of pip. For now, I've pinned |
Thanks, @ashleysommer, that works like a charm even today with Python 3.8.8 and orjson 3.5.1:
@aspin, I can only guess you are not using the bundled |
I came up with a solution combining a few of the above mentioned: FROM python:3.6-alpine
RUN echo 'manylinux2014_compatible = True' > /usr/local/lib/python3.6/_manylinux.py && \
apk add --no-cache gcompat libgcc && \
pip install "pip==20.2.2" --user && \
pip install "orjson~=3.5.1" && \
pip uninstall -y pip && \
rm /usr/local/lib/python3.6/_manylinux.py
... # Install everything else (pip is back to the image version) |
Needs PyO3/maturin#2.
The text was updated successfully, but these errors were encountered: