Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build without Python support? #87

Closed
simgt opened this issue Dec 1, 2022 · 6 comments
Closed

Build without Python support? #87

simgt opened this issue Dec 1, 2022 · 6 comments

Comments

@simgt
Copy link
Contributor

simgt commented Dec 1, 2022

I have the same issue as #67 with roughly the same configuration. Because my project is made for multiple hosts configurations, adding a dependency on Python through PyO3 without a very good reason is almost a deal breaker.

Is there any plans to make the Python bindings an optional feature?

@bwsw
Copy link
Contributor

bwsw commented Dec 6, 2022

Hello,
currently, I don't have plans to exclude PyO3 from the dependencies.

Because my project is made for multiple host configurations, adding a dependency on Python through PyO3 without a very good reason is almost a deal breaker.

I'm not sure I understand the reasoning behind your phrase. Do you mean that there is a chance that the python version mismatch on a host where you would like to deploy the code?

The reason why Python is included is that it's de-facto technology for ML/CV projects. However, if you submit PR with appropriate changes making Python optional, I accept them.

@bwsw
Copy link
Contributor

bwsw commented Dec 7, 2022

@higmo
However, the crate compiles in native Rust Docker image without extra Python packages installed. Just checked. Could you clarify the problem in more detail?

FROM rust:1.65-bullseye as chef

RUN cargo install cargo-chef --locked
#RUN apt-get update && apt-get install -y python3 build-essential python3-dev python3-pip
#RUN /usr/bin/python3 -m pip install --upgrade pip
#RUN /usr/bin/python3 -m pip install --upgrade maturin~=0.13

FROM chef AS planner
WORKDIR /opt
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
WORKDIR /opt
COPY --from=planner /opt/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
COPY .cargo/config /root/.cargo/
COPY . .

ENV RUSTFLAGS=" -C target-cpu=native -C opt-level=3"

RUN cargo build --release
#RUN maturin build --release --out dist
#RUN pip3 install dist/*.whl

@simgt
Copy link
Contributor Author

simgt commented Dec 7, 2022

Thanks for answering @bwsw. I am using similari for an object tracking pipeline that is not written in Python but in Rust and C. The models are indeed coming from Python frameworks but they are (ex)ported to a different inference engine.

In this case the binding is entirely superfluous for the runtime.

(I'll have a look at your last comment and close this issue accordingly)

@simgt
Copy link
Contributor Author

simgt commented Dec 7, 2022

I'm not sure what is happening in your last example but I can't manage to build similari without having to link against Python. I asked this question on PyO3's repo and it seems that there is no easy way to achieve what I want.

I'll see whether I can spend some time to add a feature that disables Python since you're willing to accept a PR.

@simgt simgt closed this as completed Dec 7, 2022
@bwsw
Copy link
Contributor

bwsw commented Dec 7, 2022

@higmo I assume that Python is already in the bullseye image, so during the build, it can find it and build.

@kamek-pf
Copy link

I'm in a similar situation. I don't need the python bindings, but I'm building a library that can be consumed via FFI. The resulting .so contains undefined symbols, all Python related.

undefined symbol: PyExc_BaseException	(target/release/libtracking.so)
undefined symbol: PyBool_Type	(target/release/libtracking.so)
undefined symbol: PyUnicode_AsEncodedString	(target/release/libtracking.so)
undefined symbol: PyException_SetCause	(target/release/libtracking.so)
undefined symbol: PyIter_Next	(target/release/libtracking.so)
undefined symbol: PyExc_ValueError	(target/release/libtracking.so)
undefined symbol: PyLong_FromUnsignedLongLong	(target/release/libtracking.so)
undefined symbol: PyExc_SystemError	(target/release/libtracking.so)
undefined symbol: PyGILState_Release	(target/release/libtracking.so)
undefined symbol: PyBytes_Size	(target/release/libtracking.so)
undefined symbol: PyException_GetCause	(target/release/libtracking.so)
...

Instead of fighting build issues I wanted to submit a PR to add a python feature flag as discussed here. Sadly it seems like there's currently no good way to apply pyo3 attributes conditionally , as they don't play well with cfg_attr, which is very surprising to me.

Until this is merged I don't think there's a great solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants