Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: uv bundle, uv build --release or similar to create a contained executable a la pyinstaller, py2exe #5802

Open
matterhorn103 opened this issue Aug 5, 2024 · 15 comments
Labels
wish Not on the immediate roadmap

Comments

@matterhorn103
Copy link

matterhorn103 commented Aug 5, 2024

I regularly use pyinstaller for a project, and the frequency of questions about it - or the task more generally of "compiling" an executable for distribution - around the web indicates to me that it's widely popular.

It would be incredibly cool if uv offered a uv bundle command (or whatever name) that essentially did what pyinstaller does and bundles together everything to afford an executable that contains the dependencies and interpreter and can be distributed to users as-is. And I think this falls into the realm of what things a "Cargo for Python" might be expected to be able to do.

I assume that pyinstaller, py2exe etc. are complex projects and implementing such functionality would be not trivial in the slightest, so I would have thought it would be a case of using pyinstaller itself in the background like the way that build backends are used.

Presumably there will one day be a uv build command but I assume that will be similar to rye build in that it will create an sdist or wheel. Maybe there could be a uv build --executable option, or uv build --release (by analogy to cargo build --release)?

@matterhorn103 matterhorn103 changed the title Suggestion: uv bundle or similar to create a contained executable a la pyinstaller, py2exe Suggestion: uv bundle, uv build --release or similar to create a contained executable a la pyinstaller, py2exe Aug 5, 2024
@matterhorn103
Copy link
Author

matterhorn103 commented Aug 6, 2024

Some relevant links that I'm adding to keep track of them for myself as much as anyone else:

  • It seems someone asked for the same in Rye previously
  • ofek's PyApp and Gregory Szorc's (indygreg) now-dead PyOxidizer are other, more modern executable generators that I didn't know about until now; both seem simpler than pyinstaller, and both are written in Rust
  • Gregory Szorc once did a comparison of the various alternatives to PyOxidizer
  • One missing from that list, presumably because it's not generally applicable, is the pyside6-deploy tool, but it uses Nuitka in the background in any case
  • Hatch seems to have implemented essentially this functionality, as part of a Hatch plugin that uses PyApp, and it is invoked there by having "binary" as a build target

@zanieb zanieb added the wish Not on the immediate roadmap label Aug 6, 2024
@chrisrodrigue
Copy link

To add… despite its popularity, pyinstaller suffers from slowness because has to unzip all the dependencies and the interpreter when the executable is launched. It offers users the option to display a splash image to distract them from the perceived slowness.

@chrisrodrigue
Copy link

chrisrodrigue commented Aug 6, 2024

Python to Rust transpilation would be pretty cool and there’s at least one project out there working on this:

Imagine writing idiomatic Python, transpiling it to Rust and compiling a native executable. My mind would be blown.

I don’t know how one could gracefully handle dependencies though. A lot of Python libs incorporate C extensions. Dealing with those would be extremely difficult, but there’s a DARPA program called TRACTOR working on that.

@paveldikov
Copy link
Contributor

An equivalent of pyinstaller that actually understands standard pyproject.toml-driven projects and console/GUI script entrypoints (instead of being fixated on script development, and overcompensating with its dependency detection magic) would be huge.

(somewhat related: #5653)

@my1e5
Copy link
Contributor

my1e5 commented Aug 7, 2024

I want to give a mention to Nuitka which is serving me well. It's as straightforward as running

uv add --dev nuitka
uv run python -m nuitka --standalone src/main.py

which spits out an executable.

It's not really the right tool to be including with uv, it is essentially an alternate Python interpreter which compiles Python to C. But it's very easy to add and has worked well for me. And my experience with the developers has been very good. They were quick to add support for Rye when I encountered bugs - Nuitka/Nuitka#2933

@matterhorn103
Copy link
Author

@my1e5 why do you say that Nuitka would be unsuitable for inclusion with uv?

@my1e5
Copy link
Contributor

my1e5 commented Aug 27, 2024

@matterhorn103, I guess what I mean is that Nuitka is perhaps quite an opinionated way of packaging your Python code into an executable - as it is essentially an alternate Python interpreter which compiles Python to C. Whereas tools like Pyinstaller are more like a bundler - taking the Python interpreter, Python files and dependencies and packaging them into an executable. This is definitely more straightforward than the Nuitka approach, but it does make the Pyinstaller executable very easy to un-package and see the underlying source code (see https://github.com/extremecoders-re/pyinstxtractor). Which might be a caveat needed when including certain 'executable creators' within uv - you need to make users aware that their exe can easily be un-packaged.

In terms of licensing, Nuitka is MIT licensed. But it does also have a commercial tier - which may or may not complicate matters if Astral were to want to bundle it with uv, I don't know.

@zanieb
Copy link
Member

zanieb commented Sep 3, 2024

I think this is a duplicate of #2799, though there's more discussion here.

@martimlobao
Copy link

Also want to call out pex, a tool for generating .pex files (Python EXecutable) which are self-contained zipped executable Python environments containing sources, requirements, and dependencies.

https://github.com/pex-tool/pex

@jbvsmo
Copy link
Contributor

jbvsmo commented Oct 5, 2024

I am interested in a solution like this. I think the bundler should be somewhat coupled with uv run since you want to turn any entry point into an executable.

Example

$ uv init --script foo.py
$ uv add requests --script foo.py
$ uv run foo.py
Hello from foo.py!

Then you could make an executable which is composed of

  1. bootstrap code to download uv if needed and run it
  2. target code
  3. uv executable (optional)
  4. python executable (optional from uv python)
  5. populated venv (optional)
$ uv bundle foo.py -o foo --include-uv --include-python --include-dependencies
$ ./foo 
Hello from foo.py!

This above would be a self contained binary (roughly 30MB as uv is ~12MB and python ~17MB).

If downloading stuff on the fly is ok, the bootstrap code would just download uv, then uv would download python, install dependencies and run all during 1st execution.

It should also be possible to create binaries cross platform since uv supports install/sync with --python-platform

$ uv bundle foo.py -o foo.exe --python-platform windows

In a 2nd iteration, it would be nice to be have installers such as to create .dmg (mac) or an .exe windows installer file.

@dbrtly
Copy link

dbrtly commented Oct 18, 2024

what if the command was uv run ruff --output-file ruff.uvx
with the -o/--output agument as an indicator to create the binary file rather than execute this tool right now.
and when an output file is specified, then include python by default

@stefanondisponibile
Copy link

I love this idea and @jbvsmo proposal for CLI usage.

I am a bit confused on the best choice for the bundling "backend" though, I see different options on the table and I'm not sure, how would you choose one over the other? Should maybe uv propose some protocol for each library to support bundling via uv?

This feature would be great also for bundling pyspark apps (thinking about pex support mentioned by @martimlobao here)

@AKuederle
Copy link

AKuederle commented Nov 28, 2024

I think the bootstrap option mentioned by @jbvsmo is almost achievable with the existing uv options.

You could do the following:

Within your project folder have a script (written in the scrippting language supported by your operating system) that does the following:

  1. Unmanaged installation of uv (https://docs.astral.sh/uv/configuration/installer/#unmanaged-installations) can be used to just get the uv executable in a subdir of the project folder.
  2. Setup the cache dir to be inside your project folder (https://docs.astral.sh/uv/concepts/cache/#cache-directory)
  3. Then install Python
  4. Install project deps
  5. Use uv run to execute a script within the project enviroment. The easiest option would be to just "run" the package using a __main__.py within the project

Step 1 could of course be skipped, if the uv executable is already recognized within the specified subdir. And step 3-5 are basically what happens when you run uvx. So maybe this is actually a 2 line script.

The great thing is that this script would be project independent! So someone just would need to write a script for each operating system and you could throw these scripts in your project folder.

Then you just need to distribute this project folder and have people execute the script corresponding to their operating system.

This process could then of course be wrapped further by an installer, but seems relatively straight forward to at least address the cases of "Please provide an exe".

PS: If instead of creating a shell script to run the steps above, you use the darkmagic of cosmopolitan you could even create a single binary that dropped into a python package project anywhere makes it run by "double-clicking" the file.

@jromal
Copy link

jromal commented Dec 4, 2024

For me as a short term solution it would be enough to have uv run with some options.

For example I have done a program (as module) using PySide6.
I can do uv run -m my_module from the folder above the module and it works. Fast, efficient, impressive.
But there are two things that can help as an intermediate solution before a uv bundle.

  • Read a compressed file for the full project: uv run --source my_file.zip
  • Allow an option to hide or close the terminal for qt programs: uv run --source my_file.zip --no-terminal

With this solution I could distribute on Windows the files with "Inno Setup", adding uv.exe, the zip file, and maybe a ".bat" file with a logo to execute the uv -run command.

@johnthagen
Copy link

johnthagen commented Dec 5, 2024

As much as I like Pyinstaller (to me it's best of current options), one other thing that it suffers from is that anti-virus vendors are always flagging generic Pyinstaller binaries as malware. This is (at least in part, I think) because Pyinstaller uses the same bootloader binary for all built applications, thus if someone bundles malware with Pyinstaller version X, all other users who bundle legitimate applications using Pyinstaller X can get flagged as malware too. I've seen this hit AV software across both Windows and Linux over the years. Not sure I have any ideas on how Astral could improve that issue, but wanted to bring it up as a current (very annoying) real world challenge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wish Not on the immediate roadmap
Projects
None yet
Development

No branches or pull requests