Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add installation option for portable entry points #3669

Open
ofek opened this issue May 20, 2024 · 7 comments
Open

Add installation option for portable entry points #3669

ofek opened this issue May 20, 2024 · 7 comments
Labels
configuration Settings and such enhancement New feature or request

Comments

@ofek
Copy link
Contributor

ofek commented May 20, 2024

I have a use case where I need to pre-build an entire installation which will be distributed to any number of machines. When installing packages that have entry points, the location becomes hardcoded to an absolute path.

Instead, this should be relative to the Python executable used for installation. For non-Windows systems the python-build-standalone project does this for pip for example:

#!/bin/sh
"exec" "$(dirname $0)/python3.12" "$0" "$@"
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

The trampoline logic should be equivalently simple for Windows.

Maybe this should become the default in future?
@pfmoore
Copy link
Contributor

pfmoore commented May 20, 2024

This definitely should not be the default. Entry point wrappers are built with absolute paths to the interpreter so that the wrapper can be copied or symlinked to other locations, without needing to copy the Python environment as well. This is something that's used by a lot of applications - most notably pipx, which symlinks ~/.local/bin/appname.exe to ~/.local/pipx/venvs/appenv/Scripts/appname.exe.

@charliermarsh
Copy link
Member

Makes sense, thanks for that context @pfmoore.

@charliermarsh charliermarsh added enhancement New feature or request configuration Settings and such labels May 20, 2024
@skoslowski
Copy link

I have a similar use-case. It would be great to be able to skip patching wrappers post-install.

(On Windows-systems I am using <launcher_dir> for distlib-based launchers)

@ofek ofek mentioned this issue May 21, 2024
Merged
charliermarsh pushed a commit that referenced this issue May 22, 2024
@ofek
Allow relative Python executable paths in Windows trampoline (#3717)
82820d0 
## Summary

This is a prerequisite for #3669

## Test Plan

Download one of the standalone distributions on Windows then use its
Python to run the following script and then run the scripts it creates
(only pip):

```python
from __future__ import annotations

import sys
import sysconfig
from contextlib import closing
from importlib.metadata import entry_points
from io import BytesIO
from os.path import relpath
from pathlib import Path
from tempfile import TemporaryDirectory
from zipfile import ZIP_DEFLATED, ZipFile, ZipInfo

# Change this line to your real path
LAUNCHERS_DIR = Path('C:\\Users\\ofek\\Desktop\\code\\uv\\crates\\uv-trampoline\\target\\x86_64-pc-windows-msvc\\release')
SCRIPT_TEMPLATE = """\
#!{executable}
# -*- coding: utf-8 -*-
import re
import sys
from {module} import {import_name}
if __name__ == "__main__":
    sys.argv[0] = re.sub(r"(-script\\.pyw|\\.exe)?$", "", sys.argv[0])
    sys.exit({function}())
"""


def select_entry_points(ep, group):
    return ep.select(group=group) if sys.version_info[:2] >= (3, 10) else ep.get(group, [])


def main():
    interpreters_dir = Path(sys.executable).parent
    scripts_dir = Path(sysconfig.get_path('scripts'))

    ep = entry_points()
    for group, interpreter_name, launcher_name in (
        ('console_scripts', 'python.exe', 'uv-trampoline-console.exe'),
        ('gui_scripts', 'pythonw.exe', 'uv-trampoline-gui.exe'),
    ):
        interpreter = interpreters_dir / interpreter_name
        relative_interpreter_path = relpath(interpreter, scripts_dir)
        launcher_data = (LAUNCHERS_DIR / launcher_name).read_bytes()

        for script in select_entry_points(ep, group):
            # https://github.com/astral-sh/uv/tree/main/crates/uv-trampoline#how-do-you-use-it
            with closing(BytesIO()) as buf:
                # Launcher
                buf.write(launcher_data)

                # Zipped script
                with TemporaryDirectory() as td:
                    zip_path = Path(td) / 'script.zip'
                    with ZipFile(zip_path, 'w') as zf:
                        # Ensure reproducibility
                        zip_info = ZipInfo('__main__.py', (2020, 2, 2, 0, 0, 0))
                        zip_info.external_attr = (0o644 & 0xFFFF) << 16

                        module, _, attrs = script.value.partition(':')
                        contents = SCRIPT_TEMPLATE.format(
                            executable=relative_interpreter_path,
                            module=module,
                            import_name=attrs.split('.')[0],
                            function=attrs
                        )
                        zf.writestr(zip_info, contents, compress_type=ZIP_DEFLATED)

                    buf.write(zip_path.read_bytes())

                # Interpreter path
                interpreter_path = relative_interpreter_path.encode('utf-8')
                buf.write(interpreter_path)

                # Interpreter path length
                interpreter_path_length = len(interpreter_path).to_bytes(4, 'little')
                buf.write(interpreter_path_length)

                # Magic number
                buf.write(b'UVUV')

                script_data = buf.getvalue()

            script_path = scripts_dir / f'{script.name}.exe'
            script_path.write_bytes(script_data)


if __name__ == '__main__':
    main()
```
@ofek
Copy link
Contributor Author

ofek commented May 23, 2024

FYI since #3717 this is now completely possible to implement and I do so here:

@pfmoore
Copy link
Contributor

pfmoore commented May 23, 2024

I'm confused as to what's happening here. I could test, but I have limited time right now so if someone can explain, I'd be grateful. If I uv pip install a project that has a script entry point, does that entry point by default still use an absolute path to the Python executable binary?

@charliermarsh
Copy link
Member

Yeah. We didn’t change any behavior in uv itself, except we changed our Windows trampoline to accept a relative path (though we never write one to it in uv). Ofek is then post-processing the scripts in Hatch, I think, to rewrite the absolute paths as relative.

@ofek
Copy link
Contributor Author

ofek commented May 24, 2024

Yes that is exactly correct and I can remove that entire logic once such a flag exists in UV! I posted those links mostly to assist in someone doing the implementation.

@charliermarsh charliermarsh mentioned this issue May 27, 2024
Closed
@firai firai mentioned this issue Jul 27, 2024
Open
@charliermarsh charliermarsh mentioned this issue Jul 27, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
configuration Settings and such enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pfmoore @charliermarsh @skoslowski @ofek