Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I use Poetry and Nuitka/Nuitka-Action@main, Nuitka cannot utilize the Python interpreter from the Poetry virtual environment. #55

Open
xhatt opened this issue Jun 19, 2024 · 6 comments
Labels
help wanted Extra attention is needed

Comments

@xhatt
Copy link

xhatt commented Jun 19, 2024

This is Actions file

name: Nuitka on Windows

on:
  workflow_dispatch:

jobs:
  build:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install poetry
        run: pip install poetry
      - uses: actions/setup-python@v5
        with:
          python-version-file: 'pyproject.toml'
          cache: 'poetry'
      - name: Configure Poetry
        run: poetry config virtualenvs.create false
      - name: Install dependencies
        run: |
          poetry source remove tsinghua
          poetry lock --no-update
          poetry install
          poetry show
          poetry env list
          poetry env info
      - name: Show pip list
        run: python -m pip list
      - name: Build package
        uses: Nuitka/Nuitka-Action@main
        with:
          nuitka-version: main
          script-name: main.py
          onefile: true
          enable-plugins: tk-inter
          include-data-files: |
            ./dll/*.dll=dll/
            logo.ico=logo.ico
          windows-icon-from-ico: logo.ico
          output-file: 推特助手.exe
#          windows-console-mode: true


      - name: Upload Artifacts
        uses: actions/upload-artifact@v4
        with:
          name: twitter_bot
          path: build/*.exe

python -m pip list no things

Output:

Run python -m pip list
  python -m pip list
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.1[2](https://github.com/xhatt/twitter_bot/actions/runs/9586123141/job/26433411787#step:7:2).3\x64
    PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\[3](https://github.com/xhatt/twitter_bot/actions/runs/9586123141/job/26433411787#step:7:3).12.3\x64/lib/pkgconfig
    Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.12.3\x6[4](https://github.com/xhatt/twitter_bot/actions/runs/9586123141/job/26433411787#step:7:4)
    Python2_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.12.3\x[6](https://github.com/xhatt/twitter_bot/actions/runs/9586123141/job/26433411787#step:7:6)4
    Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.12.3\x64
Package Version
------- -------
pip     24.0

but poetry show output

aiohttp                   3.9.5    Async http client/server framework (asyn...
aiohttp-socks             0.8.4    Proxy connector for aiohttp
aiosignal                 1.3.1    aiosignal: a list of registered asynchro...
altgraph                  0.17.4   Python graph (network) package
annotated-types           0.6.0    Reusable constraint types to use with ty...
async-timeout             4.0.3    Timeout context manager for asyncio prog...
attrs                     23.2.0   Classes Without Boilerplate
certifi                   2024.6.2 Python package for providing Mozilla's C...
cffi                      1.16.0   Foreign Function Interface for Python ca...
charset-normalizer        3.3.2    The Real First Universal Charset Detecto...

This will result in Nuitka being unable to obtain a valid dependency environment. I also tried to install Poetry directly in the local environment, but it doesn't seem to work either.
Thank you very much if it can be resolved, otherwise I can only use pip. Perhaps there could be a parameter that can mark where to use Python from?
——Translate through Baidu

@kayhayen
Copy link
Member

Poetry doesn't integrate with Nuitka as a build system, but that is not attempted to be used here. Windows console mode is not a boolean, but a choice of values, if you mean to have true, that would be "force", which currently is the default.

What error symptoms do you get "obtain a valid dependency environment" seems odd. The Nuitka action probably doesn't run inside a "poetry run" or so, which might be needed. I guess, it would have to use pipenv, poetry, pdm etc config files, but currently that is not implemented. I welcome PRs to do so.

@kayhayen kayhayen added the help wanted Extra attention is needed label Jun 20, 2024
@xhatt
Copy link
Author

xhatt commented Jun 20, 2024

Poetry 没有与 Nuitka 集成作为构建系统,但这里不尝试使用它。 Windows 控制台模式不是一个布尔值,而是一个值的选择,如果你的意思是 true,那就是“force”,目前这是默认值。

您收到的错误症状“获得有效的依赖环境”似乎很奇怪。 Nuitka 动作可能不会在“诗歌运行”之类的内部运行,而这可能是需要的。我想,它必须使用 pipelinev、poetry、pdm 等配置文件,但目前尚未实现。我欢迎 PR 这样做。

Thank you very much for your reply.

I looked at the action.yml file in the repository. I was wondering if it would be possible to add a parameter to run Nuitka from a specified environment. Currently, it uses python -m nuitka, but maybe it could be changed to /mypath/python -m nuitka. However, in this case, I would need to install Nuitka in my environment.

Another approach could be to provide a preceding step to install Nuitka into the specified environment. I don’t have any particularly good solutions. Yesterday, I managed to build it using pip, but it took about 45 minutes for each build. This feels too long. It might be because the CPU of the server provided by the action is too weak. I'm not sure how to speed it up; it takes about 8 minutes on my own computer.

@xhatt
Copy link
Author

xhatt commented Jun 20, 2024

This might not be an issue with Nuitka, but rather a problem with Poetry installing dependencies. If Poetry could install the packages into the local environment instead of a virtual environment, the issue would be resolved.

Perhaps tools like Poetry and Pipenv could be built into the action? This way, users wouldn't need to install them separately and could use them directly. From my research, it seems the problem is that when I used Poetry to install dependencies earlier, they were not installed into the local environment.

@kayhayen
Copy link
Member

Yes, they could and probably should be, maybe picking it up from the environment would be good. Nuitka generally would e.g. be expected to be ran like python -m pipenv run nuitka so as to not care. Entering an environment is nothing Nuitka yet does, but I would add it to Nuitka. But if the action knew it via configuration, what to run as, it would work, but it would need docs and what not.

Instead, for the action, we probably could use a --use-environment that detects a pipenv or poetry folder and stuff, this is plumbing, and if you can figure out, how to e.g. detect poetry usage in a directory and such, it would be great. Another way would be to add compiling .pyproject files, which for poetry would if it worked, do things automatically. pipenv has no interaction with those files, but a local virtualenv that can be found by asking it (if available). It's tedious and nothing I can spend time on right now to be honest. But I wish we had that.

@roger-duthie-pivotal
Copy link

I had an issue which was related to this; I feel I should add a note here to help others with similar issues. My Python project uses a pyproject.toml/poetry.lock to define the dependencies and I had a step which installed the I found that Nuitka wasn't putting the packages into the executable - specifically dotenv from the python-dotenv package.

What worked for me:

name: Build EXE using poetry

on:
  workflow_dispatch:

jobs:
  build-windows:
    runs-on: windows-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.10.10'
          architecture: 'x64'
      
      - name: Install dependencies
        env:
          POETRY_VIRTUALENVS_CREATE: false 
        run: |
          python -m pip install --upgrade pip
          python -m pip install poetry
          poetry install --no-cache --no-interaction
 
      - name: Debug Packages
        run: |
          pip list
          poetry show
    
      - name: Build Python script
        uses: Nuitka/Nuitka-Action@main
        with:
          nuitka-version: main
          script-name: [SCRIPT.PY]
          include-module: dotenv

      - name: Upload Windows Artifact
        uses: actions/upload-artifact@v3
        with:
          name: [ARCHIVE_NAME]
          path: [PATH/SCRIPT.EXE]

@roger-duthie-pivotal
Copy link

I agree that it would be great to have Nuitka be able have a configuration to use a given virtual environment. The Poetry team don't encourage the use of poetry to global and for good reasons! Having the dependencies for the target encapsulated in a virtual environment, which is then used by the build process would be cleaner. Also, it would be clearer when something is going wrong; it took me a long time to work out what was going wrong with my config (I was previously using poetry install without setting the POETRY_VIRTUALENVS_CREATE=false env var.

(Anyway, thank you for providing a great tool like Nuitka. I hope my comments might help the next user get up their learning curve faster!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants