Poetry shell; my_app works, but poetry run my_app does not

[smprather]

Hi poets--

I have had this working in the past (like last week), but after reshuffling my python installation, I started a new poetry project. When I do this, it works.

$ poetry shell
Spawning shell within /home/xxxxxx/.cache/pypoetry/virtualenvs/apl-cockpit-5QocjSzm-py3.11
. /home/xxxxxx/.cache/pypoetry/virtualenvs/apl-cockpit-5QocjSzm-py3.11/bin/activate
$ . /home/xxxxxx/.cache/pypoetry/virtualenvs/apl-cockpit-5QocjSzm-py3.11/bin/activate
(apl-cockpit-py3.11) $ apl_cockpit -vvv
3

But when I do this, it doesn't.

(apl-cockpit-py3.11) $ exit
exit
$ poetry run apl_cockpit -vvv
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/proj/xxxxxx/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1137, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'apl_cockpit.apl_cockpit'; 'apl_cockpit' is not a package

This isn't a showstopper, but it's not expected. And I imagine other things will fail, like poetry run pytest. So I would rather see it working the way it's supposed to. Thanks for the help!

Here's some more debug info.

$ poetry run type -a apl_cockpit                                                                                                                                                   
apl_cockpit is /home/xxxxxx/.cache/pypoetry/virtualenvs/apl-cockpit-5QocjSzm-py3.11/bin/apl_cockpit                                                                                

$ cat /home/xxxxxx/.cache/pypoetry/virtualenvs/apl-cockpit-5QocjSzm-py3.11/lib/python3.11/site-packages/ap                                                                         
apl_cockpit-0.1.0.dist-info/apl_cockpit.pth                                                                                                                                       
/proj/yyyyyy/xxxxxx/dev/apl_cockpit/src                                                                                                                                            

$ ls /proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/apl_cockpit.py                                                                                                            
-rw-r--r-- 1 xxxxxx eusers 295 Apr 23 22:38 /proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/apl_cockpit.py                                                                     
/home/xxxxxx/dev/apl_cockpit/src/apl_cockpit                                                                                                                                       

$ ls /proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/__init__.py                                                                                                               
-rw-r--r-- 1 xxxxxx eusers 149 Apr 23 22:38 /proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/__init__.py                                                                        
/home/xxxxxx/dev/apl_cockpit/src/apl_cockpit

$ cat /proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/__init__.py 
from pathlib import Path

with open(f"{Path(__file__).parent}/VERSION") as fh:
    __version__ = fh.read()
__version_info__ = __version__.split(".")

[Secrus]

could you share your pyproject.toml file?

[smprather]

[tool.poetry]                          
name = "apl-cockpit"                   
version = "0.1.0"                      
description = ""                       
authors = ["xxxxx"]
readme = "README.md"                   
packages = [{include = "apl_cockpit", from = "src"}]
                                       
[tool.poetry.dependencies]             
python = "^3.11"                       
click = "^8.1.3"                       
                                       
[tool.poetry.group.dev.dependencies]   
icecream = "^2.1.3"                    

[build-system]                         
requires = ["poetry-core"]             
build-backend = "poetry.core.masonry.api"
                                       
[tool.poetry.scripts]                  
apl_cockpit = "apl_cockpit.apl_cockpit:main"

[[tool.poetry.source]]
name = "internal-pypi"
url = "http://internal-pypi-server:8080/simple/"
default = true   
secondary = false

[smprather]

Here is some more info. I added a syntax error to apl_cockpit.py. So it's getting found/parsed. But when I fix the error, it goes right back to the "... is not a package" error.

foo(
def main():
    print("Hello")

$ poetry run apl_cockpit
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/proj/yyyyyy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 936, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1074, in get_code
  File "<frozen importlib._bootstrap_external>", line 1004, in source_to_code
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/proj/yyyyyy/xxxxxx/dev/apl_cockpit/src/apl_cockpit/apl_cockpit.py", line 1
    foo(
       ^
SyntaxError: '(' was never closed

[smprather]

Everything looks pretty clean. I started with a clean install of Python3.11. Then pip install pipx. Then pipx install poetry (I have a wrapper that sets the pipx env vars to make the PIPX_HOME, etc, all point to our /proj/yyyyyy space).

$ poetry --version
Poetry (version 1.4.2)

$ which poetry
/proj/yyyyyy/bin/poetry

$ ls /proj/yyyyyy/bin/poetry
lrwxrwxrwx 1 xxxxxx afcb 51 Apr 23 03:29 /proj/yyyyyy/bin/poetry -> /proj/yyyyyy/envs/pipx/venvs/poetry/bin/poetry*

$ cat /proj/yyyyyy/envs/pipx/venvs/poetry/bin/poetry
#!/proj/yyyyyy/envs/pipx/venvs/poetry/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from poetry.console.application import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

$ ls /proj/yyyyyy/envs/pipx/venvs/poetry/bin/python
lrwxrwxrwx 1 xxxxxx eusers 10 Apr 23 03:28 /proj/yyyyyy/envs/pipx/venvs/poetry/bin/python -> python3.11*

$ ls /proj/yyyyyy/envs/pipx/venvs/poetry/bin/python3.11
lrwxrwxrwx 1 epramyl eusers 32 Apr 23 03:28 /proj/yyyyyy/envs/pipx/venvs/poetry/bin/python3.11 -> /proj/yyyyyy/bin/python3.11*

$ ls /proj/yyyyyy/bin/python3.11
-rwxr-xr-x 1 xxxxxx afcb 22051152 Apr 23 23:38 /proj/yyyyyy/bin/python3.11*

[smprather]

I still had a poetry install in my home dir (~/.local/bin/poetry). I cleansed all of that and it's working now! There was some crossing of the streams, so the obvious happened, "weird undefined behavior". Thanks for the moral support @Secrus !

[smprather]

Unmarking as answer. I was able to get a simple print("Hello") to work, but uncommenting the rest of the simple startup code got me back to the same error.

[smprather]

I might have to give up. Wipe everything and start from scratch. I'm REALLY worried that when I do that, I'm going to get the same result. Everything looks correct.

[smprather]

Ok. So poetry run is parsing my apl_cockpit.py. I can do debug prints from it. It is when the entry point is attempted to launch that I get the error.

[smprather]

FOUND IT!!! I only need one apl_cockpit in the entry point. Well, there goes ~3 hours....

[tool.poetry.scripts]
#apl_cockpit = "apl_cockpit.apl_cockpit:main"   <-- FAIL
apl_cockpit = "apl_cockpit:main"                <-- WORKS!

[smprather]

The real pain was caused by poetry shell; apl_cocklpit working. That mislead me the whole time.

[smprather]

Found more. Turns out:

1. I was wrong yet again. The "fix" would only work in poetry run, but not poetry shell. So I just reversed the situation.
2. What I have FINALLY found, is that you can't have the entry script name be the same as the project name. My project name is apl_cockpit and my top level script was apl_cockpit.py. If I change apl_cockpit.py to cli.py, everything works consistently.

Is this expected? Was I supposed to know this already?

[dimbleby]

What I have FINALLY found, is that you can't have the entry script name be the same as the project name.

this is certainly not true. poetry itself is an example of a project containing a script with the same name as the project (ie both being poetry)

[smprather]

I just checked the github. The poetry entry point is defined like this. So it is not poetry.poetry:main.

[tool.poetry.scripts]
poetry = "poetry.console.application:main"

[dimbleby]

oh, you don't mean "the name of the entry script" - which to my mind is indeed poetry there, as in poetry = .... What you mean is the name of the file that contains the code?

I could believe there's some special casing around such things in poetry run, and frankly I would anti-recommend poetry shell more or less always

[smprather]

Yes, confusing. My bad. Unfortunately, I tripped over this while simultaneously uprooting my previous Python "world" and re-installing everything. So I was sure it had something to do with that, and not a native Poetry issue. There goes ~5 hours of my life.

I have a hacky little "loop_runner" script that auto-runs my WIP script every time I save a change. I thought it was a weak enough hack that I needed to use it via a poetry shell. But I just tried it using poetry run and it works. So I'll stick with that. Still, IMO, the framework needs to support poetry shell (it's there after all). If this is a limitation, it should be resolved or advertised. I think it's pretty easy to do what I did.