prefect server start raises PermissionError on Ubuntu

[marvin-robot]

Archived from the Prefect Public Slack Community

loscalzo.jony: Hi all, I've just trying to to execute "prefect server start" using "backend" server, so I supposed that this issue could be permissions on pyenv:

ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml
............
............
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.

Operating System: Ubuntu 18.04.5 LTS
Kernel: Linux 5.4.0-42-generic
Architecture: x86-64
Using pyenv with python 3.7.9

Thanks in advance!

jim: Hi Jonathan, you've installed prefect using pyenv I take it? What version of prefect do you have installed? What command did you run that resolved the above error?

loscalzo.jony: yes, wait me a second..

loscalzo.jony: With pyenv I executed:

pyenv shell 3.7.9

Then install prefect:
pip install prefect

In the end, execute
prefect backend server
prefect server start

would you like a gist with the completed error?

jim: If you could post it here, formatted appropriately between backticks that'd be quite helpful, yes. Thanks!

loscalzo.jony: Yes, of course.

I don't know if it is readable 😣

$ prefect backend server
Backend switched to server

$ prefect server start
ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml
        
Exception caught; killing services (press ctrl-C to force)
ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml
        
Traceback (most recent call last):
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/prefect/cli/server.py", line 332, in start
    ["docker-compose", "pull"], cwd=compose_dir_path, env=env
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', 'pull']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jloscalzo/.pyenv/versions/3.7.9/bin/prefect", line 8, in <module>
    sys.exit(cli())
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/site-packages/prefect/cli/server.py", line 366, in start
    ["docker-compose", "down"], cwd=compose_dir_path, env=env
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/home/jloscalzo/.pyenv/versions/3.7.9/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker-compose', 'down']' returned non-zero exit status 1.

I've just executed in any folder, (not special folder).
Installation docs doesn't say anything against that...

jim: Can you run prefect diagnostics and report back the output?

loscalzo.jony:

{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Linux-5.4.0-42-generic-x86_64-with-debian-buster-sid",
    "prefect_version": "0.13.5",
    "python_version": "3.7.9"
  }
}

And prefect is installed by pyenv here:

~/.pyenv/versions/3.7.9/lib/python3.7/site-packages/prefect/

In this folders is located the compose-file, I mean CLI should execute it

jim: Hmmm, ok, give me a bit to debug here.

loscalzo.jony: if I try to execute it manually, I receive this error:

$ docker-compose -f docker-compose.yml up
ERROR: .PermissionError: [Errno 13] Permission denied: './docker-compose.yml'

Should it up as a common cmpose-file, right?

jim: Hmmm, that sounds like it might be your problem then. What are the permissions on that file?

jim: Did you use sudo to install prefect (we recommend not doing that)?

loscalzo.jony: nono, I didn't use sudo to install prefect neither pyenv

loscalzo.jony:

-rw-r--r--  1 jloscalzo jloscalzo  3062 sep  7 17:35 docker-compose.yml-rw-r--r--  1 jloscalzo jloscalzo  3062 sep  7 17:35 docker-compose.yml

It is my user, and my user has docker-permissions (docker group)

jim: That's very odd. Perhaps it's permission issues on the docker-compose/docker side? Have you used docker-compose before on this machine, and if so has it worked then?

loscalzo.jony: yes, of course, I have running another stack in this moment 🙂

loscalzo.jony: I didn't try to install prefect with systems-python, which it were installed with ubuntu (in other words, without pyenv)

loscalzo.jony: Also, I've develop a simple script and execute it, it runs ok (python script.py)

So, I'm out of possible issues 😞

jim: So your docker-compose works fine for other projects? Or just docker? The prefect side looks right, and if you can't docker-compose up docker-compose.yml file (which should run as is) that's very odd.

loscalzo.jony: I think the same as you

jim: You wouldn't happen to have used snap to install docker/docker-compose instead of apt?

jim: A bit of googling turned up docker-archive/docker-snap#26?

loscalzo.jony: I use to have issues with docker from snap, so I've uninstalled and reinstalled from docker directly

jim: There's also https://stackoverflow.com/questions/53344380/errno-13-while-running-docker-compose-up, which I guess isn't snap specific?

loscalzo.jony: mmmm, it could be!

jim: I'm not familiar with apparmor, haven't seen this issue before.

loscalzo.jony:

docker              19.03.11                    471   latest/stable     canonical*    -docker              19.03.11                    471   latest/stable     canonical*    -

loscalzo.jony: I didn't uninstalled from snap

loscalzo.jony: I'm a lier

loscalzo.jony: 😣

jim: Ah! well hopefully one of the two links I posted above can help you :)

jim: It sounds like that's the issue then.

loscalzo.jony: Thanks a lot, I supposed that I've just uninstalling it from snap.

Sorry for not googling it 😞

jim: No problem, let us know if you run into any other issues.

chris: <@ULVA73B9P> archive “prefect server start raises PermissionError on Ubuntu”

Original thread can be found here.