SQLite not working

[Blechlawine]

I tried setting up a local instance of shynet, to test it, but the setup with SQLite did not work. I uncommented the sqlite settings in the .env template file and commented the postgres settings. I tried the command mentioned in the setup guide (docker run --env-file=.env milesmcc/shynet), which threw an error:

Performing startup checks...
Running migrations (setting up DB)...
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 209, in get_new_connection
    conn = Database.connect(**conn_params)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/shynet/./manage.py", line 21, in <module>
    main()
  File "/usr/src/shynet/./manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 92, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/loader.py", line 220, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 55, in has_table
    with self.connection.cursor() as cursor:
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 259, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 235, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 218, in ensure_connection
    with self.wrap_database_errors:
  File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/django/db/backends/sqlite3/base.py", line 209, in get_new_connection
    conn = Database.connect(**conn_params)
django.db.utils.OperationalError: unable to open database file
Migrations failed, exiting

It threw the same error without mounting the folder with the database file into the container, and it also failed with mounting the database file directly (./database/db.sqlite3:/var/local/shynet/db/db.sqlite3:rw)

This docker-compose, which should do the same if I'm not mistaken, did also fail with the same error

version: "3.0"
services:
  shynet:
    image: milesmcc/shynet
    container_name: shynet
    restart: unless-stopped
    ports:
      - 8080:8080
    volumes:
      - ./database/db.sqlite3:/var/local/shynet/db/db.sqlite3:rw
    env_file: .env

I have no idea where the problem could be, because I don't know how django handles databases.

[milesmcc]

CC @haplo

[haplo]

Sorry I'm late. @Blechlawine Shynet with SQLite is working in my personal instance, so there must be something wrong with your configuration. I suggest you check the permissions and owner of not just the DB file but the directories containing it. When I was getting it to work I discovered that the parent directory had to be writable, otherwise SQLite would fail even if the file itself was writable.

[Blechlawine]

The weird thing is that it's also not working if I don't mount the directory, then permissions should just be the default as in the docker image. Also, the parent directory is also writable by the user the container is running as, since its writable by everyone.

[Blechlawine]

One question @haplo: what os are you running?

[haplo]

One question @haplo: what os are you running?

My Shynet instance is running on Debian 11 using podman.

[Blechlawine]

Ok, so I tried running it on windows 10, with docker desktop for windows, with the engine running in wsl. I don't know if that is the problem, but in the wsl terminal the permissions looked correct, but I couldn't change them, and in windows cmd they were wrong and I couldn't change them. I dual booted into linux/ubuntu and while windows and wsl said my user owned the folder, linux dual booted said it belonged to root. And again, I can't change it... Which has apparently something to do with how the drive is mounted.
So I tried it on another drive, where I can actually change the permissions in linux. And it also throws the same error, permissions being right again. Even with 777 permissions it fails. I don't know what else I could try.

[0b11stan]

Hey, I had the same issue here, I think it's because you pull from milesmcc/shynet:latest in the docker hub.

I built the project from the Dockerfile and it worked.
5 months ago this pull request fixed the issue.
However, the latest tag in docker hub is 9 months old.

The edge tag seams to be updated more often but it's not working out of the box for me so the best solution I think is to pull the repo and build from the Dockerfile.

[Blechlawine]

I built the project from the Dockerfile and it worked.

That's a good idea, I haven't tried that yet. I'll see if it works for me. Thanks!

[Blechlawine]

Update: I tried the edge tag and got it running, but creating an admin user (command: registeradmin) failed, because password = get_random_string()(registeradmin.py, line: 23) requires a length parameter (@haplo Should I open an issue for that, or is that already fixed?) In the code I cloned to build the container, it isn't fixed.

Then I tried building the docker image from source. I couldn't find anything in the guide so I just winged it with docker build . -t shynet which didn't throw an error when building, but when trying to start a container with that image, it failed with:

shynet_main  | exec ./entrypoint.sh: no such file or directory
shynet_main exited with code 1

[0b11stan]

It is possible to run the container from source, I don't know how you do run yours but for me the following is working just fine: docker build -t shynet . && docker run -v /tmp/test:/var/local/shynet/db --env-file shynet.env shynet is starting the container.
However, I have the same issue for the registeradmin step, i think opening a new issue may be a good idea.
Edit: I opened the issue : #234

[0b11stan]

FYI: The fix has been merged

[Blechlawine]

Its working with edge docker image for me now 👍

[milesmcc]

Good!

[srad]

In which image version is the fix? I just tried to use your project and got also "django.db.utils.OperationalError: could not connect to server: No such file or directory" with the :latest tag.

[haaavk]

Try :edge tag.