ObjectDetectionCoral stops working when restarting the docker container.

[mikegleasonjr]

Area of Concern

• Server
• Behaviour of ObjectDetectionCoral
• Installer
• Runtime [e.g. Python3.7, .NET]
• Module packages [e.g. PyTorch)
• Something else

Describe the bug

1. I am running a docker container with the codeproject/ai-server image with 2 volumes (so that I can restart it without having to reinstall/reconfigure everything):

• /etc/codeproject/ai
• /app/modules

2. I installed the Object Detection (Coral) module, which worked fine with my TPU. I called the API and verified that everything was working.
3. I stopped the container
4. I started back the container
5. The module now has this error: objectdetection_coral_adapter.py: An exception occurred initialising the module: libedgetpu.so.1: cannot open shared object file: No such file or dir

Expected behavior
The module restarts properly in the docker container.

I know what the problem actually is: In ObjectDetectionCoral/install.sh, there is a line to copy the shared library in /usr/lib/x86_64-linux-gnu/ upon module installation:

cp "${moduleDirPath}/edgetpu_runtime/${edgetpu_folder}/k8/libedgetpu.so.1.0" "/usr/lib/x86_64-linux-gnu/libedgetpu.so.1.0"

Obvisouly when restarting the container, this change is lost since I do not mount /usr/lib/x86_64-linux-gnu/ as a volume to be persistent across "reboots".

I don't know if modules have a startup hook where these steps of copying the shared object and running of ldconfig could be done there?

Screenshots
N/A

Your System (please complete the following information):

• CodeProject.AI Server version: codeproject/ai-server:2.6.2
• Object Detection (Coral) version: 2.2.0

Additional context
See Expected behavior

[matthewDDennis]

How are you starting the Docker container?

• if a command line Docker run, add --restart=always. This might fix things, but might not.

[mikegleasonjr]

Hi @matthewDDennis, thanks for the help. It is my feeling that the bug report wasn't fully understood though. The bug is actually known and described in the "Expected behavior" section (at least I tried to explain it). This isn't about Docker not restarting the container.

Please feel free to ask questions if it is still misunderstood 🙏🏻

[mikegleasonjr]

Maybe the ObjectDetectionCoral module should redo some steps it does upon installation in the initialise phase of the module: https://www.codeproject.com/ai/docs/devguide/module_examples/add_python_module.html#writing-the-module

It copies files into the host at the location: /usr/lib/x86_64-linux-gnu/. Those files are obviously lost when restarting the container.

There is no mention about mounting other volumes other than the settings/module folders in the documentation: https://www.codeproject.com/ai/docs/install/running_in_docker.html#advanced-docker-launch-settings-saved-outside-of-the-container

[galperinm]

@mikegleasonjr How did you end up working around this? Did you have to mount usr/lib/x86_64-linux-gnu/ as a persistent volume?

[mikegleasonjr]

Mounting usr/lib/x86_64-linux-gnu/ as a persistent volume would mean going through hoops to first get the files from the image before the module installation since the directory would be first empty on the host (and then empty in the running container).

I felt the comprehensive bug report wasn't even read according to the comments I had so I felt helpless trying to make ppl understand what was happening and a waste of efforts to go to such an extent to file a bug report. I stopped using the module (and codeprojectai).

So yes I gave an example where some installation files were lost but I don't know if it involves a game of wack-a-mole and if something else would still be missing after a container restart. I think so, I think some apt install are being made upon installation that would be lost upon container restart.

Also the official doc says to mount the Docker image in a certain way so this would contradict what has to be done here to restore the context.

[galperinm]

Thanks for the update @mikegleasonjr. It's a shame this isn't under active development anymore given that such a major bug is unacknowledged half a year later, guess I'll need to move off of codeproject.ai as well. Surprising, I can't imagine you and I are the only two Coral + codeprojectai docker users out there.

If you feel like sharing, I'd love to know what you moved to. My intention was to use this with Frigate and Doubletake for facial recognition supporting Coral. Looks like the alternatives are dead too-- Deepstack is no longer being developed either (docker image last updated over 2 years ago), and Compreface likewise has almost a year since last update.