Docker image fails because of the missing libboost_regex.so.1.65.1 shared lib

[MohamedBassem]

Building the docker image in docker/Dockerfile.ubuntu completes successfully but whenever you try to run it, it fails with:

$ docker run -it --rm logdevice /usr/local/bin/ld-dev-cluster                                                                                             
/usr/local/bin/ld-dev-cluster: error while loading shared libraries: libboost_regex.so.1.65.1: cannot open shared object file: No such file or directory

And ldshell fails with:

root@950359196c00:/usr/local/lib# /usr/local/bin/ldshell
Traceback (most recent call last):
  File "/usr/local/bin/ldshell", line 11, in <module>
    load_entry_point('ldshell==0.1', 'console_scripts', 'ldshell')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.6/dist-packages/ldshell-0.1-py3.6.egg/ldshell/main.py", line 11, in <module>
  File "/usr/local/lib/python3.6/dist-packages/ldshell-0.1-py3.6.egg/ldshell/logdevice_plugin.py", line 11, in <module>
  File "/usr/local/lib/python3.6/dist-packages/ldshell-0.1-py3.6.egg/ldshell/autoload/commands/logsconfig.py", line 12, in <module>
ImportError: libboost_python3-py36.so.1.65.1: cannot open shared object file: No such file or directory

Running apt-get update && apt-get install libboost-regex1.65.1 libboost-python1.65.1 in the image solves the issue.

I checked the image build logs and found:

Removing pinentry-curses (1.1.0-1) ...
Removing libassuan0:amd64 (2.5.1-2) ...
Removing libboost-python1.65.1 (1.65.1+dfsg-0ubuntu5) ...
Removing libboost-regex1.65.1:amd64 (1.65.1+dfsg-0ubuntu5) ...
Removing openssh-client (1:7.6p1-4) ...

It seems that the cleanup that happens in the Dockerfile here removes some of the runtime dependencies as well.

[calebmarchent]

Yes, I can confirm we have been a little overzealous in clean-up of Dockerfile, having completed the build.

[mateusz-gozdek-sociomantic]

Maybe rather than cleaning up the dependencies, use multi step build (2nd FROM and COPY) and just copy over binaries and install runtime dependencies? This way you don't need to do any cleanups, which might be error-prone (as the issue).

[AhmedSoliman]

That sounds like a good strategy @mateusz-gozdek-sociomantic. Would you like to send a PR to address that?

[mateusz-gozdek-sociomantic]

I'd have to sing CLA I guess, which seems to be too much hassle :)

I'd split ubuntu.deps into build deps and runtime deps and then follow https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#use-multi-stage-builds

[AhmedSoliman]

@MohamedBassem are you going to send a PR for this or should we close that one?

[calebmarchent]

@MohamedBassem has passed his work-in progress changes over to me, I'll be picking this up next

[calebmarchent]

Update taking @MohamedBassem 's work forward; I now have a work in progress branch on my fork:
https://github.com/calebmarchent/LogDevice/tree/multistep_docker_ubuntu
This builds a docker image that can run the LogDevice cluster; image size: 340MB.

Next I'll add in more components to get all the tools working and do some testing.

[calebmarchent]

I have published a pull request: #23

[calebmarchent]

PR #23 is now merged; closing issue