nvidia docker v2 support

[koenlek]

Add nvidia docker v2 support

A couple of months ago, nvidia released nvidia-docker v2. As it turned out, this broke OpenGL within docker containers when using nvidia GPU. This means that tools like RVIZ wouldn't work anymore when using nvidia docker v2. Luckily, they provides a fix by now, which needs to be inserted into the docker image. Such a patched version works fine again with nvidia, but (at least for me) doesn't seem to work anymore with e.g. Intel GPU acceleration. So, for now, our company resorted to providing two image tags alway: 1 for nvidia, 1 for intel.

To patch an existing docker image, use this Dockerfile:

FROM <some_image>

RUN apt-get update && apt-get install -y --no-install-recommends \
        pkg-config \
        libxau-dev \
        libxdmcp-dev \
        libxcb1-dev \
        libxext-dev \
        libx11-dev && \
    rm -rf /var/lib/apt/lists/*

COPY --from=nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04 \
  /usr/local/lib/x86_64-linux-gnu \
  /usr/local/lib/x86_64-linux-gnu

COPY --from=nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04 \
  /usr/local/share/glvnd/egl_vendor.d/10_nvidia.json \
  /usr/local/share/glvnd/egl_vendor.d/10_nvidia.json

RUN echo '/usr/local/lib/x86_64-linux-gnu' >> /etc/ld.so.conf.d/glvnd.conf && \
    ldconfig && \
    echo '/usr/local/$LIB/libGL.so.1' >> /etc/ld.so.preload && \
    echo '/usr/local/$LIB/libEGL.so.1' >> /etc/ld.so.preload

# nvidia-container-runtime
ENV NVIDIA_VISIBLE_DEVICES \
    ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES \
    ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics

For Autoware, you need to take care of the user too, so it would become:

FROM autoware/autoware:1.6.2-kinetic
USER root
RUN apt-get update && apt-get install -y --no-install-recommends \
        pkg-config \
        libxau-dev \
        libxdmcp-dev \
        libxcb1-dev \
        libxext-dev \
        libx11-dev && \
    rm -rf /var/lib/apt/lists/*

COPY --from=nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04 \
  /usr/local/lib/x86_64-linux-gnu \
  /usr/local/lib/x86_64-linux-gnu

COPY --from=nvidia/opengl:1.0-glvnd-runtime-ubuntu16.04 \
  /usr/local/share/glvnd/egl_vendor.d/10_nvidia.json \
  /usr/local/share/glvnd/egl_vendor.d/10_nvidia.json

RUN echo '/usr/local/lib/x86_64-linux-gnu' >> /etc/ld.so.conf.d/glvnd.conf && \
    ldconfig && \
    echo '/usr/local/$LIB/libGL.so.1' >> /etc/ld.so.preload && \
    echo '/usr/local/$LIB/libEGL.so.1' >> /etc/ld.so.preload

# nvidia-container-runtime
ENV NVIDIA_VISIBLE_DEVICES \
    ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES \
    ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics

USER autoware

To build:

docker build --tag autoware/autoware:1.6.2-kinetic-nvidia .

Then to run, all you need to do is change your docker run command to something like:

# old (nvidia-docker v1):
nvidia-docker run <args> <img> <command> 

# new (nvidia-docker v2):
docker run --runtime=nvidia  <args> <img>-nvidia <command>

Could you add support for this directly from Autoware?

[amc-nu]

@koenlek thank you for your suggestion, sounds nice.
could you please create a PR targeted to the develop branch with these changes?

[koenlek]

@amc-nu : I made the PR a couple of days ago. Please go ahead and let me know if its ok like this.

[amc-nu]

Thanks @koenlek, I'll check it soon.

[twbabyduck]

Hi,
Here is the link to docker image for autoware (1.9.1-kinetic) with OpenGL support
The error of launching RVIZ should be fixed now : )

You can simply just docker pull the image for testing

docker pull twbabyduck/autoware:1.9.1-kinetic-opengl

[MouriNaruto]

I found somebody do a similar thing at autowarefoundation/autoware#1946.

Mouri