Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update armhf container to Debian 11 #332

Merged
merged 2 commits into from
Jun 26, 2023
Merged

Update armhf container to Debian 11 #332

merged 2 commits into from
Jun 26, 2023

Conversation

MaEtUgR
Copy link
Member

@MaEtUgR MaEtUgR commented Jun 22, 2023

Solved Problem

PX4/PX4-Autopilot#21737 (review)

Solution

  • Update armhf container to Debian 11 bullseye
  • Remove Fast-DDS since it's apparently not a dependency anymore

Alternatives

I also tried Debian 12 bookworm and it can be made to work but:

  • Raspberry Pi OS is currently on Debian 11. I don't know why this container is Debian based at all but was assuming it's to have less duplication e.g. on a Raspberry Pi.
  • Debian 12 enables the python 3 externally managed module protection which is not a concern in the scope of this docker container but RUN rm /usr/lib/python3.*/EXTERNALLY-MANAGED needs to be ran before using pip3.
  • Fast-DDS doesn't build, has to be updated which added more dependencies which create problems themselves... That's how I got to the idea to remove it since it's not in the PX4 Ubuntu setup script anymore.
  • The version of compiler has more warnings which make the PX4 build fail in two locations. These can be resolved but I'd like to do that separately.

Test coverage

Context

Image build and PX4 compilation output: 
 ⚙ maetugr@MaEtUgRX1  ~/PX4-containers/docker   maetugr/armhf-debian-11  make px4-dev-armhf                        
docker build -t px4io/px4-dev-armhf . -f Dockerfile_armhf
[+] Building 1.0s (13/13) FINISHED                                                                                                                                                                                                                                                                                               
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                                             0.0s
 => [internal] load build definition from Dockerfile_armhf                                                                                                                                                                                                                                                                  0.0s
 => => transferring dockerfile: 2.35kB                                                                                                                                                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/debian:bullseye                                                                                                                                                                                                                                                          0.9s
 => [1/8] FROM docker.io/library/debian:bullseye@sha256:1e5f2d70c9441c971607727f56d0776fb9eecf23cd37b595b26db7a974b2301d                                                                                                                                                                                                    0.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                           0.0s
 => => transferring context: 70B                                                                                                                                                                                                                                                                                            0.0s
 => CACHED [2/8] RUN apt-get update && apt-get -y --quiet --no-install-recommends install   bzip2   ca-certificates   ccache   clang   clang-tidy   cmake   cppcheck   curl   default-jdk-headless   dirmngr   doxygen   file   g++   g++-arm-linux-gnueabihf   gcc   gcc-arm-linux-gnueabihf   gdb   gdb-multiarch   git   0.0s
 => CACHED [3/8] RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10                                                                                                                                                                                                                               0.0s
 => CACHED [4/8] RUN pip3 install wheel setuptools                                                                                                                                                                                                                                                                          0.0s
 => CACHED [5/8] RUN pip3 install argparse argcomplete coverage cerberus empy jinja2 kconfiglib   matplotlib==3.0.* numpy packaging pkgconfig pyros-genmsg pyulog   pyyaml requests serial six toml psutil pyulog wheel jsonschema                                                                                          0.0s
 => CACHED [6/8] RUN wget -q https://downloads.sourceforge.net/project/astyle/astyle/astyle%203.1/astyle_3.1_linux.tar.gz -O /tmp/astyle.tar.gz  && cd /tmp && tar zxf astyle.tar.gz && cd astyle/src  && make -f ../build/gcc/Makefile -j$(nproc) && cp bin/astyle /usr/local/bin  && rm -rf /tmp/*                        0.0s
 => CACHED [7/8] RUN useradd --shell /bin/bash -u 1001 -c "" -m user && usermod -a -G dialout user                                                                                                                                                                                                                          0.0s
 => CACHED [8/8] COPY scripts/entrypoint.sh /usr/local/bin/entrypoint.sh                                                                                                                                                                                                                                                    0.0s
 => exporting to image                                                                                                                                                                                                                                                                                                      0.0s
 => => exporting layers                                                                                                                                                                                                                                                                                                     0.0s
 => => writing image sha256:b09f2eec659cdc62d6194f60df44cde97f3e47a1d18c96ce2410a81c361a2c4c                                                                                                                                                                                                                                0.0s
 => => naming to docker.io/px4io/px4-dev-armhf                                                                                                                                                                                                                                                                              0.0s

 ⚙ maetugr@MaEtUgRX1  ~/PX4-containers/docker   maetugr/armhf-debian-11  docker run -it --rm px4io/px4-dev-armhf
root@3f49ab46cf3a:/# git clone https://github.com/PX4/PX4-Autopilot.git
Cloning into 'PX4-Autopilot'...
remote: Enumerating objects: 442564, done.
remote: Counting objects: 100% (1381/1381), done.
remote: Compressing objects: 100% (714/714), done.
remote: Total 442564 (delta 882), reused 977 (delta 652), pack-reused 441183
Receiving objects: 100% (442564/442564), 209.52 MiB | 13.30 MiB/s, done.
Resolving deltas: 100% (326829/326829), done.
root@3f49ab46cf3a:/# cd PX4-Autopilot/

root@3f49ab46cf3a:/PX4-Autopilot# make beaglebone_blue && make emlid_navio2 && make px4_raspberrypi && make scumaker_pilotpi
-- PX4 version: v1.14.0-beta2-265-g7ed90c6d0c (1.14.0)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.2", minimum required is "3") 
-- PX4 config file: /PX4-Autopilot/boards/beaglebone/blue/default.px4board
-- PLATFORM posix
-- LINUX_TARGET y
-- TOOLCHAIN arm-linux-gnueabihf
-- ARCHITECTURE cortex-a8
-- ROMFSROOT px4fmu_common
-- ROOTFSDIR .
-- TESTING y
-- PX4 config: beaglebone_blue_default
-- PX4 platform: posix
-- PX4 lockstep: disabled
-- The CXX compiler identification is GNU 10.2.1
-- The C compiler identification is GNU 10.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-linux-gnueabihf-gcc
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/arm-linux-gnueabihf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/arm-linux-gnueabihf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- cmake build type: RelWithDebInfo
-- ccache enabled via symlink (/usr/lib/ccache/arm-linux-gnueabihf-g++ -> /usr/bin/ccache)
-- Found Java: /usr/bin/java (found version "11.0.18") 
-- ROMFS: ROMFS/px4fmu_common
-- ROMFS:  Adding boards/beaglebone/blue/init/rc.board_defaults -> /etc/init.d/rc.board_defaults
Architecture:  amd64
==> CPACK_INSTALL_PREFIX = @DEB_INSTALL_PREFIX@
-- Configuring done
-- Generating done
-- Build files have been written to: /PX4-Autopilot/build/beaglebone_blue_default
[0/1005] git submodule src/modules/uxrce_dds_client/Micro-XRCE-DDS-Client
[5/1005] git submodule src/drivers/gps/devices
[7/1005] git submodule src/modules/mavlink/mavlink
[14/1005] Generating Mavlink uAvionix: src/modules/mavlink/mavlink/message_definitions/v1.0/uAvionix.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[16/1005] Performing download step (git clone) for 'librobotcontrol'
-- librobotcontrol download command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/librobotcontrol-prefix/src/librobotcontrol-stamp/librobotcontrol-download-*.log
[19/1005] Performing configure step for 'librobotcontrol'
-- librobotcontrol configure command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/librobotcontrol-prefix/src/librobotcontrol-stamp/librobotcontrol-configure-*.log
[20/1005] Generating Mavlink common: src/modules/mavlink/mavlink/message_definitions/v1.0/common.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[21/1005] Performing build step for 'librobotcontrol'
-- librobotcontrol build command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/librobotcontrol-prefix/src/librobotcontrol-stamp/librobotcontrol-build-*.log
[68/1005] Performing configure step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project configure command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-configure-*.log
[992/1005] Performing build step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project build command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-build-*.log
[993/1005] Performing install step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project install command succeeded.  See also /PX4-Autopilot/build/beaglebone_blue_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-install-*.log
[1005/1005] Linking CXX shared library src/examples/dyn_hello/examples__dyn_hello.px4mod
-- PX4 version: v1.14.0-beta2-265-g7ed90c6d0c (1.14.0)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.2", minimum required is "3") 
-- PX4 config file: /PX4-Autopilot/boards/emlid/navio2/default.px4board
-- PLATFORM posix
-- LINUX_TARGET y
-- TOOLCHAIN arm-linux-gnueabihf
-- ARCHITECTURE cortex-a53
-- ROMFSROOT px4fmu_common
-- ROOTFSDIR .
-- TESTING y
-- PX4 config: emlid_navio2_default
-- PX4 platform: posix
-- PX4 lockstep: disabled
-- The CXX compiler identification is GNU 10.2.1
-- The C compiler identification is GNU 10.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-linux-gnueabihf-gcc
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/arm-linux-gnueabihf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/arm-linux-gnueabihf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- cmake build type: RelWithDebInfo
-- ccache enabled via symlink (/usr/lib/ccache/arm-linux-gnueabihf-g++ -> /usr/bin/ccache)
-- Found Java: /usr/bin/java (found version "11.0.18") 
-- ROMFS: ROMFS/px4fmu_common
-- ROMFS:  Adding boards/emlid/navio2/init/rc.board_defaults -> /etc/init.d/rc.board_defaults
Architecture:  amd64
==> CPACK_INSTALL_PREFIX = @DEB_INSTALL_PREFIX@
-- Configuring done
-- Generating done
-- Build files have been written to: /PX4-Autopilot/build/emlid_navio2_default
[0/1008] git submodule src/modules/uxrce_dds_client/Micro-XRCE-DDS-Client
[5/1008] git submodule src/drivers/gps/devices
[9/1008] git submodule src/modules/mavlink/mavlink
[17/1008] Generating Mavlink uAvionix: src/modules/mavlink/mavlink/message_definitions/v1.0/uAvionix.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[19/1008] Generating Mavlink common: src/modules/mavlink/mavlink/message_definitions/v1.0/common.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[56/1008] Performing configure step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project configure command succeeded.  See also /PX4-Autopilot/build/emlid_navio2_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-configure-*.log
[226/1008] Performing build step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project build command succeeded.  See also /PX4-Autopilot/build/emlid_navio2_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-build-*.log
[699/1008] Performing install step for 'libmicroxrceddsclient_project'
-- libmicroxrceddsclient_project install command succeeded.  See also /PX4-Autopilot/build/emlid_navio2_default/src/modules/uxrce_dds_client/src/libmicroxrceddsclient_project-stamp/libmicroxrceddsclient_project-install-*.log
[1008/1008] Linking CXX shared library src/examples/dyn_hello/examples__dyn_hello.px4mod
-- PX4 version: v1.14.0-beta2-265-g7ed90c6d0c (1.14.0)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.2", minimum required is "3") 
-- PX4 config file: /PX4-Autopilot/boards/px4/raspberrypi/default.px4board
-- PLATFORM posix
-- LINUX_TARGET y
-- TOOLCHAIN arm-linux-gnueabihf
-- ARCHITECTURE cortex-a53
-- ROMFSROOT px4fmu_common
-- ROOTFSDIR .
-- TESTING y
-- PX4 config: px4_raspberrypi_default
-- PX4 platform: posix
-- PX4 lockstep: disabled
-- The CXX compiler identification is GNU 10.2.1
-- The C compiler identification is GNU 10.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-linux-gnueabihf-gcc
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/arm-linux-gnueabihf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/arm-linux-gnueabihf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- cmake build type: RelWithDebInfo
-- ccache enabled via symlink (/usr/lib/ccache/arm-linux-gnueabihf-g++ -> /usr/bin/ccache)
-- Found Java: /usr/bin/java (found version "11.0.18") 
-- ROMFS: ROMFS/px4fmu_common
Architecture:  amd64
==> CPACK_INSTALL_PREFIX = @DEB_INSTALL_PREFIX@
-- Configuring done
-- Generating done
-- Build files have been written to: /PX4-Autopilot/build/px4_raspberrypi_default
[0/976] git submodule src/drivers/gps/devices
[2/976] git submodule src/modules/mavlink/mavlink
[14/976] Generating Mavlink uAvionix: src/modules/mavlink/mavlink/message_definitions/v1.0/uAvionix.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[16/976] Generating Mavlink common: src/modules/mavlink/mavlink/message_definitions/v1.0/common.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[976/976] Linking CXX shared library src/examples/dyn_hello/examples__dyn_hello.px4mod
-- PX4 version: v1.14.0-beta2-265-g7ed90c6d0c (1.14.0)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.9.2", minimum required is "3") 
-- PX4 config file: /PX4-Autopilot/boards/scumaker/pilotpi/default.px4board
-- PLATFORM posix
-- LINUX_TARGET y
-- TOOLCHAIN arm-linux-gnueabihf
-- ARCHITECTURE cortex-a53
-- ROMFSROOT px4fmu_common
-- ROOTFSDIR .
-- TESTING y
-- PX4 config: scumaker_pilotpi_default
-- PX4 platform: posix
-- PX4 lockstep: disabled
-- The CXX compiler identification is GNU 10.2.1
-- The C compiler identification is GNU 10.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-linux-gnueabihf-gcc
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/arm-linux-gnueabihf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/arm-linux-gnueabihf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- cmake build type: RelWithDebInfo
-- ccache enabled via symlink (/usr/lib/ccache/arm-linux-gnueabihf-g++ -> /usr/bin/ccache)
-- ROMFS: ROMFS/px4fmu_common
-- ROMFS:  Adding boards/scumaker/pilotpi/init/rc.board_defaults -> /etc/init.d/rc.board_defaults
Architecture:  amd64
==> CPACK_INSTALL_PREFIX = @DEB_INSTALL_PREFIX@
-- Configuring done
-- Generating done
-- Build files have been written to: /PX4-Autopilot/build/scumaker_pilotpi_default
[0/962] git submodule src/drivers/gps/devices
[2/962] git submodule src/modules/mavlink/mavlink
[14/962] Generating Mavlink uAvionix: src/modules/mavlink/mavlink/message_definitions/v1.0/uAvionix.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[16/962] Generating Mavlink common: src/modules/mavlink/mavlink/message_definitions/v1.0/common.xml
WARNING: Failed to import lxml module etree. Are lxml, libxml2 and libxslt installed? XML validation will not be performed
[962/962] Linking CXX shared library src/examples/dyn_hello/examples__dyn_hello.px4mod
root@3f49ab46cf3a:/PX4-Autopilot#

@MaEtUgR
armhf: update to Debian 11 bullseye
a81da36 
It contains a newer version of cmake which we require now.
@MaEtUgR
armhf: remove Fast-DDS from container
5677abf
@MaEtUgR MaEtUgR requested a review from dagar June 22, 2023 10:02
@MaEtUgR MaEtUgR self-assigned this Jun 22, 2023
@MaEtUgR MaEtUgR mentioned this pull request Jun 22, 2023
Merged
@MaEtUgR
Copy link
Member Author

MaEtUgR commented Jun 26, 2023

We discussed that CI should process and publish this to docker hub such that it can be used in PX4/PX4-Autopilot#21749
So it has to be merged and tagged, see

PX4-containers/.github/workflows/docker_builds.yml

Line 69 in 774b3b5

if: github.event_name == 'push' && (contains(github.ref, '/heads/master') || contains(github.ref, '/tags/'))

@MaEtUgR MaEtUgR merged commit f61afda into master Jun 26, 2023
@MaEtUgR MaEtUgR deleted the maetugr/armhf-debian-11 branch June 26, 2023 17:37
MaEtUgR added a commit to PX4/PX4-Autopilot that referenced this pull request Jun 27, 2023
@MaEtUgR
GitHub Actions: use Debian 11 to build linux targets
2320668 
This solves the CI problem related to the required cmake version
described here:
#21737 (review)

See PX4/PX4-containers#332
for the container changes.
MaEtUgR added a commit to PX4/PX4-Autopilot that referenced this pull request Jun 27, 2023
@MaEtUgR
GitHub Actions: use Debian 11 to build linux targets
e550348 
This solves the CI problem related to the required cmake version
described here:
#21737 (review)

See PX4/PX4-containers#332
for the container changes.
harrisondragoon pushed a commit to harrisondragoon/PX4-Autopilot that referenced this pull request Jun 30, 2023
@MaEtUgR @harrisondragoon
GitHub Actions: use Debian 11 to build linux targets
8a6b9b6 
This solves the CI problem related to the required cmake version
described here:
PX4#21737 (review)

See PX4/PX4-containers#332
for the container changes.
antbre pushed a commit to BioMorphic-Intelligence-Lab/PX4-Autopilot that referenced this pull request Sep 14, 2023
@MaEtUgR @antbre
GitHub Actions: use Debian 11 to build linux targets
f27deed 
This solves the CI problem related to the required cmake version
described here:
PX4#21737 (review)

See PX4/PX4-containers#332
for the container changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dagar dagar Awaiting requested review from dagar

Assignees

@MaEtUgR MaEtUgR

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@MaEtUgR