Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update lockfile formating #2402

Merged
merged 7 commits into from
Jun 9, 2021
Merged

Conversation

ruffsl
Copy link
Member

@ruffsl ruffsl commented Jun 9, 2021

Update lockfile format to cleanly list versioned package dependencies for each workspace in a readable table format, and fix up environment so that rosdep correctly resolves apt installed dependencies.

to make list of installed package more readable
space new line after each lockfile checksum
to force rosdep to resolve and list apt installed package dependencies
and to not skip them as if they are in the catkin workspace
when set of dependencies happens to be empty
to prevent rosdep from erroring out on empty var
@ruffsl ruffsl merged commit 4486a5c into ros-navigation:main Jun 9, 2021
@ruffsl
Copy link
Member Author

ruffsl commented Jun 9, 2021

This is a lot easier to read when debugging cache breaks, as well as when auditing our package dependency footprint:

cache_nonce
Release
1e632b11779dc9f8eb455cc054c8026127580b9802ff2a5c4fd7c5fc422b1d6d  /opt/ros_ws/lockfile.txt
ros_entrypoint
2021-06-09 19:01:39.000000000 +0000
49a1816fab98a9ff12ec5c0dd34a7bcb7b6248e5c75ee649441c762c48c4e7e8  /opt/ros_ws/lockfile.txt
vcs_export
repositories:
90712a086e39e431ea34650f0e37548527831a2ab58e25f0292b19448f80e079  /opt/underlay_ws/lockfile.txt
workspace_dependencies
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version       Architecture Description
+++-==============-=============-============-=================================
ii  dpkg           1.19.7ubuntu3 amd64        Debian package management system
bed49e47c911c80f3cd0e060f5b654490a2006db91929d8d3dcf1ebc2e5f5ea8  /opt/underlay_ws/lockfile.txt
vcs_export
repositories:
22e83e13abe4ae1a3fd7c7569eb4ee8f3005d70aae0338ea6e5d3894d6a08a93  /opt/overlay_ws/lockfile.txt
workspace_dependencies
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                   Version                       Architecture Description
+++-======================================-=============================-============-=========================================================================================================================================================================================
ii  dpkg                                   1.19.7ubuntu3                 amd64        Debian package management system
ii  graphicsmagick-libmagick-dev-compat    1.4+really1.3.35-1            all          image processing libraries providing ImageMagick interface
ii  lcov                                   1.14-2                        all          Summarise Code coverage information from GCOV
ii  libboost-program-options-dev:amd64     1.71.0.0ubuntu2               amd64        program options library for C++ (default version)
ii  libboost-program-options1.71.0:amd64   1.71.0-6ubuntu6               amd64        program options library for C++
ii  libceres-dev                           1.14.0-4ubuntu1.1             amd64        nonlinear least squares minimizer (development files)
ii  libeigen3-dev                          3.3.7-2                       all          lightweight C++ template library for linear algebra
ii  libgraphicsmagick++1-dev               1.4+really1.3.35-1            amd64        format-independent image processing - C++ development files
ii  libqt5core5a:amd64                     5.12.8+dfsg-0ubuntu1          amd64        Qt 5 core module
ii  libqt5gui5:amd64                       5.12.8+dfsg-0ubuntu1          amd64        Qt 5 GUI module
ii  libqt5opengl5:amd64                    5.12.8+dfsg-0ubuntu1          amd64        Qt 5 OpenGL module
ii  libqt5widgets5:amd64                   5.12.8+dfsg-0ubuntu1          amd64        Qt 5 widgets module
ii  python3-pytest                         4.6.9-1                       all          Simple, powerful testing in Python3
ii  python3-yaml                           5.3.1-1ubuntu0.1              amd64        YAML parser and emitter for Python3
ii  python3-zmq                            18.1.1-3                      amd64        Python3 bindings for 0MQ library
ii  qtbase5-dev:amd64                      5.12.8+dfsg-0ubuntu1          amd64        Qt 5 base development files
ii  ros-rolling-action-msgs                1.0.3-1focal.20210603.002131  amd64        Messages and service definitions common among all ROS actions.
ii  ros-rolling-ament-cmake                1.1.4-1focal.20210506.150302  amd64        The entry point package for the ament buildsystem in CMake.
ii  ros-rolling-ament-cmake-core           1.1.4-1focal.20210506.144148  amd64        The core of the ament buildsystem in CMake.
ii  ros-rolling-ament-cmake-gtest          1.1.4-1focal.20210506.150303  amd64        The ability to add gtest-based tests in the ament buildsystem in CMake.
ii  ros-rolling-ament-cmake-pytest         1.1.4-1focal.20210506.150304  amd64        The ability to run Python tests using pytest in the ament buildsystem in CMake.
ii  ros-rolling-ament-cmake-python         1.1.4-1focal.20210506.145838  amd64        The ability to use Python in the ament buildsystem in CMake.
ii  ros-rolling-ament-copyright            0.10.6-1focal.20210506.150401 amd64        The ability to check source files for copyright and license information.
ii  ros-rolling-ament-lint-auto            0.10.6-1focal.20210506.150307 amd64        The auto-magic functions for ease to use of the ament linters in CMake.
ii  ros-rolling-ament-lint-common          0.10.6-1focal.20210506.151701 amd64        The list of commonly used linters in the ament buildsytem in CMake.
ii  ros-rolling-angles                     1.12.4-1focal.20210506.150406 amd64        This package provides a set of simple math utilities to work with angles.
ii  ros-rolling-behaviortree-cpp-v3        3.5.6-1focal.20210605.060759  amd64        This package provides the Behavior Trees core library.
ii  ros-rolling-bond                       3.0.1-3focal.20210603.002543  amd64        A bond allows two processes, A and B, to know when the other has terminated, either cleanly or by crashing.
ii  ros-rolling-bondcpp                    3.0.1-3focal.20210605.051030  amd64        C++ implementation of bond, a mechanism for checking when another process has terminated.
ii  ros-rolling-builtin-interfaces         1.0.3-1focal.20210603.001736  amd64        A package containing message and service definitions for types defined in the OMG IDL Platform Specific Model.
ii  ros-rolling-cv-bridge                  2.2.1-1focal.20210603.004522  amd64        This contains CvBridge, which converts between ROS2 Image messages and OpenCV images.
ii  ros-rolling-eigen3-cmake-module        0.1.1-2focal.20210506.151503  amd64        Exports a custom CMake module to find Eigen3.
ii  ros-rolling-gazebo-ros-pkgs            3.5.2-4focal.20210605.085934  amd64        Interface for using ROS with the Gazebo simulator.
ii  ros-rolling-geometry-msgs              2.2.3-1focal.20210603.002727  amd64        A package containing some geometry related message definitions.
ii  ros-rolling-image-transport            3.0.0-1focal.20210605.051731  amd64        image_transport should always be used to subscribe to and publish images.
ii  ros-rolling-laser-geometry             2.2.2-4focal.20210605.051736  amd64        This package contains a class for converting from a 2D laser scan as defined by sensor_msgs/LaserScan into a point cloud as defined by sensor_msgs/PointCloud or sensor_msgs/PointCloud2.
ii  ros-rolling-launch                     0.17.0-1focal.20210506.150954 amd64        The ROS launch tool.
ii  ros-rolling-launch-ros                 0.14.2-1focal.20210605.044836 amd64        ROS specific extensions to the launch tool.
ii  ros-rolling-launch-testing             0.17.0-1focal.20210506.151505 amd64        A package to create tests which involve launch files and multiple processes.
ii  ros-rolling-launch-testing-ament-cmake 0.17.0-1focal.20210506.152456 amd64        A package providing cmake functions for running launch tests from the build.
ii  ros-rolling-lifecycle-msgs             1.0.3-1focal.20210603.001736  amd64        A package containing some lifecycle related message and service definitions.
ii  ros-rolling-map-msgs                   2.1.0-1focal.20210603.004704  amd64        This package defines messages commonly used in mapping packages.
ii  ros-rolling-message-filters            4.0.0-1focal.20210605.045120  amd64        A set of ROS2 message filters which take in messages and may output those messages at a later time, based on the conditions that filter needs met.
ii  ros-rolling-nav-msgs                   2.2.3-1focal.20210603.003741  amd64        A package containing some navigation related message and service definitions.
ii  ros-rolling-ompl                       1.5.2-1focal.20210506.150207  amd64        OMPL is a free sampling-based motion planning library.
ii  ros-rolling-osrf-pycommon              0.2.1-1focal.20210506.150154  amd64        Commonly needed Python modules, used by Python software developed at OSRF.
ii  ros-rolling-pluginlib                  5.0.0-1focal.20210525.151443  amd64        The pluginlib package provides tools for writing and dynamically loading plugins using the ROS build infrastructure.
ii  ros-rolling-rclcpp                     11.0.0-1focal.20210605.042106 amd64        The ROS client library in C++.
ii  ros-rolling-rclcpp-action              11.0.0-1focal.20210605.045118 amd64        Adds action APIs for C++.
ii  ros-rolling-rclcpp-lifecycle           11.0.0-1focal.20210605.045119 amd64        Package containing a prototype for lifecycle implementation
ii  ros-rolling-rclpy                      3.0.0-1focal.20210605.042904  amd64        Package containing the Python client.
ii  ros-rolling-resource-retriever         2.5.0-1focal.20210514.181938  amd64        This package retrieves data from url-format files such as http://, ftp://, package:// file://, etc., and loads the data into memory.
ii  ros-rolling-robot-state-publisher      2.4.3-1focal.20210605.055510  amd64        This package allows you to publish the state of a robot to tf2.
ii  ros-rolling-rosidl-default-generators  1.1.1-1focal.20210603.001553  amd64        A configuration package defining the default ROS interface generators.
ii  ros-rolling-rosidl-default-runtime     1.1.1-1focal.20210603.001553  amd64        A configuration package defining the runtime for the ROS interfaces.
ii  ros-rolling-rviz-common                8.6.0-2focal.20210605.060252  amd64        Common rviz API, used by rviz plugins and applications.
ii  ros-rolling-rviz-default-plugins       8.6.0-2focal.20210605.062524  amd64        Several default plugins for rviz to cover the basic functionality.
ii  ros-rolling-rviz-ogre-vendor           8.6.0-2focal.20210526.173749  amd64        Wrapper around ogre3d, it provides a fixed CMake module and an ExternalProject build of ogre.
ii  ros-rolling-rviz-rendering             8.6.0-2focal.20210526.180519  amd64        Library which provides the 3D rendering functionality in rviz.
ii  ros-rolling-sensor-msgs                2.2.3-1focal.20210603.003744  amd64        A package containing some sensor data related message and service definitions.
ii  ros-rolling-std-msgs                   2.2.3-1focal.20210603.002243  amd64        A package containing some standard message definitions.
ii  ros-rolling-std-srvs                   2.2.3-1focal.20210603.001737  amd64        A package containing some standard service definitions.
ii  ros-rolling-test-msgs                  1.0.3-1focal.20210603.002307  amd64        A package containing message definitions and fixtures used exclusively for testing purposes.
ii  ros-rolling-tf2                        0.18.0-1focal.20210603.004300 amd64        tf2 is the second generation of the transform library, which lets the user keep track of multiple coordinate frames over time.
ii  ros-rolling-tf2-geometry-msgs          0.18.0-1focal.20210605.055529 amd64        tf2_geometry_msgs
ii  ros-rolling-tf2-ros                    0.18.0-1focal.20210605.054819 amd64        This package contains the C++ ROS bindings for the tf2 library
ii  ros-rolling-tf2-sensor-msgs            0.18.0-1focal.20210605.060929 amd64        Small lib to transform sensor_msgs with tf.
ii  ros-rolling-visualization-msgs         2.2.3-1focal.20210603.003435  amd64        A package containing some visualization and interaction related message definitions.
ii  ros-rolling-yaml-cpp-vendor            7.1.0-1focal.20210506.150655  amd64        Wrapper around yaml-cpp, it provides a fixed CMake module and an ExternalProject build of it.
9efd0bdd2cf14621256a8a8dffa04971c8e9f170a3fd103e093173dfd7542e3e  /opt/overlay_ws/lockfile.txt

https://circle-production-customer-artifacts.s3.amazonaws.com/picard/5b21c78bcfe8df0013110d01/60c1446ca658b11f8aeb37b7-0-build/artifacts/opt/overlay_ws/lockfile.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210609T224910Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Credential=AKIAJR3Q6CR467H7Z55A%2F20210609%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=d838148079f2fb3bbf821127335b6444b0748a5c72aac0ff4e1f5eea3658346c

@ruffsl ruffsl deleted the ci-lockfile-list branch June 9, 2021 22:53
ruffsl added a commit to ruffsl/navigation2 that referenced this pull request Jul 2, 2021
* Use dpkg --list to format stdout to lockfile
to make list of installed package more readable

* Line formating
space new line after each lockfile checksum

* Specify ROS distro for rosdep update

* Strip ros distro install fom AMENT_PREFIX_PATH
to force rosdep to resolve and list apt installed package dependencies
and to not skip them as if they are in the catkin workspace

* Prevent dpkg from listing all installed pacakge
when set of dependencies happens to be empty

* Account single entry case AMENT_PREFIX_PATH

* Unset AMENT_PREFIX_PATH if only ros install path
to prevent rosdep from erroring out on empty var
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant