From b5e2015fbf4433001e52f6aacd0c883553ccbff6 Mon Sep 17 00:00:00 2001 From: Michel Hidalgo Date: Mon, 29 Jun 2020 11:16:38 -0300 Subject: [PATCH] Update Quality Declaration to QL3. (#403) Signed-off-by: Michel Hidalgo --- rmw_fastrtps_cpp/QUALITY_DECLARATION.md | 74 ++++++++++++++++++++++--- rmw_fastrtps_cpp/README.md | 2 +- 2 files changed, 67 insertions(+), 9 deletions(-) diff --git a/rmw_fastrtps_cpp/QUALITY_DECLARATION.md b/rmw_fastrtps_cpp/QUALITY_DECLARATION.md index e7fd402d6..dac4181b2 100644 --- a/rmw_fastrtps_cpp/QUALITY_DECLARATION.md +++ b/rmw_fastrtps_cpp/QUALITY_DECLARATION.md @@ -2,9 +2,9 @@ This document is a declaration of software quality for the `rmw_fastrtps_cpp` pa # `rmw_fastrtps_cpp` Quality Declaration -The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 4** category. +The package `rmw_fastrtps_cpp` claims to be in the **Quality Level 3** category. -Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 4 in REP-2004](https://www.ros.org/reps/rep-2004.html). +Below are the rationales, notes, and caveats for this claim, organized by each requirement listed in the [Package Requirements for Quality Level 3 in REP-2004](https://www.ros.org/reps/rep-2004.html). ## Version Policy [1] @@ -49,7 +49,7 @@ Following the recommended guidelines for ROS Core packages, all pull requests mu ### Continuous Integration [2.iv] -All pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers) +All pull request must pass CI on all [tier 1 platforms](https://www.ros.org/reps/rep-2000.html#support-tiers). ### Documentation Policy [2.v] @@ -72,7 +72,7 @@ The license for `rmw_fastrtps_cpp` is Apache 2.0, and a summary is in each sourc There is an automated test which runs a linter that ensures each file has a license statement. -Most recent test results can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/1525/testReport/rmw_fastrtps_cpp/copyright/) +Most recent test results can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rmw_fastrtps_cpp/copyright/) ### Copyright Statements [3.iv] @@ -80,7 +80,7 @@ The copyright holders each provide a statement of copyright in each source code There is an automated test which runs a linter that ensures each file has at least one copyright statement. -The results of the test can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/1525/testReport/rmw_fastrtps_cpp/copyright/). +The results of the test can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rmw_fastrtps_cpp/copyright/). ## Testing [4] @@ -94,7 +94,19 @@ There is no API testing in `rmw_fastrtps_cpp`. ### Coverage [4.iii] -`rmw_fastrtps_cpp` does not currently track test coverage. +`rmw_fastrtps_cpp` follows the recommendations for ROS Core packages in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#coverage), and opts to use branch coverage instead of line coverage. + +This includes: + +- tracking and reporting line coverage statistics +- achieving and maintaining a reasonable branch line coverage (90-100%) +- no lines are manually skipped in coverage calculations + +Changes are required to make a best effort to keep or increase coverage before being accepted, but decreases are allowed if properly justified and accepted by reviewers. + +Current coverage statistics can be viewed [here](https://ci.ros2.org/job/ci_linux_coverage/lastSuccessfulBuild/cobertura/src_ros2_rmw_fastrtps_rmw_fastrtps_cpp_src/). This package does not yet meet the 95% coverage guideline. + +A summary of how these statistics are calculated can be found in the [ROS 2 On-boarding guide](https://index.ros.org/doc/ros2/Contributing/ROS-2-On-boarding-Guide/#note-on-coverage-runs). ### Performance [4.iv] @@ -104,7 +116,7 @@ There is no API testing in `rmw_fastrtps_cpp`. `rmw_fastrtps_cpp` uses and passes all the standard linters and static analysis tools for a C++ package as described in the [ROS 2 Developer Guide](https://index.ros.org/doc/ros2/Contributing/Developer-Guide/#linters). -Results of the nightly linter tests can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/1525/testReport/rmw_fastrtps_cpp). +Results of the nightly linter tests can be found [here](https://ci.ros2.org/view/nightly/job/nightly_linux_release/lastBuild/testReport/rmw_fastrtps_cpp). ## Dependencies [5] @@ -140,6 +152,52 @@ Currently nightly results can be seen here: * [mac_osx_release](https://ci.ros2.org/view/nightly/job/nightly_osx_release/lastBuild/testReport/rmw_fastrtps_cpp/) * [windows_release](https://ci.ros2.org/view/nightly/job/nightly_win_rel/lastBuild/testReport/rmw_fastrtps_cpp/) -## Vulnerability Disclosure Policy [7.i] +## Security [7] + +### Vulnerability Disclosure Policy [7.i] This package conforms to the Vulnerability Disclosure Policy in [REP-2006](https://www.ros.org/reps/rep-2006.html). + +# Current status Summary + +The chart below compares the requirements in the REP-2004 with the current state of the `rmw_dds_common` package. + +|Number| Requirement| Current state | +|--|--|--| +|1| **Version policy** |---| +|1.i|Version Policy available | ✓ | +|1.ii|Stable version |✓ | +|1.iii|Declared public API|✓| +|1.iv|API stability policy|✓| +|1.v|ABI stability policy|✓| +|1.vi_|API/ABI stable within ros distribution|✓| +|2| **Change control process** |---| +|2.i| All changes occur on change request | ✓| +|2.ii| Contributor origin (DCO, CLA, etc) | ✓| +|2.iii| Peer review policy | ✓ | +|2.iv| CI policy for change requests | ✓ | +|2.v| Documentation policy for change requests | ✓ | +|3| **Documentation** | --- | +|3.i| Per feature documentation | ✓ | +|3.ii| Per public API item documentation | * | +|3.iii| Declared License(s) | ✓ | +|3.iv| Copyright in source files| ✓ | +|3.v.a| Quality declaration linked to README | ✓ | +|3.v.b| Centralized declaration available for peer review |✓| +|4| **Testing** | --- | +|4.i| Feature items tests | ✓ | +|4.ii| Public API tests | None | +|4.iii.a| Using coverage | ✓ | +|4.iii.a| Coverage policy | ✓ | +|4.iv.a| Performance tests (if applicable) | ☓ | +|4.iv.b| Performance tests policy| ✓ | +|4.v.a| Code style enforcement (linters)| ✓ | +|4.v.b| Use of static analysis tools | ✓ | +|5| **Dependencies** | --- | +|5.i| Must not have ROS lower level dependencies | ✓ | +|5.ii| Optional ROS lower level dependencies| ✓ | +|5.iii| Justifies quality use of non-ROS dependencies |✓| +|6| **Platform support** | --- | +|6.i| Support targets Tier1 ROS platforms| ✓ | +|7| **Security** | --- | +|7.i| Vulnerability Disclosure Policy | ✓ | diff --git a/rmw_fastrtps_cpp/README.md b/rmw_fastrtps_cpp/README.md index 1d517ab1a..7994cd68b 100644 --- a/rmw_fastrtps_cpp/README.md +++ b/rmw_fastrtps_cpp/README.md @@ -6,4 +6,4 @@ For more information see the repository level [README](../README.md) ## Quality Declaration -This package claims to be in the **Quality Level 4** category, see the [Quality Declaration](QUALITY_DECLARATION.md) for more details. +This package claims to be in the **Quality Level 3** category, see the [Quality Declaration](QUALITY_DECLARATION.md) for more details.