Iron changes (#368)

* move humble requirements to root Signed-off-by: Yadunund <yadunund@openrobotics.org> * Update tier 3 os labels and remove source from jammy Signed-off-by: Yadunund <yadunund@openrobotics.org> * Tmp fix for redirected links in sitemap that cause duplicate requirements Signed-off-by: Yadunund <yadunund@openrobotics.org> * Added an expectation for param delete Signed-off-by: Yadunund <yadunund@openrobotics.org> * Fix prompt for service find Signed-off-by: Yadunund <yadunund@openrobotics.org> * Update bag info with mcap Signed-off-by: Yadunund <yadunund@openrobotics.org> * Added details for component cli tests Signed-off-by: Yadunund <yadunund@openrobotics.org> * Add checks for ros1_bridge Signed-off-by: Yadunund <yadunund@openrobotics.org> * Added tests for service introspection Signed-off-by: Yadunund <yadunund@openrobotics.org> * Add check for asunc_param_client Signed-off-by: Yadunund <yadunund@openrobotics.org> * Fix duplicate py checks Signed-off-by: Yadunund <yadunund@openrobotics.org> * Add DISTRO_LABEL constant Signed-off-by: Yadunund <yadunund@openrobotics.org> * Use DISTRO const to retrieve docs Signed-off-by: Yadunund <yadunund@openrobotics.org> * Change the names of launch files to be _launch.py (#347) This matches the current best practice, which we updated in Iron. Signed-off-by: Chris Lalancette <clalancette@gmail.com> * Update the service executables demon_nodes_py test to use async. (#348) There is no '_sync' version, so this must have been a typo for the '_async' version, which does exist. Signed-off-by: Chris Lalancette <clalancette@gmail.com> * Add the holonomic parameter to turtlesim tests. (#350) Signed-off-by: Chris Lalancette <clalancette@gmail.com> * remove extra slash to output correct command (#353) Signed-off-by: Chen Lihui <lihui.chen@sony.com> * fix input (#352) Signed-off-by: Yadunund <yadunund@openrobotics.org> * Add linux label to realtime tests (#351) Signed-off-by: Yadunund <yadunund@openrobotics.org> * tag linux (#357) Signed-off-by: Yadunund <yadunund@openrobotics.org> * Fix up the multicast test to have a try and expect section. (#355) Signed-off-by: Chris Lalancette <clalancette@gmail.com> * Add in a missing component container name in a test. (#354) Signed-off-by: Chris Lalancette <clalancette@gmail.com> * Specify frame ids for static transform publisher (#358) Signed-off-by: Yadunund <yadunund@openrobotics.org> * Replace executables with link to demos readme (#365) * Replace executables with link to demos readme Signed-off-by: Yadunund <yadunund@openrobotics.org> * address feedback Signed-off-by: Yadunund <yadunund@openrobotics.org> --------- Signed-off-by: Yadunund <yadunund@openrobotics.org> * Change references to humble -> iron (#370) Signed-off-by: Chris Lalancette <clalancette@gmail.com> * Modifies the console input to test out rqt (#374) Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Adds the label to only test in linux gazebo_ros_pkgs (#392) Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Removes test cases for Windows (#394) Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Adds a clearer description for the idl message generation (#390) * Adds a clearer description of the test Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Replace executables with link to demos readme (#395) * Replace executables with link to demos readme Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Adds the full path to the launch testing stdin (#393) * Adds the full path to the launch testing stdin * Adds instructions to clone the repository Signed-off-by: Voldivh <eloyabmfcv@gmail.com> * Filter redirects in docs.ros.org sitemap (#405) * Ignore redirects Signed-off-by: Yadunund <yadunund@openrobotics.org> * Moved iron requirements to ros2_test_cases Signed-off-by: Yadunund <yadunund@openrobotics.org> * Restore garden backup Signed-off-by: Yadunund <yadunund@openrobotics.org> * Parse YAML_DISTRO_LABEL Signed-off-by: Yadunund <yadunund@openrobotics.org> * Update readme and ci Signed-off-by: Yadunund <yadunund@openrobotics.org> * Format Signed-off-by: Yadunund <yadunund@openrobotics.org> * Make YATM_DISTRO_LABEL a required param Signed-off-by: Yadunund <yadunund@openrobotics.org> --------- Signed-off-by: Yadunund <yadunund@openrobotics.org> Signed-off-by: Chris Lalancette <clalancette@gmail.com> Signed-off-by: Chen Lihui <lihui.chen@sony.com> Signed-off-by: Voldivh <eloyabmfcv@gmail.com> Co-authored-by: Yadunund <yadunund@openrobotics.org> Co-authored-by: Chris Lalancette <clalancette@gmail.com> Co-authored-by: Chen Lihui <lihui.chen@sony.com> Co-authored-by: Eloy Briceno <51831786+Voldivh@users.noreply.github.com>
audrow · Jul 13, 2023 · acc79d9 · acc79d9
1 parent 3afd3eb
commit acc79d9
Show file tree

Hide file tree

Showing 7 changed files with 60 additions and 5 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -16,6 +16,7 @@ jobs:
         env:
           YATM_TEST_CASE_CONFIG_PATH: ./example-test-case.config.yaml
           YATM_REQUIREMENTS_DIRECTORY_PATH==: ./src/requirements/validator
+          YATM_DISTRO_LABEL: rolling
           GITHUB_REPO_OWNER: audrow
           GITHUB_REPO_NAME: yatm
           GITHUB_TOKEN: dummy_key

diff --git a/README.md b/README.md
@@ -103,6 +103,7 @@ Here are the steps to using this package:
 
 - `YATM_REQUIREMENTS_DIRECTORY_PATH`: The absolute path to a directly containing YAML files files that describe testing requirements. See [Creating Requirements](#creating-requirements) for more details.
 - `YATM_OUTPUT_DIRECTORY_PATH` (OPTIONAL): The absolute path to a directory where YATM should output test cases to. If unspecified, a `generated-files/` directory is created in the current working directory.
+- `YATM_DISTRO_LABEL`: When generating requirements from ros2-docs, this should be set to the distro name, eg. `rolling`. Any Github tickets created will also append this value as a "label" for the ticket if it is non-empty.
 - `GITHUB_REPO_OWNER`: The name of the github organization or owner containing the repository below.
 - `GITHUB_REPO_NAME`: The name of repository where Issue tickets for test cases should be opened.
 - `GITHUB_TOKEN`: Your Github Personal Access token which has the ability to read and write to the repository above.

diff --git a/scripts/executable-features.csv b/scripts/executable-features.csv
@@ -12,7 +12,7 @@ intra_process_demo,"camera_node, watermark_node, image_view_node"
 composition,linktime_composition
 quality_of_service_demo_cpp,"lifespan, liveliness, deadline"
 quality_of_service_demo_py,"lifespan, liveliness, deadline"
-lifecycle,lifecycle_demo.launch.py
+lifecycle,lifecycle_demo_launch.py
 tf2_ros,"tf2_echo, tf2_monitor, static_transform_publisher"
 ros1_bridge,"dynamic_bridge, ros1_bridge*, both directions"
 examples_rclcpp_*,"minimal_publisher: publisher_lambda, publisher_member_function, publisher_not_composable"
@@ -36,8 +36,8 @@ tlsf_cpp,tlsf_allocator_example
 rttest,rttest_plot
 dummy_map_server,dummy_map_server
 dummy_sensors,"dummy_joint_states, dummy_laser"
-dummy_robot_bringup,"dummy_robot_bringup.launch.py, dummy_robot_bringup_launch.xml, dummy_robot_bringup_launch.yaml "
+dummy_robot_bringup,"dummy_robot_bringup_launch.py, dummy_robot_bringup_launch.xml, dummy_robot_bringup_launch.yaml "
 rviz2,rviz2
 gazebo_ros_pkgs,gazebo --verbose /opt/ros/rolling/share/gazebo_plugins/worlds/gazebo_ros_camera_distortion_barrel_demo.world
 rqt,rqt
-rosbag2,"ros2 bag record -a, ros2 bag record <topic1> <topic2> <topic3>, ros2 bag info <bag_file>, ros2 bag play <bag_file>"
+rosbag2,"ros2 bag record -a, ros2 bag record <topic1> <topic2> <topic3>, ros2 bag info <bag_file>, ros2 bag play <bag_file>"
diff --git a/src/constants.ts b/src/constants.ts
@@ -7,6 +7,7 @@ import type Repo from './test-cases/db/github/__types__/Repo'
 // =================================================================================================
 // Define the names of the environment variables that will be retrieved.
 const envSchema = z.object({
+  YATM_DISTRO_LABEL: z.string(),
   YATM_TEST_CASE_CONFIG_PATH: z.string(),
   YATM_REQUIREMENTS_DIRECTORY_PATH: z.string(),
   YATM_OUTPUT_DIRECTORY_PATH: z
@@ -62,6 +63,8 @@ export const REPOSITORY: Repo = {
   name: ENV.GITHUB_REPO_NAME,
 }
 
+export const DISTRO_LABEL = ENV.YATM_DISTRO_LABEL
+
 const packageXml = JSON.parse(
   fs.readFileSync(join(process.cwd(), 'package.json'), 'utf8'),
 )

diff --git a/src/requirements/generator/ros2-docs/ros2-docs.ts b/src/requirements/generator/ros2-docs/ros2-docs.ts
@@ -3,14 +3,15 @@ import fs from 'fs'
 import yaml from 'js-yaml'
 import {join} from 'path'
 import urlParse from 'url-parse'
+import {DISTRO_LABEL} from '../../../constants'
 import validateRequirements from '../../validator/validate-requirements'
 import type Requirement from '../../__types__/Requirement'
 import {errorIfFileExists} from '../utils'
 import getFromSiteMap from './get-pages-from-sitemap'
 
 async function makeDocumentationRequirementFiles(
   outputDirectory: string,
-  distro = 'rolling',
+  distro = DISTRO_LABEL,
   baseUrl = 'https://docs.ros.org/en/',
   sections: string[] = ['Install', 'Tutorials', 'How-to-guide'],
   documentationLabel = 'docs',

diff --git a/src/test-cases/db/github/test-case-to-gh-issue.ts b/src/test-cases/db/github/test-case-to-gh-issue.ts
@@ -1,3 +1,4 @@
+import {DISTRO_LABEL} from '../../../constants'
 import testCaseToMd from '../../markup/test-case-to-md'
 import type TestCase from '../../__types__/TestCase'
 import type GithubIssue from './__types__/GithubIssue'
@@ -9,7 +10,9 @@ export default function testCaseToGithubIssue(testCase: TestCase) {
   }
   labels.push(...Object.values(testCase.dimensions))
   labels.push(`generation-${testCase.generation}`)
-
+  if (DISTRO_LABEL !== '') {
+    labels.push(DISTRO_LABEL)
+  }
   const outIssue: GithubIssue = {
     title: testCase.name,
     body: testCaseToMd(testCase),

diff --git a/src/test-cases/generator/generate-test-cases.ts b/src/test-cases/generator/generate-test-cases.ts
@@ -42,6 +42,7 @@ export default function generateTestCases({
 function filterRequirements(
   requirements: Requirement[],
   filters: RequirementFilter | RequirementFilter[],
+  remove_duplicates = true,
 ) {
   if (!Array.isArray(filters)) {
     filters = [filters]
@@ -76,6 +77,51 @@ function filterRequirements(
       })
     }
   }
+  // Remove requirements with duplicate names by selecting the requirement
+  // with the greater number of labels.
+  const original_count: number = requirements.length
+  if (remove_duplicates) {
+    const requirementsWithSameNameMap: {[name: string]: Requirement[]} = {}
+    requirements.forEach((requirement) => {
+      if (!requirementsWithSameNameMap[requirement.name]) {
+        const requirementsWithSameName = requirements.filter(
+          (r) => r.name === requirement.name,
+        )
+        if (requirementsWithSameName.length > 1) {
+          requirementsWithSameNameMap[requirement.name] =
+            requirementsWithSameName
+        }
+      }
+    })
+
+    Object.entries(requirementsWithSameNameMap).forEach(
+      ([, requirementsWithSameName]) => {
+        const requirementsToRemove: Requirement[] =
+          requirementsWithSameName.sort((r1, r2) => {
+            if (r1.labels && r2.labels && r1.labels.length > r2.labels.length) {
+              return 1
+            }
+
+            if (r1.labels && r2.labels && r1.labels.length < r2.labels.length) {
+              return -1
+            }
+            return 0
+          })
+        // Pop the last element which is the requirement we want to keep.
+        // Delete remaining requirements.
+        requirementsToRemove.pop()
+        requirementsToRemove.forEach((r: Requirement) => {
+          const index = requirements.indexOf(r)
+          if (index > -1) {
+            requirements.splice(index, 1)
+          }
+        })
+      },
+    )
+    const final_count: number = requirements.length
+    console.log('Removed %i duplicates', original_count - final_count)
+  }
+
   return requirements
 }