ci: error out if no riot hashes are found when running test suites (#…

…7051)

If no riot hashes are found for some reason, `scripts/run-test-suite`
should error and exit.

This also adds a bit of logging for fun.

Example of failure due to no matching hashes:
```
root@docker-desktop:~/project# scripts/run-test-suite romainlikethelettucewithoutthee
No riot hashes found for pattern: romainlikethelettucewithoutthee
```

Example of run with matching hashes:
```
root@docker-desktop:~/project# scripts/run-test-suite ci_visibility
Found 6 riot hashes.
Running riot hash: 12a04f4
INFO:riot.riot:Generating virtual environments for interpreters Interpreter(_hint='3.12')
...
```

## Checklist

- [x] Change(s) are motivated and described in the PR description.
- [x] Testing strategy is described if automated tests are not included
in the PR.
- [x] Risk is outlined (performance impact, potential for breakage,
maintainability, etc).
- [x] Change is maintainable (easy to change, telemetry, documentation).
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed. If no release note is required, add label
`changelog/no-changelog`.
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/)).
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist

- [x] Title is accurate.
- [x] No unnecessary changes are introduced.
- [x] Description motivates each change.
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes unless absolutely necessary.
- [x] Testing strategy adequately addresses listed risk(s).
- [x] Change is maintainable (easy to change, telemetry, documentation).
- [x] Release note makes sense to a user of the library.
- [x] Reviewer has explicitly acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment.
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
- [x] If this PR touches code that signs or publishes builds or
packages, or handles credentials of any kind, I've requested a review
from `@DataDog/security-design-and-guidance`.
- [x] This PR doesn't touch any of that.

scripts/run-test-suite

@@ -2,15 +2,33 @@
 
 CHECKPOINT_FILENAME="latest-success-commit"
 RIOT_PATTERN=${1}
-if [[ -v CIRCLECI ]]; then
-    RIOT_HASHES=$(riot list --hash-only $RIOT_PATTERN | sort | circleci tests split)
-else
-    RIOT_HASHES=$(riot list --hash-only $RIOT_PATTERN | sort)
-fi
 DDTRACE_FLAG=$([ -v _CI_DD_API_KEY ] && echo '--ddtrace')
 COVERAGE_FLAG=$([[ "${2:-false}" == false ]] && echo '--no-cov')
 DDTEST_CMD=$([[ ${3} == "1" ]] && echo "./scripts/ddtest")
 
+RIOT_HASHES=( $(riot list --hash-only $RIOT_PATTERN | sort) )
+echo "Found ${#RIOT_HASHES[@]} riot hashes: ${RIOT_HASHES[@]}"
+if [[ ${#RIOT_HASHES[@]} -eq 0 ]]; then
+    echo "No riot hashes found for pattern: $RIOT_PATTERN"
+    if [[ -v CIRCLECI ]]; then
+        circleci step halt
+    fi
+    exit 1
+fi
+
+if [[ -v CIRCLECI ]]; then
+    # circleci tests splits expects one test per line
+    RIOT_HASHES=( $( printf '%s\n' "${RIOT_HASHES[@]}"  | circleci tests split) )
+    if [[ ${#RIOT_HASHES[@]} -eq 0 ]]; then
+        echo "No riot hashes found after split, halting."
+        circleci step halt
+        exit 0
+    fi
+    echo "${#RIOT_HASHES[@]} hashes split for CircleCI: ${RIOT_HASHES[@]}"
+fi
+
+
+
 set -e
 
 if ! [[ -v CIRCLECI && $CIRCLE_BRANCH =~ [0-9]\.x ]]; then
@@ -26,7 +44,8 @@ if ! [[ -v CIRCLECI && $CIRCLE_BRANCH =~ [0-9]\.x ]]; then
     fi
 fi
 
-for hash in $RIOT_HASHES; do
+for hash in ${RIOT_HASHES[@]}; do
+    echo "Running riot hash: $hash"
     if ! $DDTEST_CMD riot -P -v run --exitfirst --pass-env -s $hash $DDTRACE_FLAG $COVERAGE_FLAG; then
         if [[ -v CIRCLECI ]]; then
             circleci step halt