Feature/3658/automate sonarqube project create and scanning (#3694)

* feat: Add automated sonarqube and sonarscanner analysis scripts * chore: Update SonarQube and SonarScanner analysis script * chore: update password flow * chore: fix print in password flow * chore: Add dependency_checker.sh script for checking and installing dependencies * chore: add variable promting to change the default while running script * feat: Add automated sonarqube and sonarscanner analysis scripts * chore: Update SonarQube and SonarScanner analysis script * chore: update password flow * chore: fix print in password flow * chore: Add dependency_checker.sh script for checking and installing dependencies * chore: add variable promting to change the default while running script * fix: url not being encoded correctly * chore: Update SonarQube and SonarScanner analysis script * chore: Update SonarQube and SonarScanner analysis script * rafactor: Rafactor folder structure * chore: Add release automation script for Linux and macOS * chore: Add push trigger to release automation script * chore: Update release automation script to include source code in ZIP archive * chore: Update SonarQube and SonarScanner analysis script * chore: prompt the user for installing * chore: Update release automation script to use new file names for Linux and macOS * chore: remove release automation script to use new file names for Linux and macOS * fix: fix script not working on macos because of unsupported bash features * refactor: fix the project base dir and the sonarscanner arguments * fix macos not recognising command * feat: add dynamic wait * fix: improve dep checker * feat: add spinners to see when loading * fix: Check if docker Daemon is running * chore: add docs * docs: add how to docs to github pages * feat: Add menu functionality to select steps and improve user experience * refactor: Refactor dependency checking and add documentation - Refactor the dependency checking code in the `dependency_checker.sh` script to improve readability and maintainability. - Add documentation to the `README.md` file explaining how to use the script with default values, custom configurations, and reusable commands. * chore: remove output * refactor: Update menu navigation instructions in helpers.sh * refactor: Update menu navigation instructions in helpers.sh * chore: update docs * chore: update docs * doc --------- Co-authored-by: IhsenBouallegue <IhsenBouallegue@gmail.com>
MaibornWolff · Sep 13, 2024 · 857aeb1 · 857aeb1
1 parent ba8ebb0
commit 857aeb1
Show file tree

Hide file tree

Showing 8 changed files with 1,153 additions and 0 deletions.
diff --git a/gh-pages/_posts/how-to/2024-09-05-automated-sonar-analysis.md b/gh-pages/_posts/how-to/2024-09-05-automated-sonar-analysis.md
@@ -0,0 +1,105 @@
+---
+categories:
+  - How-to
+tags:
+  - sonarimport
+  - analysis
+title: Automated SonarQube Analysis
+---
+
+# SonarQube & CodeCharta Automation Script
+
+## Overview
+
+This script automates the setup and analysis processes for SonarQube and CodeCharta on Linux and MacOS. It handles:
+
+1. **SonarQube Project Setup**: Creates a SonarQube project and optionally resets the default 'admin' password.
+2. **Source Code Analysis**: Runs SonarScanner to analyze the project's source code.
+3. **CodeCharta Analysis**: Performs a CodeCharta analysis based on the scanned data.
+
+You can choose to use default values or provide custom configurations when running the script. To skip prompts and use default values, use the `-s` flag. After execution, the script will print a reusable command with the provided configurations, which you can use next time to skip prompts.
+
+## Script Execution
+
+1. **Introduction**: Displays the purpose of the script and usage instructions.
+2. **Prompt for Configuration**: If the `-s` flag is not used, prompts for the following:
+   - Project Key
+   - Project Name
+   - SonarQube Admin Password
+   - Directory Path for Scanning
+3. **Build and Display Reusable Command**: After gathering inputs (whether via flags or prompts), the script builds a reusable command reflecting the provided configurations and prints it at the end for future use.
+4. **Encode Project Key and Name**: URL-encodes the project key and name for safe usage.
+5. **Run Steps**:
+   - Ensure SonarQube is running.
+   - Reset SonarQube admin password.
+   - Clean up the previous SonarQube project.
+   - Revoke existing token.
+   - Create a new SonarQube project and generate a token.
+   - Run SonarScanner for code analysis.
+   - Perform CodeCharta analysis.
+   - Run final cleanup.
+
+## Usage
+
+### Parameters
+
+| Parameter               | Description                                                                        |
+| ----------------------- | ---------------------------------------------------------------------------------- |
+| `-k <project_key>`      | Set the project key for SonarQube.                                                 |
+| `-n <project_name>`     | Set the project name for SonarQube.                                                |
+| `-p <new_password>`     | Set the new SonarQube admin password.                                              |
+| `-d <project_basedir>`  | Set the directory containing the project to be scanned.                            |
+| `-u <host_sonar_url>`   | Set the URL for the SonarQube host.                                                |
+| `-t <sonar_token_name>` | Set the token name for SonarQube authentication.                                   |
+| `-s`                    | Skip all prompts and use either default values or the flags passed in the command. |
+| `-h`                    | Show the help message for the script and exit.                                     |
+
+> **USAGE:**
+>
+> ```shell
+> run_analysis.sh [-h] [-s] [-k <project_key>] [-n <project_name>] [-p <new_password>] [-d <project_basedir>] [-u <host_sonar_url>] [-t <sonar_token_name>]
+> ```
+
+### Default Execution
+
+These commands assume you are in the root of the project.
+For MacOS users, you should have brew installed and bash updated.
+
+```shell
+# For MacOS you need to give execution permission to the script
+chmod +x ./script/automated_sonar_analysis/run_analysis.sh
+```
+
+No need to pass anything the script will prompt as needed.
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh
+```
+
+### Skip Prompts
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -s
+```
+
+### Custom Execution with Flags
+
+You can provide flags to customize the execution. In this case, it will skip the prompt and use the provided parameter.
+
+For example:
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -k "custom_project_key" -n "Custom Project Name" -p "new_password" -d "/path/to/codebase"
+```
+
+### Reusable Command
+
+After running the script, it will display a command you can use to execute the script with the same parameters without prompting next time. This allows for easy reuse of the configurations you provided during the first run.
+
+Example reusable command generated:
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -k "custom_project_key" -n "Custom Project Name" -p "new_password" -d "/path/to/codebase" -u "http://localhost:9000" -t "codecharta_token"
+```
+
+This command will automatically use the values you previously provided, making future executions more efficient.
diff --git a/script/automated_sonar_analysis/README.md b/script/automated_sonar_analysis/README.md
@@ -0,0 +1,96 @@
+# SonarQube & CodeCharta Automation Script
+
+## Overview
+
+This script automates the setup and analysis processes for SonarQube and CodeCharta on Linux and MacOS. It handles:
+
+1. **SonarQube Project Setup**: Creates a SonarQube project and optionally resets the default 'admin' password.
+2. **Source Code Analysis**: Runs SonarScanner to analyze the project's source code.
+3. **CodeCharta Analysis**: Performs a CodeCharta analysis based on the scanned data.
+
+You can choose to use default values or provide custom configurations when running the script. To skip prompts and use default values, use the `-s` flag. After execution, the script will print a reusable command with the provided configurations, which you can use next time to skip prompts.
+
+## Script Execution
+
+1. **Introduction**: Displays the purpose of the script and usage instructions.
+2. **Prompt for Configuration**: If the `-s` flag is not used, prompts for the following:
+   - Project Key
+   - Project Name
+   - SonarQube Admin Password
+   - Directory Path for Scanning
+3. **Build and Display Reusable Command**: After gathering inputs (whether via flags or prompts), the script builds a reusable command reflecting the provided configurations and prints it at the end for future use.
+4. **Encode Project Key and Name**: URL-encodes the project key and name for safe usage.
+5. **Run Steps**:
+   - Ensure SonarQube is running.
+   - Reset SonarQube admin password.
+   - Clean up the previous SonarQube project.
+   - Revoke existing token.
+   - Create a new SonarQube project and generate a token.
+   - Run SonarScanner for code analysis.
+   - Perform CodeCharta analysis.
+   - Run final cleanup.
+
+## Usage
+
+### Parameters
+
+| Parameter               | Description                                                                        |
+| ----------------------- | ---------------------------------------------------------------------------------- |
+| `-k <project_key>`      | Set the project key for SonarQube.                                                 |
+| `-n <project_name>`     | Set the project name for SonarQube.                                                |
+| `-p <new_password>`     | Set the new SonarQube admin password.                                              |
+| `-d <project_basedir>`  | Set the directory containing the project to be scanned.                            |
+| `-u <host_sonar_url>`   | Set the URL for the SonarQube host.                                                |
+| `-t <sonar_token_name>` | Set the token name for SonarQube authentication.                                   |
+| `-s`                    | Skip all prompts and use either default values or the flags passed in the command. |
+| `-h`                    | Show the help message for the script and exit.                                     |
+
+> **USAGE:**
+>
+> ```shell
+> run_analysis.sh [-h] [-s] [-k <project_key>] [-n <project_name>] [-p <new_password>] [-d <project_basedir>] [-u <host_sonar_url>] [-t <sonar_token_name>]
+> ```
+
+### Default Execution
+
+These commands assume you are in the root of the project.
+For MacOS users, you should have brew installed and bash updated.
+
+```shell
+# For MacOS you need to give execution permission to the script
+chmod +x ./script/automated_sonar_analysis/run_analysis.sh
+```
+
+No need to pass anything the script will prompt as needed.
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh
+```
+
+### Skip Prompts
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -s
+```
+
+### Custom Execution with Flags
+
+You can provide flags to customize the execution. In this case, it will skip the prompt and use the provided parameter.
+
+For example:
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -k "custom_project_key" -n "Custom Project Name" -p "new_password" -d "/path/to/codebase"
+```
+
+### Reusable Command
+
+After running the script, it will display a command you can use to execute the script with the same parameters without prompting next time. This allows for easy reuse of the configurations you provided during the first run.
+
+Example reusable command generated:
+
+```bash
+./script/automated_sonar_analysis/run_analysis.sh -k "custom_project_key" -n "Custom Project Name" -p "new_password" -d "/path/to/codebase" -u "http://localhost:9000" -t "codecharta_token"
+```
+
+This command will automatically use the values you previously provided, making future executions more efficient.
diff --git a/script/automated_sonar_analysis/analysers.sh b/script/automated_sonar_analysis/analysers.sh
@@ -0,0 +1,90 @@
+#!/bin/bash
+
+# Run SonarScanner in the container and capture output
+run_sonarscanner() {
+    echo "🔍 Running SonarScanner..."
+
+    # Print start of dimmed output
+    echo -e "\033[2m"  # Start dimming the text
+
+    # Run the Docker container with SonarScanner and display dimmed output
+    docker run --rm -it \
+      --network $NETWORK_NAME \
+      -v "$PROJECT_BASEDIR:/usr/src" \
+      -w /usr/src \
+      sonarsource/sonar-scanner-cli \
+      sonar-scanner \
+      -Dsonar.token=$token \
+      -Dsonar.host.url="$CONTAINER_SONAR_URL"
+
+    # Stop dimming after the Docker command completes
+    echo -e "\033[0m"  # Reset to normal text
+
+    if [ $? -ne 0 ]; then
+        echo "❌ SonarScanner analysis failed."
+        exit 1
+    fi
+
+    echo "✅ SonarScanner analysis complete."
+
+    wait_for_data_processing
+}
+
+wait_for_data_processing() {
+    start_spinner "⏳ Waiting for the data to be fully uploaded to SonarQube..." &
+    spinner_pid=$!
+
+    interval=2          # Check every 2 seconds
+    waited=0
+
+    while true; do
+        response=$(curl -s -u $SONAR_USER:$SONAR_PASSWORD -w "\n%{http_code}" "$HOST_SONAR_URL/api/ce/component?component=$PROJECT_KEY")
+
+        http_status=$(echo "$response" | tail -n1)
+        response_body=$(echo "$response" | head -n1)
+
+        check_response "$http_status" "$response_body" "SonarQube data processing failed."
+
+        status=$(echo "$response_body" | jq -r '.current.status')
+
+        if [ "$status" == "SUCCESS" ]; then
+            # Stop spinner if data processing is complete
+            stop_spinner "$spinner_pid"
+            echo -e "\n✅ Data has been fully uploaded and processed by SonarQube!"
+            break
+        elif [ "$waited" -ge "$TIMEOUT_PERIOD" ]; then
+            stop_spinner "$spinner_pid"
+            echo -e "\n❌ SonarQube did not finish processing the data within $TIMEOUT_PERIOD seconds."
+            exit 1
+        fi
+
+        sleep "$interval"
+        waited=$((waited + interval))
+    done
+}
+
+
+# Run CodeCharta analysis using docker run
+run_codecharta_analysis() {
+    echo "📊 Running CodeCharta analysis..."
+
+    # Print start of dimmed output
+    echo -e "\033[2m"  # Start dimming the text
+
+    # Use the correct hostname 'sonarqube' and execute the analysis
+    docker run --rm -it --network "$NETWORK_NAME" --name codecharta-analysis \
+      -v "$PROJECT_BASEDIR:$PROJECT_BASEDIR" \
+      -w "$PROJECT_BASEDIR" \
+      codecharta/codecharta-analysis \
+      ccsh sonarimport "$CONTAINER_SONAR_URL" "$PROJECT_KEY" "--user-token=$token" "--output-file=$PROJECT_BASEDIR/sonar.cc.json" "--merge-modules=false"
+
+    # Stop dimming after the Docker command completes
+    echo -e "\033[0m"  # Reset to normal text
+
+    if [ $? -ne 0 ]; then
+        echo "❌ CodeCharta analysis failed."
+        exit 1
+    fi
+
+    echo "✅ CodeCharta analysis complete. Output stored in $PROJECT_BASEDIR/sonar.cc.json.gz"
+}
diff --git a/script/automated_sonar_analysis/cleanup.sh b/script/automated_sonar_analysis/cleanup.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Cleanup function: Stop and remove containers and network
+cleanup() {
+    echo "🧹 Cleaning up..."
+    docker stop $SONAR_CONTAINER_NAME 2>/dev/null
+    docker rm $SONAR_CONTAINER_NAME 2>/dev/null
+    docker network rm $NETWORK_NAME 2>/dev/null
+    echo "🧹 Cleanup complete."
+}