Feature/3658/automate sonarqube project create and scanning

[IhsenBouallegue]

Automate sonarqube project creation and scanning

Please read the CONTRIBUTING.md before opening a PR.

{Issue/Closes}: #3658

Description

This script solves the pong process of using sonarqube and CC to analyse. It automates all manual steps except creating a sonar config file for your project.
There is docs under the new script directory and a new how-to.

Definition of Done

A PR is only ready for merge once all the following acceptance criteria are fulfilled:

• Changes have been manually tested
• All TODOs related to this PR have been closed
• There are automated tests for newly written code and bug fixes
• All bugs discovered while working on this PR have been submitted as issues (if not already an open issue)
• Documentation (GH-pages, analysis/visualization READMEs, parser READMEs, --help, etc.) has been updated (almost always necessary except for bug fixes)
• CHANGELOG.md has been updated

Screenshots or gifs

🔧 Welcome to the SonarQube & CodeCharta Automation Script 🔧
------------------------------------------------------------
This script automates the process of:
1. Setting up a SonarQube project and resetting the default 'admin' password if needed.
2. Running SonarScanner to analyze your projects source code.
3. Conducting a CodeCharta analysis of the scanned data.
------------------------------------------------------------

🔑 Enter the Project Key (default: maibornwolff-gmbh_codecharta_visualization):
📛 Enter the Project Name (default: CodeCharta Visualization):
🔒 Enter the new password for the SonarQube admin user (default: newadminpassword):
📁 Enter the directory path to be scanned (default: /mnt/c/Users/IhsenB/Documents/repos/codecharta/visualization):

Running:
  1) Ensure SonarQube Running
  2) Reset SonarQube Password
  3) Clean Up Previous Project
  4) Revoke Token
  5) Create Project and Generate Token
  6) Run SonarScanner
  7) Run CodeCharta Analysis
  8) Final Cleanup
✅ jq is already installed.
✅ Docker is already installed.
✅ Docker Daemon is running.
✅ Docker image 'sonarsource/sonar-scanner-cli' is already available locally.
✅ Docker image 'codecharta/codecharta-analysis' is already available locally.
✅ Docker image 'sonarqube:community' is already available locally.
🎉 All dependencies are installed and required Docker images are available.
🔧 Creating Docker network sonarnet...
72cad88ab5240eb351f948a5185dfb0cbd5e199feac9ce2983b8c10257b08903
🚀 Starting SonarQube container...
3cd5f16bbb9aa5a4a26d0320e6ba753f04225c70cdf6a217fafd50193369d19c
⏳ Checking SonarQube status... ⠧ 
✅ SonarQube is ready!
🔍 Testing SonarQube credentials: Username='admin', Password='admin'
✅ Default credentials are valid. Proceeding to change the password...
✅ Password has been successfully changed to the new password.
🧹 Cleaning up previous SonarQube project...
❗️ Project not found, skipping deletion.
🧹 Revoking existing SonarQube token...
✅ Token revoked successfully.
🔍 Checking if project 'maibornwolff-gmbh_codecharta_visualization' already exists in SonarQube...   
🚀 Creating project in SonarQube...
✅ Project created successfully.
🔍 Checking if token 'codecharta_token' already exists...
🔑 Generating token...
✅ Token generated: squ_eff11128606ae2e9d4e17a6db658bce7da7fc0df
🔍 Running SonarScanner...

INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /usr/src/sonar-project.properties
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.11 Alpine (64-bit)
INFO: Linux 5.15.153.1-microsoft-standard-WSL2 amd64
INFO: User cache: /opt/sonar-scanner/.sonar/cache
INFO: Analyzing on SonarQube server 10.6.0.92116
INFO: Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)        
INFO: Load global settings
INFO: Load global settings (done) | time=108ms
INFO: Server id: 147B411E-AZHXVbcbZaH9wFKx9gqz
INFO: Loading required plugins
INFO: Load plugins index
INFO: Load plugins index (done) | time=86ms
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=378ms
INFO: Process project properties
INFO: Process project properties (done) | time=18ms
INFO: Project key: maibornwolff-gmbh_codecharta_visualization
INFO: Base dir: /usr/src
INFO: Working dir: /tmp/.scannerwork
INFO: Load project settings for component key: 'maibornwolff-gmbh_codecharta_visualization'
INFO: Load project settings for component key: 'maibornwolff-gmbh_codecharta_visualization' (done) | time=33ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=138ms
WARN: SCM provider autodetection failed. Please use "sonar.scm.provider" to define SCM of your project, or disable the SCM Sensor in the project settings.
INFO: Load active rules
INFO: Load active rules (done) | time=6255ms
INFO: Load analysis cache
INFO: Load analysis cache (404) | time=12ms
INFO: Preprocessing files...
INFO: 4 languages detected in 815 preprocessed files
INFO: 1485 files ignored because of inclusion/exclusion patterns
INFO: Loading plugins for detected languages
INFO: Load/download plugins
INFO: Load/download plugins (done) | time=471ms
INFO: Load project repositories
INFO: Load project repositories (done) | time=26ms
INFO: Indexing files...
INFO: Project configuration:
INFO:   Excluded sources: **/app/puppeteer.helper.ts, **/node_modules/**, **/*.spec.ts, **/*.spec.js, **/*.e2e.ts, **/*.e2e.js, **/*.po.ts, **/app/codeCharta/util/testUtils/*, **/*.spec.ts
INFO:   Included tests: **/*.spec.ts
INFO:   Excluded tests: **/app/puppeteer.helper.ts, **/node_modules/**, **/*.spec.ts, **/*.spec.js, **/*.e2e.ts, **/*.e2e.js, **/*.po.ts, **/app/codeCharta/util/testUtils/*
INFO:   Excluded sources for coverage: **/app/puppeteer.helper.ts, **/node_modules/**, **/*.spec.ts, **/*.spec.js, **/*.e2e.ts, **/*.e2e.js, **/*.po.ts, **/*.html, **/*.scss, **/app/codeCharta/util/dataMocks.ts, **/app/codeCharta/util/testUtils/*
INFO:   Excluded sources for duplication: **/app/codeCharta/util/dataMocks.ts
INFO: 814 files indexed
INFO: Quality profile for css: Sonar way
INFO: Quality profile for json: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for web: Sonar way
INFO: ------------- Run sensors on module CodeCharta Visualization
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=24ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=1266ms
INFO: Sensor JaCoCo XML Report Importer [jacoco]
INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml       
INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer     
INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
INFO: Sensor IaC CloudFormation Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC CloudFormation Sensor [iac] (done) | time=201ms
INFO: Sensor IaC AzureResourceManager Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC AzureResourceManager Sensor [iac] (done) | time=108ms
INFO: Sensor Java Config Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor Java Config Sensor [iac] (done) | time=14ms
INFO: Sensor JavaScript/TypeScript analysis [javascript]
INFO: Detected os: Linux arch: amd64 alpine: true. Platform: UNSUPPORTED
INFO: Using Node.js executable: 'node'.
INFO: Memory configuration: OS (15719 MB), Node.js (2096 MB).
INFO: Found 1 tsconfig.json file(s): [/usr/src/tsconfig.json]
INFO: Creating TypeScript program
INFO: 590 source files to be analyzed
INFO: TypeScript configuration file /usr/src/tsconfig.json
INFO: 0/590 files analyzed, current file: app/codeCharta/ui/attributeSideBar/util/showAttributeTypeSelector.selector.ts
INFO: 0/590 files analyzed, current file: app/codeCharta/ui/attributeSideBar/util/showAttributeTypeSelector.selector.ts
INFO: 0/590 files analyzed, current file: app/codeCharta/ui/attributeSideBar/util/showAttributeTypeSelector.selector.ts
INFO: 0/590 files analyzed, current file: app/codeCharta/ui/attributeSideBar/util/showAttributeTypeSelector.selector.ts
INFO: Starting analysis with current program
INFO: 23/590 files analyzed, current file: app/codeCharta/state/store/appSettings/scaling/scaling.actions.ts
INFO: 85/590 files analyzed, current file: app/codeCharta/util/fileNameHelper.ts
INFO: 148/590 files analyzed, current file: app/codeCharta/ui/attributeSideBar/attributeSideBarSecondaryMetrics/attributeSideBarSecondaryMetrics.component.ts
INFO: 205/590 files analyzed, current file: app/codeCharta/ui/labelledColorPicker/readableColorForBackground.pipe.ts
INFO: 265/590 files analyzed, current file: app/codeCharta/state/store/appSettings/isLoadingMap/isLoadingMap.reducer.ts
INFO: 324/590 files analyzed, current file: app/codeCharta/ui/ribbonBar/artificialIntelligence/selectors/util/riskProfileHelper.ts
INFO: 380/590 files analyzed, current file: app/codeCharta/ui/searchPanel/mapTreeView/mapTreeViewLevel/mapTreeViewLevel.component.ts
INFO: 438/590 files analyzed, current file: app/codeCharta/ui/ribbonBar/colorSettingsPanel/colorSettingsPanel.component.ts
INFO: 492/590 files analyzed, current file: app/codeCharta/services/3DExports/3DPreview/CreateGeometryStrategies/createBaseplateGeometryStrategy.ts
INFO: 536/590 files analyzed, current file: app/codeCharta/ui/toolBar/toolBar.module.ts
INFO: 575/590 files analyzed, current file: app/codeCharta/state/effects/updateFileSettings/utils/blacklist.merger.ts
INFO: Analyzed 590 file(s) with current program
INFO: 590/590 source files have been analyzed
INFO: Hit the cache for 0 out of 590
INFO: Miss the cache for 590 out of 590: ANALYSIS_MODE_INELIGIBLE [590/590]
INFO: Sensor JavaScript/TypeScript analysis [javascript] (done) | time=173664ms
INFO: Sensor JavaScript inside HTML analysis [javascript]
INFO: 107 source files to be analyzed
INFO: 107/107 source files have been analyzed
INFO: Hit the cache for 0 out of 107
INFO: Miss the cache for 107 out of 107: ANALYSIS_MODE_INELIGIBLE [107/107]
INFO: Sensor JavaScript inside HTML analysis [javascript] (done) | time=1848ms
INFO: Sensor CSS Rules [javascript]
INFO: 173 source files to be analyzed
INFO: 173/173 source files have been analyzed
INFO: Hit the cache for 0 out of 0
INFO: Miss the cache for 0 out of 0
INFO: Sensor CSS Rules [javascript] (done) | time=2336ms
INFO: Sensor CSS Metrics [javascript]
INFO: Sensor CSS Metrics [javascript] (done) | time=270ms
INFO: Sensor IaC Docker Sensor [iac]
INFO: 0 source files to be analyzed
INFO: 0/0 source files have been analyzed
INFO: Sensor IaC Docker Sensor [iac] (done) | time=22ms
INFO: Sensor TextAndSecretsSensor [text]
INFO: Available processors: 22
INFO: Using 22 threads for analysis.
INFO: Using git CLI to retrieve untracked files
WARN: Analyzing only language associated files, make sure to run the analysis inside a git repository to make use of inclusions specified via "sonar.text.inclusions"
INFO: 777 source files to be analyzed
INFO: 777/777 source files have been analyzed
INFO: Sensor TextAndSecretsSensor [text] (done) | time=1856ms
INFO: ------------- Run sensors on project
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=84ms
INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
INFO: CPD Executor 314 files had no CPD blocks
INFO: CPD Executor Calculating CPD for 382 files
INFO: CPD Executor CPD calculation finished (done) | time=147ms
INFO: Analysis report generated in 2457ms, dir size=3.2 MB
INFO: Analysis report compressed in 640ms, zip size=2.2 MB
INFO: Analysis report uploaded in 114ms
INFO: ANALYSIS SUCCESSFUL, you can find the results at: http://sonarqube:9000/dashboard?id=maibornwolff-gmbh_codecharta_visualization
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://sonarqube:9000/api/ce/task?id=2f3dcd03-1126-4cae-ab8f-ff078dc5f263
INFO: Analysis total time: 3:24.329 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 3:26.847s
INFO: Final Memory: 15M/80M
INFO: ------------------------------------------------------------------------

✅ SonarScanner analysis complete.
⏳ Waiting for the data to be fully uploaded to SonarQube... ⠇ 
✅ Data has been fully uploaded and processed by SonarQube!
📊 Running CodeCharta analysis...

Fetching SonarQube Version...
Found SonarQube version 10.6
[main] INFO de.maibornwolff.codecharta.util.Logger - Get values for metrics [accepted_issues, blocker_violations, branch_coverage, bugs, class_complexity, classes, code_smells, cognitive_complexity, comment_lines, comment_lines_density, complexity, complexity_in_classes, complexity_in_functions, conditions_to_cover, confirmed_issues, coverage, critical_violations, duplicated_blocks, duplicated_files, duplicated_lines, duplicated_lines_density, false_positive_issues, file_complexity, files, function_complexity, functions, generated_lines, generated_ncloc, high_impact_accepted_issues, info_violations, line_coverage, lines, lines_to_cover, major_violations, minor_violations, ncloc, new_accepted_issues, new_blocker_violations, new_branch_coverage, new_bugs, new_code_smells, new_conditions_to_cover, new_coverage, new_critical_violations, new_development_cost, new_duplicated_blocks, new_duplicated_lines, new_duplicated_lines_density, new_info_violations, new_line_coverage, new_lines, new_lines_to_cover, new_major_violations, new_minor_violations, new_security_hotspots, new_security_hotspots_reviewed, new_security_hotspots_reviewed_status, new_security_hotspots_to_review_status, new_sqale_debt_ratio, new_uncovered_conditions, new_uncovered_lines, new_violations, new_vulnerabilities, open_issues, prioritized_rule_issues, projects, public_api, public_documented_api_density, public_undocumented_api, pull_request_fixed_issues, reopened_issues, security_hotspots, security_hotspots_reviewed, security_hotspots_reviewed_status, security_hotspots_to_review_status, skipped_tests, sqale_debt_ratio, statements, test_errors, test_failures, test_success_density, tests, unanalyzed_c, unanalyzed_cpp, uncovered_conditions, uncovered_lines, violations, vulnerabilities].
   8% [========>                                                                                     
  16% [================>                                                                             
  25% [=========================>                                                                    
  33% [=================================>                                                            
  41% [=========================================>                                                    
  50% [==================================================>                                           
  58% [==========================================================>                                   
  66% [==================================================================>                           
  75% [===========================================================================>                  
  83% [===================================================================================>          
  91% [===========================================================================================>   100% [====================================================================================================>] 12/12 Files, ETA: 00:00:00
763 of 763 components processed...
Created Project with 763 leaves.
[main] INFO de.maibornwolff.codecharta.util.Logger - Created output file at /mnt/c/Users/IhsenB/Documents/repos/codecharta/visualization/sonar.cc.json.gz

✅ CodeCharta analysis complete. Output stored in /mnt/c/Users/IhsenB/Documents/repos/codecharta/visualization/sonar.cc.json.gz
🧹 Cleaning up...
sonarqube
sonarqube
sonarnet
🧹 Cleanup complete.

To run this script again without prompts, use the following command:
./run_analysis.sh -k "maibornwolff-gmbh_codecharta_visualization" -n "CodeCharta Visualization" -p "newadminpassword" -d "/mnt/c/Users/IhsenB/Documents/repos/codecharta/visualization" -u "http://localhost:9000" -t "codecharta_token"

You will have a file called sonar.cc.json.gz. Import it in CodeCharta and it should look normal.

PS: It should be run in WSL or macOS (no Windows support)

PS2: Ensure you enabled this if you are using rancher desktop so WSL can access docker.

[sonarcloud]

Quality Gate passed for 'CodeCharta Visualization'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarcloud]

Quality Gate passed for 'CodeCharta Analysis'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[Nereboss]

LGTM!