Skip to content

Commit

Permalink
Documentation adjustments (#734)
Browse files Browse the repository at this point in the history
  • Loading branch information
pressayuliya authored May 3, 2024
1 parent 2c06e52 commit 8ba6ea7
Show file tree
Hide file tree
Showing 7 changed files with 27 additions and 32 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ test automation reports to monitor test execution, identify issues, and analyze
| Report | Description | Widget |
|-------------------------------| --|-------------|
| **Summary report** | An overview of test execution: tests run, passed, and failed. | [Overall statistics](/dashboards-and-widgets/OverallStatistics) <br/> <br/> [Launches table](/dashboards-and-widgets/LaunchesTable) <br/> <br/> [Passing rate summary](/dashboards-and-widgets/PassingRateSummary) |
| **Detailed test case report** | Specific details about individual test cases: status (pass/fail), execution time, and associated issues/errors. | [Most failed test-cases table (TOP-50)](/dashboards-and-widgets/MostFailedTestCasesTableTop50/) <br/> <br/> [Flaky test cases table (TOP-50)](/dashboards-and-widgets/FlakyTestCasesTableTop50/) <br/> <br/> [Unique bugs table](/dashboards-and-widgets/UniqueBugsTable/) |
| **Detailed test case report** | Specific details about individual test cases: status (Passed/Failed), execution time, and associated issues/errors. | [Most failed test-cases table (TOP-50)](/dashboards-and-widgets/MostFailedTestCasesTableTop50/) <br/> <br/> [Flaky test cases table (TOP-50)](/dashboards-and-widgets/FlakyTestCasesTableTop50/) <br/> <br/> [Unique bugs table](/dashboards-and-widgets/UniqueBugsTable/) |
| **Trend analysis report** | The trends in test results over time. | [Most failed test-cases table (TOP-50)](/dashboards-and-widgets/MostFailedTestCasesTableTop50/) <br/> <br/> [Flaky test cases table (TOP-50)](/dashboards-and-widgets/FlakyTestCasesTableTop50/) <br/> <br/> [Failed cases trend chart](/dashboards-and-widgets/FailedCasesTrendChart/)<br/> <br/> [Passing rate summary](/dashboards-and-widgets/PassingRateSummary) |
| **Failure analysis report** | The failed test cases and their details. | [Most failed test-cases table (TOP-50)](/dashboards-and-widgets/MostFailedTestCasesTableTop50/) <br/> <br/> [Flaky test cases table (TOP-50)](/dashboards-and-widgets/FlakyTestCasesTableTop50/) <br/> <br/> [Failed cases trend chart](/dashboards-and-widgets/FailedCasesTrendChart/)<br/> <br/> [Most popular pattern table (TOP-20)](/dashboards-and-widgets/MostPopularPatternTableTop20/) |
| **Execution history report** | The historical data of test runs. | [Passing rate summary](/dashboards-and-widgets/PassingRateSummary) <br/> <br/> [Launches duration chart](/dashboards-and-widgets/LaunchesDurationChart/) <br/> <br/> [Investigated percentage of launches](/dashboards-and-widgets/InvestigatedPercentageOfLaunches/) <br/> <br/> [Test-cases growth trend chart](/dashboards-and-widgets/TestCasesGrowthTrendChart/) |
Expand All @@ -24,7 +24,7 @@ ReportPortal provides various metrics for improving quality assurance and the ov

| Metric | Description | Widget |
|-----------------------------------| --|------------|
| **Pass/Fail rates** | The percentage of test cases that are passing or failing during test execution. | [Passing rate per launch](/dashboards-and-widgets/PassingRatePerLaunch/) <br/> <br/> [Passing rate summary](/dashboards-and-widgets/PassingRateSummary/) <br/> <br/> [Launch statistics chart](/dashboards-and-widgets/LaunchStatisticsChart/) <br/> <br/> [Overall statistics](/dashboards-and-widgets/OverallStatistics/) <br/> <br/> [Launch execution and issue statistic](/dashboards-and-widgets/LaunchExecutionAndIssueStatistic/) |
| **Passed/Failed rates** | The percentage of test cases that are passing or failing during test execution. | [Passing rate per launch](/dashboards-and-widgets/PassingRatePerLaunch/) <br/> <br/> [Passing rate summary](/dashboards-and-widgets/PassingRateSummary/) <br/> <br/> [Launch statistics chart](/dashboards-and-widgets/LaunchStatisticsChart/) <br/> <br/> [Overall statistics](/dashboards-and-widgets/OverallStatistics/) <br/> <br/> [Launch execution and issue statistic](/dashboards-and-widgets/LaunchExecutionAndIssueStatistic/) |
| **Execution time** | The time it takes to execute test suites or individual test cases. | [Launches duration chart](/dashboards-and-widgets/LaunchesDurationChart/) <br/> <br/> [Most time-consuming test cases widget (TOP-20)](/dashboards-and-widgets/MostTimeConsumingTestCasesWidgetTop20/) |
| **Defect density** | The number of defects found per unit of code or test case provides a quantitative perspective on the software quality of a given system under test. | [Component health check](/dashboards-and-widgets/ComponentHealthCheck/) |
| **Test case success rate** | The percentage of successful test cases out of the total executed highlights the reliability of the test. | [Passing rate per launch](/dashboards-and-widgets/PassingRatePerLaunch/) <br/> <br/> [Passing rate summary](/dashboards-and-widgets/PassingRateSummary/) <br/> <br/> [Launch statistics chart](/dashboards-and-widgets/LaunchStatisticsChart/) <br/> <br/> [Overall statistics](/dashboards-and-widgets/OverallStatistics/) <br/> <br/> [Launch execution and issue statistic](/dashboards-and-widgets/LaunchExecutionAndIssueStatistic/) |
Expand Down
4 changes: 2 additions & 2 deletions docs/plugins/ManagePlugins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,8 @@ Details on how to configure a specific plugin please find in the [Integration se
* [integration with E-mail server](/plugins/EmailServer)
* [integration with LDAP](/reportportal-configuration/authorization/LDAP)
* [integration with Active directory](/reportportal-configuration/authorization/ActiveDirectory)
* [integration with Okta SAML](../reportportal-configuration/authorization/SamlProviders/OktaSaml.mdx)
* [integration with Azure SAML](../reportportal-configuration/authorization/SamlProviders/AzureSaml.mdx)
* [integration with Okta SAML](/reportportal-configuration/authorization/SamlProviders/OktaSaml)
* [integration with Azure SAML](/reportportal-configuration/authorization/SamlProviders/AzureSaml)

> **Use case 1:**
>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ How is status calculated:
* For "Amount of issues" rule: if a number of To Investigate issues in the analyzed launch is more than allowable To Investigate level
* For "New Failure": if a baseline is not found in the system

If you get this status, you can proceed with launch analysis (or choose another baseline) and rerun Quality Gates. For that check the section ### How to recalculate Qulaity Gates.
If you get this status, you can proceed with launch analysis (or choose another baseline) and rerun Quality Gates. For that check the section [How to recalculate Quality Gates](/quality-gates/AssessmentOfTestResultsUsingQualityGates#how-to-recalculate-quality-gates).

### Timeout
Specially for integration with CI\CD, Quality Gates has parameter ```Timeout```. If a launch whose status should be sent to a pipeline, gets UNDEFINED status, the system uses a value from ```Timeout```. Default ```Timeout``` equals to 2 hours. It means, that after 2 hours after launch finish, the system force recaluculats Quality Gate Status and defined status.
Expand Down
2 changes: 1 addition & 1 deletion docs/quality-gates/DeleteQualityGates.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ If you want to delete Quality Gate:
* Open ```Project Settings> Quality Gates```
* Click on ```the trash``` on the Quality Gate rule
* The system removes Quality Gate from the DB
* All Quality Gate report for this rule also will be deleted form the system
* All Quality Gate report for this rule also will be deleted from the system
4 changes: 2 additions & 2 deletions docs/reportportal-configuration/IntegrationViaPlugin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Users can enhance ReportPortal by adding additional integrations with:
* [E-mail server](/plugins/EmailServer)
* [LDAP](/reportportal-configuration/authorization/LDAP)
* [Active Directory](/reportportal-configuration/authorization/ActiveDirectory)
* [Okta SAML](./authorization/SamlProviders/OktaSaml.mdx)
* [Azure SAML](./authorization/SamlProviders/AzureSaml.mdx)
* [Okta SAML](/reportportal-configuration/authorization/SamlProviders/OktaSaml)
* [Azure SAML](/reportportal-configuration/authorization/SamlProviders/AzureSaml)

<MediaViewer src={require('./img/Integrations.png')} alt="Integrations with our test automation results dashboard" />

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,12 @@ sidebar_label: Okta SAML
# Okta SAML

:::important
Before starting, please read about the [SAML provider](Overview.mdx).
Before starting, please read about the [SAML provider](/reportportal-configuration/authorization/SamlProviders/Overview).
:::

This guide will help you to configure the connection between ReportPortal and Okta.

There is no comprehensive guide for Okta configuration, but you can use it as
a reference for the general steps. To get more information about Okta configuration,
read the [official documentation](https://developer.okta.com/docs/guides/saml-application-setup/overview/).
You can use it as high-level steps for reference. To get more information about Okta configuration, read the [official documentation](https://developer.okta.com/docs/guides/saml-application-setup/overview/).

SAML plugin is available in ReportPortal on the Plugins page.

Expand All @@ -21,7 +19,7 @@ SAML plugin is available in ReportPortal on the Plugins page.
1. Okta account
2. ReportPortal instance
3. Access to the ReportPortal instance and Okta account
4. Available connection between Okta and ReportPortal.
4. Available connection between Okta and ReportPortal

## Configuration Okta

Expand Down Expand Up @@ -57,7 +55,7 @@ In the `Single sign-on URL` field, enter the following URL:
```url
https://<host>/uat/saml/sp/SSO/alias/report-portal-sp
```
In my case, I deployed ReportPortal on the local machine, so I used the following URL:
In the example below, ReportPortal was deployed on the local machine using the following URL:

```url
http://192.168.49.2/uat/saml/sp/SSO/alias/report-portal-sp
Expand All @@ -70,10 +68,10 @@ report.portal.sp.id
```

You can change this identifier when you deploy ReportPortal. More information
about the identifier you can find in the [SAML Overview](Overview.mdx#identifier).
about the identifier you can find in the [SAML Overview](/reportportal-configuration/authorization/SamlProviders/Overview#identifier).

In the `Name ID format` field, select `EmailAddress`. You can use the UNSPECIFIED format as well.
More information about the Name ID format you can find in the [SAML Overview](Overview.mdx#reportportal-configuration).
More information about the Name ID format you can find in the [SAML Overview](/reportportal-configuration/authorization/SamlProviders/Overview#reportportal-configuration).

#### Fill in the Attribute Statements

Expand All @@ -91,8 +89,7 @@ Click `Next`.

<MediaViewer src={require('./img/okta-saml/okta-step-2.2.jpg')} alt="Provide feedback" />

In my case, I picked the `I'm an Okta customer adding an internal app` option
and picked in the `App type` field that 'This is an internal app that we have created'.
In this case, the 'I'm an Okta customer adding an internal app' option was selected, along with the 'This is an internal app that we have created' option in the `App type` field.

Click `Finish`.

Expand All @@ -114,22 +111,21 @@ configure the ReportPortal SAML plugin.
:::note
You can't provision a user from Okta to ReportPortal if you already have
a pre-created internal user in ReportPortal with the same email and a different
login. Read more about [JIT provisioning](Overview.mdx#user-jit-provisioning)
login. Read more about [JIT provisioning](/reportportal-configuration/authorization/SamlProviders/Overview#user-jit-provisioning)
:::

You can add a user by clicking on the `Assignments` tab and then clicking on
the `Assign` button.

In my case, a user with the email `baba_yaga@continental.com` doesn't exist in
the deployed ReportPortal instance.
In the case above, a user with the email baba_yaga@continental.com doesn't exist in the deployed ReportPortal instance.

## ReportPortal configuration

### Step 1: Open the SAML plugin

<MediaViewer src={require('./img/okta-saml/okta-step-4.0.png')} alt="Open the SAML plugin" />

Log-in to the ReportPortal by Admin account and go to the `Administration` -> `Plugins` -> `SAML`.
Log-in to the ReportPortal by Admin account and go to the `Administrate` -> `Plugins` -> `SAML`.

### Step 2: Add a new integration

Expand All @@ -148,13 +144,12 @@ In the `Identity provider name ID` field, enter the
`EmailAddress` format for the `Name ID format` in the Okta configuration.
If you choose the `UNSPECIFIED` format, keep this field empty.

You can find more information about the Name ID format in thes
[SAML Overview](Overview.mdx#reportportal-configuration).
You can find more information about the Name ID format in the
[SAML Overview](/reportportal-configuration/authorization/SamlProviders/Overview#reportportal-configuration)

#### Provider name

In the `Provider name` field, enter any name you want. In my case, I used `Okta`.

In the `Provider name` field, enter any name you want, for example `Okta`.

#### Metadata URL

Expand All @@ -164,7 +159,7 @@ In the `Metadata URL` field, paste the copied Okta metadata URL.

You need to map the attributes from the Okta configuration to the ReportPortal.

1. Enter the `email` attribute name in the' Email' field.
1. Enter the `email` attribute name in the 'Email' field.
2. In the `First name` field, enter the `firstName` attribute name.
3. In the `Last name` field, enter the `lastName` attribute name.

Expand All @@ -176,7 +171,7 @@ In the `RP callback URL` field, enter the URL:
https://<host>/uat
```

In my case, I used the following URL: `http://192.168.49.2/uat`.
In the case below, the following URL was used: `http://192.168.49.2/uat`.

#### Create the integration

Expand All @@ -194,9 +189,9 @@ Log-out from the ReportPortal and go to login page. You will see the

<MediaViewer src={require('./img/okta-saml/okta-step-4.5.png')} alt="Login with SAML" />

Click on the butto, and you will be redirected to the Okta login page.
Click on the button, and you will be redirected to the Okta login page.

<MediaViewer src={require('./img/okta-saml/okta-step-4.6.png')} alt="User JIT provisioning" />

After successful login, SAML plugin will create the user to the ReportPortal
and you can see the user in the `Users` section on the `Administration` page.
and you can see the user in the `Users` section on the `Administrate` page.
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ the ReportPortal.

:::tip
There are detailed manuals for the configuration of
[Azure SAML](AzureSaml.mdx)
and [Okta SAML](OktaSaml.mdx).
[Azure SAML](/reportportal-configuration/authorization/SamlProviders/AzureSaml)
and [Okta SAML](/reportportal-configuration/authorization/SamlProviders/OktaSaml).
:::

- SAML 2.0 version
Expand Down

0 comments on commit 8ba6ea7

Please sign in to comment.