Skip to content

Commit

Permalink
Inclusive lg changes (kyma-project#3501)
Browse files Browse the repository at this point in the history
  • Loading branch information
Karolina Zydek authored Apr 1, 2021
1 parent 0577457 commit 6ce574c
Show file tree
Hide file tree
Showing 19 changed files with 60 additions and 60 deletions.
2 changes: 1 addition & 1 deletion .github/pull-request-template.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!-- Thank you for your contribution. Before you submit the pull request:
1. Follow contributing guidelines, templates, the recommended Git workflow, and any related documentation.
2. Read and submit the required Contributor Licence Agreements (https://github.com/kyma-project/community/blob/master/contributing/02-contributing.md#agreements-and-licenses).
2. Read and submit the required Contributor Licence Agreements (https://github.com/kyma-project/community/blob/main/contributing/02-contributing.md#agreements-and-licenses).
3. Test your changes and attach their results to the pull request.
4. Update the relevant documentation.
-->
Expand Down
2 changes: 1 addition & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Code of conduct

Each contributor and maintainer of this project agrees to follow the [community Code of Conduct](https://github.com/kyma-project/community/blob/master/CODE_OF_CONDUCT.md) that relies on the CNCF Code of Conduct. Read it to learn about the agreed standards of behavior, shared values that govern our community, and details on how to report any suspected Code of Conduct violations.
Each contributor and maintainer of this project agrees to follow the [community Code of Conduct](https://github.com/kyma-project/community/blob/main/CODE_OF_CONDUCT.md) that relies on the CNCF Code of Conduct. Read it to learn about the agreed standards of behavior, shared values that govern our community, and details on how to report any suspected Code of Conduct violations.
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@

## Overview

To contribute to this project, follow the rules from the general [CONTRIBUTING.md](https://github.com/kyma-project/community/blob/master/CONTRIBUTING.md) document in the `community` repository.
To contribute to this project, follow the rules from the general [CONTRIBUTING.md](https://github.com/kyma-project/community/blob/main/CONTRIBUTING.md) document in the `community` repository.
6 changes: 3 additions & 3 deletions development/tools/cmd/pjtester/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ ProwJob tester is a tool for testing changes to ProwJobs and scripts in the `tes

The workhorse for testing ProwJobs is a tool written in Go called `pjtester`. It's available in the `prow-tools` Docker image.

`pjtester` is executed by the presubmit job `pre-master-test-infra-pjtester`. This presubmit job is triggered when something changes under the `test-infra` repository virtual path `vpath/pjtester.yaml`.
`pjtester` is executed by the presubmit job `pre-master-test-infra-pjtester`. This presubmit job is triggered when something changes under the `test-infra` repository virtual path `vpath/pjtester.yaml`.
It is configured by the **run-if-changed** option:
```bash
run_if_changed: "^(vpath/pjtester.yaml)"
Expand All @@ -20,7 +20,7 @@ By default, `pjtester` disables all reporting for the ProwJob. That means no Sla

Details from `pjtester.yaml` and from the ProwJob environment variables are used to construct the specification of the ProwJob to test. `pjtester` uses the environment variables created by Prow for the presubmit job which identify the pull request and its commit hash in the `test-infra` repository. The generated ProwJob to test then uses the `test-infra` code from the pull request's head, ensuring that the latest code is under test.

For presubmit jobs, Prow requires the pull request's head SHA, pull request number, and pull request author set in the ProwJob refs. In the `pjtester.yaml file`, you must specify a pull request number for a repository against which a tested Prow job is running. If you don't specify one, `pjtester` will find a pull request for the `master` branch (`HEAD`) and use its details for the presubmit refs.
For presubmit jobs, Prow requires the pull request's head SHA, pull request number, and pull request author set in the ProwJob refs. In the `pjtester.yaml file`, you must specify a pull request number for a repository against which a tested Prow job is running. If you don't specify one, `pjtester` will find a pull request for the `main` branch (`HEAD`) and use its details for the presubmit refs.

Finally, `pjtester` creates the ProwJob on the production Prow instance. The ProwJob name for which you triggered the test is prefixed with `{YOUR_GITHUB_USER}_test_of_prowjob_`.

Expand All @@ -32,7 +32,7 @@ ProwJob tester is a tool running on Prow. You need a working Prow instance to us

## Installation

To make `pjtester` work for you, you need to compile it and build an image with its binary included. This is done by the `post-test-infra-prow-tools` ProwJob. It builds and pushes the `prow-tools` image.
To make `pjtester` work for you, you need to compile it and build an image with its binary included. This is done by the `post-test-infra-prow-tools` ProwJob. It builds and pushes the `prow-tools` image.
Next, you must add a presubmit job to trigger `pjtester` execution. This is done by the `pre-master-test-infra-pjtester` ProwJob.

## Configuration
Expand Down
8 changes: 4 additions & 4 deletions development/tools/cmd/rendertemplates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The `config.yaml` file and the data files specify the following for the Render T
- The name and location of the output files
- Values it must use to generate the output files

### Data for templates
### Data for templates

The Render Templates passes data used to generate files from templates in the **$.Values** and **$.Global** variables. The values of these variables are created from the `config.yaml` file and data files stored in the `templates/data` directory. The **Global** variable holds data from the **global** key. The Render Templates generates the values of the **Values** variable from scratch for each Prow job. **Values** variable is generated by merging ConfigSets. Each ConfigSet is a map defined in one of three possible places:

Expand All @@ -21,9 +21,9 @@ globalSets:
image: "eu.gcr.io/kyma-project/test-infra/bootstrap:v20200831-e46c648b"
```
ConfigSets defined in **globalSets** hold data used to generate multiple files. A good example of such usage is the `image_bootstrap` global ConfigSet which defines a bootstrap image to use in Prow jobs.
ConfigSets defined in **globalSets** hold data used to generate multiple files. A good example of such usage is the `image_bootstrap` global ConfigSet which defines a bootstrap image to use in Prow jobs.


- **Local ConfigSets** defined under the **localSets** parameter for each **to** key in the `config.yaml` file or in data files in the `templates/data` directory:

```yaml
Expand Down
2 changes: 1 addition & 1 deletion docs/prow/crier.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ If the job is still in the testing phase, you can set `optional: true`.

## Migration from Plank to GitHub reporter

First, you need to disable GitHub reporting in Plank. To do that, add the `--skip-report=true` flag to the Plank [deployment](https://github.com/kyma-project/test-infra/blob/master/prow/cluster/components/11-plank_deployment.yaml).
First, you need to disable GitHub reporting in Plank. To do that, add the `--skip-report=true` flag to the Plank [deployment](https://github.com/kyma-project/test-infra/blob/main/prow/cluster/components/11-plank_deployment.yaml).

Before migrating, upgrade your Plank to a version that includes the commit [`2118178`](https://github.com/kubernetes/test-infra/pull/10975/commits/211817826fc3c4f3315a02e46f3d6aa35573d22f).

Expand Down
8 changes: 4 additions & 4 deletions docs/prow/prow-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ There are different kinds of plugins that react to GitHub events forwarded by th
Prow plugins applied for the Kyma project include:
- **trigger** that matches the received event against the job configuration from the `jobs` folder. If it finds the match, it creates a new Prow job resource.
- **cat** that checks if there is a new GitHub event for a `/meow` comment on a PR. If it finds it, it adds a cat image to the related PR. For that purpose, it uses the GitHub token available as a Kubernetes Secret.
- **config-updater** that reads the configuration from `config.yaml`, `plugins.yaml`, and the `jobs` folder, and updates it on the production cluster after the merge to the `master` branch. This plugin is only configured for the `test-infra` repository.
- **config-updater** that reads the configuration from `config.yaml`, `plugins.yaml`, and the `jobs` folder, and updates it on the production cluster after the merge to the `main` branch. This plugin is only configured for the `test-infra` repository.

## Prow jobs
Different build jobs are specified in the `jobs` folder per repository. Each of them uses different kind of trigger conditions. Depending on the trigger, a component becomes active to create a Prow-specific Prow job resource that represents a given job execution. At a later time, a real Pod gets created by the Plank based on the Pod specification provided in the `jobs` folder. Inside the Pod, a container executes the actual build logic. When the process is finished, the Sinker component cleans up the Pod.
Expand All @@ -64,16 +64,16 @@ Build logs are archived by Plank on GCS in a dedicated bucket. The bucket is con

## Generate development artifacts

There are two jobs that generate artifacts which allow you to install Kyma on a cluster either from the `master` branch or from a pull request changes:
There are two jobs that generate artifacts which allow you to install Kyma on a cluster either from the `main` branch or from a pull request changes:
- `pre-master-kyma-development-artifacts`
- `post-master-kyma-development-artifacts`

>**NOTE:** For pull requests, the job is executed only if the introduced changes have an impact on the installed Kyma version.
All artifacts are stored in the publicly available bucket under the `gs://kyma-development-artifacts/` location. The bucket has a defined lifecycle management rule to automatically delete files older than 60 days. These are the exact artifacts locations:
* For pull requests: `gs://kyma-development-artifacts/PR-<number>`
* For changes to the `master` branch: `gs://kyma-development-artifacts/master-<commit_sha>`
* For the latest changes in the master branch: `gs://kyma-development-artifacts/master`
* For changes to the `main` branch: `gs://kyma-development-artifacts/master-<commit_sha>`
* For the latest changes in the `main` branch: `gs://kyma-development-artifacts/master`

A directory with artifacts consists of the following files:
- `kyma-installer-cluster.yaml` to deploy Kyma installer
Expand Down
2 changes: 1 addition & 1 deletion docs/prow/prow-installation-on-forks.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Verify if the Prow installation was successful.

## Configure the webhook

After Prow installs successfully, you must [configure the webhook](https://github.com/kyma-project/community/blob/master/guidelines/repository-guidelines/01-new-repository-settings.md#add-webhooks) to enable the GitHub repository to send events to Prow.
After Prow installs successfully, you must [configure the webhook](https://github.com/kyma-project/community/blob/main/guidelines/repository-guidelines/01-new-repository-settings.md#add-webhooks) to enable the GitHub repository to send events to Prow.

## Configure Prow

Expand Down
4 changes: 2 additions & 2 deletions docs/prow/prow-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ When you define jobs for Prow, the **name** parameter of the job must follow one
You can extend the name of the job with a suffix to indicate the job's purpose. For example, write `pre-master-kyma-integration`.

Add `{prefix}` in front of all presubmit and postsubmit jobs. Use:
- `pre-master` for presubmit jobs that run against the `master` branch.
- `post-master` for postsubmit jobs that run against the `master` branch.
- `pre-master` for presubmit jobs that run against the `main` branch.
- `post-master` for postsubmit jobs that run against the `main` branch.
- `pre-rel{release-number}` for presubmit jobs that run against the release branches. For example, write `pre-rel06-kyma-components-api-controller`.


Expand Down
6 changes: 3 additions & 3 deletions docs/prow/prow-k8s-testgrid.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ The PRs created by the bot are visible [here](https://github.com/kubernetes/test

## Adding new dashboard

The dashboards' configuration is stored in the [testgrid-default.yaml](https://github.com/kyma-project/test-infra/tree/master/prow/testgrid-default.yaml) file.
This file is automatically generated from the [template testgrid-default.yaml](https://github.com/kyma-project/test-infra/blob/master/templates/templates/testgrid-default.yaml) file. See the example of a dashboard configuration file:
The dashboards' configuration is stored in the [testgrid-default.yaml](https://github.com/kyma-project/test-infra/tree/main/prow/testgrid-default.yaml) file.
This file is automatically generated from the [template testgrid-default.yaml](https://github.com/kyma-project/test-infra/blob/main/templates/templates/testgrid-default.yaml) file. See the example of a dashboard configuration file:
```yaml
dashboards:
# kyma
Expand All @@ -32,7 +32,7 @@ Follow these steps to add a new dashboard:
1. Add a new dashboard name in the `dashboards` field inside a template file. **Dashboards must have a dashboard group name as a prefix**.
2. Add the previously added dashboard to the corresponding **dashboard_name** inside the **dashboard_groups** field.
3. Generate a new config file using the [rendertemplates](https://github.com/kyma-project/test-infra/tree/master/development/tools/cmd/rendertemplates) tool and check if the config file generated correctly.
3. Generate a new config file using the [rendertemplates](https://github.com/kyma-project/test-infra/tree/main/development/tools/cmd/rendertemplates) tool and check if the config file generated correctly.

## Adding ProwJob to the TestGrid

Expand Down
30 changes: 15 additions & 15 deletions docs/prow/prow-secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,32 +9,32 @@ This document lists all types of Secrets used in the `kyma-prow` and `workload-k

## kyma-prow cluster

| Prow Secret | Description |
| :---------- | :---------------- |
| **hmac-token**| Used for validating GitHub webhooks. It is manually generated using the `openssl rand -hex 20` command.|
| **oauth-token**| Personal access token called `prow-production` used by the `kyma-bot` GitHub user. |
| Prow Secret | Description |
| :---------- | :---------------- |
| **hmac-token**| Used for validating GitHub webhooks. It is manually generated using the `openssl rand -hex 20` command.|
| **oauth-token**| Personal access token called `prow-production` used by the `kyma-bot` GitHub user. |
|**sap-slack-bot-token**| Token for publishing messages to the SAP CX workspace. Find more information [here](https://api.slack.com/docs/token-types#bot). |
|**workload-cluster**| Stores the `workload-kyma-prow` cluster certificate. |
| **slack-token** | OAuth token for the Slack bot user. It is used by Crier. |

## workload-kyma-prow cluster

| Secret | Description |
| :---------- | :---------------- |
| **whitesource** keys | Copied directly from the bucket when executing a job. These Secrets are not stored on the cluster. |
| Secret | Description |
| :---------- | :---------------- |
| **whitesource** keys | Copied directly from the bucket when executing a job. These Secrets are not stored on the cluster. |
| **github-integration** | Used to authorize GitHub applications configured in the `kyma-project` organization. See the **[OAuth Apps](https://developer.github.com/apps/building-oauth-apps/)** section in GitHub.|
| **sa-*** | Service Accounts used in pipelines. Find more information [here](/docs/prow/authorization.md).|
| **kyma-website-bot-zenhub-token** | ZenHub token for the `kyma-website-bot` account.|
| **kyma-bot-github-token**| Personal access token called `prow-job` used by the `kyma-bot` GitHub user.|
| **kyma-guard-bot-github-token** | Personal access token for the `kyma-guard-bot` GitHub account.|
| **sa-*** | Service Accounts used in pipelines. Find more information [here](/docs/prow/authorization.md).|
| **kyma-website-bot-zenhub-token** | ZenHub token for the `kyma-website-bot` account.|
| **kyma-bot-github-token**| Personal access token called `prow-job` used by the `kyma-bot` GitHub user.|
| **kyma-guard-bot-github-token** | Personal access token for the `kyma-guard-bot` GitHub account.|
| **kyma-bot@sap.com**| Stores credentials to the `kyma-bot` GitHub account. |
| **kyma-bot-npm-token** | Token for publishing npm packages in the `npmjs.com` registry. The `kyma-bot` user credentials are used to authenticate to the registry. The Secret is used by the `post-master-varkes` ProwJob. |
| **gardener-kyma-prow-kubeconfig** | Kubeconfig file that allows connection to the `kyma-prow` Gardener project.|
| **slack-nightly-token**| Token that allows the stability checker to push notifications to Slack. |
| **gardener-kyma-prow-kubeconfig** | Kubeconfig file that allows connection to the `kyma-prow` Gardener project.|
| **slack-nightly-token**| Token that allows the stability checker to push notifications to Slack. |
| **sap-slack-bot-token** | Token for publishing messages to the SAP CX workspace. Find more information [here](https://api.slack.com/docs/token-types#bot).|
| **kyma-alerts-slack-api-url** | Token for publishing messages to the SAP CX workspace. It is used by nightly and weekly ProwJobs.|
| **neighbors-alerts-slack-api-url** | Publishes alerts to the private `neighbors` Slack channel.|
| **kyma-azure-credential-*** | Azure subscription and service principal credentials. |
| **kyma-website-bot-*** | Personal access token of the `kyma-website-bot` GitHub account. It is responsible for publishing the `kyma-project.io` website. |
| **slack-webhook-kyma-ci-force** | Slack webhook that points to the #kyma-ci-force channel. It is used by [alertmanager](https://github.com/kyma-project/test-infra/blob/master/docs/prow/prow-monitoring.md).|
| **slack-webhook-kyma-prow-alerts** | Slack webhook that points to the #kyma-prow-alerts channel. It is used by [alertmanager](https://github.com/kyma-project/test-infra/blob/master/docs/prow/prow-monitoring.md).|
| **slack-webhook-kyma-ci-force** | Slack webhook that points to the #kyma-ci-force channel. It is used by [alertmanager](https://github.com/kyma-project/test-infra/blob/main/docs/prow/prow-monitoring.md).|
| **slack-webhook-kyma-prow-alerts** | Slack webhook that points to the #kyma-prow-alerts channel. It is used by [alertmanager](https://github.com/kyma-project/test-infra/blob/main/docs/prow/prow-monitoring.md).|
10 changes: 5 additions & 5 deletions docs/prow/quality-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ SELECT
, NTH_VALUE(timestamp, 2)
OVER w1 AS previous_timestamp
, format("https://status.build.kyma-project.io/?job=%s", k8s_pod_prow_k8s_io_job) as link
FROM
FROM
(
SELECT
MIN(CAST(REGEXP_EXTRACT(textPayload, r"(\d{1,2})") as int64)) as val
Expand All @@ -49,9 +49,9 @@ You can check this report [here](https://datastudio.google.com/open/1TmjzxgO8yTG

## Bugs and regression metrics

[GithubStats application](https://github.com/kyma-project/test-infra/tree/master/development/tools/cmd/githubstats)
[GithubStats application](https://github.com/kyma-project/test-infra/tree/main/development/tools/cmd/githubstats)
is used to gather data on bugs and regressions. It is executed as a [Prow job](https://status.build.kyma-project.io/?job=github-stats) once a day. The data are grabbed by Stackdriver export and forwarded to BigQuery.
A JSON object is automatically flattened into multiple columns.
A JSON object is automatically flattened into multiple columns.

The Prow job creates the following output:
```
Expand Down Expand Up @@ -83,7 +83,7 @@ The Prow job creates the following output:
The following SQL statement pre-formats the data:

```
select
select
CAST(jsonPayload.issues.open.totalcount as int64) as open
, CAST(jsonPayload.issues.closed.totalcount as int64) as closed
, CAST(jsonPayload.issues.open.bugs as int64) as open_bugs
Expand All @@ -92,7 +92,7 @@ select
, jsonPayload.timestamp
, CONCAT(jsonPayload.owner, "/", jsonPayload.repository) as repository
from `sap-kyma-prow.stats_github.stdout`
order by jsonPayload.timestamp desc
order by jsonPayload.timestamp desc
```

This query is used as a data source for a report created in DataStudio.
Expand Down
2 changes: 1 addition & 1 deletion docs/prow/workload-clusters.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Workload clusters:
- Use separate subnets for nodes, Pods, and services.

```gcloud container clusters list
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
NAME LOCATION MAIN_VERSION MAIN_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
trusted-workload-kyma-prow europe-west3 1.14.10-gke.36 _____________ n1-standard-4 1.14.10-gke.36 3 RUNNING
untrusted-workload-kyma-prow europe-west3 1.14.10-gke.36 _____________ n1-standard-4 1.14.10-gke.36 2 RUNNING
```
Expand Down
Loading

0 comments on commit 6ce574c

Please sign in to comment.