Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update token configuration instructions in readme #188

Merged
merged 1 commit into from
Feb 22, 2024
Merged

Update token configuration instructions in readme #188

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 44 additions & 23 deletions runfo/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,42 @@
# runfo

This is an abbreviation for "runtime info." It's a tool that provides quick
This is an abbreviation for "runtime info." It's a tool that provides quick
summary status of builds from the dotnet/runtime repository.

## Authentication
In order to use the `tests` command you will need to provide a personal access
token to the tool. These can be obtained by visitting the following site:
In order to use `runfo` you will need to provide personal access
tokens to the tool. These can be obtained by visitting the following site:

- https://dnceng.visualstudio.com/_usersSettings/tokens
- AzDO: https://dev.azure.com/dnceng/_usersSettings/tokens
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This URL matches the one indicated in settings.txt

- Helix: https://helix.dot.net/Account/Tokens

The token can be passed to the tool in two ways:
The tokens can be passed to the tool in two ways:

1. By using the `-token` command line argument
2. Using the `%RUNFO_AZURE_TOKEN%` environment variable
1. By saving your tokens in `%LOCALAPPDATA%/runfo/settings.txt`. Example of the file contents:

## Build filtering
All commands that search builds use the same set of arguments to define the
```
azdo-token=YOUR_AZDO_TOKEN
helix-token=YOUR_HELIX_TOKEN
helix-base-uri=https://helix.dot.net/
```

2. By passing the token arguments in the command line. Example:

```
runfo --azdo-token=YOUR_AZDO_TOKEN --helix-token=YOUR_HELIX_TOKEN --helix-base-uri=https://helix.dot.net/ [additional arguments]
```

3. Using the `%RUNFO_AZURE_TOKEN%` and `%RUNFO_HELIX_TOKEN%` environment variables. There is no environment variable for the helix base uri (the default `https://helix.dot.net/` will be used). Example:

```cmd
set RUNFO_AZURE_TOKEN=YOUR_AZDO_TOKEN
set RUNFO_HELIX_TOKEN=YOUR_HELIX_TOKEN

runfo [additional arguments]
```

## Build filtering
All commands that search builds use the same set of arguments to define the
set of builds being searched:

- `definition`: the build definition id or name to get builds from
Expand All @@ -25,7 +46,7 @@ set of builds being searched:
- `after`: filter to builds after the given date
- `before`: filter to builds before the given date

The common pattern is to search the most recent hundred builds in a given
The common pattern is to search the most recent hundred builds in a given
definition to find occurances of a failure. For example:

```cmd
Expand All @@ -36,14 +57,14 @@ This will search the last 100 builds from the runtime build definition for any
failures that have the text "Central Directory Record".

## Commands
These are the most common commands used to search for failures. More are
These are the most common commands used to search for failures. More are
available and all have help available by using the `-help` argument.

### search-timeline
This command will search the timeline of the builds looking for a particular
piece of text. There are two ways to filter the results:

- `name`: only search records matching this name. Default is search all
- `name`: only search records matching this name. Default is search all
records
- `value`: find records whose issues match the following string

Expand All @@ -56,8 +77,8 @@ For example here is how we commonly filtered the Docker 126 exit code issue.
The `-markdown` argument will print the output using markdown tables.

### tests
This dumps the test information for the provided builds. Essentially it will
enumerate all of the builds and dump test failure information based on the
This dumps the test information for the provided builds. Essentially it will
enumerate all of the builds and dump test failure information based on the
provided grouping. The grouping can be changed by passing the following
arguments to the `-group` switch:

Expand All @@ -75,12 +96,12 @@ netcoreapp5.0-Linux-Release-arm64-CoreCLR_release-(Alpine.38.Arm64.Open)Ubuntu.1
netcoreapp5.0-Linux-Release-arm64-CoreCLR_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-bfcd90a-20200127194925
Builds 4
Test Cases 16
<lots of clipped data>
<lots of clipped data>
```

Notice that the Alpine.38.Arm64 job failed on 5 builds which is also the
amount we’re limiting the results too. So pretty good bet this configuration is
busted in some way that requires investigation.
Notice that the Alpine.38.Arm64 job failed on 5 builds which is also the
amount we’re limiting the results too. So pretty good bet this configuration is
busted in some way that requires investigation.

When investigating a particular test failure you can use `-name` to limit the
results to test failures matching the provided name regex.
Expand All @@ -97,22 +118,22 @@ https://helix.dot.net/api/2019-06-17/jobs/6afd25d6-b672-4525-bcb3-92be7581046a/w
https://helix.dot.net/api/2019-06-17/jobs/3cd49f06-a2f6-4a87-bda4-d33be9b16f83/workitems/System.Runtime.Tests/files/console.7fdd181f.log
```

Going to change tests to have this info soon as well.
Going to change tests to have this info soon as well.

## FAQ
### Why the name runfo?
I’m terrible at naming things. The tool was meant for “Runtime Information” so
I’m terrible at naming things. The tool was meant for “Runtime Information” so
I shortened it to runfo cause I’m bad at naming.

### Why this over the CI Council Dashboard
CI council dashboard is meant to represent overall repository CI health. It
takes into account bigger issues like publishing, core-eng infra issues, etc …
CI council dashboard is meant to represent overall repository CI health. It
takes into account bigger issues like publishing, core-eng infra issues, etc …
It also only follows a subset of the build definitions that we maintain. I
wanted a quick and dirty tool for looking at test failures only as that’s the
biggest source of flakiness that we directly control as a team.

### I looked at the code and you should be ashamed of the hackery!
Indeed I am ashamed. I’ve used more SelectMany calls and Tuples in this
Indeed I am ashamed. I’ve used more SelectMany calls and Tuples in this
tool as I did in the rest of my career combined :smile:


Expand Down
Loading