Skip to content

Commit

Permalink
Set -useWarnings default to true
Browse files Browse the repository at this point in the history
  • Loading branch information
Marc M. Adkins committed Aug 13, 2024
1 parent ef7284f commit a87ac9f
Show file tree
Hide file tree
Showing 13 changed files with 28 additions and 27 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
run: go test -bench=. bench/*.go > ${{ runner.temp }}/bench.txt

- name: Verify Go logging libraries
run: go test -v ./verify -args -useWarnings > ${{ runner.temp }}/verify.txt
run: go test -v ./verify > ${{ runner.temp }}/verify.txt

- name: Run Server & Get Pages
run: go run cmd/server/*.go -bench=${{ runner.temp }}/bench.txt -verify=${{ runner.temp }}/verify.txt & scripts/pages
Expand Down
1 change: 0 additions & 1 deletion TODO.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# TODO

* Make -useWarnings default to true, fix various doc to match
* Warning levels in top-level README.md
? Default chart size on score page to smallest one that contains all items
? Workflow keep-alive
Expand Down
2 changes: 1 addition & 1 deletion bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ a global resource to the other tests ([described below](#testmain)).

Run the handler verification tests installed in this repository with:
```shell
go test -bench=. bench/*.go -args -useWarnings
go test -bench=. bench/*.go
```

It is not necessary to use the `-benchmem` argument to generate memory statistics.
Expand Down
2 changes: 1 addition & 1 deletion cmd/server/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The flags are:
-language value
One or more language tags to be tried, defaults to US English.
-useWarnings
Show warning instead of known errors
Show warning instead of known errors, defaults true
-verify string
Load verification data from path (optional)
Expand Down
4 changes: 2 additions & 2 deletions cmd/tabular/tabular.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ The flags are:
Load benchmark data from path (optional)
-language value
One or more language tags to be tried, defaults to US English.
-useWarnings
Show warning instead of known errors
-useWarnings=<bool>
Show warning instead of known errors, defaults true
-verify string
Load verification data from path (optional)
Expand Down
3 changes: 2 additions & 1 deletion infra/warning/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
// - In test code check the `WarningManager` for applicable warnings.
// - In warning-specific code use `AddWarning` to note the condition exists
// or `UnusedWarning` to note that the warning is redundant.
// - Run the tests with the `-useWarnings` flag to invoke warning code.
// - Run the tests with the `-useWarnings` flag set to true to invoke warning code.
// The `useWarnings` flag is turned on by default.
// Without this flag the `WarningManager` will never flag warning code
// and test assertions will raise conventional errors.
package warning
10 changes: 6 additions & 4 deletions infra/warning/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,19 @@ import (
// Warnings mechanism to trade test failure for warning list at end of tests.

// useWarnings is the flag value for enabling warning instead of known errors.
// Command line setting:
// The default value for this flag is true.
// In the rare event that warnings should be disabled
// (resulting in actual errors in the test harness and no warning results) use:
//
// go test ./... -args -useWarnings
// go test ./... -args -useWarnings=false
//
// This flag will automatically set WarnLevelCase.
// Other behavior must be activated in specific handler test managers, for example:
//
// sLogSuite := &test.SlogTestSuite{Creator: &SlogCreator{}}
// sLogSuite.WarnOnly(test.WarnMessageKey)
// suite.Run(t, slogSuite)
var useWarnings = flag.Bool("useWarnings", false, "Show warning instead of known errors")
var useWarnings = flag.Bool("useWarnings", true, "Show warning instead of known errors")

// -----------------------------------------------------------------------------

Expand Down Expand Up @@ -115,7 +117,7 @@ func (mgr *Manager) WarnOnly(w *Warning) {
// -----------------------------------------------------------------------------
// Calls to be made during testing.

// AddUnused adds a Unused warning to the results list.
// AddUnused adds an Unused warning to the results list.
// The warning added is Unused and the extra text is the name of the specified warning.
func (mgr *Manager) AddUnused(w *Warning, logRecordJSON string) {
mgr.AddWarning(Unused, w.Name, logRecordJSON)
Expand Down
2 changes: 1 addition & 1 deletion scripts/bench
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

clear
mkdir -p /tmp/go-slog
go test -bench=. bench/*.go -args -useWarnings | # Run benchmark tests \
go test -bench=. bench/*.go | # Run benchmark tests \
tee /dev/tty | # Show progress to user in real time \
tee /tmp/go-slog/bench.txt | # Save benchmark output \
go run cmd/tabular/tabular.go | # Process JSON to table \
Expand Down
3 changes: 1 addition & 2 deletions scripts/tests
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#!/bin/bash

# Run all tests in the current repository except for the slog verification tests.
# The verification tests require the -useWarnings flag which blows up other tests.

clear
go test -skip=TestVerify -v ./... "$@"
go test -v ./... "$@"
4 changes: 2 additions & 2 deletions scripts/verify
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/bin/bash

# Run slog verification tests with -useWarnings.
# Run slog verification tests.
# Save verification output to /tmp/go-slog/verify.txt.

clear
mkdir -p /tmp/go-slog
go test -v ./verify -args -useWarnings | # Run verification tests \
go test -v ./verify | # Run verification tests \
tee /tmp/go-slog/verify.txt # Save benchmark output
4 changes: 2 additions & 2 deletions template/go_slot_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
//
// Run verification suite:
//
// go test -v -args -useWarnings
// go test -v
//
// Run Benchmark suite:
//
// go test -v -run=none -bench=. -args -useWarnings
// go test -v -run=none -bench=.
package template

import (
Expand Down
14 changes: 6 additions & 8 deletions verify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,25 +80,22 @@ a global resource to the other tests ([described below](#testmain)).

Run the handler verification tests installed in this repository with:
```shell
go test -v ./verify -args -useWarnings
go test -v ./verify
```

On an operating system that supports `bash` scripts you can use
the [`scripts/verify`](https://github.com/madkins23/go-slog/blob/main/scripts/verify) script.

**Note**: running `go test ./... -args -useWarnings` will fail as
the other tests in the repository don't recognize the `-useWarnings` flag.

#### Test Flags

There are two flags defined for testing the verification code:
* `-debug=<level>`
Sets an integer level for showing any `test.Debugf()` statements in the code.
* `-useWarnings`
Activates the warning system (see [**Warnings**](#warnings)) below.
Without this flag the tests fail on errors in the usual manner.
When this flag is present tests succeed and warnings are presented
in the `go test` output.
As of 2024-08-13 this flag is set to `true` by default
so that all tests succeed and warnings are presented in the `go test` output.
When set to `false` the tests fail on errors in the usual manner.

## Creators

Expand Down Expand Up @@ -242,7 +239,8 @@ The test harness that drives verification has some limitations.
limited number of handlers for which tests have been configured.
If your handler comes up with a new error condition for which there are tests but no warning
you can either fix your handler or file a ticket.
* The `-useWarnings` flag tends to result in the results being buried in the normal `go test` output.
* The `-useWarnings` flag (which is `true` by default)
tends to result in the results being buried in the normal `go test` output.
This can be fixed by implementing a global [`TestMain()`](#testmain) function.
* Warnings will only be visible when running `go test` if the `-v` flag is used.

Expand Down
4 changes: 3 additions & 1 deletion verify/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Supporting files:

The verification test suite is based on
[`stretchr/testify/suite.Suite`](https://pkg.go.dev/github.com/stretchr/testify/suite).
Running that code without the `-useWarnings` flag will result in
Running that code with the `-useWarnings=false` flag will result in
error messages generated by the test suite.
These messages can be confusing and difficult to parse.

Expand All @@ -77,6 +77,8 @@ a secondary mechanism using "warnings" is used.
The [code that manages verification warnings](https://github.com/madkins23/go-slog/blob/main/infra/warnings.go)
is currently located in the `infra` package.

As of 2024-08-13 the `-useWarnings` flag is set to `true` by default.

### Usage

The following is a basic format for adding a warning to a pre-existing test:
Expand Down

0 comments on commit a87ac9f

Please sign in to comment.