Testing And Coverage

Testing and Coverage

• Introduction
• Running Tests
• Writing Tests
  • Helpers
• Code Coverage

Introduction

Cesil has an extensive test suite, using the XUnit framework, in the Cesil.Tests project. The test suite aims for complete coverage of all branches and configuration options, as well as error cases.

Cesil.Tests is built without nullable reference types (that is, it is 'null oblivious') to make testing appropriate detection of null errors simpler.

Running Tests

Tests can be run with any runner that understands XUnit tests. Two such runners are Visual Studio's built-in runner, and dotnet test.

Certain cases are only explored in DEBUG builds, as they depend on instrumentation that is removed in RELEASE builds.

Writing Tests

Cesil.Tests has a number of helpers that automate testing of certain functionality in conjunction with helpers in the Cesil project. Any tests written should use the appropriate helpers to ensure that all special functionality is thoroughly tested.

Automated functionality tests cover:

• Leaking IMemoryOwner<char>
• Small buffer configurations
• Fast-path short circuiting to avoid awaits
• Cancellation
• Disposal

For ease of use, using static Cesil.Tests.Helpers; is declared in many test files.

Helpers

There are eight helpers, one for each combination of "sync/async", "static/dynamic" and "read/write":

• Helpers.RunAsyncDynamicReaderVariants
• Helpers.RunAsyncDynamicWriterVariants
• Helpers.RunAsyncReaderVariants
• Helpers.RunAsyncWriterVariants
• Helpers.RunSyncWriterVariants
• Helpers.RunSyncReaderVariants
• Helpers.RunSyncDynamicReaderVariants
• Helpers.RunSyncDynamicWriterVariants

Every helper will check for leaks using an instrumented MemoryPool<char>.

Every helper will run with a modified Options that reduces certain buffers to their minimum size.

Every async helper will automate testing of different fast-path transition points. This relies upon using AsyncTestHelper.IsCompletedSuccessfully(...) in Cesil code.

Every async helper will also automate testing that cancellation leaves the created reader/writer in a "poisoned" state. It will also test that cancellation doesn't leave any resources undisposed. This relies upon using AwaitHelper.CheckCancellation(...) and AwaitHelper.ConfigureCancellableAwait(...) in Cesil code.

The IDisposalTests.IDisposable and IDisposalTests.IAsyncDisposable tests ensure that their respective interfaces are correctly implemented. "Correctly implemented" means that double-disposal doesn't error and that all public members throw an ObjectDisposedException if accessed after being disposed. These tests rely on types implementing ITestableDisposable and ITestableAsyncDisposable instead of directly implementing IDisposable and IDisposableAsync.

Code Coverage

Cesil uses Coverlet to produce test coverage reports.

In order to produce a report, run RunCodeCoverage.ps1 in Cesil's root directory, this will place a report in the TestCoverageResults folder.

Cesil aims for complete coverage of all branches and lines, however certain edge cases prevent complete coverage from being achieved:

• await hides a couple branches, often resulting in partial branch coverage (this appears as a yellow line in reports)
• Certain "Impossible" cases must still be accounted for in control flow, but cannot be reached via tests
  • Use the ImpossibleException methods on Throw in these cases, to signal that coverage is not expected

Any new functionality should have tests that fully cover it committed, subject to the above caveats.