Skip to content

Commit

Permalink
Update docs and cleanup some chaos API (#1914)
Browse files Browse the repository at this point in the history
  • Loading branch information
martintmk authored Jan 23, 2024
1 parent 54becab commit 74f0a72
Show file tree
Hide file tree
Showing 13 changed files with 145 additions and 97 deletions.
8 changes: 4 additions & 4 deletions docs/chaos/behavior.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ The behavior chaos strategy is designed to inject custom behaviors into system o
// To use a custom delegated for injected behavior
var optionsWithBehaviorGenerator = new ChaosBehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05
};

// To get notifications when a behavior is injected
var optionsOnBehaviorInjected = new ChaosBehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05,
OnBehaviorInjected = static args =>
Expand All @@ -43,7 +43,7 @@ new ResiliencePipelineBuilder().AddChaosBehavior(optionsWithBehaviorGenerator);
new ResiliencePipelineBuilder<HttpResponseMessage>().AddChaosBehavior(optionsOnBehaviorInjected);

// There are also a handy overload to inject the chaos easily
new ResiliencePipelineBuilder().AddChaosBehavior(0.05, RestartRedisVM);
new ResiliencePipelineBuilder().AddChaosBehavior(0.05, RestartRedisAsync);
```
<!-- endSnippet -->

Expand All @@ -62,7 +62,7 @@ var pipeline = new ResiliencePipelineBuilder()
})
.AddChaosBehavior(new ChaosBehaviorStrategyOptions // Chaos strategies are usually placed as the last ones in the pipeline
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05
})
Expand Down
36 changes: 31 additions & 5 deletions docs/chaos/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,36 @@

![Simmy](../media/simmy-logo.png)

## Usage

<!-- snippet: chaos-usage -->
```cs
var builder = new ResiliencePipelineBuilder<HttpResponseMessage>();

// First, configure regular resilience strategies
builder
.AddConcurrencyLimiter(10, 100)
.AddRetry(new RetryStrategyOptions<HttpResponseMessage> { /* configure options */ })
.AddCircuitBreaker(new CircuitBreakerStrategyOptions<HttpResponseMessage> { /* configure options */ })
.AddTimeout(TimeSpan.FromSeconds(5));

// Finally, configure chaos strategies if you want to inject chaos.
// These should come after the regular resilience strategies.
// 2% of invocations will be injected with chaos
const double InjectionRate = 0.02;

builder
.AddChaosLatency(InjectionRate, TimeSpan.FromMinutes(1)) // Inject a chaos latency to executions
.AddChaosFault(InjectionRate, () => new InvalidOperationException("Injected by chaos strategy!")) // Inject a chaos fault to executions
.AddChaosOutcome(InjectionRate, () => new HttpResponseMessage(System.Net.HttpStatusCode.InternalServerError)) // Inject a chaos outcome to executions
.AddChaosBehavior(0.001, cancellationToken => RestartRedisAsync(cancellationToken)); // Inject a chaos behavior to executions
```
<!-- endSnippet -->

> [!NOTE]
> It is usual to place the chaos strategy as the last strategy in the resilience pipeline. By placing the chaos strategies as last, they subvert the usual outbound call at the last minute, substituting their fault or adding extra latency, etc. The existing resilience strategies - further out in the `ResiliencePipeline` - still apply, so you can test how the Polly resilience strategies you have configured handle the chaos/faults injected by Simmy.
## Motivation

There are a lot of questions when it comes to chaos engineering and making sure that a system is actually ready to face the worst possible scenarios:
Expand Down Expand Up @@ -34,14 +64,10 @@ Chaos strategies (formerly known as Monkey strategies) are in essence a [Resilie
| Strategy | Reactive | What does the strategy do? |
|-------------------------|----------|----------------------------------------------------------------------|
| [Fault](fault.md) | No | Injects exceptions in your system. |
| [Result](result.md) | Yes | Substitute results to fake outcomes in your system. |
| [Outcome](outcome.md) | Yes | Injects fake outcomes (results or exceptions) in your system. |
| [Latency](latency.md) | No | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | No | Allows you to inject *any* extra behaviour, before a call is placed. |

## Usage

It is usual to place the chaos strategy as the last strategy in the resilience pipeline. By placing the chaos strategies as last, they subvert the usual outbound call at the last minute, substituting their fault or adding extra latency, etc. The existing resilience strategies - further out in the `ResiliencePipeline` - still apply, so you can test how the Polly resilience strategies you have configured handle the chaos/faults injected by Simmy.

## Common options across strategies

All the strategies' options implement the [`ChaosStrategyOptions`](xref:Polly.Simmy.ChaosStrategyOptions) class as it contains the basic configuration for every chaos strategy.
Expand Down
1 change: 0 additions & 1 deletion docs/chaos/outcome.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
## About

- **Options**:
- [`ChaosOutcomeStrategyOptions`](xref:Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions)
- [`ChaosOutcomeStrategyOptions<T>`](xref:Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions`1)
- **Extensions**: `AddChaosOutcome`
- **Strategy Type**: Reactive
Expand Down
4 changes: 2 additions & 2 deletions docs/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@
items:
- name: Fault
href: chaos/fault.md
- name: Result
href: chaos/result.md
- name: Outcome
href: chaos/outcome.md
- name: Latency
href: chaos/latency.md
- name: Behavior
Expand Down
24 changes: 11 additions & 13 deletions src/Polly.Core/PublicAPI.Unshipped.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ Polly.Simmy.Behavior.OnBehaviorInjectedArguments.Context.get -> Polly.Resilience
Polly.Simmy.Behavior.OnBehaviorInjectedArguments.OnBehaviorInjectedArguments() -> void
Polly.Simmy.Behavior.OnBehaviorInjectedArguments.OnBehaviorInjectedArguments(Polly.ResilienceContext! context) -> void
Polly.Simmy.ChaosBehaviorPipelineBuilderExtensions
Polly.Simmy.ChaosStrategyOptions.Enabled.get -> bool
Polly.Simmy.ChaosStrategyOptions.Enabled.set -> void
Polly.Simmy.ChaosStrategyOptions.EnabledGenerator.get -> System.Func<Polly.Simmy.EnabledGeneratorArguments, System.Threading.Tasks.ValueTask<bool>>?
Polly.Simmy.ChaosStrategyOptions.EnabledGenerator.set -> void
Polly.Simmy.ChaosStrategyOptions.InjectionRate.get -> double
Polly.Simmy.ChaosStrategyOptions.InjectionRate.set -> void
Polly.Simmy.ChaosStrategyOptions.InjectionRateGenerator.get -> System.Func<Polly.Simmy.InjectionRateGeneratorArguments, System.Threading.Tasks.ValueTask<double>>?
Polly.Simmy.ChaosStrategyOptions.InjectionRateGenerator.set -> void
Polly.Simmy.ChaosStrategyOptions.Randomizer.get -> System.Func<double>!
Polly.Simmy.ChaosStrategyOptions.Randomizer.set -> void
Polly.Simmy.EnabledGeneratorArguments
Polly.Simmy.EnabledGeneratorArguments.Context.get -> Polly.ResilienceContext!
Polly.Simmy.EnabledGeneratorArguments.EnabledGeneratorArguments() -> void
Expand Down Expand Up @@ -71,18 +81,6 @@ Polly.Simmy.ChaosStrategy<T>.ChaosStrategy(Polly.Simmy.ChaosStrategyOptions! opt
Polly.Simmy.ChaosStrategy<T>.ShouldInjectAsync(Polly.ResilienceContext! context) -> System.Threading.Tasks.ValueTask<bool>
Polly.Simmy.ChaosStrategyOptions
Polly.Simmy.ChaosStrategyOptions.ChaosStrategyOptions() -> void
Polly.Simmy.ChaosStrategyOptions<TResult>
Polly.Simmy.ChaosStrategyOptions<TResult>.Enabled.get -> bool
Polly.Simmy.ChaosStrategyOptions<TResult>.Enabled.set -> void
Polly.Simmy.ChaosStrategyOptions<TResult>.EnabledGenerator.get -> System.Func<Polly.Simmy.EnabledGeneratorArguments, System.Threading.Tasks.ValueTask<bool>>?
Polly.Simmy.ChaosStrategyOptions<TResult>.EnabledGenerator.set -> void
Polly.Simmy.ChaosStrategyOptions<TResult>.InjectionRate.get -> double
Polly.Simmy.ChaosStrategyOptions<TResult>.InjectionRate.set -> void
Polly.Simmy.ChaosStrategyOptions<TResult>.InjectionRateGenerator.get -> System.Func<Polly.Simmy.InjectionRateGeneratorArguments, System.Threading.Tasks.ValueTask<double>>?
Polly.Simmy.ChaosStrategyOptions<TResult>.InjectionRateGenerator.set -> void
Polly.Simmy.ChaosStrategyOptions<TResult>.ChaosStrategyOptions() -> void
Polly.Simmy.ChaosStrategyOptions<TResult>.Randomizer.get -> System.Func<double>!
Polly.Simmy.ChaosStrategyOptions<TResult>.Randomizer.set -> void
Polly.Simmy.ChaosOutcomePipelineBuilderExtensions
Polly.Simmy.Outcomes.OnOutcomeInjectedArguments<TResult>
Polly.Simmy.Outcomes.OnOutcomeInjectedArguments<TResult>.Context.get -> Polly.ResilienceContext!
Expand All @@ -106,7 +104,7 @@ Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions<TResult>.OnOutcomeInjected.set
Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions<TResult>.OutcomeGenerator.get -> System.Func<Polly.Simmy.Outcomes.OutcomeGeneratorArguments, System.Threading.Tasks.ValueTask<Polly.Outcome<TResult>?>>!
Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions<TResult>.OutcomeGenerator.set -> void
Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions<TResult>.ChaosOutcomeStrategyOptions() -> void
static Polly.Simmy.ChaosBehaviorPipelineBuilderExtensions.AddChaosBehavior<TBuilder>(this TBuilder! builder, double injectionRate, System.Func<System.Threading.Tasks.ValueTask>! behavior) -> TBuilder!
static Polly.Simmy.ChaosBehaviorPipelineBuilderExtensions.AddChaosBehavior<TBuilder>(this TBuilder! builder, double injectionRate, System.Func<System.Threading.CancellationToken, System.Threading.Tasks.ValueTask>! behavior) -> TBuilder!
static Polly.Simmy.ChaosBehaviorPipelineBuilderExtensions.AddChaosBehavior<TBuilder>(this TBuilder! builder, Polly.Simmy.Behavior.ChaosBehaviorStrategyOptions! options) -> TBuilder!
static Polly.Simmy.Fault.FaultGenerator.implicit operator System.Func<Polly.Simmy.Fault.FaultGeneratorArguments, System.Threading.Tasks.ValueTask<System.Exception?>>!(Polly.Simmy.Fault.FaultGenerator! generator) -> System.Func<Polly.Simmy.Fault.FaultGeneratorArguments, System.Threading.Tasks.ValueTask<System.Exception?>>!
static Polly.Simmy.ChaosFaultPipelineBuilderExtensions.AddChaosFault<TBuilder>(this TBuilder! builder, double injectionRate, System.Func<System.Exception?>! faultGenerator) -> TBuilder!
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ public static class ChaosBehaviorPipelineBuilderExtensions
/// <returns>The same builder instance.</returns>
/// <exception cref="ArgumentNullException">Thrown when <paramref name="builder"/> is <see langword="null"/>.</exception>
/// <exception cref="ValidationException">Thrown when the options produced from the arguments are invalid.</exception>
public static TBuilder AddChaosBehavior<TBuilder>(this TBuilder builder, double injectionRate, Func<ValueTask> behavior)
public static TBuilder AddChaosBehavior<TBuilder>(this TBuilder builder, double injectionRate, Func<CancellationToken, ValueTask> behavior)
where TBuilder : ResiliencePipelineBuilderBase
{
Guard.NotNull(builder);
Expand All @@ -28,7 +28,7 @@ public static TBuilder AddChaosBehavior<TBuilder>(this TBuilder builder, double
{
Enabled = true,
InjectionRate = injectionRate,
BehaviorAction = (_) => behavior()
BehaviorAction = args => behavior(args.Context.CancellationToken)
});
}

Expand Down
54 changes: 0 additions & 54 deletions src/Polly.Core/Simmy/ChaosStrategyOptions.TResult.cs

This file was deleted.

50 changes: 46 additions & 4 deletions src/Polly.Core/Simmy/ChaosStrategyOptions.cs
Original file line number Diff line number Diff line change
@@ -1,11 +1,53 @@
namespace Polly.Simmy;
using System.ComponentModel.DataAnnotations;

#pragma warning disable CS8618 // Required members are not initialized in constructor since this is a DTO, default value is null
namespace Polly.Simmy;

/// <summary>
/// The options associated with the <see cref="ChaosStrategy"/>.
/// </summary>
public abstract class ChaosStrategyOptions : ChaosStrategyOptions<object>
public abstract class ChaosStrategyOptions : ResilienceStrategyOptions
{
}
/// <summary>
/// Gets or sets the injection rate for a given execution, which the value should be between [0, 1] (inclusive).
/// </summary>
/// <remarks>
/// Defaults to 0.001, meaning one in a thousand executions/0.1%. Either <see cref="InjectionRateGenerator"/> or this property is required.
/// </remarks>
[Range(ChaosStrategyConstants.MinInjectionThreshold, ChaosStrategyConstants.MaxInjectionThreshold)]
public double InjectionRate { get; set; } = ChaosStrategyConstants.DefaultInjectionRate;

/// <summary>
/// Gets or sets the injection rate generator for a given execution, which the value should be between [0, 1] (inclusive).
/// </summary>
/// <remarks>
/// Defaults to <see langword="null"/>. Either <see cref="InjectionRate"/> or this property is required.
/// When this property is <see langword="null"/> the <see cref="InjectionRate"/> is used.
/// </remarks>
public Func<InjectionRateGeneratorArguments, ValueTask<double>>? InjectionRateGenerator { get; set; }

/// <summary>
/// Gets or sets the enable generator that indicates whether or not the chaos strategy is enabled for a given execution.
/// </summary>
/// <remarks>
/// Defaults to <see langword="null"/>. Either <see cref="Enabled"/> or this property is required.
/// When this property is <see langword="null"/> the <see cref="Enabled"/> is used.
/// </remarks>
public Func<EnabledGeneratorArguments, ValueTask<bool>>? EnabledGenerator { get; set; }

/// <summary>
/// Gets or sets a value indicating whether or not the chaos strategy is enabled for a given execution.
/// </summary>
/// <remarks>
/// Defaults to <see langword="false"/>. Either <see cref="EnabledGenerator"/> or this property is required.
/// </remarks>
public bool Enabled { get; set; }

/// <summary>
/// Gets or sets the Randomizer generator instance that is used to evaluate the injection rate.
/// </summary>
/// <remarks>
/// The default randomizer is thread safe and returns values between 0.0 and 1.0.
/// </remarks>
[Required]
public Func<double> Randomizer { get; set; } = RandomUtil.Instance.NextDouble;
}
2 changes: 1 addition & 1 deletion src/Polly.Core/Simmy/EnabledGeneratorArguments.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#pragma warning disable CA1815 // Override equals and operator equals on value types

/// <summary>
/// Defines the arguments for the <see cref="ChaosStrategyOptions{TResult}.EnabledGenerator"/>.
/// Defines the arguments for the <see cref="ChaosStrategyOptions.EnabledGenerator"/>.
/// </summary>
public readonly struct EnabledGeneratorArguments
{
Expand Down
2 changes: 1 addition & 1 deletion src/Polly.Core/Simmy/InjectionRateGeneratorArguments.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#pragma warning disable CA1815 // Override equals and operator equals on value types

/// <summary>
/// Defines the arguments for the <see cref="ChaosStrategyOptions{TResult}.InjectionRateGenerator"/>.
/// Defines the arguments for the <see cref="ChaosStrategyOptions.InjectionRateGenerator"/>.
/// </summary>
public readonly struct InjectionRateGeneratorArguments
{
Expand Down
10 changes: 4 additions & 6 deletions src/Snippets/Docs/Chaos.Behavior.cs
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,19 @@ internal static partial class Chaos
{
public static void BehaviorUsage()
{
static ValueTask RestartRedisVM() => ValueTask.CompletedTask;

#region chaos-behavior-usage
// To use a custom delegated for injected behavior
var optionsWithBehaviorGenerator = new ChaosBehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05
};

// To get notifications when a behavior is injected
var optionsOnBehaviorInjected = new ChaosBehaviorStrategyOptions
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05,
OnBehaviorInjected = static args =>
Expand All @@ -38,7 +36,7 @@ public static void BehaviorUsage()
new ResiliencePipelineBuilder<HttpResponseMessage>().AddChaosBehavior(optionsOnBehaviorInjected);

// There are also a handy overload to inject the chaos easily
new ResiliencePipelineBuilder().AddChaosBehavior(0.05, RestartRedisVM);
new ResiliencePipelineBuilder().AddChaosBehavior(0.05, RestartRedisAsync);
#endregion

#region chaos-behavior-execution
Expand All @@ -53,7 +51,7 @@ public static void BehaviorUsage()
})
.AddChaosBehavior(new ChaosBehaviorStrategyOptions // Chaos strategies are usually placed as the last ones in the pipeline
{
BehaviorAction = static args => RestartRedisVM(),
BehaviorAction = static args => RestartRedisAsync(args.Context.CancellationToken),
Enabled = true,
InjectionRate = 0.05
})
Expand Down
Loading

0 comments on commit 74f0a72

Please sign in to comment.