Polly and HttpClientFactory

Using Polly with HttpClient factory from ASPNET Core 2.1

TL;DR HttpClient factory in ASPNET Core 2.1 provides a way to pre-configure instances of HttpClient which apply Polly policies to every outgoing call (among other benefits).

What is HttpClient factory?

From ASPNET Core 2.1, Polly integrates with IHttpClientFactory. HttpClient factory is a factory that simplifies the management and usage of HttpClient in four ways. It:

• allows you to name and configure logical HttpClients. For instance, you may configure a client that is pre-configured to access the github API;

• manages the lifetime of HttpClientMessageHandlers to avoid some of the pitfalls associated with managing HttpClient yourself (the disposing-it-too-often-can-cause-socket-exhaustion but also only-using-a-singleton-can-miss-DNS-updates aspects);

• provides configurable logging (via ILogger) for all requests and responses performed by clients created with the factory;

• provides a simple API for adding middleware to outgoing calls, be that for logging, authorisation, service discovery, or resilience with Polly.

The Microsoft early announcement speaks more to these topics, and Steve Gordon's quartet of blog posts (1; 2; 3; 4) are also an excellent read for deeper background and some great worked examples. UPDATE: The official documentation is also now out.

Using Polly with IHttpClientFactory

Step 1 Reference the ASPNET Core 2.1 packages

Have your project grab the ASPNET Core 2.1 packages from nuget. You'll typically need the AspNetCore metapackage, and the extension package Microsoft.Extensions.Http.Polly.

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.0" />
    <PackageReference Include="Microsoft.Extensions.Http.Polly" Version="2.1.0" />
  </ItemGroup>

</Project>

Note: later versions of these packages may be available when you read this.

Step 2 Configure a client with Polly policies, in Startup

Define a named HttpClient configuration

In your standard Startup.ConfigureServices(...) method, start by configuring a named client as below:

public void ConfigureServices(IServiceCollection services)
{
    // Configure a client named as "GitHub", with various default properties.
    services.AddHttpClient("GitHub", client =>
    {
        client.BaseAddress = new Uri("https://api.github.com/");
        client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
    });

    // ...
}

(We've used magic strings for clarity, but of course you can obtain those from config or declare them once as consts.)

We'll focus on configuring this with Polly policies, but there are many more options for configuring the named HttpClient which you can read about from the official doco, or Steve Gordon or Scott Hanselman. To keep the examples in this post shorter, we've used named clients, but the documentation and blogs above also cover how to use typed clients, which offer the advantages of strong-typing and allowing you to build call overloads focused on your specific needs.

Fluently extend that client configuration with Polly policies

To apply Polly policies, you simply extend the above example with some fluent configuration:

services.AddHttpClient("GitHub", client =>
{
    client.BaseAddress = new Uri("https://api.github.com/");
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
})
.AddTransientHttpErrorPolicy(builder => builder.WaitAndRetryAsync(new[]
{
    TimeSpan.FromSeconds(1),
    TimeSpan.FromSeconds(5),
    TimeSpan.FromSeconds(10)
}));

This example creates a policy which will handle typical transient faults, retrying the underlying http request up to 3 times if necessary. The policy will apply a delay of 1 second before the first retry; 5 seconds before a second retry; and 10 seconds before the third.

The overload .AddTransientHttpErrorPolicy(...) is one of a number of options, which we'll look at after covering the basics.

Step 3 Consume the configured HttpClient

For completeness, here's an example of consuming the configured HttpClient. For a named client (as the above example), take an IHttpClientFactory by dependency injection at the usage site. Then use that factory to obtain an HttpClient configured to the specification you defined in Startup:

public class MyController : Controller
{
    private readonly IHttpClientFactory _httpClientFactory;

    public MyController(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public Task<IActionResult> SomeAction()
    {
        // Get an HttpClient configured to the specification you defined in StartUp.
        var client = _httpClientFactory.CreateClient("GitHub"); 

        return Ok(await client.GetStringAsync("/someapi"));
    }
}

Again, Steve Gordon's and Scott Hanselman's blogs give richer examples, including if you prefer typed clients.

How are the Polly policies applied?

The policy or policies configured on your HttpClient are applied to outbound calls by Polly-based DelegatingHandlers.

This means the policies will be applied to all outgoing calls through that configured HttpClient.

If you've tried in the past to hand-craft retries outside calls to HttpClient.SendAsync(...) which pass in an HttpRequestMessage, you may have discovered that the HttpRequestMessage passed in cannot be reused once sent (doing so raises an InvalidOperationException). The DelegatingHandler approach avoids this problem.

A DelegatingHandler is simply middleware for an outbound http call: see Steve Gordon's third blog post for a great introduction to how delegating handlers work.

Configuring the Polly policies

Using .AddTransientHttpErrorPolicy(...)

Let's look at the example from Step 2 above again:

services.AddHttpClient("GitHub", client =>
{
    client.BaseAddress = new Uri("https://api.github.com/");
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
})
.AddTransientHttpErrorPolicy(builder => builder.WaitAndRetryAsync(new[]
{
    TimeSpan.FromSeconds(1),
    TimeSpan.FromSeconds(5),
    TimeSpan.FromSeconds(10)
}));

This uses a new convenience method, .AddTransientHttpErrorPolicy(...). This configures a policy to handle errors typical of Http calls:

• Network failures (System.Net.Http.HttpRequestException)
• HTTP 5XX status codes (server errors)
• HTTP 408 status code (request timeout)

Using .AddTransientHttpErrorPolicy(...) pre-configures what the policy handles. The builder => builder clause then specifies how the policy will handle those faults.

In the builder => builder clause you can choose any reactive policy from Polly's offerings: a retry strategy (as in the above example), circuit-breaker or fallback policy.

The choice in .AddTransientHttpErrorPolicy(...) to handle HttpRequestException, HTTP 5xx, HTTP 408 is a convenience option, but not mandatory. If that error filter doesn't suit your needs - which you should think through - you can extend the definition of errors to handle, or build an entirely bespoke Polly policy.

Using any policy configured via the traditional Polly syntax

Overloads are also available taking any IAsyncPolicy<HttpResponseMessage>, so you can define and apply any kind of policy: you specify both the what to handle and how to handle.

This example demonstrates .AddPolicyHandler(...) to add a policy where we coded our own specification of faults to handle:

var retryPolicy = Policy.Handle<HttpRequestException>()
    .OrResult<HttpResponseMessage>(response => MyCustomResponsePredicate(response))
    .WaitAndRetryAsync(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(10)
    }));

services.AddHttpClient(/* etc */)
    .AddPolicyHandler(retryPolicy);

As well as Polly's reactive policies (such as retry and circuit-breaker), these overloads mean you can also use proactive policies such as timeout:

var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10);

services.AddHttpClient(/* etc */)
    .AddPolicyHandler(timeoutPolicy);

All calls through HttpClient return an HttpResponseMessage, so the policies configured must be of type IAsyncPolicy<HttpResponseMessage>. Non-generic policies IAsyncPolicy can also be converted to IAsyncPolicy<HttpResponseMessage> with a simple convenience method:

var timeoutPolicy = Policy.TimeoutAsync(10);

services.AddHttpClient(/* etc */)
    .AddPolicyHandler(timeoutPolicy.AsAsyncPolicy<HttpResponseMessage>());

Extending the convenience .AddTransientHttpErrorPolicy(...) definition

The definition of errors handled by .AddTransientHttpErrorPolicy(...) is also available from a Polly extension package, Polly.Extensions.Http (github; nuget).

Using this allows you to take the base specification of errors to handle (HttpRequestException, HTTP 5xx, HTTP 408) and extend it. For example, the policy configured below would handle status code 429 additionally:

using Polly.Extensions.Http; // After installing the nuget package: Polly.Extensions.Http

// ..

var policy = HttpPolicyExtensions
  .HandleTransientHttpError() // HttpRequestException, 5XX and 408
  .OrResult(response => (int)response.StatusCode == 429) // RetryAfter
  .WaitAndRetryAsync(/* etc */);

Applying multiple policies

All overloads for configuring policies can also be chained to apply multiple policies:

services.AddHttpClient(/* etc */)
    .AddTransientHttpErrorPolicy(builder => builder.WaitAndRetryAsync(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(10)
    }))
    .AddTransientHttpErrorPolicy(builder => builder.CircuitBreakerAsync(
        handledEventsAllowedBeforeBreaking: 3,
        durationOfBreak: TimeSpan.FromSeconds(30)
    ));

What order are multiple policies applied in?

When you configure multiple policies (as in the above example), the policies are applied to each call from outer (first-configured) to inner (last-configured) order.

In the above example, the call will:

1. first be placed through the (outer) retry policy, which will in turn:
2. place the call through the (inner) circuit-breaker, which in turn:
3. makes the underlying http call.

The sequencing of policies in this example was chosen because the circuit-breaker may change state in one of those periods (1, 5 or 10 seconds) when the retry policy is waiting between tries. The circuit-breaker is configured 'inside' the retry, so that the circuit state is tested again as part of the action of making a retry.

The above example applies two policies (retry and circuit-breaker), but any number is possible. A common useful combination might be to apply a retry, a circuit-breaker, and a timeout-per-try (see below).

How does this compare to PolicyWrap?

For those familiar with Polly's PolicyWrap, configuring multiple policies with the pattern shown above is entirely equivalent to using a PolicyWrap. All the usage recommendations in the PolicyWrap wiki apply.

Combining PolicyHttpMessageHandler with other DelegatingHandlers

Likewise, if you combine PolicyHttpMessageHandler with other DelegatingHandlers, consider whether the policy handlers should be 'inside' or 'outside' the other delegating handlers in the middleware pipeline you construct. The sequence in which DelegatingHandlers are applied corresponds to the sequence you configure them in after the .AddHttpClient(/* etc */) call.

Selecting policies dynamically

Overloads of .AddPolicyHandler(...) exist allowing you to select policies dynamically based on the request.

One use case for this is to apply different policy behaviour for endpoints which are not idempotent. POST operations typically are not idempotent. PUT operations should be idempotent, but may not be for a given API (there is no substitute for knowing the behaviour of the API you are calling). So, you might want to define a strategy which retries for GET requests but not for other http verbs:

var retryPolicy = HttpPolicyExtensions
    .HandleTransientHttpError()
    .WaitAndRetryAsync(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(10)
    });
var noOpPolicy = Policy.NoOpAsync().AsAsyncPolicy<HttpResponseMessage>();

services.AddHttpClient(/* etc */)
    // Select a policy based on the request: retry for Get requests, noOp for other http verbs.
    .AddPolicyHandler(request => request.Method == HttpMethod.Get ? retryPolicy : noOpPolicy);

The above example uses NoOp policy for http verbs other than GET. NoOp policy simply executes the underlying call 'as is', without any additional policy behaviour.

Selecting policies from a PolicyRegistry

Polly also provides PolicyRegistry as a central store for policies you might reuse in multiple places in your application. Overloads of .AddPolicyHandler(...) exist allowing you to select a policy from the registry.

The following example registers a PolicyRegistry with the IServiceCollection, adds some policies to the registry, and then defines two logical clients using different policies from the registry.

var registry = services.AddPolicyRegistry();

registry.Add("defaultretrystrategy", 
    HttpPolicyExtensions.HandleTransientHttpError().WaitAndRetryAsync(/* etc */));
registry.Add("defaultcircuitbreaker", 
    HttpPolicyExtensions.HandleTransientHttpError().CircuitBreakerAsync(/* etc */));

services.AddHttpClient(/* etc */)
    .AddPolicyHandlerFromRegistry("defaultretrystrategy");
    
services.AddHttpClient(/* etc */)
    .AddPolicyHandlerFromRegistry("defaultretrystrategy")
    .AddPolicyHandlerFromRegistry("defaultcircuitbreaker");

More complex use cases for PolicyRegistry include dynamically updating the policies in your registry from an external source, to facilitate dynamic reconfiguration of policies during running.

Configuring policies to use services registered with DI, such as ILogger<T>

You may want to configure a policy which makes use of other services registered for Dependency Injection. A typical example would be to configure a policy whose policy hooks access an ILogger<T> configured with DI.

An .AddPolicyHandler(...) overload exists allowing you to configure a policy which can resolve services from IServiceProvider when the policy is created.

Because the typical .NET Core logging pattern prefers generic ILogger<T>, this approach plays best with typed clients.

services.AddHttpClient<MyServiceHttpClient>(/* etc */)
    .AddPolicyHandler((services, request) => HttpPolicyExtensions.HandleTransientHttpError()
        .WaitAndRetryAsync(new[]
        {
            TimeSpan.FromSeconds(1),
            TimeSpan.FromSeconds(5),
            TimeSpan.FromSeconds(10)
        },             
        onRetry: (outcome, timespan, retryAttempt, context) =>
        {
            services.GetService<ILogger<MyServiceHttpClient>>()
                .LogWarning("Delaying for {delay}ms, then making retry {retry}.", timespan.TotalMilliseconds, retryAttempt);
        }
        ));

Use Case: Applying timeouts

HttpClient already has a Timeout property, but how does this apply when a retry policy is in use? And where does Polly's TimeoutPolicy fit?

• HttpClient.Timeout will apply as an overall timeout to each entire call through HttpClient, including all tries and waits between retries.
• To apply a timeout-per-try, configure a RetryPolicy before a Polly TimeoutPolicy.

In this case, you may want the retry policy to retry if any individual try timed out. To do this, make the retry policy handle the TimeoutRejectedException which Polly's timeout policy throws.

This example uses the Polly.Extensions.Http package described earlier, to extend the convenience error set (HttpRequestException, HTTP 5XX, and HTTP 408) with extra handling:

using Polly.Extensions.Http;

var retryPolicy = HttpPolicyExtensions
    .HandleTransientHttpError()
    .Or<TimeoutRejectedException>() // thrown by Polly's TimeoutPolicy if the inner call times out
    .WaitAndRetryAsync(new[]
        {
            TimeSpan.FromSeconds(1),
            TimeSpan.FromSeconds(5),
            TimeSpan.FromSeconds(10)
        });

var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(10); // Timeout for an individual try

serviceCollection.AddHttpClient("GitHub", client =>
{
    client.BaseAddress = new Uri("https://api.github.com/");
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
    client.Timeout = TimeSpan.FromSeconds(60); // Overall timeout across all tries
})
.AddPolicyHandler(retryPolicy)
.AddPolicyHandler(timeoutPolicy); // We place the timeoutPolicy inside the retryPolicy, to make it time out each try.

If you configure TimeoutPolicy outside RetryPolicy

If you have configured the retry and timeout policy in the other order (configuring timeoutPolicy before, thus outside, the retryPolicy), that TimeoutPolicy will instead act as an overall timeout for the whole operation (just as HttpClient.Timeout does), not as a timeout-per-try. This is a natural consequence of the way multiple policies act as nested steps in a middleware pipeline.

Use Case: scoping CircuitBreakers and Bulkheads

Policy instances applied to a named HttpClient configuration are shared across all calls through that HttpClient configuration.

For the stateful policy circuit breaker, this means that all calls through a named HttpClient configured with a circuit-breaker will share that same circuit state.

This usually plays well with HttpClients configured via HttpClient factory, because those HttpClients typically define a common BaseAddress, meaning all calls are to some endpoint on that same BaseAddress. In that case, we might expect that if one endpoint on BaseAddress is unavailable, others will be too. The scoping then plays well: if calls to one endpoint through that HttpClient configuration break the circuit, the circuit will also be broken for others.

If, however, this 'shared' scoping of the circuit-breaker is not appropriate for your scenario, define separate named HttpClient instances and configure each with a separate circuit-breaker policy instance.

The same consideration applies if you use Polly's other stateful policy, Bulkhead. With a Bulkhead policy applied to a named HttpClient configuration, the Bulkhead capacity will be shared across all calls placed through that HttpClient.

Use Case: CachePolicy

Polly CachePolicy can be used in a DelegatingHandler configured via IHttpClientFactory. Polly is generic (not tied to Http requests), so at time of writing, the Polly CachePolicy determines the cache key to use from the Polly.Context. This can be set on an HttpRequestMessage request immediately prior to placing the call through HttpClient, by using an extension method: (add using Polly; to access the extension method)

request.SetPolicyExecutionContext(new Polly.Context("CacheKeyToUseWithThisRequest")); 

Using CachePolicy with HttpClientFactory thus also requires that you use overloads on HttpClient which take an HttpRequestMessage as an input parameter.

Some additional considerations flow from the fact that caching with Polly CachePolicy in a DelegatingHandler caches at the HttpResponseMessage level.

Is caching at the HttpResponseMessage level the right fit?

If the HttpResponseMessage is the end content you wish to re-use (perhaps to re-serve in whole or in part), then caching at the HttpResponseMessage level may be a good fit.

In cases such as calling to a webservice to obtain some serialized data which will then be deserialized to some local types in your app, HttpResponseMessage may not be the optimal granularity for caching.

In these cases, caching at the HttpResponseMessage level implies subsequent cache hits repeat the stream-read and deserialize-content operations, which is unnecessary from a performance perspective.

It may be more appropriate to cache at a level higher-up - for example, cache the results of stream-reading and deserializing to the local types in your app.

Considerations when caching HttpResponseMessage

• The HttpResponseMessage can contain HttpContent which behaves like a forward-only stream - you can only read it once. This can mean that when CachePolicy retrieves it from cache the second time, the stream cannot be re-read unless you also reinitialise the stream pointer.

• Consider de-personalisation and timestamping. Personal information (if any) and timestamps from a cached result may not be appropriate to re-supply to later requesters.

• Exercise care to only cache 200 OK responses. Consider using code such as response.EnsureSuccessStatusCode(); to ensure that only successful responses pass to the cache policy. Or you can use a custom ITtlStrategy as described here.

Use Case: Exchanging information between policy execution and calling code

An execution-scoped instance of the class Polly.Context travels with every execution through a Polly policy. The role of this class is to provide context and to allow the exchange of information between the pre-execution, mid-execution, and post-execution phases.

For executions through HttpClients configured with Polly via HttpClientFactory, you can use the extension method HttpRequestMessage.SetPolicyExecutionContext(context), prior to execution, to set the Polly.Context that will be used with the Http call. Context has dictionary-semantics, allowing you to pass any arbitrary data.

var context = new Polly.Context();
context["MyCustomData"] = foo;

HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, requestUri);
request.SetPolicyExecutionContext(context); 

var response = await client.SendAsync(request, cancellationToken);

Polly passes that Context instance as an input parameter to any delegate hooks such as onRetry configured on the policy. For example, the HttpClient may have been pre-configured with a policy:

var retryPolicy = HttpPolicyExtensions
    .HandleTransientHttpError()
    .WaitAndRetryAsync(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(10)
    },
    onRetryAsync: async (outcome, timespan, retryCount, ctx) => {
        /* Do something with ctx["MyCustomData"] */
        // ...
    });

Delegate hooks may also set information on Context:

var retryPolicy = HttpPolicyExtensions
    .HandleTransientHttpError()
    .WaitAndRetryAsync(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(10)
    },
    onRetryAsync: async (outcome, timespan, retryCount, ctx) => {
        ctx["RetriesInvoked"] = retryCount;
        // ...
    });

And this information can be read from the context after execution:

var response = await client.SendAsync(request, cancellationToken);

var context = response.RequestMessage?.GetPolicyExecutionContext(); // (if not already held in a local variable)
if (context?.TryGetValue("RetriesInvoked", out int? retriesNeeded) ?? false)
{
    // Do something with int? retriesNeeded
}

Note that the context from HttpRequestMessage.GetPolicyExecutionContext() is only available post-execution if you used HttpRequestMessage.SetPolicyExecutionContext(Context) to set a context prior to execution.

IHttpClientFactory and Polly Roadmap

Going forward, Polly vNext and a future ASPNET Core release intend to integrate policy logging with HttpClient factory, so that policies applied to clients configured via HttpClient factory will log - with no extra user configuration needed - to the same loggers used by the HttpClient instance.