Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExceptionHandler middleware mutes http_server_request_duration_seconds_count metrics on exceptions #52766

Closed
1 task done
davidroth opened this issue Dec 12, 2023 · 2 comments
Labels
area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlesware ✔️ Resolution: Duplicate Resolved as a duplicate of another issue Status: Resolved

Comments

@davidroth
Copy link

davidroth commented Dec 12, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

Consider the following .NET 8 minimal api application with OTEL metrics enabled.

builder.Services.AddProblemDetails();

builder.Services
        .AddOpenTelemetry()
        .WithMetrics(metrics =>
        {
            metrics.AddAspNetCoreInstrumentation();
            metrics.AddPrometheusExporter();
        })
....

app.MapGet("/crash", () =>
{
    throw new InvalidOperationException("An error occured");
})

When I run the application in Developer mode, I see that the http_server_request_duration_seconds_count metrics is emmited for the failed request and also enriched because of this line of code: DiagnosticsTelemetry.ReportUnhandledException(_logger, context, ex);.

This is the enriched metric:

http_server_request_duration_seconds_count{error_type="System.InvalidOperationException",http_request_method="GET",http_response_status_code="500",http_route="/crash",network_protocol_version="2",url_scheme="https"} 2 1702390457828

So http_server_request_duration_seconds_count is increased for every invocation on the "/crash" route and also enriched with the error_type.

Now, as soon as I start the app with the "Production" environment, where the following line kicks in:

if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler(new ExceptionHandlerOptions() { });
 }

the http_server_request_duration_seconds_count metric is now longer emitted for the failed requests.

Expected Behavior

I dont know why ExceptionHandlerMiddlewareImpl "mutes" the http_server_request_duration_seconds_count metric for endpoints with exceptions. Maybe I'm missing something and it makes sense, but for me the behaviour doesn't make sense.

Is this possibly a bug or as designed?

Steps To Reproduce

Run the following app:

using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

builder.Services
        .AddOpenTelemetry()
        .WithMetrics(metrics =>
        {
            metrics.AddAspNetCoreInstrumentation();
            metrics.AddPrometheusExporter();
        });

builder.Services.AddProblemDetails();

var app = builder.Build();

app.UseSwagger();
app.UseSwaggerUI();
app.UseExceptionHandler(new ExceptionHandlerOptions() { });

app.UseHttpsRedirection();
app.MapPrometheusScrapingEndpoint();

app.MapGet("/crash", () =>
{
    if (true)
    {
        throw new InvalidOperationException("XY");
    }
    return "OK";
})
.WithName("Crash")
.WithOpenApi();

app.Run();
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <InvariantGlobalization>true</InvariantGlobalization>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.0" />
    <PackageReference Include="OpenTelemetry.Exporter.Prometheus.AspNetCore" Version="1.7.0-rc.1" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.6.0-rc.1" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="6.4.0" />
  </ItemGroup>

</Project>
  1. Invoke the "/crash" endpoint multiple times
  2. Open /metrics endpoint and search for http_server_request_duration_seconds_count metric

.NET Version

.NET 8

@dotnet-issue-labeler dotnet-issue-labeler bot added the area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlesware label Dec 12, 2023
@BrennanConroy
Copy link
Member

Looks like a dupe of #52648

@davidroth
Copy link
Author

Duplicate of #52648

@ghost ghost locked as resolved and limited conversation to collaborators Feb 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlesware ✔️ Resolution: Duplicate Resolved as a duplicate of another issue Status: Resolved
Projects
None yet
Development

No branches or pull requests

2 participants