Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sometimes when auditing to CosmosDB a CosmosException occurs #625

Closed
wouter-b opened this issue Sep 15, 2023 · 5 comments
Closed

Sometimes when auditing to CosmosDB a CosmosException occurs #625

wouter-b opened this issue Sep 15, 2023 · 5 comments

Comments

@wouter-b
Copy link

Describe the bug
Sometimes when auditing to CosmosDB a CosmosException occurs in
Audit.AzureCosmos.Providers.AzureCosmosDataProvider+d__39.MoveNext

Expected behavior
Audit exceptions should not stop the code flow

Libraries (specify the Audit.NET extensions being used including version):
For example:

  • Audit.EntityFramework.Core 21.0.3
  • Audit.NET 21.0.3
  • Audit.NET.AzureCosmos 21.0.3
  • Audit.NET.JsonNewtonsoftAdapter 21.0.3
  • Audit.WebApi.Core 21.0.3

Target .NET framework:
For example:

  • .NET 6.0

Additional context

Response status code does not indicate success: RequestTimeout (408);
Microsoft.Azure.Cosmos.CosmosException:
at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode (Microsoft.Azure.Cosmos.Client, Version=3.35.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage (Microsoft.Azure.Cosmos.Client, Version=3.35.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.Azure.Cosmos.ContainerCore+d__561.MoveNext (Microsoft.Azure.Cosmos.Client, Version=3.35.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) at Microsoft.Azure.Cosmos.ClientContextCore+<RunWithDiagnosticsHelperAsync>d__401.MoveNext (Microsoft.Azure.Cosmos.Client, Version=3.35.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Microsoft.Azure.Cosmos.ClientContextCore+d__30`1.MoveNext (Microsoft.Azure.Cosmos.Client, Version=3.35.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.AzureCosmos.Providers.AzureCosmosDataProvider+d__39.MoveNext (Audit.NET.AzureCosmos, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.Core.AuditScope+d__41.MoveNext (Audit.NET, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.Core.AuditScope+d__35.MoveNext (Audit.NET, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.EntityFramework.DbContextHelper+d__23.MoveNext (Audit.EntityFramework.Core, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.EntityFramework.DbContextHelper+d__40.MoveNext (Audit.EntityFramework.Core, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.EntityFramework.DbContextHelper+d__37.MoveNext (Audit.EntityFramework.Core, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Audit.EntityFramework.AuditDbContext+d__57.MoveNext (Audit.EntityFramework.Core, Version=21.0.3.0, Culture=neutral, PublicKeyToken=571d6b80b242c87e)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Fiberworkz.Database.FiberworkzDbContext+d__92.MoveNext (Fiberworkz.Database, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /home/runner/work/fiberworkz/fiberworkz/Fiberworkz.Database/FiberworkzDbContext.cs:75)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at ConnectedApp.ConnectedAppUpdateService+d__11.MoveNext (ConnectedApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /home/runner/work/fiberworkz/fiberworkz/ConnectedApp.Domain/ConnectedAppUpdateService.cs:135)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at ConnectedApp.ConnectedAppUpdateService+d__10.MoveNext (ConnectedApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: /home/runner/work/fiberworkz/fiberworkz/ConnectedApp.Domain/ConnectedAppUpdateService.cs:85)
@thepirat000
Copy link
Owner

Looks like you're getting a timeout from cosmosDB when trying to save the audit event.

Is your audit event too large?

Have you seen Diagnose and troubleshoot Azure Cosmos DB .NET SDK request timeout exceptions? Maybe you need to customize the timeout.

@wouter-b
Copy link
Author

You could be right about the audit event size. I'll dive into that.
Is there any way I could catch the Exception? So, I can log the exception and continue.

@thepirat000
Copy link
Owner

Also, you could implement your own data provider, by inheriting from AzureCosmosDataProvider and adding your custom try/catch mechanism in the Insert Audit Event logic.

For example:

public class AzureCosmosCustomProvider : AzureCosmosDataProvider
{
    public AzureCosmosCustomProvider() : base() { }
    public AzureCosmosCustomProvider(Action<IAzureCosmosProviderConfigurator> config) : base(config) { }

    public override object InsertEvent(AuditEvent auditEvent)
    {
        try
        {
            return base.InsertEvent(auditEvent);
        }
        catch (CosmosException ex) when (ex.StatusCode == HttpStatusCode.RequestTimeout)
        {
            // timed out
            return null;
        }
    }

    public override async Task<object> InsertEventAsync(AuditEvent auditEvent, CancellationToken cancellationToken = default)
    {
        try
        {
            return await base.InsertEventAsync(auditEvent, cancellationToken);
        }
        catch (CosmosException ex) when (ex.StatusCode == HttpStatusCode.RequestTimeout)
        {
            // timed out
            return null;
        }
    }
}

Then you can set up the data provider like this:

Audit.Core.Configuration.Setup()
    .Use(new AzureCosmosCustomProvider(_ => _
        .Endpoint(...)
        .Database(...)
        .Container(...)
        .AuthKey(...)));

@thepirat000
Copy link
Owner

If you're looking for a retry mechanism. check this: #372 (comment)

@wouter-b
Copy link
Author

A big thanks, your suggestions will solve my issue 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@thepirat000 @wouter-b