Azure SignalR ASP.NET (Full Framework) running on Azure App Service Pv3 results in excessive application logs #1837
Assignees
Labels
investigating
Issue in investigating
Comments
|
I am facing the similar issue when running on Azure App Service Pv3 with Full Framework. Around 5000 event logs in one minute trace taken from the Azure App Service via Diagnostic Tools. Would love to have workaround other than dependency resolver.
|
|
Thanks @vicancy for thumbs up and looking into it. Wanted to add additional context. As you might already know this, these events are also source of major allocations. Allocations from a recent trace of slowdown - related to ETW events. I wonder how many and what sources it adds by default? After spending weeks chasing the cause of slowdowns and high memory. I think ETW Trace writes are part of the bigger problem. as I am getting help from runtime team to track down in the linked issue. Excuse me if the tagging other issue is unrelated. Using Dependency Resolver: That's why requesting for other workaround to avoid the trace writes without DR. If I set the Additional Context - Allocations and Diagnostic Settings
Then when Drilling down for string allocations. For byte[] drill down Finally, here is how AzureSignalR Diagnostic Settings are configured. I try to find the events logged in the portal but was not able to. I guess these logs are lost but I see it in the PerfView as noted above.
Automated analysis suggestion - I tried removing the default trace listener but it did not have any impact.
An example SignalR Thread - It also have a call to |
|
I suggest just working around this by overriding the logging infrastructure to log to a null logger. public class AzureSignalRDependencyResolver : IDependencyResolver
{
private readonly IDependencyResolver _resolver;
public AzureSignalRDependencyResolver(IDependencyResolver resolver) => _resolver = resolver;
public void Dispose() => _resolver.Dispose();
public object GetService(Type serviceType) => serviceType == typeof(ILoggerFactory) ? NullLoggerFactory.Instance : _resolver.GetService(serviceType);
public IEnumerable<object> GetServices(Type serviceType) => _resolver.GetServices(serviceType);
public void Register(Type serviceType, Func<object> activator) => _resolver.Register(serviceType, activator);
public void Register(Type serviceType, IEnumerable<Func<object>> activators) => _resolver.Register(serviceType, activators);
}
public class Startup
{
public void Configuration(IAppBuilder app)
{
var hubConfiguration = new HubConfiguration();
hubConfiguration.Resolver = new AzureSignalRDependencyResolver(hubConfiguration.Resolver);
app.MapAzureSignalR("", hubConfiguration);
}
} |
|
Thank you @maskati will try out the workaround. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There is an issue when using a combination of the following:
The issue results from:
ILoggerFactorythat configures an internal EventSource logger (AddEventSourceLogger)AppServiceAppLogscategory logging enabled consuming all platform ETW events and shipping them to Azure MonitorThe end result is a system that logs several million debug logs per day to Azure Monitor.
The way I see it the actual bug is a combination of bugs in both Azure SignalR for ASP.NET library and Azure App Service Windows hosting infrastructure:
Microsoft.Azure.SignalR.AspNetshould not be creating its own internal logger factory with an event source logger over which the application has no control. Applications always create their own logging configuration and specify the desired targets. I understand the compatibility reason why it is done like this when integrating from full framework withMicrosoft.Extensions.Logging, but it should be opt-in with some sort of flag, and the normal usage should result in the use ofNullLogger.Microsoft.WindowsAzure.WebSites.Diagnostics.AzureMonitorTraceListener) as well as the 100 logs/minute threshold implemented by the internalMicrosoft.WindowsAzure.WebSites.Diagnostics.AzureMonitorTraceThrottler.For now my workaround is to configure SignalR with a
HubConfigurationand customIDependencyResolverwrappingHubConfiguration.Resolverand forIDependencyResolver.GetService(Type serviceType)when serviceType =ILoggerFactoryreturningNullLoggerFactory.Instance.The text was updated successfully, but these errors were encountered: