Skip to content

NLog cloud logging with Azure function or AWS lambda

Jump to bottom Edit New page
Rolf Kristensen edited this page Nov 23, 2024 · 38 revisions

Logging in the cloud

Standard log files doesn't always work well in the cloud. It can be difficult to access the log files after the function / lambda has completed.

The easy solution is to perform logging to these locations:

Then update the configuration for the function / lambda to perform capture of the output and save it to cloud storage. The execution of the function / lambda will then not be bothered with any network issues to the cloud storage.

The NLog JsonLayout can then be configured to enable structured logging.

See also: NLog Console vs. AddConsole

Optimize logging performance

The cloud hosting environment will try and reuse the same initialized cloud function instance for multiple calls. This allows one to optimize the function by keeping initialization code into the constructor or with static class variables.

Things that can benefit from being shared across multiple calls:

  • Logging Framework
  • Database connection pool

Disable auto loading of NLog extension for NLog v4

NLog v4 can be slow to initialize because it scans for assemblies that might be relevant for loading NLog.config. The startup overhead (5-10 ms) from the scanning is not an issue for normal service-application. But for cloud functions then it is a long time to wait.

One can disable the scanning with this line, before having created the first NLog Logger-object:

NLog.Config.ConfigurationItemFactory.Default = new 
   NLog.Config.ConfigurationItemFactory(typeof(NLog.ILogger).GetTypeInfo().Assembly);

NLog v5 no longer automatically scans for NLog extensions assemblies for faster startup by default.

Writing to ILambdaContext ILambdaLogger in AWS

var lambdaLogger = lambaContext.Logger;
var targetLayout = new NLog.Layouts.SimpleLayout("${message}${exception:format=tostring}");  // Can also be JsonLayout
var loggerTarget = new NLog.Targets.MethodCallTarget("MyTarget", (logEvent,parms) => lambdaLogger.LogLine(parms[0].ToString()));
loggerTarget.Parameters.Add(new NLog.Targets.MethodCallParameter(targetLayout);
var nlogConfig = new NLog.Config.LoggingConfiguration();
nlogConfig.AddRuleForAllLevels(loggerTarget);
NLog.LogManager.Configuration = nlogConfig;
var nlogLogger = NLog.LogManager.GetCurrentClassLogger();
nlogLogger.Info("Hello World");

NLog v5 allows you to use the Fluent Configuration API

var lambdaLogger = lambaContext.Logger;
NLog.LogManager.Setup().LoadConfiguration(cfg => {
    var targetLayout = new NLog.Layouts.SimpleLayout("${message}${exception:format=tostring}");  // Can also be JsonLayout
    var loggerTarget = new NLog.Targets.MethodCallTarget("MyTarget", (logEvent,parms) => lambdaLogger.LogLine(parms[0].ToString()));
    loggerTarget.Parameters.Add(new NLog.Targets.MethodCallParameter(targetLayout);
    cfg.ForLogger().FilterMinLevel(LogLevel.Info).WriteTo(loggerTarget);
});
var nlogLogger = NLog.LogManager.GetCurrentClassLogger();
nlogLogger.Info("Hello World");

Configure NLog as logging provider in Azure

Register NLog using FunctionsStartup in Startup.cs, and ensure correct NLog shutdown:

[assembly: FunctionsStartup(typeof(MyNamespace.Startup))]

namespace MyNamespace
{
    public class Startup : FunctionsStartup
    {
        private readonly NLog.Logger Logger;

        public Startup()
        {
            Logger = LogManager.Setup()
               .SetupExtensions(e => e.AutoLoadAssemblies(false))
               .LoadConfigurationFromFile("nlog.config", optional: false)
               .LoadConfiguration(builder => builder.LogFactory.AutoShutdown = false)
               .GetCurrentClassLogger();
        }

        public override void Configure(IFunctionsHostBuilder builder)
        {
            // ... other setup code
            builder.Services.AddLogging((loggingBuilder) =>
            {
                // loggingBuilder.SetMinimumLevel(LogLevel.Trace); // Update default MEL-filter
                loggingBuilder.AddNLog(new NLogProviderOptions() { ShutdownOnDispose = true });
            });
        }
    }
}

Setup filtering for Microsoft Extension Logging (MEL) in host.json:

{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "MyNamespace": "Information"
    }
  }
}

See also Use dependency injection in .NET Azure Functions

Initialize NLog in Azure Functions v2

There is an optimization in Azure Functions v2 so it will allow reuse of static variables. This means first thread will initialize, and all new threads will reuse.

When using NLog directly (Without [FunctionsStartup] as shown above), then it is recommended to make use of static NLog.Logger like this:

namespace MyServiceBusFunctions
{
   static readonly NLog.Logger Log = NLog.LogManager.GetLogger("MyFunction");

   [FunctionName( "MyFunction" )]
   public static void Run( [ServiceBusTrigger( "TestTopicName", "TestSubscriptionName", Connection = "TestSubscriptionConnectionString" )] string messageContents, ExecutionContext context, int deliveryCount, DateTime enqueuedTimeUtc, string messageId )
   {
       Log.Info("Hello World");
   }
}

Writing to Azure Diagnostics Log Stream

The Azure Diagnostics Log Stream ships any information written to files ending in .txt, .log, or .htm that are stored in the /home/LogFiles directory (d:/home/logfiles).

It is just a matter of pointing the NLog-FileTarget to the folder d:/home/logfiles/application, and the AppService will automatically stream it.

<nlog>
     <targets async="true">
         <!-- Environment Variable %HOME% matches D:/Home -->
         <target type="file" name="appfile" filename="${environment:HOME:cached=true}/logfiles/application/app-${shortdate}-${machinename}-${processid}.txt" />
     </targets>
     <rules>
         <logger name="*" minLevel="Debug" writeTo="appFile" />
     </rules>
</nlog>

Notice that Function-Execution-Logs will not be supported for Azure Function App / Azure Web App. Instead one should consider ApplicationInsights.

Redirect NLog to Microsoft Extension ILogger in Azure

If you have a library that depends on NLog-Logging, and not interested in using NLog as Logging Provider. Then you can setup NLog to forward output to a standard Microsoft ILogger using MicrosoftILoggerTarget.

MicrosoftILoggerTarget can help:

var loggerTarget = new NLog.Extensions.Logging.MicrosoftILoggerTarget(azureILogger);
loggerTarget.Layout = new NLog.Layouts.SimpleLayout("${message}${exception:format=tostring}");  // Can also be JsonLayout
var nlogConfig = new NLog.Config.LoggingConfiguration();
nlogConfig.AddRuleForAllLevels(loggerTarget);
NLog.LogManager.Configuration = nlogConfig;
var nlogLogger = NLog.LogManager.GetCurrentClassLogger();
nlogLogger.Info("Hello World");

NLog v5 allows you to use the Fluent Configuration API

NLog.LogManager.Setup().LoadConfiguration(cfg => {
    var loggerTarget = new NLog.Extensions.Logging.MicrosoftILoggerTarget(azureILogger);
    loggerTarget.Layout = new NLog.Layouts.SimpleLayout("${message}${exception:format=tostring}");  // Can also be JsonLayout
    cfg.ForLogger().FilterMinLevel(LogLevel.Info).WriteTo(loggerTarget);
});
var nlogLogger = NLog.LogManager.GetCurrentClassLogger();
nlogLogger.Info("Hello World");

This will redirect the output of NLog into the Azure ILogger, so it will be stored where the Azure Function is configure to store its logging.

Writing directly to cloud storage

It is possible for the lambda / function to write directly to the cloud storage, but it might be more fragile. The cloud usually has high latency so the execution might be affected by timeouts/retries etc. Make sure to use the matching region of the cloud storage. Very important to flush before exit.

See the different NLog Integrations (Cloud)

Add a custom footer
Add a custom sidebar
Clone this wiki locally