AWS Lambda in .net 6.0 create some files in temp storage when it fails with "Exceeded maximum allowed payload size" exception #1540(short issue description)

[asn1997]

Describe the bug

AWS Lambda in .net 6.0 create some files in temp storage when it fails with "Exceeded maximum allowed payload size" exception.
It create folowing file "/tmp/core.dotnet.8" and it will not be deleted and this impact the next execution in same Lambda Instance.

We use NewtonsoftJson as Input and Output formator so it use disk space internally when the data is more than 30KB.

So if we invoke Lambda after this error "Exceeded maximum allowed payload size" and the request goes to same instance and if the response is more than 30 KB it tries to create a temp file in /tmp folder then it throws this error

"ClassName":"System.IO.IOException","Message":"No space left on device : '/tmp/ASPNETCORE_5a294445-5cfd-4db9-8049-d2ef44559710.tmp'","Data":{},"InnerException":null,"HelpURL":null,"StackTraceString":" at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan1 buffer, Int64 fileOffset)\n at System.IO.Strategies.OSFileStreamStrategy.Write(Byte[] buffer, Int32 offset, Int32 count)\n at Microsoft.AspNetCore.WebUtilities.PagedByteBuffer.MoveTo(Stream stream)\n at Microsoft.AspNetCore.WebUtilities.FileBufferingWriteStream.Write(Byte[] buffer, Int32 offset, Int32 count)\n at Microsoft.AspNetCore.WebUtilities.HttpResponseStreamWriter.Write(String value)\n at Newtonsoft.Json.Utilities.JavaScriptUtils.WriteEscapedJavaScriptString(TextWriter writer, String s, Char delimiter, Boolean appendDelimiters, Boolean[] charEscapeFlags, StringEscapeHandling stringEscapeHandling, IArrayPool1 bufferPool, Char[]& writeBuffer)\n at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name, Boolean escape)\n at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)\n at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)\n at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)\n at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)\n at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)\n at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)\n at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)\n at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()\n--- End of stack trace from previous location ---\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)\n at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)\n at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":28,"Source":"System.Private.CoreLib","WatsonBuckets":null

this happens because disk space is full because of this file "/tmp/core.dotnet.8"

Expected Behavior

Temp files like "/tmp/core.dotnet.8" which got created after this exception "Exceeded maximum allowed payload size" should be deleted

Current Behavior

Temp files like "/tmp/core.dotnet.8" which got created after this exception "Exceeded maximum allowed payload size" are not deleted

Reproduction Steps

Invoke the Lambda to return response which is greater than 6MB 3 to 4 times in a way that all request should go to the same instance of the Lambda, Then invoke the lambda with request or request more than 30 KB this error will occor

Possible Solution

Temp files like "/tmp/core.dotnet.8" which got created after this exception "Exceeded maximum allowed payload size" should be deleted

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.Core 3.7.107.8

Targeted .NET Platform

.NET 6

Operating System and version

AWS Lambda AmazonLinux x86_64

[bhoradc]

Hi @asn1997,

Thank you for notifying us. I have reopened the previous issue associated with this. We'll take a look at it and get back to you soon.

I will go ahead and close this out in favor of #1540.

Regards,
Chaitanya

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.