CryptoStream calling flush when only reads have been performed #61325
Comments
dotnet-issue-labeler
bot
added
area-System.IO
untriaged
|
Tagging subscribers to this area: @dotnet/area-system-io Issue DetailsDescriptionDeserializing JSON from DeflateStream/ZLibStream with inner stream of CryptoStream seems to cause the CryptoStream to call Flush on its inner stream while disposing. If the inner stream does not permit writes (we are only supposed to be reading from it!), it can lead to a exception to be thrown. This is most likely related to the breaking changed around partial and zero-byte reads in DeflateStream, GZipStream, and CryptoStream. Reproduction Stepsusing System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<T> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
return await JsonSerializer.DeserializeAsync<T>(deflate);
}
MemoryStream json = await Serialize("Hello World!");
string output = await Deserialize<string>(new WrapperStream(json));
Console.WriteLine(output);Expected behaviorThe code should run just fine and not call Flush. Actual behaviorThe code calls Flush and throws NotSupportedException. Regression?Regression from .NET 5. Known WorkaroundsNo response ConfigurationRunning on .NET 6. Other informationNo response
|
|
Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks Issue DetailsDescriptionDeserializing JSON from DeflateStream/ZLibStream with inner stream of CryptoStream seems to cause the CryptoStream to call Flush on its inner stream while disposing. If the inner stream does not permit writes (we are only supposed to be reading from it!), it can lead to a exception to be thrown. This is most likely related to the breaking changed around partial and zero-byte reads in DeflateStream, GZipStream, and CryptoStream. Reproduction Stepsusing System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<T> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
return await JsonSerializer.DeserializeAsync<T>(deflate);
}
MemoryStream json = await Serialize("Hello World!");
string output = await Deserialize<string>(new WrapperStream(json));
Console.WriteLine(output);
public sealed class WrapperStream : MemoryStream
{
public WrapperStream(MemoryStream stream) : base(stream.ToArray())
{
}
public override void Flush() => throw new NotSupportedException();
public override Task FlushAsync(CancellationToken cancellationToken) => throw new NotSupportedException();
}Expected behaviorThe code should run just fine and not call Flush. Actual behaviorThe code calls Flush and throws NotSupportedException. Regression?Regression from .NET 5. Known WorkaroundsNo response ConfigurationRunning on .NET 6. Other informationNo response
|
|
Why is flush being called at all? Flushing is an expensive operation that is not automatically required. Normally, flushing should be initiated by application code. Only the application knows how to correctly orchestrate stream flushing. |
|
Stream.Flush{Async} is expected to nop even when a stream has been used only for reads. From the docs: This also isn't a regression in CryptoStream or DeflateStream. You can see, for example, this throws the same NotSupportedException on both .NET 5 and .NET Framework 4.8: using System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Threading;
using System.Threading.Tasks;
internal class Program
{
static void Main()
{
using (var base64Transformer = new FromBase64Transform())
using (var cryptoStream = new CryptoStream(new WrapperStream(new MemoryStream()), base64Transformer, CryptoStreamMode.Read, leaveOpen: true))
using (var deflate = new DeflateStream(cryptoStream, CompressionMode.Decompress))
{
}
}
public sealed class WrapperStream : MemoryStream
{
public WrapperStream(MemoryStream stream) : base(stream.ToArray()) { }
public override void Flush() => throw new NotSupportedException();
public override Task FlushAsync(CancellationToken cancellationToken) => throw new NotSupportedException();
}
}I believe what's different here in your example between .NET 5 and .NET 6 is that in .NET 5 JsonSerializer.DeserializeAsync was reading until EOF whereas in .NET 6 it doesn't appear to be doing so. When it reads until EOF, that causes CryptoStream.Dispose to not follow the code path that flushes, but if it hasn't read until EOF, CryptoStream.Dispose will (and always has) flush. You can see that by running this tweak to your repro... this outputs True on .NET 5 but False on .NET 6: using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<T> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
using DeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
return await JsonSerializer.DeserializeAsync<T>(deflate);
}
var s = new TrackingStream(await Serialize("Hello World!"));
await Deserialize<string>(s);
Console.WriteLine(s.ReadToEof);
class TrackingStream : MemoryStream
{
public bool ReadToEof;
public TrackingStream(MemoryStream stream) : base(stream.ToArray()) { }
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
{
int r = await base.ReadAsync(buffer, cancellationToken);
if (r == 0) ReadToEof = true;
return r;
}
}cc: @dotnet/area-system-text-json for comment on whether that change in reading is by design |
|
Tagging subscribers to this area: @dotnet/area-system-text-json Issue DetailsDescriptionDeserializing JSON from DeflateStream/ZLibStream with inner stream of CryptoStream seems to cause the CryptoStream to call Flush on its inner stream while disposing. If the inner stream does not permit writes (we are only supposed to be reading from it!), it can lead to a exception to be thrown. This is most likely related to the breaking changed around partial and zero-byte reads in DeflateStream, GZipStream, and CryptoStream. Reproduction Stepsusing System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<T> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
return await JsonSerializer.DeserializeAsync<T>(deflate);
}
MemoryStream json = await Serialize("Hello World!");
string output = await Deserialize<string>(new WrapperStream(json));
Console.WriteLine(output);
public sealed class WrapperStream : MemoryStream
{
public WrapperStream(MemoryStream stream) : base(stream.ToArray())
{
}
public override void Flush() => throw new NotSupportedException();
public override Task FlushAsync(CancellationToken cancellationToken) => throw new NotSupportedException();
}Expected behaviorThe code should run just fine and not call Flush. Actual behaviorThe code calls Flush and throws NotSupportedException. Regression?Regression from .NET 5. Known WorkaroundsNo response ConfigurationRunning on .NET 6. Other informationNo response
|
|
@stephentoub I ran your reproduction on the debugger, and it seems that it's an issue with Here's a reproduction demonstrating that this is the case: using System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
var memoryStream = new InstrumentedMemoryStream(await Serialize("x"));
memoryStream.Position = 0;
(InstrumentedDeflateStream deflateStream, _) = await Deserialize<string>(memoryStream);
Console.WriteLine($"{memoryStream.ReadToEof}, {deflateStream.ReadToEof}"); // False, True
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<(InstrumentedDeflateStream, T?)> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
using InstrumentedDeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
var result = await JsonSerializer.DeserializeAsync<T>(deflate);
return (deflate, result);
}
class InstrumentedMemoryStream : MemoryStream
{
public bool ReadToEof;
public InstrumentedMemoryStream(MemoryStream stream) : base(stream.ToArray()) { }
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
{
int r = await base.ReadAsync(buffer, cancellationToken);
if (r == 0) ReadToEof = true;
return r;
}
}
class InstrumentedDeflateStream : DeflateStream
{
public bool ReadToEof;
public InstrumentedDeflateStream(Stream stream, CompressionMode mode) : base(stream, mode) { }
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
{
int r = await base.ReadAsync(buffer, cancellationToken);
if (r == 0) ReadToEof = true;
return r;
}
} |
|
Tagging subscribers to this area: @dotnet/area-system-io-compression Issue DetailsDescriptionDeserializing JSON from DeflateStream/ZLibStream with inner stream of CryptoStream seems to cause the CryptoStream to call Flush on its inner stream while disposing. If the inner stream does not permit writes (we are only supposed to be reading from it!), it can lead to a exception to be thrown. This is most likely related to the breaking changed around partial and zero-byte reads in DeflateStream, GZipStream, and CryptoStream. Reproduction Stepsusing System;
using System.IO;
using System.IO.Compression;
using System.Security.Cryptography;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
static async Task<MemoryStream> Serialize<T>(T test)
{
MemoryStream stream = new();
using ToBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Write, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Compress);
await JsonSerializer.SerializeAsync(deflate, test);
return stream;
}
static async Task<T> Deserialize<T>(Stream stream)
{
using FromBase64Transform base64Transformer = new();
await using CryptoStream cryptoStream = new(stream, base64Transformer, CryptoStreamMode.Read, leaveOpen: true);
await using DeflateStream deflate = new(cryptoStream, CompressionMode.Decompress);
return await JsonSerializer.DeserializeAsync<T>(deflate);
}
MemoryStream json = await Serialize("Hello World!");
string output = await Deserialize<string>(new WrapperStream(json));
Console.WriteLine(output);
public sealed class WrapperStream : MemoryStream
{
public WrapperStream(MemoryStream stream) : base(stream.ToArray())
{
}
public override void Flush() => throw new NotSupportedException();
public override Task FlushAsync(CancellationToken cancellationToken) => throw new NotSupportedException();
}Expected behaviorThe code should run just fine and not call Flush. Actual behaviorThe code calls Flush and throws NotSupportedException. Regression?Regression from .NET 5. Known WorkaroundsNo response ConfigurationRunning on .NET 6. Other informationNo response
|
Description
Deserializing JSON from DeflateStream/ZLibStream with inner stream of CryptoStream seems to cause the CryptoStream to call Flush on its inner stream while disposing. If the inner stream does not permit writes (we are only supposed to be reading from it!), it can lead to a exception to be thrown.
This is most likely related to the breaking changed around partial and zero-byte reads in DeflateStream, GZipStream, and CryptoStream.
Reproduction Steps
Expected behavior
The code should run just fine and not call Flush.
Actual behavior
The code calls Flush and throws NotSupportedException.
Regression?
Regression from .NET 5.
Known Workarounds
No response
Configuration
Running on .NET 6.
Other information
No response
The text was updated successfully, but these errors were encountered: