Crashes when loading large (32k) image.

[piranha771]

Prerequisites

• I have written a descriptive issue title
• I have verified that I am running the latest version of ImageSharp
• I have verified if the problem exist in both DEBUG and RELEASE mode
• I have searched open and closed issues to ensure it has not already been reported

Description

When a image is very large this exception gets thrown:

System.ArgumentOutOfRangeException: 'Value -8386560 must be greater than or equal to 0.
Parameter name: length'

Steps to Reproduce

Load an image with 32786x16384 pixels

System Configuration

• ImageSharp version: beta 7

• Other ImageSharp packages and versions: core beta 8

• Environment (Operating system, version and so on):
  Win10 1903
  48GB RAM
  i7-6700k

• .NET Framework version:
  4.7.2

[supcry]

I have exactly the same problem but in another environment.

GNU/Linux, 32GB RAM, .Net Core v3.0.

Exception=System.ArgumentOutOfRangeException: Value -11595144 must be greater than or equal to 0. Parameter name: length at SixLabors.Guard.ThrowArgumentOutOfRangeException(String parameterName, String message) at SixLabors.Guard.MustBeGreaterThanOrEqualTo[TValue](TValue value, TValue min, String parameterName) at SixLabors.Memory.ArrayPoolMemoryAllocator.Allocate[T](Int32 length, AllocationOptions options) at SixLabors.ImageSharp.Memory.MemoryAllocatorExtensions.Allocate2D[T](MemoryAllocator memoryAllocator, Int32 width, Int32 height, AllocationOptions options) at SixLabors.ImageSharp.Formats.Jpeg.Components.Decoder.JpegComponent.Init() at SixLabors.ImageSharp.Formats.Jpeg.Components.Decoder.JpegFrame.InitComponents() at SixLabors.ImageSharp.Formats.Jpeg.JpegDecoderCore.ProcessStartOfFrameMarker(Int32 remaining, JpegFileMarker& frameMarker, Boolean metadataOnly) at SixLabors.ImageSharp.Formats.Jpeg.JpegDecoderCore.ParseStream(Stream stream, Boolean metadataOnly) at SixLabors.ImageSharp.Formats.Jpeg.JpegDecoderCore.Decode[TPixel](Stream stream) at SixLabors.ImageSharp.Formats.Jpeg.JpegDecoder.Decode[TPixel](Configuration configuration, Stream stream) at SixLabors.ImageSharp.Formats.Jpeg.JpegDecoder.Decode(Configuration configuration, Stream stream)

[supcry]

#805

[antonfirsov]

It's a duplicate issue indeed, therefore it will be closed so we can keep our issue set clean. But first, a question!
@piranha771 @supcry it would be a very useful information for us if you could tell a bit more about your use-cases. What problem does your app/code try to solve? Is it desktop or server? Thanks!

[supcry]

I have big continuous pictures in jpeg-format from customer with typical sizes 500-1000 MB and dimensions 30000x60000px. My task is to process them (read jpeg, random read pixels, resize, get random rect, etc) for machine learning. Yes, I can split them manually in blocks by GIMP and then process through pipe of batches, but it would be great to do this through .net-code.

I've tried work with such big pictures through several .net-libraries:

• Accord.Imaging,
• Aspose.Imaging,
• Quamotion.TurboJpegWrapper,
• GrapeCity.Documents.Imaging,
• and SixLabors.ImageSharp,
• and others,

but none of them can process big pictures on GNU/Linux, faulting in memory allocation or even arithmetic overflow.

Also, I can't use System.Drowing, because it hasn't Linux-support.

[dlemstra]

Have you tried Magick.NET? That library will swap to disk when necessary and should work in your situation.

[supcry]

It works! Thanks!

[kleisauke]

@supcry Alternatively, you can try NetVips, the C# binding for libvips. This is a streaming image processing library, so rather than manipulating huge objects in memory, it builds pipelines and then runs them in parallel, streaming images in a series of small regions.

It has an fetch feature that is useful for machine learning, see:
libvips/pyvips#100 (comment)

[JimBobSquarePants]

Let's please focus on removing constraints within this library.

[JimBobSquarePants]

Closing as a duplicate of #805