ImageMagick_jll v6.9.12-0 is breaking for PNG and TIFF

[johnnychen94]

julia> versioninfo()
Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  CPU: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 8

using ImageMagick_jll v6.9.12-0 (JuliaPackaging/Yggdrasil#3424):

julia> using ImageCore, ImageMagick

julia> img = Gray{N0f8}.(reshape(collect(0.0:0.01:0.49), 5, 10))
5×10 Array{Gray{N0f8},2} with eltype Gray{N0f8}:
 Gray{N0f8}(0.0)    Gray{N0f8}(0.051)  Gray{N0f8}(0.102)  Gray{N0f8}(0.149)  Gray{N0f8}(0.2)    Gray{N0f8}(0.251)  Gray{N0f8}(0.298)  Gray{N0f8}(0.349)  Gray{N0f8}(0.4)    Gray{N0f8}(0.451)
 Gray{N0f8}(0.012)  Gray{N0f8}(0.059)  Gray{N0f8}(0.11)   Gray{N0f8}(0.161)  Gray{N0f8}(0.212)  Gray{N0f8}(0.259)  Gray{N0f8}(0.31)   Gray{N0f8}(0.361)  Gray{N0f8}(0.412)  Gray{N0f8}(0.459)
 Gray{N0f8}(0.02)   Gray{N0f8}(0.071)  Gray{N0f8}(0.122)  Gray{N0f8}(0.169)  Gray{N0f8}(0.22)   Gray{N0f8}(0.271)  Gray{N0f8}(0.322)  Gray{N0f8}(0.369)  Gray{N0f8}(0.42)   Gray{N0f8}(0.471)
 Gray{N0f8}(0.031)  Gray{N0f8}(0.078)  Gray{N0f8}(0.129)  Gray{N0f8}(0.18)   Gray{N0f8}(0.231)  Gray{N0f8}(0.278)  Gray{N0f8}(0.329)  Gray{N0f8}(0.38)   Gray{N0f8}(0.431)  Gray{N0f8}(0.478)
 Gray{N0f8}(0.039)  Gray{N0f8}(0.09)   Gray{N0f8}(0.141)  Gray{N0f8}(0.188)  Gray{N0f8}(0.239)  Gray{N0f8}(0.29)   Gray{N0f8}(0.341)  Gray{N0f8}(0.388)  Gray{N0f8}(0.439)  Gray{N0f8}(0.49)

julia> ImageMagick.save("tmp.png", img)

julia> ImageMagick.load("tmp.png")
5×10 Array{Gray{N0f8},2} with eltype Gray{N0f8}:
 Gray{N0f8}(0.0)    Gray{N0f8}(0.051)  Gray{N0f8}(0.122)  Gray{N0f8}(0.129)  Gray{N0f8}(0.2)   Gray{N0f8}(0.259)  Gray{N0f8}(0.298)  Gray{N0f8}(0.349)  Gray{N0f8}(0.38)   Gray{N0f8}(0.439)
 Gray{N0f8}(0.0)    Gray{N0f8}(0.051)  Gray{N0f8}(0.122)  Gray{N0f8}(0.18)   Gray{N0f8}(0.2)   Gray{N0f8}(0.259)  Gray{N0f8}(0.298)  Gray{N0f8}(0.349)  Gray{N0f8}(0.431)  Gray{N0f8}(0.439)
 Gray{N0f8}(0.0)    Gray{N0f8}(0.078)  Gray{N0f8}(0.122)  Gray{N0f8}(0.18)   Gray{N0f8}(0.22)  Gray{N0f8}(0.259)  Gray{N0f8}(0.322)  Gray{N0f8}(0.349)  Gray{N0f8}(0.431)  Gray{N0f8}(0.49)
 Gray{N0f8}(0.051)  Gray{N0f8}(0.078)  Gray{N0f8}(0.129)  Gray{N0f8}(0.18)   Gray{N0f8}(0.22)  Gray{N0f8}(0.259)  Gray{N0f8}(0.322)  Gray{N0f8}(0.38)   Gray{N0f8}(0.431)  Gray{N0f8}(0.49)
 Gray{N0f8}(0.051)  Gray{N0f8}(0.122)  Gray{N0f8}(0.129)  Gray{N0f8}(0.2)    Gray{N0f8}(0.22)  Gray{N0f8}(0.298)  Gray{N0f8}(0.322)  Gray{N0f8}(0.38)   Gray{N0f8}(0.439)  Gray{N0f8}(0.49)

julia> ImageMagick.load("tmp.png") == img
false

julia> using PNGFiles # v0.3.9

julia> img == open("tmp.png") do io
           PNGFiles.load(io)
       end
true

If reverted to ImageMagick_jll@6.9.10-12+3, it is:

julia> img = Gray{N0f8}.(reshape(collect(0.0:0.01:0.49), 5, 10));

julia> ImageMagick.save("tmp.png", img)

julia> ImageMagick.load("tmp.png") == img
true

cc: @staticfloat @timholy

[johnnychen94]

It turns out this version also breaks TIFF loading:

julia> using ImageMagick, TestImages

julia> filepath = testimage("camera", download_only=true)
"/Users/jc/.julia/artifacts/e752bdc739f34d02e79c7fa834bc2f2e0d71c7e0/cameraman.tif"

julia> ImageMagick.load(filepath)
ERROR: /Users/jc/.julia/artifacts/e752bdc739f34d02e79c7fa834bc2f2e0d71c7e0/cameraman.tif: Null count for "Tag 34022" (type 1, writecount -3, passcount 1). `_TIFFVSetField' @ error/tiff.c/TIFFErrors/540
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] error(wand::MagickWand)
   @ ImageMagick ~/.julia/packages/ImageMagick/b8swT/src/libmagickwand.jl:187
 [3] readimage
   @ ~/.julia/packages/ImageMagick/b8swT/src/libmagickwand.jl:274 [inlined]
 [4] load_(file::String, permute_horizontal::Bool; ImageType::Type, extraprop::String, extrapropertynames::Nothing, view::Bool)
   @ ImageMagick ~/.julia/packages/ImageMagick/b8swT/src/ImageMagick.jl:146
 [5] load_ (repeats 2 times)
   @ ~/.julia/packages/ImageMagick/b8swT/src/ImageMagick.jl:139 [inlined]
 [6] #load#30
   @ ~/.julia/packages/ImageMagick/b8swT/src/ImageMagick.jl:126 [inlined]
 [7] load(::String)
   @ ImageMagick ~/.julia/packages/ImageMagick/b8swT/src/ImageMagick.jl:126
 [8] top-level scope
   @ REPL[8]:1

@staticfloat if possible, could you help investigate this?

[simeonschaub]

FYI: bisected the test failures to ImageMagick/ImageMagick6@a1a71cb

Edit: TIFF loading is a separate issue unfortunately, so simply reverting that commit is not enough

[ViralBShah]

The PNG stuff here seems to work correctly now with ImageMagick 6.9.12. The TIFF stuff still has the same error. There's also 6.9.13 upstream, which perhaps fixes the TIFF issue. Is that worth trying?

On M1, the bigger issue is that 6.9.10 is too old and doesn't work at all.

[giordano]

There's also 6.9.13 upstream, which perhaps fixes the TIFF issue. Is that worth trying?

I simply tried

diff --git a/Project.toml b/Project.toml
index 9e47aa8..fb6cacf 100644
--- a/Project.toml
+++ b/Project.toml
@@ -11,7 +11,7 @@ InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
 [compat]
 FileIO = "1"
 ImageCore = "0.9, 0.10"
-ImageMagick_jll = "= 6.9.11"
+ImageMagick_jll = "= 6.9.13"
 julia = "1.6"
 
 [extras]

but I still get

issue #206: Error During Test at /Users/mose/.julia/dev/ImageMagick/test/constructed_images.jl:359
  Got exception outside of a @test
  /Users/mose/.julia/artifacts/0fca6f56665d8dfa11bad924ab67e24e6fdbc0d7/cameraman.tif: Null count for "Tag 34022" (type 1, writecount -3, passcount 1). `_TIFFVSetField' @ error/tiff.c/TIFFErrors/542

For the record, our ImageMagick_jll@6.9.13 currently corresponds to upstream version 6.9.13-6: https://github.com/JuliaPackaging/Yggdrasil/blob/24010c93241559f44b967496e65782a9d18729f0/I/ImageMagick/ImageMagick%406/build_tarballs.jl#L5. I guess someone will have to report the issue upstream with a C reproducer, otherwise no one will look into this.

[giordano]

Actually, this seems to be ImageMagick/ImageMagick#2015, which is apparently a bug in libtiff, still open: https://gitlab.com/libtiff/libtiff/-/issues/110. ImageMagick/ImageMagick#2015 (comment) suggests

I used the workaround of first running the images through tiff2rgba (which strips those unknown tags, apparently), and then running convert on them.

Maybe we can do something similar? What's tiff2rgba?

[mrAceT]

I have the same problem.

I decided to upgrade ImageMagick 6.9.12-93 Q16 x86_64 17898 to ImageMagick 6.9.13-14 Q16 x86_64 18442

But that had no effect.

On an old test server I have ImageMagick 6.8.9-9 Q16 x86_64 which is working perfectly.. quite frustrating!

[ViralBShah]

Is it possible that PNGFiles.jl is doing something that is non-standard or buggy that is leading to this issue?

For example: JuliaIO/PNGFiles.jl#48

[dgleich]

Okay, there are two different issues here. The start of the issue was the PNG file breaking. Then it also moved to include the TIFF file breaking.

On the new main version that I updated to ImageMagick 7.1, it fixes the PNG file issue. But leaves the TIFF file issue.

[dgleich]

Notes on the tiff issue:

https://imagemagick.org/discourse-server/viewtopic.php?t=36263

ImageMagick/ImageMagick#7185

https://gitlab.com/libtiff/libtiff/-/issues/532

https://stackoverflow.com/questions/64239390/imagemagick-identify-fails-for-tiff-files

So this seems to be a recurring issues for ImageMagick and libtiff.

My plan at the moment is to make a list of these tags and just setup the ImageMagick reader to ignore them. This should broadly work. They aren't 'file supported'

[ViralBShah]

Addressed in 1411a8a

[ViralBShah]

Closing since we are now on ImageMagick 7 on master.