HDR color mapping is way off compared to QuickTime

[sarahkadar]

I really, badly want this to work but the HDR mapping is just... not great. Video is way too over-contrasty to the point of unwatchable. Attached is an example, the first is IINA, the second is QuickTime:

[sarahkadar]

This is a dark scene but I chose it intentionally because it shows the color difference as well as missing a lot of subtleties in the black part.

[CarterLi]

I can't test it without the video source.

Please try this older version to see if it works better: https://github.com/iina-plus/iina/releases/tag/v1.2.0-13 also

[sarahkadar]

@CarterLi is there a way I can contact you in private? I can send you the video source that way

I tried this with several versions of IINA already

[CarterLi]

No. Cut it into 3 secs and upload it somewhere

[xose]

Possibly related: Trying to play the HDR test pattern from https://www.youtube.com/watch?v=NlAsAuBtmps

Attaching it here reencoded as HEVC so it can also be played in Quicktime: https://user-images.githubusercontent.com/662304/146276378-e7a3777a-4edd-4261-b8b8-04717c202c43.mov

The video shows rectangles of different sizes and cycles though multiple brightness levels. Both Youtube and Quicktime correctly display all the brightness level changes. IINA however displays most brightness levels at 100% brightness. The description text is also brighter in IINA.

ffmpeg log with video details

Input #0, matroska,webm, from 'Real 4K HDR Test Pattern - Brightness stress test (Chromecast Ultra) [NlAsAuBtmps].webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:03:03.00, start: 0.000000, bitrate: 166 kb/s
  Stream #0:0(eng): Video: vp9 (Profile 2), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
    Side data:
      Content Light Level Metadata, MaxCLL=4000, MaxFALL=300
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2649,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.009900, max_luminance=4000.000000
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> hevc (libx265))
Press [q] to stop, [?] for help
x265 [info]: HEVC encoder version 3.4+31-6722fce1f
x265 [info]: build info [Mac OS X][clang 13.0.0][32 bit][noasm] 10bit
x265 [info]: using cpu capabilities: none!
x265 [info]: Main 10 profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 10 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 3 / wpp(34 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing lslices=8 deblock sao
Output #0, mov, to 'hdr_test2.mov':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0(eng): Video: hevc (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 12288 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 libx265
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
      Content Light Level Metadata, MaxCLL=4000, MaxFALL=300
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2649,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.009900, max_luminance=4000.000000
frame= 4392 fps=8.2 q=36.0 Lsize=    2111kB time=00:03:02.87 bitrate=  94.6kbits/s speed=0.34x
video:2058kB audio:0kB subtitle:0kB other streams:0kB global headers:2kB muxing overhead: 2.598028%
x265 [info]: frame I:     21, Avg QP:30.68  kb/s: 2058.51
x265 [info]: frame P:   1554, Avg QP:32.97  kb/s: 114.99
x265 [info]: frame B:   2817, Avg QP:35.39  kb/s: 63.63
x265 [info]: Weighted P-Frames: Y:1.1% UV:0.0%
x265 [info]: consecutive B-frames: 38.0% 7.7% 22.4% 1.2% 30.7%

[CarterLi]

Possibly related: Trying to play the HDR test pattern from https://www.youtube.com/watch?v=NlAsAuBtmps

Attaching it here reencoded as HEVC so it can also be played in Quicktime: https://user-images.githubusercontent.com/662304/146276378-e7a3777a-4edd-4261-b8b8-04717c202c43.mov

The video shows rectangles of different sizes and cycles though multiple brightness levels. Both Youtube and Quicktime correctly display all the brightness level changes. IINA however displays most brightness levels at 100% brightness. The description text is also brighter in IINA.

This video source uses Display-P3 primaries. Display-P3 was disabled in change 2d48204, and is restored now.

Please test my latest binary.

[xose]

Tested with v1.2.0-17. The inspector now has the correct colorspace (DisplayP3_PQ, vs TUR_2100_PQ on the previous -15 build) but the behavior did not change: it still goes to full brightness after a single level change.

I found the video from OP and tested on both builds. It displays incorrectly on both the -17 and -15 builds, but works fine on QuickTime. I cut a 1sec portion corresponding to the screenshot without reencoding: https://user-images.githubusercontent.com/662304/147021128-47b16b6b-01f3-43c8-a621-f91c6502c099.mov

[CarterLi]

Tested with v1.2.0-17. The inspector now has the correct colorspace (DisplayP3_PQ, vs TUR_2100_PQ on the previous -15 build) but the behavior did not change: it still goes to full brightness after a single level change.

Language                                 : English
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0099 cd/m2, max: 4000 cd/m2
Maximum Content Light Level              : 4000 cd/m2
Maximum Frame-Average Light Level        : 300 cd/m2
Codec configuration box                  : hvcC

This video uses max luminance of 4000 nits, which is much higher than the luminance that Apple XDR Display supports ( 1600 nits ). In this case, tone mapping algorithm must be enabled.

The best option is the native edrMetadata. However it is supported by Metal layer only while IINA still uses OGL layer so we can't use it.

The second option is using MPV for software conversion. --tone-mapping and --target-peak must be set in IINA advance config.

We can't enable them automatically at runtime before we can detect the max luminance supported by user's display dynamically somehow.

They will break SDR sources, use it at your own risk.

[avdimdv]

It seems that disabling advanced settings with the toggle don't have any effect. If apply target-peak=1600 display will be way dim in SDR content. Only removing mpv option will revert to default.

[CarterLi]

It seems that disabling advanced settings with the toggle don't have any effect. If apply target-peak=1600 display will be way dim in SDR content. Only removing mpv option will revert to default.

Seems a bug of IINA. I will have a look of it

[akash329d]

I still feel the colors mapping are off for some reason as well, with this sample the colors are clearly off compared to any other HDR player (Infuse, Quicktime, iOS video players, HDR TV). I know I've brought this up before but still not sure if the color spaces are being interpreted correctly or if there's some other issue. IINA displays the green as much less saturated than it should be compared to Quicktime/Infuse. (This issue might be isolated to certain displays though? I see no difference on an external display yet on my Macbook [14in M1 Pro miniLED] display it looks incorrect).

[sarahkadar]

yeah fwiw I also have a 14" M1 Pro and I still see the same issue

Last time I checked, Infuse was also inferior to QuickTime but they may have fixed that

[CarterLi]

colors mapping are off for some reason doesn't help. We can't make IINA better without knowing what the some reason is.

You may contribute IINA without touching IINA's code. Since IINA uses MPV, IINA passes HDR related flags directly to MPV. You may compile MPV yourself with the EDR/HDR patch applied. Then use whatever flags / switches / options to make MPV work as you expect. Comment these flags / switches / options here.

The flags that IINA is using currently and you may start with ( for 240-1000nits test ): --macos-output-csp=display_p3_pq --target-trc=pq --target-prim=display_p3

[akash329d]

colors mapping are off for some reason doesn't help. We can't make IINA better without knowing what the some reason is.

You may contribute IINA without touching IINA's code. Since IINA uses MPV, IINA passes HDR related flags directly to MPV. You may compile MPV yourself with the EDR/HDR patch applied. Then use whatever flags / switches / options to make MPV work as you expect. Comment these flags / switches / options here.

The flags that IINA is using currently and you may start with ( for 240-1000nits test ): --macos-output-csp=display_p3_pq --target-trc=pq --target-prim=display_p3

It matches if I set the target and output as BT2020 for iina (haven’t tested with mpv, but should be the same. Not sure if that’s correct behavior as we’ve discussed earlier for mastering display primaries). Though all the other players I’ve tested seem to ignore the mastering display primaries it seems? They’re all displaying it in BT2020 since the colors seem to match. Will test it on my 14” miniLED internal monitor as well later tonight. Only have access to an older MacBook atm.

[akash329d]

Will test it on my 14” miniLED internal monitor as well later tonight.

Quick update, tested it and behavior matches (looks incorrect in IINA).

Also tested the file with madVR on windows, and it matches the behavior of displaying it in BT2020 and not P3 (which matches the behavior of Quicktime as well).

[low-batt]

HDR is being discussed in the IINA Telegram group. I've been suggesting people provide feedback here.

The latest HDR comment from "Tim" is:

It's still not enough good, we should focus the function of EDR / HDR in next_gpu (libplacebo in mpv)
libplacebo

[CarterLi]

It's still not enough good, we should focus the function of EDR / HDR in next_gpu (libplacebo in mpv)
libplacebo

I'm aware of it. However MPV doesn't support libplacebo core on mac currently: haasn/libplacebo#111 (comment)

[CarterLi]

Now we will always use bt.2020 as suggested by @ValZapod

[ViRo3]

Samples here are still shown in quite a bad manner. (Sorry to bring this sample up again)

[CarterLi]

Samples here are still shown in quite a bad manner. (Sorry to bring this sample up again)

I don't know how to fix that. Sorry.

[ViRo3]

@ValZapod Sample also has issues.

[ViRo3]

Lots of samples here broken. Seems to be related to HLG somehow? @CarterLi could I help out in any way on a non technical basis?

[CarterLi]

@ViRo3

You may contribute IINA without touching IINA's code. Since IINA uses MPV, IINA passes HDR related flags directly to MPV. You may compile MPV yourself with the EDR/HDR patch applied. Then use whatever flags / switches / options to make MPV work as you expect. Comment these flags / switches / options here.

The flags that IINA is using currently and you may start with ( for 240-1000nits test ): --macos-output-csp=display_p3_pq --target-trc=pq --target-prim=display_p3

[csdesigns]

Apologies for the delay in getting these samples to you (photos taken with external camera).

Here is an example of how QT X renders a particular scene in question:

Versus IINA Plus:

The sample encoded file is here:
https://user-images.githubusercontent.com/22551919/156405304-4e76386b-579b-4897-9783-e8cec4ccc25f.mp4

For the most part, scenes are generally rendered roughly within 1-2% subjectively of each other when comparing on a MBP 16" M1 Max in HDR or XDR Display modes. IMHO, that is more than acceptable given all of the other benefits offered by IINA. But this particular scene is far apart and quite distracting.

[ViRo3]

https://andytather.net/Panda/Files/Video/HDR/Dolby%20Vision/LG_2_DEMO_4K_L_H_03_Daylight.mkv does not trigger as EDR anymore and playback completely breaks if you just go back to a point while its playing.

[CarterLi]

https://andytather.net/Panda/Files/Video/HDR/Dolby%20Vision/LG_2_DEMO_4K_L_H_03_Daylight.mkv does not trigger as EDR anymore and playback completely breaks if you just go back to a point while its playing.

Got 403 forbidden

[CarterLi]

Got 403 forbidden

You need to use the site and download from there. https://andytather.net/Panda/VideoGalleryPage.aspx

This file is not tagged as PQ, so this is correct behaviour. Besides nothing can play dolby vision profile 4, it was long time ago deprecated. Also this file is not mkv, it is ts.

$ mediainfo /Users/carter/Downloads/LG_2_DEMO_4K_O_H_2_Dolby\ Trailer.mkv
General
ID                                       : 1 (0x1)
Complete name                            : /Users/carter/Downloads/LG_2_DEMO_4K_O_H_2_Dolby Trailer.mkv
Format                                   : MPEG-TS
File size                                : 298 MiB
Duration                                 : 1 min 17 s
Overall bit rate mode                    : Constant
Overall bit rate                         : 32.1 Mb/s
FileExtension_Invalid                    : ts m2t m2s m4t m4s tmf ts tp trp ty

Video
ID                                       : 33 (0x21)
Menu ID                                  : 2 (0x2)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : Dolby Vision, Version 1.0, dvhe.04.06, BL+EL+RPU
Codec ID                                 : 36
Duration                                 : 1 min 16 s
Bit rate                                 : 29.7 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.150
Stream size                              : 273 MiB (91%)

It's not bt.2020 so no.

I tested libplacebo and HDR wasn't enabled too.

[CarterLi]

For the most part, scenes are generally rendered roughly within 1-2% subjectively of each other when comparing on a MBP 16" M1 Max in HDR or XDR Display modes. IMHO, that is more than acceptable given all of the other benefits offered by IINA. But this particular scene is far apart and quite distracting.

To get this less yelllow picture --target-trc=pq --target-prim=bt.2020 works for me vs --target-trc=pq --target-prim=display-p3

On windows at least where the swapchain is bt.2020 and thus you need to target it.

IINA does uses bt.2020 for this content

[CarterLi]

What do you mean?

I mean IINA passes --target-trc=pq --target-prim=bt.2020 to MPV for this file

[CarterLi]

IINA will set it to bt.2020/pq ( actually itur_2100_PQ )

https://github.com/iina-plus/iina/blob/develop/iina/VideoView.swift#L336

[blackhole9307]

HDR brightness and color are way off
I am using macbook air m2, and iina latest version app..when I turn on HDR option on HDR videos the brightness, contrast of the video is dropping to very minimum after I turn off HDR option for the video then the brightness and contrast getting good again....and in HDR mode the minimum brightness and color are getting good again when I switch to another application
The colors are glitching when I turn on HDR ....
please help me bro...