Add HDR display output support

[Calinou]

• Related to Implement color profile management (ICC) #903
  and Cross-platform HDR support #1004.

Describe the project you are working on

The Godot editor 🙂

Describe the problem or limitation you are having in your project

While Godot internally renders in HDR when using the Forward+ and Mobile rendering methods, it currently does not support outputting to an HDR display in a way that preserves the dynamic range.

This makes the image look more dull than it could be on displays that support HDR, particularly those with a good HDR implementation (OLED and miniLED panels).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add HDR display output support to Godot.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

This would be exposed as a display/window/hdr/enabled project setting that defaults to false. It can be toggled at runtime using DisplayServer.window_set_hdr_output_enabled(), which allows for per-window control of HDR output.

Likewise, a maximum luminance cap in nits can be defined using display/window/hdr/max_luminance and DisplayServer.window_set_hdr_output_max_luminance(). This would be set to a value around 1000 nits by default.

Additionally, a paperwhite setting in nits should be defined somewhere. In addition to a project setting (likely defaulting to a value around 200 nits), it would make the most sense as a per-window property, but per-viewport might be more technically feasible. This setting affects the brightness of UI elements so that they don't become extremely bright compared to white elements of the 3D scene. By default, all 2D node rendering would be affected by this, but there should be a property to make certain nodes ignore the paperwhite tonemapping and make them render with full bightness (for 2D games making use of HDR output).

The documentation should emphasize that the HDR toggle, the luminance cap and paperwhite settings should be exposed to users in settings menus, as they need to be adjusted to suit the user's monitor.

There should also be a way for unshaded materials to base themselves on paperwhite, so that Sprite3D and Label3D used for UI purposes are not overly bright in HDR. I suppose this would be a BaseMaterial3D property or a shader render mode. In the interest of compatibility, we may not be able to enable this by default in Sprite3D due to VFX usage (where HDR display can be intended), but for Label3D, we may be able to safely default to this.

Lastly, there should be a DisplayServer method to query whether a screen has HDR enabled, so that you can gray out an HDR checkbox in a game if HDR is disabled on the screen. DisplayServer should also have a FEATURE_HDR constant which would be true on supported platforms, false otherwise.

The implementation of HDR output will require OS-specific changes to each rendering driver and display server, so that the HDR metadata can be set during the window and swapchain creation process.

Platforms that can be supported:

• Windows Support output to HDR monitors godot#94496
• macOS
• Linux
• Android (is it possible for games to use HDR output?)
• iOS (is it possible for games to use HDR output?)

Some caveats:

• On Windows, using Windows 11 is recommended as it has an HDR Calibration tool and overall better HDR support.
• On Linux, HDR output requires Wayland. It can't be achieved using X11 due to limitations in that display server.
  • Steam Deck OLED is configured to support HDR, although I don't know if it works with native Linux games.
• When using the Mobile rendering method, this will need Allow the Mobile rendering method to use 16 bpc HDR instead of 10 bpc #7878 to benefit from a greater exposure range and color buffer precision. Otherwise, HDR output support will likely have to be disabled when the current RGB10A2 precision is used (Forward+ uses RGBAH = 16 bpc).
• HDR output won't be supported when using the Compatibility rendering method, due to the limited exposure range and color buffer precision.
• The implementation would initially target 3D games, but in the long run, it should also be possible to create 2D games that support HDR output.
• Improved color grading support would be hugely beneficial to HDR displays, possibly more so than SDR. See Expand adjustments into more proper color grading #6613.

See the settings exposed by the Control HDR mod for an example of a best-in-class HDR implementation (related video):

control_hdr_mod_settings.mp4

Interesting, that UI seems to use the term "paperwhite" in a different way, and has a dedicated setting for the brightness of UI and HUD elements.

Additional recommendations from @alvinhochun:

HGIG recommendations: https://www.hgig.org/doc/ForBetterHDRGaming.pdf

Tone mapping:

• Assume monitors to not tone map the input.
  • i.e. ST2084 PQ input signal refers to absolute luminance.
  • Values out of range will be clipped.
• Monitors shall provide parameters such as MaxLuminance / MaxFFLuminance / MinLuminance.
• Console/OS shall provide built-in HDR calibration tools to adjust the above parameters if needed.
  • Windows HDR Calibration is not supported on Windows 10!
  • Shall the game provide its own calibration in case the OS does not support it?
• Console/OS API shall provide a way to query the above parameters (calibrated).
• Games shall perform tone mapping according to the above parameters.
• SDR white point / mid-tone brightness:
  • May be configurable by the player.
  • Default to the OS SDR brightness setting?
    • Note that the setting is per-monitor.
  • Otherwise, may default to 203 nits (matching BT.2408's definition of diffuse white).
    • Typical recommendation for photo editing is 80~120 nits, but for gaming or even general usage this may be too dark in practice.
  • Lower SDR brightness gives more HDR headroom, which is important for FakeHDR DisplayHDR 400.

Colour gamut:

• HDR monitors are typically also wide gamut (read: wider than sRGB).
• High DCI-P3 coverage is typical.
• Usually not as wide as AdobeRGB.
• Monitors don't get close to full BT.2020 (BT.2100) gamut. It is only used for signalling.
• If the game is produced in sRGB:
  • All colours are assumed to be in BT.709 colour gamut.
  • SDR output does not require gamut mapping.
    • Depending on colour management implementation, may want to expand to the native gamut of the monitor.
  • For HDR output, the final image is converted to BT.2020 without gamut expansion.
    • Optionally, the gamut may be expanded to DCI-P3 (D65) since some people prefer the oversaturated colours.
    • Generic colour adjustments may be provided.
• If the game is produced in DCI-P3 (D65 or not):
  • All colours are assumed to be in DCI-P3 colour gamut.
  • For SDR output, gamut compression may be required.
    • Clipping may be an option, but this is likely to give unsatisfactory results.
      • https://bottosson.github.io/posts/gamutclipping/
  • For HDR output, the final image is converted to BT.2020 without gamut expansion.
    • Since the monitor gamut may not have full DCI-P3 coverage, gamut mapping may be required.
    • If monitor has a wider gamut than DCI-P3, gamut expansion may be an option.
    • Generic colour adjustments may be provided.

Additional slideshow on HDR support in Frostbite: https://www.ea.com/frostbite/news/high-dynamic-range-color-grading-and-display-in-frostbite

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

This can't be implemented with an add-on.

Third-party workarounds such as Special K and RTX HDR exist, but they are limited in what they can do (especially when it comes to UI handling). These tools are also only available on Windows.

[masterthdev]

Android (is it possible for games to use HDR output?)

It is possible with EGL extensions but hardware support wasn't very good two years ago. I am not sure about Vulkan, it has similar extensions but they may not work on Android. I have a simple OpenGL implementation here: https://github.com/masterthdev/android-hdr-output/blob/main/HdrDisplayTest.kt