GDExtension: passing Image to ImageTexture with ImageTexture::update is too slow

[DetermLZ]

Godot version

4.0.2-stable

System information

Win 11, Intel i7-12700, RTX 3060Ti

Issue description

Hello.

I am currently working on image rendering with GPU with C++ GDExtension. An important step in the project is passing Image data to ImageTexture before uploading to the shader.

But I found this conversion takes too much time in each frame with low efficiency. For a 1920x1080 image, it takes about 32 to 38 milliseconds to update the ImageTexture. While in Godot 3.5, the same process takes less than 1 millisecond.

Any help will be appreciated.

Steps to reproduce

1. clone godot-cpp repository
2. checkout version 4.0.2-stable
3. go to folder godot-cpp\test\src and replace 'example.cpp' and 'example.h' with the following files
   example.zip
4. run 'scons' to build the GDExteion dll file and run the demo project

Minimal reproduction project

PLease follow the above descriptions to reproduce the project.

[MGilleronFJ]

For a 1920x1080 image, it takes about 32 to 38 milliseconds to update the ImageTexture. While in Godot 3.5, the same process takes less than 1 millisecond.

This is quite surprising. I assume you measured the time it takes for the update function to complete, but maybe in Godot 3 the actual update isn't happening at this function call and occurs later internally?
I have also found that when uploading graphics data in general sometimes the renderer stalls at very unfortunate timings, which can either go very fast or very slowly (up to a whole frame of waiting because the GPU has to get in sync).
I'm curious about the cause of update's performance in Godot 4 though.

Also, I see you use GDExtension. Does the issue happen with GDScript? I wouldn't expect a difference (also would make it easier to test). If it does make a big difference however, that would be a GDExtension issue. But otherwise I think GDExtension is unrelated to this.

Also you could try with other renderers, to see if that's one in particular.

[bitsawer]

Also, some image formats have to be converted internally before they are used, depending on renderer. Try using FORMAT_RGBAF instead of FORMAT_RGBF and just ignore the alpha channel. This might avoid having to convert the data every time you update the texture.

[DetermLZ]

Also, some image formats have to be converted internally before they are used, depending on renderer. Try using FORMAT_RGBAF instead of FORMAT_RGBF and just ignore the alpha channel. This might avoid having to convert the data every time you update the texture.

Thanks so much! This is actually what I need.
After changing the format to FORMAT_RGBAF, the conversion time surprisingly decreased to 2 or 3 milliseconds.

But still, I think this is a bug for GDExtension, since this can still occur in some situations.

Edited:
I also tried to convert the FORMAT_RGBF format image data to FORMAT_RGBAF with Image::convert before uploading ImageTexture. And this step took almost the same time as uploading FORMAT_RGBF format Image to ImageTexture, while update took 2 or 3 milliseconds.
Is it reasonable to believe ImageTexture::update converts FORMAT_RGBF to FORMAT_RGBAF before copying the data? And the format of ImageTexture must be FORMAT_RGBAF?

[lyuma]

Still reproducible on Godot 4.1 . RGB format is way slower than equivalent R, RG and RGBA formats.

Here is my GDScript MRP:

extends Sprite2D
var image: Image
var image_texture: ImageTexture

# Called when the node enters the scene tree for the first time.
func _ready():
	image = Image.create(4096,4096,false,Image.FORMAT_RGBA8)
	image.fill(Color.TURQUOISE)
	texture = ImageTexture.create_from_image(image)
	print(texture)
	self.texture = texture


# Called every frame. 'delta' is the elapsed time since the previous frame.
func _process(_delta: float):
	var time_start: int = Time.get_ticks_usec()
	texture.update(image)
	var time_end: int = Time.get_ticks_usec()
	print(str(0.001 * (time_end - time_start)) + " ms")

Data for 4kx4k texture:

RF: 21ms
RGF: 48ms
RGBF: 650ms
RGBAF: 100ms

RH: 10ms
RGH: 20ms
RGBH: 800ms
RGBAH: 48ms

R8: 6.0ms
RG8: 12ms
RGB8: 51ms
RGBA8: 21ms

You can see the RGB formats consistently stand out as outliers.

perhaps caused by misalignment as the GPU will want to align them to multiples of 4 (though I'm surprised about RGBF, since the floating point type should guarantee they are word aligned)

I'm just surprised that RGB is up to 10 times slower than the equivalent RGBA

[Zylann]

Would there be a relation to the fact RGB doesnt actually exist as a format in the rendering driver? Therefore requiring conversion, which is a per-pixel operation adding a component of padding