[d3d9] fix opCompositeExtract out of bound

[AlexBozhenk]

Closes: #3293

Signed-by: Oleksii Bozhenko oleksii.bozhenko@globallogic.com

[DadSchoorse]

Can you explain why this is nessecary? Fixed function code isn't easy to understand.

The commit message should be [d3d9] ... instead of spirv: ..., by the way.

[AlexBozhenk]

@DadSchoorse Yep! As mentioned in #3293 (comment) we have out of bound during compilation to spirv. Disassembling the shader shows that the var is float[4]

%26 = OpCompositeConstruct %v4float %23 %24 %float_0 %float_0
...
%48 = OpLoad %v4float %in_Texcoord0
%49 = OpSelect %v4float %45 %26 %48
...
%108 = OpCompositeExtract %float %49 6

In the fixed function we trying to get the last element. So, all I've changed relates to array size.

[misyltoad]

This is wrong, TCI_PASSTHRU doesn't set count = 4 because it's meant to use the size of the texcoord passed in.

Is this required to close the other issue? If so I imagine there is something else going on, but this is definitely not the right solution!

https://learn.microsoft.com/en-us/windows/win32/direct3d9/d3dtexturetransformflags

[misyltoad]

Are you hitting a case here where count == 0? Curious... That might be the D3DTTFF_DISABLE case that is missing some handling.

[AlexBozhenk]

Not really. I'm hitting count == 7. Which is more than texcoord size

[AlexBozhenk]

For case if we have count smaller than 4 can suggest this:

Suggested change

	count = 4;
	count = std::min(count, static_cast<uint32_t>(4));

[AlexBozhenk]

one more way is to prevent D3DTTFF_DISABLE if we have count >= 4 here. Obviously, it doesn't make sense to do opCompositeExtract if we don't opCompositeInsert(which won't do if count >= 4)

[AlexBozhenk]

count == 7 && texcoordCount == 0
Good for you ;)

[misyltoad]

If texcoordCount == 0, we probably want to treat it as NINED3DTSS_TCI_DISABLE right?

[AlexBozhenk]

Sry, what is NINED3DTSS_TCI_DISABLE ?
are you talking about D3DTTFF_DISABLE?

[misyltoad]

Yeah, I had a look at what Gallium Nine (see link above) did and it disabled it in that case. So I imagine that's probably what we should do.

[AlexBozhenk]

Updated. Pls, check if it's what you meant :)

[misyltoad]

Once we solve what is going on with that random count = 4, I am happy to merge this.