Packing data into UV channels in Tangent space

[Xceptionull]

Blender Version: 2.93 LTS
Malt Version: Latest (is there a version number somewhere?)

I'm trying to pass vertex data from a different mesh into the shader to use in displacement calculation, but I also want the general shape to be deformable by bones. In order to do this, I figured the easiest way would be to convert this data into tangent space, then convert it back to world space in the shader.

In python:

def matrix_from_loop(loop):
    bitangent = loop.bitangent_sign * loop.normal.cross(loop.tangent)
    return mathutils.Matrix((loop.tangent, bitangent, loop.normal))

model = bpy.context.object.matrix_world.to_3x3()
matrix = matrix_from_loop(amesh.loops[index]) @ model
matrix.transpose()
offset = (amesh_co - bmesh_co) @ matrix
uv_tb.data[index].uv = offset.xy
uv_n0.data[index].uv = offset.z, 0

In shader:

vec3 data_tangent = vec3(S.uv[1], S.uv[2].x);
mat3 TBN = mat3(TANGENT[0], BITANGENT[0], NORMAL);
vec3 data_world = data_tangent * TBN;

Video showing expected result, followed by actual result, then with precomputed tangents (why did the normals flip?):

2021-08-13.16-53-00_edit.mp4

I tried looking into how Malt computes tangents and found this in Common.glsl:

if(in_tangent0.xyz == vec3(0)) //No precomputed tangents
    {
        vec3 axis = vec3(0,0,1);
        axis = abs(dot(axis, in_normal)) < 0.99 ? axis : vec3(1,0,0);
        vec3 tangent = normalize(cross(axis, in_normal));
        tangent = transform_normal(MODEL, tangent);
        
        for(int i = 0; i < 4; i++) TANGENT[i] = tangent;
    }
    else
    {
        TANGENT[0]= transform_normal(MODEL, in_tangent0.xyz);
        TANGENT[1]= transform_normal(MODEL, in_tangent1.xyz);
        TANGENT[2]= transform_normal(MODEL, in_tangent2.xyz);
        TANGENT[3]= transform_normal(MODEL, in_tangent3.xyz);
    }

I know that Tangents are calculated using the UV Map in Blender, so I figured I just need to calculate them the same way as Malt to fix the issue, like so:

def get_tangent(normal):
    axis = mathutils.Vector((0, 0, 1))
    axis = axis if abs(axis.dot(normal)) < 0.99 else mathutils.Vector((1, 0, 0))
    return axis.cross(normal).normalized()

def matrix_from_loop(loop):
    tangent = get_tangent(loop.normal)
    bitangent = loop.bitangent_sign * loop.normal.cross(tangent)
    return mathutils.Matrix((tangent, bitangent, loop.normal))

This didn't give me the right result either. It got worse, in fact. Well, even if the computation is different, using precomputed tangents should work in theory but, it doesn't. Might have something to do with calc_tangents() flipping the normals for some reason.

Any ideas on what I'm doing wrong here? I feel like I'm making it way more complicated than it needs to be.

[pragma37]

Long story short:

1. If you want to use tangents, always use pre-computed tangents. The current calculation when there are no pre-computed tangents is a pretty bad workaround and will be replaced soon.

2. In Blender and Malt matrix multiplication is right to left:

// Wrong
vector * matrix
vector @ matrix
// Right
matrix * vector
matrix @ vector

[Xceptionull]

I don't know how I got by without knowing the matrix evaluation order. Another thing I'd like to add is that it seems like in GLSL matrices are initialized with vectors as columns, but in Blender they are rows. This means I have to transpose the matrix in the shader too, which I wasn't doing (EDIT: Actually, I have it backwards, I don't need to transpose any matrix).

Also, I don't know if it's only on my end, but the mesh_loop.bitangent_sign is giving the wrong value. I placed arrow objects to orient with tangent space per vertex and got this:

Inverting the bitangent sign gives me the result I'd expect:

Is this why the shader has the front and back faces flipped? Seems likely to me, but I don't know enough about what's going on under the hood to say for sure. Anyway, aside from the lighting, my issue is fixed, so I'll leave it at that. Thanks for the help.

EDIT: I edited MaltMeshes.py to invert the bitangent sign, and that fixed the lighting for me: