Segfault in arrow substrait integration tests since latest version update

[jorisvandenbossche]

In the Arrow project we have a nightly CI build testing with the substrait python package, and that started failing today with the update to 0.4.1. I didn't further investigate, do don't know if it's an issue with substrait or with the Arrow consumer (although I think it probably isn't an issue with this package itself), but opening an issue here as well for visibility (arrow issue -> apache/arrow#37932)

[gforsyth]

Hmm, does Arrow also have substrait C++ generated code? I've hit segfaults before when C++ and Python have different versions of the "same" protos.

It is weird that 0.4.1 would cause this breakage, though, since the only update is to an extension yaml file

[jorisvandenbossche]

Ah, yes, also Arrow C++ builds against a specific version of substrait. And at the moment we have

ARROW_SUBSTRAIT_BUILD_VERSION=v0.27.0

arrow/cpp/thirdparty/versions.txt

In that case, the Arrow integration tests should maybe pin the substrait-python version as well, to ensure matching substrait versions?

[gforsyth]

I think that's probably a good idea. Just removes one more avenue where things can go wrong