Handle PyArrow CVE-2023-47248

[roeap]

Environment

Delta-rs version: *

Binding: python

Environment:

• Cloud provider: n.a.
• OS: n.a.
• Other:

---

Bug

There is a critical CVE out for all pyarrow versions between >= 0.14 < 14.0.1. Not sure if we can force all downstream consumers to upgrade to > 14. There is a hotfix package available as well for other versions that we could conditionally install and load.

What you expected to happen:

How to reproduce it:

More details:

[r3stl355]

Current dependency is "pyarrow>=8" so it will install the latest (14.0.1) but won't touch 14.0.0 unless the dependency is changed. I'm happy to give it a try if you think this could be changed to "pyarrow>=14.0.1". I can also add a runtime version check to prevent prevent running if 14.0.0 is installed but I don't know if that's a good idea if the user may have installed the patch

[wjones127]

but won't touch 14.0.0 unless the dependency is changed

It's not just 14.0.0. Basically every release before that also has the same bug.

I don't think the relevant codepath is used in by our library, so I don't think we have strong reason to take any action. I think it's on our users to upgrade to a fixed pyarrow version. I do wish that Arrow project would consider releasing bugfix releases going back to 8.0.0, but looks like they found that too difficult.

[r3stl355]

It's not just 14.0.0. Basically every release before that also has the same bug.

I didn't realise it also holds for previous versions, CVE description is misleading as it says "versions 0.14.0 to 14.0.0". EDIT - I don't think I'm really good at Spot the Difference game 🤦‍♂️

[roeap]

I don't think the relevant codepath is used in by our library, so I don't think we have strong reason to take any action.

I guess from my work critical CVEs are just triggering 😆 - thinking about it you are right though .. as we are not directly affected, downstream consumers should handle that...

[wjones127]

I didn't realise it also holds for previous versions, CVE description is misleading as it says "versions 0.14.0 to 14.0.0"

What do you mean? "versions 0.14.0 to 14.0.0" clearly includes 1.0.0, 8.0.0, etc.

[r3stl355]

🤦 that 14 in both, I clearly could not spot a difference 🤦

[roeap]

this is handled.