Unable to query partitioned tables

[cmackenzie1]

Environment

Delta-rs version: 0.9.0

Binding:

Environment:

• Cloud provider: N/A
• OS: macOS 13.3.1
• Other:

---

Bug

What happened:

Trying to query a partitioned table using datafusion is failing.

What you expected to happen:

Should return the results matching the partition filter.

How to reproduce it:

1. Create a partitioned table (replicated using https://github.com/delta-io/delta-rs/tree/main/rust/tests/data/delta-0.8.0-partitioned)
2. Add data to table
3. Query table using SELECT * FROM t WHERE year = 2020

Internal("The type of Dictionary(UInt16, Utf8) Eq Utf8 of binary physical should be same")

More details:

Trying again where the partition column type is string yields no results and no error. I think the resulting parquet file is missing the partition column as data.

[roeap]

Thanks for reporting, there were some changes in how datafusion handles partitioned columns.

I'll look into it.

[cmackenzie1]

Thanks, @roeap. Let me know how I can help. I've started building a project with delta-rs at work, so happy to help out.

I see in datafusion_tests.ts that the value is cast during query. Do you know if type coercion now supported, so the casting can be omitted?

[roeap]

I've started building a project with delta-rs at work, so happy to help out.

Excellent - we are always happy for any contributions. Very much like you already did with the great test cases!