fix: schema adapter doesn't map partial batches correctly

[alexwilcoxson-rel]

Description

When pushdown_filters on DataFusion SessionConfig is enabled for Parquet, map_partial_batch on the schema adapter is called with a batch that is evaluating the RowFilter created by pushdown_filters. The batch only contains the column(s) in the predicate for the RowFilter. If table_schema contains not null column then the cast will fail.

Example error:

thread 'delta_datafusion::tests::delta_scan_supports_pushdown' panicked at crates/core/src/delta_datafusion/mod.rs:2284:41:
called `Result::unwrap()` on an `Err` value: ArrowError(ExternalError(External(SchemaError("Could not find column col_2"))), None)

This change only uses from table_schema the columns that exist in the batch.

See https://docs.rs/datafusion/latest/datafusion/datasource/schema_adapter/trait.SchemaMapper.html#tymethod.map_partial_batch

Related Issue(s)

Documentation

[alexwilcoxson-rel]

@HawaiianSpork FYI