Changelog

rust-v0.8.0 (2023-03-10)

Implemented enhancements:

Fixed bugs:

File pruning does not occur on partition columns #1175
Bug: Error loading Delta table locally #1157
Deltalake 0.7.0 with s3 feature compliation error due to rusoto_dynamodb version conflict #1191
Writing from a Delta table scan using WriteBuilder fails due to missing object store #1186

Merged pull requests:

build(deps): bump datafusion #1217 (roeap)
Implement pruning on partition columns #1179 (Blajda)
feat: enable passing storage options to Delta table builder via Datafusion's CREATE EXTERNAL TABLE #1043 (gruuya)
feat: typed commit info #1207 (roeap)
add boolean, date, timestamp & binary partition types #1180 (marijncv)
feat: extend configuration handling #1206 (marijncv)
fix: load command for local tables #1205 (roeap)
Enable passing Datafusion session state to WriteBuilder #1187 (gruuya)
chore: increment dynamodb_lock version #1202 (wjones127)
fix: update out-of-date doc about datafusion #1183 (xudong963)
feat: move and update Optimize operation #1154 (roeap)
add test for extract_partition_values #1159 (marijncv)
fix typo #1166 (spebern)
chore: remove star dependencies #1139 (wjones127)

Implemented enhancements:

Fixed bugs:

DeltaTable DataFusion TableProvider does not support filter pushdown #1064
DeltaTable DataFusion scan does not prune files properly #1063
deltalake.DeltaTable constructor hangs in Jupyter #1093
Transaction log JSON formatting issue when writing data via Python bindings #1017
crates.io entry is missing link to rustdoc documentation #1076
URL Registered with ObjectStore registry is different from url in DeltaScan #1018
Not able to connect to Azure Storage with client id/secret #977
Deltalake 0.5 crate s3 feature dynamodb version mismatch #973
Overwrite mode does not work with Azure #939
Use Chrono without default features #914
cargo test does not run due to tls conflict #985
Azure SAS authorization fails with <AuthenticationErrorDetail>Signature fields not well formed. #910

Merged pull requests:

Make rustls default across all packages #1097 (wjones127)
Implement filesystem check #1103 (Blajda)
refactor: move vacuum command to operations module #1045 (roeap)
feat: enable passing storage options to Delta table builder via DataFusion's CREATE EXTERNAL TABLE #1043 (gruuya)
feat: improve storage location handling #1065 (roeap)
Fix to support UTC timezone #1022 (andrei-ionescu)
feat: harmonize and simplify storage configuration #1052 (roeap)
feat: expose function to get table of add actions #1033 (wjones127)
fix: change unexpected field logging level to debug #1112 (houqp)
fix: datafusion predicate pushdown and dependencies #1071 (roeap)
fix: azure sas key url encoding #1036 (roeap)
Add provisional workaround to support CDC #1039 #1042 (Fazzani)
improve debuggability of json ser/de errors #1119 (houqp)
Add an example of writing to a delta table with a RecordBatch #1085 (rtyler)
minor: optimize partition lookup for vacuum loop #1120 (houqp)
Add missing documentation metadata to Cargo.toml #1077 (johnbatty)
add test for null_count_schema_for_fields #1135 (marijncv)
add test for min_max_schema_for_fields #1122 (marijncv)
add test for get_boolean_from_metadata #1121 (marijncv)
add test for left_larger_than_right #1110 (marijncv)
Add test for: to_scalar_value #1086 (marijncv)
Fix typo in delta-inspect #1072 (byteink)
chore: update datafusion #1114 (roeap)

Implemented enhancements:

Support Apache Arrow DataFusion 15 #1020
Python package: Loosen version requirements for maturin #1004
Remove Cargo.lock from library crates and add Cargo.lock to binary ones #1000
More frequent Rust releases #969
Thoughts on adding read_delta to pandas #869
Add the support of the AWS_PROFILE environment variable for S3 #986 (fvaleye)

Fixed bugs:

Azure SAS signatures ending in "=" don't work #1003
Fail to compile deltalake crate, need to update dynamodb_lock in crates.io #1002
error reading delta table to pandas: runtime dropped the dispatch task #975
MacOS arm64 wheels are generated incorrectly #972
Overwrite creates new file #960
The written delta file has corrupted structure #956
Write mode doesn't work with Azure storage #955
Python: We don't error on reader protocol v2 #886
Cannot open a deltatable in S3 using AWS_PROFILE based credentials from a local machine #855

Merged pull requests:

Support DataFusion 15 #1021 (andrei-ionescu)
fix truncating signature on SAS #1007 (damiondoesthings)
Loosen version requirement for maturin #1005 (gyscos)
Update .gitignore and add/remove Cargo.lock when appropriate #1001 (iajoiner)
fix: get azure client secret from config #981 (roeap)
feat: check invariants in write command #980 (roeap)
Add a new release github action for Python binding: macos with universal2 wheel #976 (fvaleye)
Bump version of the Python binding to 0.6.4 #970 (fvaleye)
Handle pandas timestamps #958 (hayesgb)
test(python): add azure integration tests #912 (wjones127)

* This Changelog was automatically generated by github_changelog_generator