pyiceberg-0.7.1

What's Changed

• Fix delete to trace existing manifests when a data file is partially rewritten by @Fokko in #1046
• Fix 'to_arrow_batch_reader' to respect the limit input arg by @kevinjqliu in #1042
• Fix correctness of applying positional deletes on Merge-On-Read tables by @sungwy in #1026
• Fix overwrite when filtering data by @ndrluis in #1023
• Bug fix for deletes across multiple partition specs on partition evolution by @sungwy in #984
• Fix evolving the table and writing in the same transaction by @HonahX in #985
• Fix scans when result is empty by @grobgl in #997
• Fix ListNamespace response in REST Catalog by @ndrluis in #995
• Exclude Python 3.9.7 from list of supported versions by @ndrluis in #526
• Allow setting write.parquet.row-group-limit by @Fokko in #1016
• Allow setting write.parquet.page-row-limit by @Fokko in #1017
• Fix pydantic warning during commit by @ndrluis in #972

Full Changelog: pyiceberg-0.7.0...pyiceberg-0.7.1

pyiceberg-0.7.0

What's Changed

• Build: Bump getdaft from 0.2.14 to 0.2.15 by @dependabot in #434
• Build: Bump cryptography from 42.0.0 to 42.0.2 by @dependabot in #440
• docs: Add missing release steps by @Fokko in #443
• Build: Bump moto from 5.0.1 to 5.0.2 by @dependabot in #447
• Build: Bump mkdocs-material from 9.5.9 to 9.5.10 by @dependabot in #448
• Make the snapshot creation part of the Transaction by @Fokko in #446
• Send X-Iceberg-Access-Delegation header to signal support for vended credentials/remote signing by @nastra in #436
• Retry with new Access Token on 419 response by @anupam-saini in #340
• Reuse commit-uuid as the write-uuid by @Fokko in #437
• Update NameMapping on update_schema() by @sungwy in #441
• Feat: Implement create_table_if_not_exists by @hussein-awala in #415
• Build: Bump coverage from 7.4.1 to 7.4.2 by @dependabot in #457
• Build: Bump getdaft from 0.2.15 to 0.2.16 by @dependabot in #456
• Accept pyarrow LargeListType and FixedSizeListType by @hussein-awala in #458
• Bump pre-commit and such by @Fokko in #442
• docstring: Fix missing commit by @Fokko in #432
• Improve error message in case of a mismatch by @Fokko in #352
• Cleanup conftest, remove LocalOutputFile by @kevinjqliu in #468
• Fix InMemoryCatalog Catalog commit operation by @anupam-saini in #470
• enable set hadoop ugi for hive catalog by @j7nhai in #472
• Raise exception if namespace does not exist in load_namespace_properties for Sql Catalog by @rushilshah1 in #477
• Add Support for Custom Header Configurations in RESTCatalog by @geruh in #467
• rest: Set OAuth Content-Type header explicitly by @Fokko in #478
• Partition Evolution by @amogh-jahagirdar in #245
• Fix retrying logic by @Fokko in #480
• Remove unused catalog from integration test by @kevinjqliu in #481
• add github add to check md link by @kevinjqliu in #324
• Sort Order update by @anupam-saini in #476
• Make issued_token_type optional to support OAuth2 Client Credential Flow by @flyrain in #466
• Update table metadata throughout transaction by @Fokko in #471
• Allow non-string typed values in table properties by @kevinjqliu in #469
• Construction of filenames for partitioned writes by @jqin61 in #453
• Remove extraneous import by @Fokko in #485
• Default spark session timezone to UTC in test by @kevinjqliu in #494
• Fix dead links in docs by @kevinjqliu in #493
• Update bug isse template release list by @ndrluis in #496
• add rest scope in the config documentation by @himadripal in #495
• Make scope configurable by @himadripal in #484
• Tests should explicitly check for schema_id by @kevinjqliu in #487
• add support for glue.id by @jrouly in #490
• [Bug Fix] cast None current-snapshot-id as -1 for Backwards Compatibility by @sungwy in #473
• Make optional oauth configurable by @himadripal in #486
• Disable Spark Catalog caching for integration tests by @kevinjqliu in #501
• Set table properties with dictionary by @kevinjqliu in #503
• Imports decouple by @ndrluis in #505
• Allow setting non-string typed values in set_properties by @kevinjqliu in #504
• [Bug fix] update name mapping in Transaction.update_schema by @sungwy in #508
• [Bug Fix] Allow Partition data to be nullable in ManifestEntry by @sungwy in #509
• Allow fsspec up to 2025.1 by @bolkedebruin in #510
• Build: Bump pypa/cibuildwheel from 2.16.5 to 2.17.0 by @dependabot in #517
• Decouple imports reported by mypy linter by @ndrluis in #519
• build: Move back to the mmh3 by @Fokko in #460
• Improve the InMemory Catalog Implementation by @kevinjqliu in #289
• Add table_exists method to Catalog by @anupam-saini in #512
• Add StrictMetricsEvaluator by @Fokko in #518
• Add Data Files from Parquet Files to UnPartitioned Table by @sungwy in #506
• Fix CommitTableRequest serialisation by @kdbhiggins in #525
• Add partition stats in snapshot summary by @jqin61 in #521
• UUID literal to binary and fixed by @sebpretzer in #529
• Adding a new dev dep, deptry by @kevinjqliu in #528
• Add as_arrow() to Schema class by @ndrluis in #532
• Change Append/Overwrite API to accept snapshot properties by @Gowthami03B in #419
• Fix Glue Integration test by @HonahX in #536
• Add Snapshots table metadata by @Fokko in #524
• add_files support partitioned tables by @sungwy in #531
• [Bug Fix] Fix TableMetadataV1 Validators by @HonahX in #544
• Fix race condition on Table.scan with limit by @kevinjqliu in #545
• Add Strict projection by @Fokko in #539
• Fix the Avro tests by @Fokko in #552
• On write operation, cast data to Iceberg Table's pyarrow schema by @kevinjqliu in #523
• Bin-pack Writes Operation into multiple parquet files, and parallelize writing WriteTasks by @kevinjqliu in #444
• Bump version to 0.6.1 by @HonahX in #561
• Minor fixes, #523 followup by @kevinjqliu in #563
• Call as_arrow() call in overwrite by @kevinjqliu in #565
• Remove as visitors import by @Fokko in #567
• Tests: Make Spark optional for testing by @Fokko in #568
• [CI FIx] Use Docker Compose V2 by @HonahX in #575
• typealias for table version by @MehulBatra in #566
• Disallow default header to be overwritten by @whynick1 in #577
• [Doc] Update how-to-release.md by @HonahX in #576
• Support CreateTableTransaction in Glue and Rest by @HonahX in #498
• Move writes to Transaction by @sungwy in #571
• Add entries metadata table by @Fokko in #551
• Partitioned Append on Identity Transform by @jqin61 in #555
• Implement getstate and setstate on PyArrowFileIO and FsSpecFileIO so that they can be pickled by @amogh-jahagirdar in #543
• [Bug Fix] Allow HiveCatalog to create table with TimestamptzType by @HonahX in #585
• Change DataScan to accept Metadata and io by @Fokko in #581
• Read: fetch file_schema directly from pyarrow_to_schema by @HonahX in #597
• Support Time Travel in InspectTable.entries by @sungwy in #599
• [Bug Fix] HiveCatalog's _commit_table need to refresh and update the metadata in a ...

PyIceberg 0.6.1

Patch release:

• Fail to create version 1 table with non-empty partition-spec and sort-order
• Hive Catalog cannot create table with TimestamptzType field
• Fail to read parquet file with special characters in column names
• Hive Catalog commit consistency issue
• docutils=0.21 installation issue

Full Changelog: https://github.com/apache/iceberg-python/commits/pyiceberg-0.6.1

PyIceberg 0.6.0

What's Changed

• Python: Migrate from iceberg to iceberg-python by @Fokko in #3
• Build: Bump duckdb from 0.8.1 to 0.9.0 by @dependabot in #4
• Build: Bump mkdocs-section-index from 0.3.7 to 0.3.8 by @dependabot in #5
• Build: Bump mkdocstrings-python from 1.7.0 to 1.7.1 by @dependabot in #6
• Build: Bump pydantic from 2.3.0 to 2.4.2 by @dependabot in #7
• Build: Bump psycopg2-binary from 2.9.7 to 2.9.8 by @dependabot in #8
• Build: Bump moto from 4.2.4 to 4.2.5 by @dependabot in #9
• Build: Bump mkdocs-material from 9.4.1 to 9.4.2 by @dependabot in #10
• Build: Bump rich from 13.5.3 to 13.6.0 by @dependabot in #11
• Build: Bump typing-extensions from 4.7.1 to 4.8.0 by @dependabot in #12
• Build: Bump griffe from 0.36.2 to 0.36.4 by @dependabot in #13
• Build: Bump urllib3 from 1.26.16 to 1.26.17 by @dependabot in #36
• Update how to release by @Fokko in #34
• pydantic exclude 2.4.0, 2.4.1 by @syun64 in #38
• Add logic to generate a new snapshot-id by @Fokko in #37
• Fix the TableIdentifier by @Fokko in #44
• Convert the Logical to Physical map to a visitor by @Fokko in #43
• Build: Bump mkdocstrings-python from 1.7.1 to 1.7.2 by @dependabot in #52
• Build: Bump fastavro from 1.8.3 to 1.8.4 by @dependabot in #51
• Build: Bump pypa/cibuildwheel from 2.16.0 to 2.16.2 by @dependabot in #47
• Build: Bump psycopg2-binary from 2.9.8 to 2.9.9 by @dependabot in #49
• Build: Bump coverage from 7.3.1 to 7.3.2 by @dependabot in #50
• Build: Bump cython from 3.0.2 to 3.0.3 by @dependabot in #48
• Docs: Fix repo name and url by @manuzhang in #54
• Run integration tests with Iceberg 1.4.0 by @Fokko in #56
• Add logic for table format-version updates by @Fokko in #55
• Disable merge-commit and enforce linear history by @Fokko in #57
• Construct a writer tree by @Fokko in #40
• Add method and property around sequence-numbers by @Fokko in #60
• Fix column rename doc example to reflect correct API by @cabhishek in #59
• Expression: Part of the expression is ignored when multiple and/or expressions are specified by @amogh-jahagirdar in #65
• Fix Iceberg to Avro Schema Conversion: Fixed, Decimal, UUID by @HonahX in #53
• allow override env-variables in load_catalog by @bdilday in #45
• Make next_sequence_number private by @Fokko in #62
• Check for empty responses by @Fokko in #69
• Fix Arrow fixed type by @Fokko in #70
• Bump version to 0.5.1 by @Fokko in #68
• Add spec_id back to data file by @puchengy in #63
• Build: Bump ray from 2.7.0 to 2.7.1 by @dependabot in #77
• Build: Bump griffe from 0.36.4 to 0.36.5 by @dependabot in #76
• Build: Bump mypy-boto3-glue from 1.28.36 to 1.28.63 by @dependabot in #75
• Build: Bump mkdocstrings-python from 1.7.2 to 1.7.3 by @dependabot in #74
• Build: Bump moto from 4.2.5 to 4.2.6 by @dependabot in #73
• Remove python working directory by @Fokko in #71
• Don't fail on warning when releasing by @Fokko in #80
• Remove example since it is deprecated by @Fokko in #79
• Build: Bump urllib3 from 1.26.17 to 1.26.18 by @dependabot in #84
• Doc: Fix "Verifying Checksums" script in verify-release.md by @HonahX in #82
• Make to_arrow function capable of handling parquet files with sanitized name due to Avro restirction by @puchengy in #83
• Require full expression parse match by @danielcweeks in #88
• Fix NotStartsWith negation by @danielcweeks in #92
• Fix some broken commands and URLs in the docs by @hussein-awala in #89
• Update like statements to reflect sql behaviors by @danielcweeks in #91
• Fix equality of bound expressions by @Fokko in #95
• Build: Bump mkdocs-material from 9.4.2 to 9.4.6 by @dependabot in #100
• Build: Bump pytest-mock from 3.11.1 to 3.12.0 by @dependabot in #99
• Build: Bump sqlalchemy from 2.0.21 to 2.0.22 by @dependabot in #98
• Build: Bump griffe from 0.36.5 to 0.36.7 by @dependabot in #97
• Build: Bump adlfs from 2023.9.0 to 2023.10.0 by @dependabot in #96
• Replace old %-formatted by f-strings by @hussein-awala in #93
• Fix literal predicate equality check by @danielcweeks in #94
• Fix the nullability of snapshot-id on AssertRefSnapshotId by @Fokko in #103
• Build: Bump werkzeug from 2.3.7 to 3.0.1 by @dependabot in #105
• Api docs refactor by @mobley-trent in #106
• Fixed typos by @whisk in #108
• Build: Bump duckdb from 0.9.0 to 0.9.1 by @dependabot in #114
• Build: Bump pre-commit from 3.4.0 to 3.5.0 by @dependabot in #113
• Build: Bump mkdocs-material from 9.4.6 to 9.4.7 by @dependabot in #111
• Build: Bump pytest from 7.4.2 to 7.4.3 by @dependabot in #112
• Build: Bump moto from 4.2.6 to 4.2.7 by @dependabot in #110
• fix: partition evaluator thread safety by @skellys in #115
• Run dependabot daily by @Fokko in #66
• Build: Bump griffe from 0.36.7 to 0.36.9 by @dependabot in #118
• Build: Bump cython from 3.0.3 to 3.0.5 by @dependabot in #122
• Build: Bump sqlalchemy from 2.0.22 to 2.0.23 by @dependabot in #125
• Build: Bump zstandard from 0.21.0 to 0.22.0 by @dependabot in #120
• Build: Bump fastavro from 1.8.4 to 1.9.0 by @dependabot in #119
• Refactor Arrow schema conversion by @Fokko in #117
• Build: Bump pyarrow from 13.0.0 to 14.0.0 by @Fokko in #126
• Build: Bump mkdocs-material-extensions from 1.2 to 1.3 by @dependabot in #128
• Add flake8-pie to ruff by @Fokko in #86
• Update pre-commit by @Fokko in #85
• Bump version to 0.6.0 by @Fokko in #72
• Build: Bump mypy-boto3-glue from 1.28.63 to 1.28.77 by @dependabot in #130
• Catch warning in PyLint tests by @Fokko in #33
• Build: Bump mkdocs-material from 9.4.7 to 9.4.8 by @dependabot in #131
• Fix Github Pages path by @Fokko in #133
• Build: Bump pyarrow from 14.0.0 to 14.0.1 by @dependabot in #136
• Add list-refs cli command by @amogh-jahagirdar in #137
• Docs: Add section on pandas by @Fokko in #138
• Build: Bump mkdocstrings-python from 1.7.3 to 1.7.4 by @dependabot in #142
  ...

PyIceberg 0.5.1

Patch release:

• Part of the expression is ignored when multiple and/or expressions are specified
• Update like statements to reflect sql behaviors

Full Changelog: https://github.com/apache/iceberg-python/commits/pyiceberg-0.5.1