Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build: Bump parquet from 1.13.1 to 1.14.3 #11264

Merged
merged 1 commit into from
Oct 21, 2024

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Oct 6, 2024

Bumps parquet from 1.13.1 to 1.14.3.
Updates org.apache.parquet:parquet-avro from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-avro's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-avro's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Updates org.apache.parquet:parquet-column from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-column's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-column's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Updates org.apache.parquet:parquet-hadoop from 1.13.1 to 1.14.3

Release notes

Sourced from org.apache.parquet:parquet-hadoop's releases.

Apache Parquet Java 1.14.3

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.3 RC2

What's Changed

  • GH-3007: Ensure version specific Jackson classes are shaded
  • GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encoding
  • GH-3021: Upgrade Avro dependency to 1.11.4

Apache Parquet Java 1.14.2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC2

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • GH-2992: Gate LocalTimestamp references in AvroSchemaConverter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet Java 1.14.2 RC1

What's Changed

  • GH-2948: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • GH-2956: Use avro SchemaBuilder API to convert record
  • GH-2935: Avoid double close of ParquetFileWriter
  • PARQUET-1126: Fix NPE when using the AvroParquetReader.Builder with LocalInputFile
  • PARQUET-1126: Write unencrypted Parquet files without Hadoop
  • PARQUET-2472: Close in finally block in ParquetFileWriter#end

Apache Parquet 1.14.1

What's Changed

... (truncated)

Changelog

Sourced from org.apache.parquet:parquet-hadoop's changelog.

Parquet

Version 1.14.1

Release Notes - Parquet - Version 1.14.1

Bug

  • PARQUET-2468 - ParquetMetadata.toPrettyJSON throws exception on file read when LOG.isDebugEnabled()
  • PARQUET-2498 - Hadoop vector IO API doesn't handle empty list of ranges

Version 1.14.0

Release Notes - Parquet - Version 1.14.0

Bug

  • PARQUET-2260 - Bloom filter bytes size shouldn't be larger than maxBytes size in the configuration
  • PARQUET-2266 - Fix support for files without ColumnIndexes
  • PARQUET-2276 - ParquetReader reads do not work with Hadoop version 2.8.5
  • PARQUET-2300 - Update jackson-core 2.13.4 to a version without CVE PRISMA-2023-0067
  • PARQUET-2325 - Fix parquet-cli's dictionary subcommand to work with FIXED_LEN_BYTE_ARRAY
  • PARQUET-2329 - Fix wrong help messages of parquet-cli subcommands
  • PARQUET-2330 - Fix convert-csv to show the correct position of the invalid record
  • PARQUET-2332 - Fix unexpectedly disabled tests to be executed
  • PARQUET-2336 - Add caching key to CodecFactory
  • PARQUET-2342 - Parquet writer produced a corrupted file due to page value count overflow
  • PARQUET-2343 - Fixes NPE when rewriting file with multiple rowgroups
  • PARQUET-2348 - Recompression/Re-encrypt should rewrite bloomfilter
  • PARQUET-2354 - Apparent race condition in CharsetValidator
  • PARQUET-2363 - ParquetRewriter should encrypt the V2 page header

... (truncated)

Commits
  • b5e376a [maven-release-plugin] prepare release apache-parquet-1.14.3-rc2
  • 7d970b7 GH-3021: Upgrade Avro dependency (#3022)
  • 4257337 [maven-release-plugin] prepare for next development iteration
  • cf1efcc [maven-release-plugin] prepare release apache-parquet-1.14.3-rc0
  • b1475a7 GH-3013: Fix potential ClassCastException at reading DELTA_BYTE_ARRAY encodin...
  • aec24e7 MINOR: Don't run all the tests on a release (#2999)
  • 2734728 GH-3007: Ensure version specific Jackson classes are shaded (#3017)
  • 7b6753d MINOR: fix version of parquet-plugins
  • dde627c Prepare for next development iteration
  • 2245b30 [maven-release-plugin] prepare for next development iteration
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot dependabot bot added dependencies Pull requests that update a dependency file java Pull requests that update Java code labels Oct 6, 2024
@findepi
Copy link
Member

findepi commented Oct 12, 2024

@dependabot rebase

@dependabot dependabot bot force-pushed the dependabot/gradle/parquet-1.14.3 branch from b9f6809 to 1d3d3e5 Compare October 12, 2024 19:13
@nastra nastra closed this Oct 18, 2024
@nastra nastra force-pushed the dependabot/gradle/parquet-1.14.3 branch from 1d3d3e5 to 9d58865 Compare October 18, 2024 07:16
@dependabot dependabot bot deleted the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:16
@nastra nastra restored the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:18
@nastra nastra deleted the dependabot/gradle/parquet-1.14.3 branch October 18, 2024 07:20
@nastra nastra reopened this Oct 18, 2024
@github-actions github-actions bot added the flink label Oct 18, 2024
@@ -217,27 +217,27 @@ public void testPrimitiveColumns() throws Exception {

Row binaryCol =
Row.of(
52L,
55L,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like column sizes got slightly larger:

Parquet 1.13.1

% parquet-cli/1.14.3/bin/parquet column-size old-version.parquet
binaryCol-> Size In Bytes: 52 Size In Ratio: 0.08695652
intCol-> Size In Bytes: 71 Size In Ratio: 0.1187291
decimalCol-> Size In Bytes: 85 Size In Ratio: 0.14214046
fixedCol-> Size In Bytes: 44 Size In Ratio: 0.073578596
booleanCol-> Size In Bytes: 32 Size In Ratio: 0.053511705
stringCol-> Size In Bytes: 79 Size In Ratio: 0.13210702
floatCol-> Size In Bytes: 71 Size In Ratio: 0.1187291
longCol-> Size In Bytes: 79 Size In Ratio: 0.13210702
doubleCol-> Size In Bytes: 85 Size In Ratio: 0.14214046

Parquet 1.14.3

% parquet-cli/1.14.3/bin/parquet column-size new-version.parquet
binaryCol-> Size In Bytes: 55 Size In Ratio: 0.085403726
intCol-> Size In Bytes: 77 Size In Ratio: 0.11956522
decimalCol-> Size In Bytes: 91 Size In Ratio: 0.14130434
fixedCol-> Size In Bytes: 47 Size In Ratio: 0.072981365
booleanCol-> Size In Bytes: 36 Size In Ratio: 0.055900622
stringCol-> Size In Bytes: 85 Size In Ratio: 0.13198757
floatCol-> Size In Bytes: 77 Size In Ratio: 0.11956522
longCol-> Size In Bytes: 85 Size In Ratio: 0.13198757
doubleCol-> Size In Bytes: 91 Size In Ratio: 0.14130434

@nastra nastra requested a review from findepi October 18, 2024 08:59
@findepi
Copy link
Member

findepi commented Oct 18, 2024

thank you @nastra !

@nastra nastra merged commit b8c2b20 into main Oct 21, 2024
51 of 89 checks passed
@Fokko Fokko mentioned this pull request Oct 30, 2024
RussellSpitzer added a commit to RussellSpitzer/iceberg that referenced this pull request Nov 4, 2024
This reverts commit b8c2b20.

apache/parquet-java#3040
Was discovered by @pan3793 in Parquet 1.14.(0,1,2,3).
RussellSpitzer added a commit that referenced this pull request Nov 4, 2024
RussellSpitzer added a commit to RussellSpitzer/iceberg that referenced this pull request Nov 4, 2024
RussellSpitzer added a commit that referenced this pull request Nov 4, 2024
Fokko added a commit to Fokko/iceberg that referenced this pull request Nov 8, 2024
Fokko added a commit that referenced this pull request Nov 20, 2024
* Revert "Revert "Build: Bump parquet from 1.13.1 to 1.14.3 (#11264)" (#11462)"

This reverts commit 7cc16fa.

* Bump to Parquet 1.14.4

* Lookup sizes instead

* Update build.gradle
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file flink java Pull requests that update Java code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants