-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: reduce included dependencies for Iceberg implementation #5668
Conversation
extensions/iceberg/build.gradle
Outdated
@@ -23,6 +23,7 @@ dependencies { | |||
|
|||
Classpaths.inheritIcebergCore(project) | |||
Classpaths.inheritIcebergHadoop(project) | |||
Classpaths.inheritAvro(project, JavaPlugin.IMPLEMENTATION_CONFIGURATION_NAME) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the specific error when this was not included?
It seems like we should be marking this as runtimeOnly instead? Maybe best to move into one of the added-back-in dependencies with a "because" reason in inheritParquetHadoopConfiguration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error is:
/Users/larrybooker/dhc/lab-reduce-dep/extensions/iceberg/src/main/java/io/deephaven/iceberg/layout/IcebergKeyValuePartitionedLayout.java:91: error: cannot access IndexedRecord
final PartitionData partitionData = (PartitionData) df.partition();
^
class file for org.apache.avro.generic.IndexedRecord not found
/Users/larrybooker/dhc/lab-reduce-dep/extensions/iceberg/src/main/java/io/deephaven/iceberg/layout/IcebergKeyValuePartitionedLayout.java:94: error: cannot access SpecificData
final Object colValue = partitionData.get(colData.index);
^
class file for org.apache.avro.specific.SpecificData not found
Note: /Users/larrybooker/dhc/lab-reduce-dep/extensions/iceberg/src/main/java/io/deephaven/iceberg/util/IcebergCatalogAdapter.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
2 errors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting; I wonder why iceberg doesn't declare this as an API dependency... I might file an issue with them.
extensions/iceberg/build.gradle
Outdated
@@ -23,6 +23,7 @@ dependencies { | |||
|
|||
Classpaths.inheritIcebergCore(project) | |||
Classpaths.inheritIcebergHadoop(project) | |||
Classpaths.inheritAvro(project, JavaPlugin.IMPLEMENTATION_CONFIGURATION_NAME) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting; I wonder why iceberg doesn't declare this as an API dependency... I might file an issue with them.
No description provided.