Added ability to generate unique paths for tables #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Also, add an integration test to TestMemorySmoke
Some connector might have partial support for DELETE statement.
sendUpdate for the next request can be triggered after currentRequest future is done but before UpdateResponseHandler callback execution. This can overwrite value of currentRequestStartNanos before the callback has recorded the stats for the completed request. The next request may also miss the value of sendPlan set in the callback execution in this scenario and send the plan an extra time.
- inline field - remove obvious comments - limit access to field - remove explicit default constructor
Fixes trinodb#8268 The problem was caused by multiple rows having the same (writeId, bucket, rowId). In order to fix this it is necessary to ensure unique row IDs across writers. To achieve it different writers will have separated id ranges in the split assigned to them
Co-authored-by: Ashhar Hasan <hashhar_dev@outlook.com>
SPI cannot use Guava's toImmutableList
Add option to configure many SystemSessionPropertiesProviders in the SessionPropertyManager
Not all connector tests run within containers so any operation that deletes data from the TPCH tables can change state of the testing infrastructure leading to hard to diagnose failures. So we create temporary tables to verify if deletes are supported or not.
Hoists "any row" replication and nullBlock.isNull logic out of the PartitionedOutputOperator.PagePartitioner inner loop logic
Trino will cast (if possible) storage table columns if they don't match materialized view column types.
Before the change, Trino was not able to read from an Iceberg v1 table in which a partitioning field was removed via Spark. In v1 tables, removed partitioning fields are replaced with `void` transformation to keep field ordinal numbers unchanged, and `void` transformation was not supported.
In v1 tables, removed partitioning fields are replaced with `void` transformation to keep field ordinal numbers unchanged, and `void` transformation was not supported when writing. This commit adds support for writing with `void` transformation and, implicitly, for creating tables with such transformation.
Some databases are case-insensitive (MySQL, SQL Server) while others sort textual types differently compared to Trino (PostgreSQL). For such databases pushdown of aggregation functions when the grouping set includes a textual type can lead to incorrect results. So we prevent aggregation pushdown for such cases. We also prevent pushdown for functions whose results depend on sort order (min/max) when the input is a textual type.
They are currently not supported. Previously, Iceberg would use `HiveMetastoreModule` and thus inherited metastore configuration from Hive connector. `IcebergMetastoreModule` existed for validation purposes. Besides thrift and file, Iceberg inherited 'support' for glue (which actually didn't work at runtime) and alluxio metastores (which was not intentional). The commit copies `HiveMetastoreModule` into `IcebergMetastoreModule` so that the latter class defines metastore configuration for Iceberg. The Glue metastore option is currently not supported, but will be added in the future. The Alluxio metastore option is dropped.
We started to hitting trinodb#8719 without any explicit change on our side (the world has changed). Disabling oauth2 to not to distract other areas of development. While, in the same time running investigation what has changed and how to mitigate it.
Before the change, it was not intuitive that `CharValueWriter` is suitable for `varbinary` data. The new name better describes what the writer is doing, and that it doesn't assume slice's contents to represent characters.
It should have been deleted in ced0253.
In order to prevent introducing regressions a test that counts accesses to file system is added. The test differentiate between different files and accesses types. Additioanlly this pr removes queryId from information provided by TrackingFileIoProvider as it is no longer needed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
trinodb#5632