Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Hive connector test with HMS #14743

Closed
wants to merge 5 commits into from

Conversation

findepi
Copy link
Member

@findepi findepi commented Oct 25, 2022

Will fix #14741

- assert that schema is not being implicitly created
  - assert also for connectors that do not support table creation
@cla-bot cla-bot bot added the cla-signed label Oct 25, 2022
@findepi
Copy link
Member Author

findepi commented Oct 25, 2022

The test being added here is a bit similar to #8826, except I am not trying to use HDFS. I don't know how to use HDFS reliably outside of product tests.
(The HDFS is still being spin up, this would be improved by #14561)

The test should be runnable in an environment with shared resources, so
test objects need to have randomized names.
The method exists to allow running tests in environments that require a
schema to have certain properties (e.g. location pointing to an S3
bucket). Apply `createSchemaSql()` to places where it was still missing.
Comment on lines +81 to +71
// TODO (https://github.com/trinodb/trino/issues/14745) adding columns may overwrite concurrent addition of columns (or some other operations)
// because adding columns currently consist of "read table, derive new table object, persist" with no locking (pessimistic nor optimistic)
throw new SkipException("The test may or may not fail, because there is a concurrency issue");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@findepi
Copy link
Member Author

findepi commented Oct 26, 2022

prefix extracted to #14771

ci / test (plugin/trino-hive) (pull_request) Cancelled after 60m

:(

@findepi
Copy link
Member Author

findepi commented Oct 27, 2022

It looks like the TestHiveHmsMetastoreMinioConnectorTest is just very slow. Didn't look for the cause yet, but eg

2022-10-25T12:19:40.2659744Z 2022-10-25T07:19:40.238-0500 INFO pool-3-thread-2 io.trino.testng.services.LogTestDurationListener Test io.trino.plugin.hive.TestHiveHmsMetastoreMinioConnectorTest.testPartitionPerScanLimit took 4.71m

generally grep logs for LogTestDurationListener.

@mosabua
Copy link
Member

mosabua commented Jan 11, 2024

Is this still in progress @findepi ?

@findepi findepi closed this Mar 25, 2024
@findepi findepi deleted the findepi/foobar-schema branch March 25, 2024 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Bad error message creating a Hive table in a schema that does not exist
2 participants