Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Flink] Move partition column fill to native io #254

Merged
merged 9 commits into from
Jun 21, 2023

Conversation

xuchen-plus
Copy link
Contributor

@xuchen-plus xuchen-plus commented Jun 15, 2023

  1. Move filling partition column values to native io
  2. Simplify returned row construction as requested query schema
  3. Fix minicluster shutdown warning in tests
  4. Fix some tests fail occassionally because of not setting ENABLE_CHECKPOINTS_AFTER_TASKS_FINISH
  5. Fix test cases running on Windows

Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
@xuchen-plus xuchen-plus added flink flink support into lakesoul native-io labels Jun 15, 2023
Signed-off-by: chenxu <chenxu@dmetasoul.com>
@xuchen-plus xuchen-plus force-pushed the flink_read_column_proj branch from 95d2e8c to f54981e Compare June 15, 2023 10:20
Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: chenxu <chenxu@dmetasoul.com>
@xuchen-plus xuchen-plus merged commit e85f26b into lakesoul-io:main Jun 21, 2023
@xuchen-plus xuchen-plus deleted the flink_read_column_proj branch June 21, 2023 06:52
F-PHantam pushed a commit to F-PHantam/LakeSoul that referenced this pull request Jul 3, 2023
* fix column proj pushdown in flink

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* use explicit class level minicluster for test

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* use tmp dir property for test table paths

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* fix ci test stream config

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* fix npe in reader

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* fix cdc test verify

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* add flink sink verify in cdc test

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* fix row kind not correctly set in flink reader

Signed-off-by: chenxu <chenxu@dmetasoul.com>

* fix flink stream read cases

Signed-off-by: chenxu <chenxu@dmetasoul.com>

---------

Signed-off-by: chenxu <chenxu@dmetasoul.com>
Co-authored-by: chenxu <chenxu@dmetasoul.com>
Signed-off-by: fphantam <dongf@dmetasoul.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flink flink support into lakesoul native-io
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants