Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Global Automatic Disaggregated Compaction Service #212

Merged
merged 23 commits into from
May 11, 2023
Merged

Implement Global Automatic Disaggregated Compaction Service #212

merged 23 commits into from
May 11, 2023

Conversation

F-PHantam
Copy link
Contributor

add auto compaction task and fix meta info submit bug when different operations occurrence simultaneous.

close #202

F-PHantam added 20 commits April 13, 2023 18:47
@F-PHantam
add auto compaction task
887b597
@F-PHantam
accurate update and compact data read-write consistency
fe276b7
@F-PHantam
handle readPartition is null case
525ab44
@F-PHantam
deal with readPartition is the current partition case
fcaf4f5
@F-PHantam
Optimize update conflict processing
84be590
@F-PHantam
abstract method
69048ec
@F-PHantam
deal with reading partitions changed bug when compacting all partitions
d5fb220
@F-PHantam
fix data consistence bug when update/compaction contains other versio…
ea741d9 
…n operations between read and write
@F-PHantam
supply ParquetScan condition for compacted partition
6cfad4a
@F-PHantam
avoid compact the same table and partition simultaneously
2b9d115
@F-PHantam
auto compact task deals with partition is null case
225f398
@F-PHantam
meta_init sql add trigger and function
fffbadf
@F-PHantam
consider compaction operation with other middle operation case when u…
40798e0 
…pdate encounter compaction
@F-PHantam
tmp task test
ba777d8
@F-PHantam
Merge branch 'main' into auto_compaction_task
198bd2d
@F-PHantam
update aws config error key
469b298
@F-PHantam
benchmark update url when input args
726731e
@F-PHantam
change conditions to avoid useless compaction execution
2b484a6
@F-PHantam
remove debug code info
872ab2c
@F-PHantam
Merge remote-tracking branch 'origin/main' into auto_compaction_task
5d06171
@F-PHantam F-PHantam added enhancement New feature or request spark spark support into lakesoul labels May 10, 2023
@xuchen-plus xuchen-plus merged commit 13a4f42 into main May 11, 2023
@xuchen-plus xuchen-plus deleted the auto_compaction_task branch May 11, 2023 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xuchen-plus xuchen-plus xuchen-plus approved these changes

@Ceng23333 Ceng23333 Awaiting requested review from Ceng23333

@moresun moresun Awaiting requested review from moresun

@dmetasoul01 dmetasoul01 Awaiting requested review from dmetasoul01

Assignees

@F-PHantam F-PHantam

Labels
enhancement New feature or request spark spark support into lakesoul
Projects
2.3.0 Release
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement Global Automatic Disaggregated Compaction Service
2 participants
@F-PHantam @xuchen-plus