Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HUDI-5078] Fixing determination of table service for metadata calls #7037

Merged
merged 1 commit into from
Dec 12, 2022
Merged

[HUDI-5078] Fixing determination of table service for metadata calls #7037

merged 1 commit into from
Dec 12, 2022

Conversation

nsivabalan
Copy link
Contributor

Change Logs

While applying commit metadata to metadata table, there is an argument "isTableService" to determine whether the triggering action is a table service or not. We mistakenly considered any replace commit as table service. Right fix is, only clustering is table service and other replace commits are not.

Impact

Metadata compaction will make progress with replace commit actions like INSERT_OVERRIDE after this patch.

Risk level (write none, low medium or high below)

low.

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change

N/A

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@nsivabalan nsivabalan added the priority:major degraded perf; unable to move forward; potential bugs label Oct 22, 2022
@nsivabalan nsivabalan mentioned this pull request Oct 22, 2022
Closed
@nsivabalan nsivabalan changed the title [HUDI-5078] Is table service replace commit [HUDI-5078] Fixing determination of table service for metadata calls Oct 24, 2022
xushiyan
xushiyan reviewed Nov 4, 2022
View reviewed changes
Copy link
Member

@xushiyan xushiyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Have a comment on the API.

hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java Outdated Show resolved Hide resolved
...client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseActionExecutor.java Outdated Show resolved Hide resolved
...di-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java Outdated Show resolved Hide resolved
@nsivabalan nsivabalan force-pushed the isTableServiceReplaceCommit branch from b2d84de to 40a2546 Compare November 7, 2022 07:37
@nsivabalan
Copy link
Contributor Author

@xushiyan : feel free to take another look.

@nsivabalan nsivabalan force-pushed the isTableServiceReplaceCommit branch from 40a2546 to 6b24d08 Compare November 7, 2022 07:39
@apache apache deleted a comment from hudi-bot Nov 8, 2022
@codope codope added the metadata metadata table label Nov 29, 2022
@codope codope force-pushed the isTableServiceReplaceCommit branch from 6b24d08 to 2ce44c5 Compare November 29, 2022 13:39
danny0405
danny0405 requested changes Nov 30, 2022
View reviewed changes
Copy link
Contributor

@danny0405 danny0405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to construct the instant everywhere.

@codope codope added priority:critical production down; pipelines stalled; Need help asap. and removed priority:major degraded perf; unable to move forward; potential bugs labels Dec 1, 2022
@nsivabalan nsivabalan force-pushed the isTableServiceReplaceCommit branch from 2ce44c5 to c3cf14a Compare December 2, 2022 19:59
@nsivabalan
Copy link
Contributor Author

nsivabalan commented Dec 2, 2022 •
edited
Loading

@danny0405 :
I did not want to introduce another overloaded method for

ClusteringUtils.getClusteringPlan(

So, have kept it just for that use-case. have removed elsehwere.

@nsivabalan
Copy link
Contributor Author

@xushiyan @danny0405 : addressed all reviews

@nsivabalan nsivabalan added the release-0.12.2 Patches targetted for 0.12.2 label Dec 6, 2022
xushiyan
xushiyan approved these changes Dec 9, 2022
View reviewed changes
Copy link
Member

@xushiyan xushiyan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only 1 suggestion for code improvement

hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstant.java Show resolved Hide resolved
@nsivabalan nsivabalan force-pushed the isTableServiceReplaceCommit branch from c3cf14a to afb7e32 Compare December 9, 2022 21:42
@hudi-bot
Copy link

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

danny0405
danny0405 approved these changes Dec 12, 2022
View reviewed changes
Copy link
Contributor

@danny0405 danny0405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@nsivabalan nsivabalan merged commit 4e05ca8 into apache:master Dec 12, 2022
nsivabalan added a commit that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (#7037)
38e5a35
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
1c1843d
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
c3ee14d
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
cc3aa8a
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
43c200a
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
2851b9a
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
8a1c332
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
b4021a3
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
7cc42e3
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
7538e7e
alexeykudinkin pushed a commit to onehouseinc/hudi that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
4085f27
alexeykudinkin pushed a commit that referenced this pull request Dec 14, 2022
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (#7037)
1743d05
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Apr 5, 2023
@nsivabalan
[HUDI-5078] Fixing isTableService for replace commits (apache#7037)
51efff3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@codope codope codope left review comments

@xushiyan xushiyan xushiyan approved these changes

@danny0405 danny0405 danny0405 approved these changes

Assignees

@xushiyan xushiyan

Labels
metadata metadata table priority:critical production down; pipelines stalled; Need help asap. release-0.12.2 Patches targetted for 0.12.2
Projects
Hudi PR Support
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@nsivabalan @hudi-bot @xushiyan @danny0405 @codope