Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](rowset meta) Fix rowset meta size relation #41022

Merged
merged 3 commits into from
Oct 31, 2024
Merged

[fix](rowset meta) Fix rowset meta size relation #41022

merged 3 commits into from
Oct 31, 2024

Conversation

Yukang-Lian
Copy link
Collaborator

Proposed changes

Issue #40879 step 2

make rowset total size = rowset data size + rowset index size

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Yukang-Lian
Copy link
Collaborator Author

Tests will be added to verify the correction of this PR.

@Yukang-Lian
Copy link
Collaborator Author

run buildall

1 similar comment
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

gavinchou
gavinchou requested changes Sep 19, 2024
View reviewed changes
Copy link
Contributor

@gavinchou gavinchou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls add regression test to ensure the correctness from this PR on

cloud/src/meta-service/meta_service_txn.cpp Show resolved Hide resolved
@github-actions github-actions bot removed approved Indicates a PR has been approved by one committer. labels Sep 19, 2024
@Yukang-Lian Yukang-Lian changed the title [Refactor](rowset meta) Refactor rowset meta size releation [Refactor](rowset meta) Refactor rowset meta size relation Sep 19, 2024
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.32% (9590/25699)
Line Coverage: 28.70% (79264/276147)
Region Coverage: 28.17% (41030/145669)
Branch Coverage: 24.79% (20905/84318)
Coverage Report: http://coverage.selectdb-in.cc/coverage/eb9d0914c4a0d7001bce4b79a05c08475afece22_eb9d0914c4a0d7001bce4b79a05c08475afece22/report/index.html

@zhannngchen
Copy link
Contributor

pls add regression test to ensure the correctness from this PR on

+1

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.26% (9605/25775)
Line Coverage: 28.68% (79521/277234)
Region Coverage: 28.13% (41117/146193)
Branch Coverage: 24.77% (20960/84632)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d2e24449e52101031adb42897de52eb536c64d77_d2e24449e52101031adb42897de52eb536c64d77/report/index.html

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.28% (9607/25773)
Line Coverage: 28.70% (79534/277151)
Region Coverage: 28.14% (41128/146133)
Branch Coverage: 24.78% (20964/84594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fd0aa240aee27f50c09fbe530d76cf0a8b564088_fd0aa240aee27f50c09fbe530d76cf0a8b564088/report/index.html

@Yukang-Lian
Copy link
Collaborator Author

run p0

@gavinchou gavinchou changed the title [Refactor](rowset meta) Refactor rowset meta size relation [fix](rowset meta) Fix rowset meta size relation Sep 25, 2024
gavinchou
gavinchou previously approved these changes Sep 25, 2024
View reviewed changes
w41ter
w41ter previously approved these changes Sep 25, 2024
View reviewed changes
@Yukang-Lian Yukang-Lian dismissed stale reviews from w41ter and gavinchou via a011c26 September 26, 2024 09:43
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.31% (9635/25823)
Line Coverage: 28.70% (79717/277712)
Region Coverage: 28.14% (41224/146514)
Branch Coverage: 24.76% (20993/84776)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6738ab2113c9d357e6ad15b998fcc45b9b65155b_6738ab2113c9d357e6ad15b998fcc45b9b65155b/report/index.html

gavinchou
gavinchou previously approved these changes Sep 27, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.89% (9849/25993)
Line Coverage: 29.15% (82036/281442)
Region Coverage: 28.43% (42331/148919)
Branch Coverage: 25.01% (21504/85976)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a01574636453e5fa42a75555f88c42b7029b8120_a01574636453e5fa42a75555f88c42b7029b8120/report/index.html

@Yukang-Lian
Copy link
Collaborator Author

run p0

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 31, 2024
@gavinchou gavinchou self-requested a review October 31, 2024 06:32
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.92% (9851/25981)
Line Coverage: 29.16% (82064/281418)
Region Coverage: 28.44% (42360/148947)
Branch Coverage: 25.01% (21520/86048)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fac0681dea34b4664a43552ab37fbffb8ef17e15_fac0681dea34b4664a43552ab37fbffb8ef17e15/report/index.html

@Yukang-Lian
Copy link
Collaborator Author

run cloud_p0

@Yukang-Lian
Copy link
Collaborator Author

run external

@Yukang-Lian
Copy link
Collaborator Author

run p0

csun5285
csun5285 reviewed Oct 31, 2024
View reviewed changes
be/src/cloud/cloud_meta_mgr.cpp
total_inverted_index_size += file_size;
}
}
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (rs_meta.tablet_schema()->has_inverted_index())

be/src/olap/tablet.cpp Show resolved Hide resolved
be/src/runtime/load_stream_writer.cpp Show resolved Hide resolved
@gavinchou gavinchou merged commit d1b448f into apache:master Oct 31, 2024
34 of 36 checks passed
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Nov 7, 2024
@Yukang-Lian
[fix](rowset meta) Fix rowset meta size relation (apache#41022)
937d1d6 
Issue apache#40879 step 2
make rowset total size = rowset data size + rowset index size
@Yukang-Lian Yukang-Lian mentioned this pull request Nov 7, 2024
Merged
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Nov 11, 2024
@Yukang-Lian
[fix](rowset meta) Fix rowset meta size relation (apache#41022)
7ff4f70 
Issue apache#40879 step 2
make rowset total size = rowset data size + rowset index size
Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Nov 11, 2024
@Yukang-Lian
[fix](rowset meta) Fix rowset meta size relation (apache#41022)
a6ae649 
Issue apache#40879 step 2
make rowset total size = rowset data size + rowset index size
gavinchou pushed a commit that referenced this pull request Nov 12, 2024
@Yukang-Lian
[cherry-pick](branch-3.0) Pick "[fix](rowset meta) Fix rowset meta si…
b5b8e49 
…ze relation (#41022)" (#43414)

Pick #41022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring left review comments

@csun5285 csun5285 csun5285 requested changes

@bobhan1 bobhan1 bobhan1 approved these changes

@gavinchou gavinchou gavinchou approved these changes

@w41ter w41ter w41ter left review comments

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@Yukang-Lian @doris-robot @zhannngchen @gavinchou @w41ter @bobhan1 @csun5285 @dataroaring