Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](code-style)add spotless plugin #25033

Merged
merged 18 commits into from
Nov 6, 2023
Merged

Conversation

liugddx
Copy link
Member

@liugddx liugddx commented Oct 1, 2023

Proposed changes

https://lists.apache.org/thread/x0r5lndj1b0dxypm3v2k0dnkvl416szl

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@liugddx
Copy link
Member Author

liugddx commented Oct 1, 2023

run buildall

1 similar comment
@liugddx
Copy link
Member Author

liugddx commented Oct 1, 2023

run buildall

@liugddx liugddx closed this Oct 1, 2023
@liugddx liugddx reopened this Oct 1, 2023
@liugddx
Copy link
Member Author

liugddx commented Oct 1, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.68 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.6 seconds inserted 10000000 Rows, about 349K ops/s
storage size: 17162408950 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.06 seconds
stream load tsv: 579 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162342930 Bytes

@morningman morningman self-assigned this Oct 5, 2023
@liugddx
Copy link
Member Author

liugddx commented Oct 6, 2023

run buildall

@liugddx
Copy link
Member Author

liugddx commented Oct 6, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.65 seconds
stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162467000 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.72 seconds
stream load tsv: 582 seconds loaded 74807831229 Bytes, about 122 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162353310 Bytes

@liugddx
Copy link
Member Author

liugddx commented Oct 7, 2023

run buildall

@liugddx
Copy link
Member Author

liugddx commented Oct 7, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.98 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162279212 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.06 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162374888 Bytes

Copy link
Contributor

@morrySnow morrySnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does new checker's rules TOTALLY same as original checkstyle rules? if not same, please desc what different between two of them. there are too many check to avoid wrong code and ensure easy reading in checkstyle rules, u should impl all of them in new checker before remove checkstyle rules.
furthermore, u should explain the purpose of replacing spotless with checkstyle

@liugddx
Copy link
Member Author

liugddx commented Oct 7, 2023

does new checker's rules TOTALLY same as original checkstyle rules? if not same, please desc what different between two of them. there are too many check to avoid wrong code and ensure easy reading in checkstyle rules, u should impl all of them in new checker before remove checkstyle rules. furthermore, u should explain the purpose of replacing spotless with checkstyle

Other rules can use spotless directly. If you have any other verification rules that you are interested in, please send them to me.

@morrySnow
Copy link
Contributor

checkstyle is not just for code format check. it is also check some code logic check. for example,

  • ensure no fall through branch in switch statement
  • ensure switch statement has default branch
  • name name overload method should put togather
  • must have java doc for long method
  • ...

does spotless do same check now?

@liugddx liugddx marked this pull request as draft October 16, 2023 23:55
@liugddx liugddx dismissed stale reviews from morrySnow and morningman via 7d1f22a November 1, 2023 15:40
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 1, 2023
@liugddx
Copy link
Member Author

liugddx commented Nov 1, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.04 seconds
stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17161971553 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.84 seconds
stream load tsv: 559 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17161945301 Bytes

@morningman
Copy link
Contributor

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.05 seconds
stream load tsv: 570 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162246391 Bytes

@liugddx
Copy link
Member Author

liugddx commented Nov 3, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.47 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162330237 Bytes

@liugddx
Copy link
Member Author

liugddx commented Nov 3, 2023

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.74 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17162120025 Bytes

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 3, 2023
Copy link
Contributor

github-actions bot commented Nov 3, 2023

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit d088cba into apache:master Nov 6, 2023
26 of 27 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 6, 2023
morningman added a commit to morningman/doris that referenced this pull request Nov 7, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 8, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 8, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 13, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 13, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants