[Code Health] clang-tidy cleanup, part 1

[msiddhu]

Contributes to #2053

Changes

Please provide a brief description of the changes here.

Added the .clang-tidy configuration file. Solved some warnings using auto FIX-IT and some manually. Suppressed some warnings where needed (mostly in test cases, headers, and recursive functions).

Started with configurations that are a good starting point, inspired by Google Cloud clang-tidy and AWS Palace clang-tidy.

More checks could be enabled in further cleanups.

Points to note:

1. A lot of pass by value to const pass by reference.
2. Addition and removal of rvalue references where ever possible and needed.
3. String format changes such as .append instead of + , '\n' instead of std::endl , added missed comma's
4. Removing using namespace which are not used in source code.
5. Added exceptions where it was needed and clang-tidy warnings where thrown.
6. Changing from recursive to iterative ( metrics circular buffer, for the other function (json) used suppress warnings)
7. long to int64 Google Coding conventions

Cleanup in following Areas:

1. API: tests
2. EXAMPLES
3. EXPORTERS
   • Elasticsearch
   • Memory
   • OStream
   • OTLP
   • Prometheus
   • Zipkin
4. HTTP Client and its tests
5. SDK: Traces, Logs, Metrics and their tests.

Please review the differences in indented block as they'll be shown as complete block making it difficult to find differences. Hopefully this cleanup may provide some performance improvements.

Remaining Work:

More cleanup, Integrate in CI .

For significant contributions please make sure you have completed the following items:

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[codecov]

Codecov Report

Attention: Patch coverage is 83.33333% with 7 lines in your changes missing coverage. Please review.

Project coverage is 87.67%. Comparing base (497eaf4) to head (d72b8c8).
Report is 98 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2990      +/-   ##
==========================================
+ Coverage   87.12%   87.67%   +0.55%     
==========================================
  Files         200      190      -10     
  Lines        6109     5853     -256     
==========================================
- Hits         5322     5131     -191     
+ Misses        787      722      -65     

Files	Coverage Δ
exporters/memory/test/in_memory_span_data_test.cc	100.00% <ø> (ø)
...de/opentelemetry/exporters/ostream/span_exporter.h	100.00% <ø> (ø)
exporters/ostream/src/log_record_exporter.cc	97.06% <100.00%> (-0.08%)	⬇️
exporters/ostream/src/metric_exporter.cc	90.91% <ø> (-0.89%)	⬇️
exporters/ostream/src/span_exporter.cc	87.50% <100.00%> (-0.61%)	⬇️
exporters/ostream/test/ostream_span_test.cc	100.00% <100.00%> (ø)
...clude/opentelemetry/sdk/metrics/sync_instruments.h	100.00% <ø> (ø)
...k/include/opentelemetry/sdk/trace/tracer_context.h	100.00% <ø> (ø)
sdk/src/common/global_log_handler.cc	70.84% <100.00%> (-16.66%)	⬇️
sdk/src/logs/event_logger.cc	70.59% <100.00%> (ø)
... and 17 more

... and 100 files with indirect coverage changes

[lalitb]

@msiddhu Can you update the description with the rationale behind these changes? Seems some of the changes in the API could result in ABI incompatibility.

[msiddhu]

I'm trying to add clang-tidy into CI. Creating a cleanup commit. I'm having tough time deciding what clang-tidy checks should be present. We'll be back in a couple of days after testing it.

clang-tidy cleanup: #2053
clang-tidy ci: #2054

[marcalff]

Thanks for the cleanup.

Reopening and setting as draft, we definitively don't want this work to be forgotten ;-)

[marcalff]

@msiddhu Thanks for working on this, this cleanup is much needed.

From experience in similar cleanup for warnings:

• https://github.com/open-telemetry/opentelemetry-cpp/pulls?q=is%3Apr+is%3Aclosed+author%3Amarcalff+warning

the complexity in resolving the cleanup is not so much individual changes, but making sure the overall task makes progress, including with reviews and merges.

Several activities are needed:

• fix existing warnings, sure,
• deliver fixes in workable patches, that can be reviewed,
• implement some CI to detect warnings in the code.

The clang-tidy cleanup will be an iterative process, the best is to do it by small size chunks, instead in one massive PR.

Please proceed as follows:

• (1), do not ever use force-push. This destroys the previous history on the PR, and resets the review to the very beginning. Do several commits instead, which is easier to review, and also easier to rollback if needed. The number of commits does not matter (see [RFC] Poc config yaml #2518), because the PR will be merged with a squash at the end.

• (2), this PR has some merge conflicts, because another cleanup is already in progress, with include-what-you-use, which also touches a lot of files. Please resolve the existing merge conflicts for this PR (it should be trivial). We can hold on on iwyu to avoid generating more conflicts for a while, to give time to clang-tidy to progress.

• (3), and it is a very weird thing to say: the content of the .clang-tidy file (which warning to report) is not important ... at this point. The starting list of settings looks good. What is more important is to merge the .clang-tidy file and have some CI to report warnings, to get the process to bootstrap. Once warnings are reported in CI, subsequent PRs can fix them, and the list of warnings to enforce can grow other time.

• (4), as @lalitb already pointed out, the content of api/* is very sensitive, and will take much longer to review. Because of this, changes touching api/* are better done in a separate PR, to avoid blocking progress for everything else. Note that changes to the api tests, as in api/test/* are ok, that part is not sensitive to ABI breaks, so just keep the fixes already implemented there.

• (5), reviewers need some time to look at changes, so the PR should not constantly change with more fixes, as it delays the review ever more. Since this PR already touches 100 files and 1K LOC, this is a good chunk to look at already, so please do not add more fixes to "part 1". Once part 1 is merged, more fixes can be implemented in a different PR of course.

From a quick look at the PR current content, the review should be fast for this, so I think we can aim to merge this chunk soon. Please resolve the merge conflicts and flag the PR as ready to review.

Again, thanks for taking this up.

[marcalff]

Not sure why CI is stuck. First thing is to resolve conflicts anyway, the next commit should restart CI.

[msiddhu]

@marcalff Thanks for your comments.

1. Noted. I'll avoid force push from now on.
2. Resolved the merge conflicts.
3. Noted. I had a doubt about the wanted to ask about CI, The CI will only run on the files which are touched in the particular PR right? Because running clang-tidy on every file(full project except some components) in every commit will take very long time I feel. At Least for me its taking long time to run clang-tidy.
4. Noted. For this PR I removed the changes in API. Will create a new PR with those changes soon.
5. Noted.

Question: Should the CI for clang-tidy be provided in this PR or new PR for that?

Still one check is failing (Bazel noexcept). I'll resolve that soon and make the PR ready to review.

Please let me know if anything is needed. Thanks.

[msiddhu]

@marcalff
I think the hard CI check using the no-exceptions flag is failing. Please help me with this. What could the solutions be?

Marking it ready to review.

[lalitb]

I think the hard CI check using the no-exceptions flag is failing. Please help me with this. What could the solutions be?

Was there any need to add try-catch for OTELResourceDetector::Detect()? The current CI is strict enough not to allow adding exception in the code - for both API and SDK.

PS - Would be good to discuss in the community meeting if the enforcement should be only for the API (which is required to ensure exceptions don't cross the ABI boundary).

[msiddhu]

Yes, There is a possibility to get exception in OTELResourceDetector::Detect(). I understood the CI.

So, I have added exceptions currently to following:

• Examples: OTLP
• Exporter: Zipkin,
• SDK
  • Logger
  • Metrics - Aggregations
  • Resource - Resource Detector

Please let me know the decision. Happy to make changes. Thanks.

[lalitb]

Are all the compilers wrong here, is the CI not strict enough, or is clang-tidy wrong ?

I think both are correct, the static analysis just goes a step further:

/tmp$ echo "#include <stdexcept>

void bar() {
    throw std::runtime_error(\"An error occurred\");
}

void foo() noexcept {
    bar(); 
}

int main() {
    foo();
    return 0;
}" > ./test_exception.cpp
$
$ g++ -Wall -Wextra -Werror -std=c++17 test_exception.cpp -o example
$ clang-tidy test_exception.cpp -checks='-*,bugprone-exception-escape' -- -std=c++17
2 warnings generated.
/tmp/test_exception.cpp:7:6: warning: an exception may be thrown in function 'foo' which should not throw exceptions [bugprone-exception-escape]
void foo() noexcept {
     ^
/tmp/test_exception.cpp:11:5: warning: an exception may be thrown in function 'main' which should not throw exceptions [bugprone-exception-escape]
int main() {
    ^
$

[marcalff]

Thanks for the cleanup.

This patch contains a lot of very good fixes, and also will improve performances with a lot of object copy that are avoided.

See some changes around CURL, it really needs to use the long type.

About try/catch blocks with exceptions, a bigger discussion is needed with maintainers to see how best to proceed, but given the topic, all the exception related work should probably be done separately.

[msiddhu]

If method foo() is flagged noexcept, and calls method bar() that can raise an exception, I would expect the compiler to raise plenty of errors about that.

noexcept is for compiler optimizations, it's just tell compiler that this method won't throw any exception. But when there is an other function call it may which throws exceptions the function with noexcept will call program to terminate.

Yes, I feel about exception handling we should discuss more as they would reduce performance (Stack unwinding, Control flow).

I'll remove the changes related to try/catch blocks and create a new PR after the discussion.

Thanks @marcalff for taking time to review the long PR. Now I understood what I have done wrong. I'll update PR and have comments some comments soon.

About the previous comments:

1. Noted the comments in [CI] Add a clang-tidy build #2051
2. Running clang-tidy is taking ~20mins (4 core 8th-gen I7) with parallel processing for full source( using the same FIND from format.sh)
   eval $FIND | xargs -P $NUM_PROCESSORS -n 1 bash -c 'run_clang_tidy "$@"' _

run_clang_tidy.sh.txt
build_project.sh.txt

[marcalff]

LGTM, thanks for the cleanup.

[marcalff]

Waiting for:

• tiny fix in zipskin_exporter.cc
• possible comments from other maintainers

and will merge after that.