Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Make build environment parallel (Windows) #2080

Merged
merged 3 commits into from
Apr 1, 2023
Merged

[CI] Make build environment parallel (Windows) #2080

merged 3 commits into from
Apr 1, 2023

Conversation

cngzhnp
Copy link
Contributor

@cngzhnp cngzhnp commented Mar 30, 2023

Fixes #2077

Changes

Please provide a brief description of the changes here.

For significant contributions please make sure you have completed the following items:

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed

@codecov
Copy link

codecov bot commented Mar 30, 2023 •
edited
Loading

Codecov Report

Merging #2080 (c45edf4) into main (0139cb4) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2080   +/-   ##
=======================================
  Coverage   87.19%   87.19%           
=======================================
  Files         166      166           
  Lines        4784     4784           
=======================================
  Hits         4171     4171           
  Misses        613      613

@cngzhnp cngzhnp force-pushed the fix/make_windows_build_env_parallel branch from 134f5e3 to d63808d Compare March 30, 2023 13:12
@cngzhnp cngzhnp marked this pull request as ready for review March 30, 2023 14:59
@cngzhnp cngzhnp requested a review from a team March 30, 2023 14:59
lalitb
lalitb reviewed Mar 30, 2023
View reviewed changes
ci/do_ci.ps1 Outdated
@@ -40,7 +40,7 @@ switch ($action) {
if ($exit -ne 0) {
exit $exit
}
cmake --build .
cmake --build . -j
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, Will CMake automatically use Ninja as build system, or we need to configure it explicitly?
The build time is reduced from 14 min to 10 min for Windows, so there seems to be some level of parallelism.

Copy link
Member

@owent owent Mar 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add -G Ninja to use ninja to build.
In my understanding, -j also works for Visual Studio Generator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, not automatically. Ninja generator (one of the CMake generators) which needs to be given explicitly to CMake.

Also, I tried to put Ninja as a generator to CMake at first but a few tests are failed due to missing Ninja packages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With my solution (getting number of physical cores from Powershell command), I assume that it takes less time than before, what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it looks good to me. The title of the PR says Ninja, so got confused :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sorry for inconvenience. After discussion on issue, I decided to use Ninja but it does not work at all. So, I will update the PR title and do you want me to change commit message either?

@cngzhnp cngzhnp changed the title Make Windows build environment parallel with help of Ninja Make Windows build environment parallel Mar 30, 2023
@esigo
Copy link
Member

esigo commented Mar 30, 2023

Are we sure we have enough RAM on the runners to use all the cores?

@cngzhnp
Copy link
Contributor Author

cngzhnp commented Mar 30, 2023

Are we sure we have enough RAM on the runners to use all the cores?

What do you mean by that? I do not understand the relation between RAM and parallel build. If you say that, one core must be isolated for other jobs on the system, I can get it but RAM sorry I do not. Could you please elaborate more?

@esigo
Copy link
Member

esigo commented Mar 30, 2023

parallel builds require more RAM for build. The more core you use for build the more RAM you should have. Normal GHA runners can't do more than 3 parallel jobs. I'm not sure what setup we have here.

@lalitb lalitb added the ok-to-merge The PR is ok to merge (has two approves or raised by a maintainer/approver and has one approve) label Apr 1, 2023
@marcalff marcalff merged commit 9bcfcb9 into open-telemetry:main Apr 1, 2023
@marcalff marcalff changed the title Make Windows build environment parallel [CI] Make build environment parallel (Windows) Apr 12, 2023
@chuanchan1116 chuanchan1116 mentioned this pull request Apr 19, 2023
Closed
ays7 added a commit to ays7/opentelemetry-cpp that referenced this pull request May 18, 2023
@ays7
Merge v1.9.0
d44532c 
* commit '7887d32da60f54984a597abccbb0c883f3a51649': (82 commits)
  [RELEASE] Release version 1.9.0 (open-telemetry#2091)
  Use sdk_start_ts for MetricData start_ts for instruments having cumulative aggregation temporality. (open-telemetry#2086)
  [SEMANTIC CONVENTIONS] Upgrade to version 1.20.0 (open-telemetry#2088)
  [EXPORTER] Add OTLP HTTP SSL support (open-telemetry#1793)
  Make Windows build environment parallel (open-telemetry#2080)
  make some hints (open-telemetry#2078)
  Make some targets parallel in CI pipeline (open-telemetry#2076)
  [Metrics SDK] Implement Forceflush for Periodic Metric Reader (open-telemetry#2064)
  Upgraded semantic conventions to 1.19.0 (open-telemetry#2017)
  Bump actions/stale from 7 to 8 (open-telemetry#2070)
  Include directory path added for Zipkin exporter example (open-telemetry#2069)
  Ignore more warning of generated protobuf files than not included in `-Wall` and `-Wextra` (open-telemetry#2067)
  Add `ForceFlush` for all `LogRecordExporter`s and `SpanExporter`s. (open-telemetry#2000)
  Remove unused 'alerting' section from prometheus.yml in examples (open-telemetry#2055)
  Clean warnings in ETW exporters (open-telemetry#2063)
  Fix default value of `OPENTELEMETRY_INSTALL_default`. (open-telemetry#2062)
  [EXPORTER] GRPC endpoint scheme should take precedence over OTEL_EXPORTER_OTLP_TRACES_INSECURE (open-telemetry#2060)
  Fix view names in Prometheus example (open-telemetry#2034)
  Fix some docs typo (open-telemetry#2057)
  Checking indices before dereference (open-telemetry#2040)
  ...

# Conflicts:
#	exporters/ostream/CMakeLists.txt
#	sdk/src/metrics/state/metric_collector.cc
#	sdk/src/metrics/state/temporal_metric_storage.cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@owent owent owent left review comments

@ThomsonTan ThomsonTan ThomsonTan approved these changes

@lalitb lalitb lalitb approved these changes

Assignees
No one assigned
Labels
ok-to-merge The PR is ok to merge (has two approves or raised by a maintainer/approver and has one approve)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Some of CI build targets are not compiled in parallel (Windows)
6 participants
@cngzhnp @esigo @ThomsonTan @lalitb @owent @marcalff