Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Update to use CircleCI 2.0 #31

Conversation

blowekamp
Copy link
Member

Change-Id: Ic3880bb5ba0da9572299a4fa66688e5fd36ffab1

@blowekamp
Copy link
Member Author

@thewtex I am trying out the "resource_class" field in CircleCI 2.0. But we are still getting the following message:

Configurable resource class is not enabled in your project. Please contact your CSM person or our support team to whitelist your project.

Any ideas?

@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch 2 times, most recently from cfa5ffa to a8d1b64 Compare August 29, 2017 19:29
@thewtex
Copy link
Member

thewtex commented Aug 29, 2017

@blowekamp thanks for the update!

I will check with CircleCI support.

@thewtex
Copy link
Member

thewtex commented Aug 30, 2017

@blowekamp please re-try the build.

@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch 2 times, most recently from e4db7cb to 3a452f7 Compare August 30, 2017 13:15
@blowekamp
Copy link
Member Author

Disabled CircleCI for blowekamp/ITK, enable "Build forked pull requests" for InsightSoftwareConsortium/ITK. Now we are getting the following error message:

The machine executor doesn't support configurable resource class.

Progress....

@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch 2 times, most recently from b4aa242 to 9151b25 Compare August 30, 2017 16:22
@thewtex
Copy link
Member

thewtex commented Aug 30, 2017

Getting there.

A few ideas:

  • resource_class may mean distcc is no longer required
  • to use distcc with CircleCI 2.0, the CircleCI Workflow will likely need to be configured
  • ctest -VV will give information and possibly avoid timeout
  • output timeout is configurable with CircleCI 2.0 and can be extended

@blowekamp
Copy link
Member Author

Thank you for the suggestions.

Yes, the goal is to use no longer use distcc and just use one "container". The circleci.cmake file in the dashboard branch was always adding distcc as a launcher, which required distcc configuration and problems. I have patched it on my repo only use distcc when configured. This is used with the most current try.

I'm going to extent the no output timeout for the run command. However, I have concerns that the timeout was due to running out of memory during linking or smoother resource limitation with parallelization.

progress...

@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch 2 times, most recently from 460dd35 to 2a5459a Compare August 30, 2017 18:09
@blowekamp
Copy link
Member Author

running nproc via ssh on the container yielded 32, and ninja was doing it's best to used all 32 cores. The timeout was likely due to termination of the processes by a job manager. I have manually added a -j 5 to the build command.

@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch 5 times, most recently from 5a2d55f to 1e9e2e0 Compare August 31, 2017 15:40
@blowekamp
Copy link
Member Author

The last build was successful. Just tweaking the JUnit reporting.

With the "large" resource of 4 cores and 8 GB of memory we are able to build and test ITK in ~45 min, compares to 1h37m with 3 containers and distcc.

This looks like a good place to start with the CircleCI 2.0 testing, so can merge and make use of additional features such as a custom docker image, or cache latter.

What do you think?

Change-Id: Ic3880bb5ba0da9572299a4fa66688e5fd36ffab1
@blowekamp blowekamp force-pushed the UpdateToCircleCI2.0 branch from 1e9e2e0 to 3711c99 Compare August 31, 2017 16:58
@thewtex
Copy link
Member

thewtex commented Aug 31, 2017

Fantastic! Yes, this is great.

@blowekamp
Copy link
Member Author

#32 Must be merge first to the dashboard branch, before this.

@thewtex
Copy link
Member

thewtex commented Sep 6, 2017

@thewtex thewtex closed this Sep 6, 2017
ihnorton pushed a commit to ihnorton/ITK that referenced this pull request Mar 2, 2018
Francois Budin (4):
      BUG: Remove CMAKE_INSTALL_PREFIX from install directory
      ENH: Improve usage of JAVA_HOME and SCIFIO_PATH environment variables
      ENH: Uses https instead of http to download jre and *.jars
      ENH: Remove extra slash (or backslash) in paths to facilitate concatenation

Matt McCormick (3):
      Merge pull request InsightSoftwareConsortium#31 from fbudin69500/remove_CMAKE_INSTALL_PREFIX
      Merge pull request InsightSoftwareConsortium#32 from fbudin69500/ENH_Uses_https_to_download_jre_and_jars
      Merge pull request InsightSoftwareConsortium#33 from fbudin69500/Improve_Path_handling

Change-Id: I407603553947c5c88e0c29b6db4f3990217b6936
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants