Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Catch2 unit tests OOM when using the JUnit reporter #25

Open
xyzsam opened this issue Jul 31, 2020 · 0 comments
Open

Catch2 unit tests OOM when using the JUnit reporter #25

xyzsam opened this issue Jul 31, 2020 · 0 comments
Labels
bug Something isn't working

Comments

@xyzsam
Copy link
Member

xyzsam commented Jul 31, 2020

The JUnit reporters in Catch2 require the entire test to be run and all data to be buffered in memory before anything is written. For the large tiling tests, this can exceed several GB of RAM usage, causing tests to OOM in continuous integration.

Catch2 has an alternative XML format that, with a bit of work, should be convertible into JUnit XML. Filing this issue to keep track of it. For now we're going to just disable JUnit reports so CI can pass.
@xyzsam xyzsam added the bug Something isn't working label Jul 31, 2020
xyzsam added a commit to xyzsam/smaug that referenced this issue Jul 31, 2020
@xyzsam
tests: Don't use Catch2 JUnit reporters.
40dd185 
JUnit reporters cause the tests to OOM in CI. For now, just capture the
contents of stdout/stderr, as before, and dump them on test failure. In
the future, we can consider converting Catch2 XML reports to JUnit
reports.

Issue harvard-acc#25.
xyzsam added a commit to xyzsam/smaug that referenced this issue Jul 31, 2020
@xyzsam
Use CircleCI for continuous integration testing.
474a639 
The CircleCI configuration includes build, unit testing, and downloading
a recently built gem5-aladdin binary so that we can run gem5 simulations
in the future as part of the CI flow. We don't produce JUnit reports as
that causes the tests to OOM.

Issue harvard-acc#25.

TESTED=verified CircleCI pipeline passes.
@xyzsam xyzsam mentioned this issue Jul 31, 2020
Merged
xyzsam added a commit that referenced this issue Jul 31, 2020
@xyzsam
tests: Migrate from Travis to CircleCI.
c6f246a 
CircleCI will give us better scalability for larger integration tests
than Travis at lower cost. The CircleCI configuration includes build,
unit testing, and downloading a recently built gem5-aladdin binary so
that we can run gem5 simulations in the future as part of the CI flow.
We don't produce JUnit reports as that causes the tests to OOM.

Issue #25.

TESTED=verified CircleCI pipeline passes.
xyzsam added a commit that referenced this issue Jul 31, 2020
@xyzsam
tests: Migrate from Travis to CircleCI. (#26)
27bf495 
CircleCI gives us easier Docker integration and better scalability for larger
integration tests than Travis at lower cost. The CircleCI configuration
includes build, unit testing, and downloading a recently built gem5-aladdin
binary so that we can run gem5 simulations in the future as part of the CI
flow. We don't produce JUnit reports as that causes the tests to OOM.

Issue #25.

TESTED=verified CircleCI pipeline passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@xyzsam