Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add RegressionTest error message for when possible non-deterministic behavior is detected. #883

Merged
merged 2 commits into from
Aug 14, 2024
Merged

Add RegressionTest error message for when possible non-deterministic behavior is detected. #883

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
793768e
Add RegressionTest error message for when possible non-deterministic …
Aug 8, 2024
ec19417
Changes from PR feedback.
Aug 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/test/java/emissary/test/core/junit5/RegressionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,7 @@ protected void setupPayload(final IBaseDataObject payload, final Document answer
@Override
protected void checkAnswers(final Document answers, final IBaseDataObject payload,
final List<IBaseDataObject> attachments, final String tname) {
RegressionTestUtil.checkAnswers(answers, payload, actualSimplifiedLogEvents, attachments, place.getClass().getName(), getDecoders());
RegressionTestUtil.checkAnswers(answers, payload, actualSimplifiedLogEvents, attachments, place.getClass().getName(), getDecoders(),
generateAnswers());
}
}
7 changes: 5 additions & 2 deletions src/test/java/emissary/test/core/junit5/RegressionTestUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ public final class RegressionTestUtil {
*/
public static final String THROWABLE_MESSAGE_NAME = "throwableMessage";

public static final String NON_DETERMINISTIC_ERROR_MESSAGE =
"\nNOTE: Since 'generateAnswers' is true, these differences could indicate non-deterministic processing in the tested code path\n";

/**
* XML builder to read XML answer file in
*/
Expand Down Expand Up @@ -119,7 +122,7 @@ public static Path getTestResx() {
}

public static void checkAnswers(final Document answers, final IBaseDataObject payload, final List<SimplifiedLogEvent> actualSimplifiedLogEvents,
final List<IBaseDataObject> attachments, final String placeName, final ElementDecoders decoders) {
final List<IBaseDataObject> attachments, final String placeName, final ElementDecoders decoders, final boolean generateAnswers) {
final Element root = answers.getRootElement();
final Element parent = root.getChild(ANSWERS);

Expand All @@ -130,7 +133,7 @@ public static void checkAnswers(final Document answers, final IBaseDataObject pa
final String differences = PlaceComparisonHelper.checkDifferences(expectedIbdo, payload, expectedAttachments,
attachments, placeName, DIFF_CHECK);

assertNull(differences, differences);
assertNull(differences, generateAnswers ? differences + NON_DETERMINISTIC_ERROR_MESSAGE : differences);

final List<SimplifiedLogEvent> expectedSimplifiedLogEvents = getSimplifiedLogEvents(parent);

Expand Down