Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update xUnit to display output on failures #8147

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Update xUnit to display output on failures #8147

wants to merge 4 commits into from

Conversation

bkhouri
Copy link
Contributor

@bkhouri bkhouri commented Nov 29, 2024 •
edited
Loading

Modify the XCTest xUnit failure message to display the test result.

Motivation:

The generated xUnit XML file is not helpful when tests fail as it contains the message "failure", which is redundant with the test results.

Modifications:

Update the XML output to display the result output instead of static "failure" message. The failure message may contain irrelevant output, but it's better than the current message of failure.

Result:

swift test

Requires: swiftlang/swift#78300

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from b06c5d3 to 20c2b2d Compare November 29, 2024 20:32
@bkhouri bkhouri marked this pull request as ready for review December 2, 2024 12:51
@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 2, 2024

@grynspan : Could I please get your feedback on this change?

@grynspan
Copy link
Contributor

grynspan commented Dec 2, 2024

As we had discussed last week with @plemarquand, this is probably not the approach we want to take.

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch 6 times, most recently from 4f1349c to b8da895 Compare December 3, 2024 04:33
grynspan
grynspan reviewed Dec 3, 2024
View reviewed changes
Sources/_InternalTestSupport/SwiftPMProduct.swift Outdated
@@ -82,7 +82,8 @@ extension SwiftPM {
public func execute(
_ args: [String] = [],
packagePath: AbsolutePath? = nil,
env: Environment? = nil
env: Environment? = nil,
errorIfCommandUnsuccessful: Bool = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure there's a more elegant way to do this, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely. Can you offer a suggestion?

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from b8da895 to 8a1d353 Compare December 12, 2024 23:26
@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 12, 2024

@swift-ci please test

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from 8a1d353 to 91b6163 Compare December 13, 2024 02:06
@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 13, 2024

@swift-ci please test

@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 16, 2024

@swift-ci please test

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from 91b6163 to 12fd457 Compare December 16, 2024 22:22
@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 16, 2024

@swift-ci please test

@bkhouri bkhouri mentioned this pull request Dec 18, 2024
Merged
@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from 12fd457 to 4f29d57 Compare December 20, 2024 01:55
@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 20, 2024

@swift-ci please test

@bkhouri
Copy link
Contributor Author

bkhouri commented Dec 20, 2024

@swift-ci please test linux

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 5, 2025

@swift-ci please test linux

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 10, 2025

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test

@bkhouri bkhouri mentioned this pull request Jan 10, 2025
Merged
@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 10, 2025

@swift-ci please test windows

@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch 2 times, most recently from 0446827 to c7c13a1 Compare January 10, 2025 05:47
@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 10, 2025

@swift-ci Please test

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 10, 2025

@swift-ci please test windows

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 10, 2025

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test

@bkhouri
Update xUnit to display output on failures (XCTest only)
719c979 
For XCTest, the generated xUnit XML file is not helpful when tests fail
as it contains the message "failure", which is redundant with the test
results.

Add a hidden flag (--xunit-message-failure) that will update the XML
output to dipslay the result output instead of static "failure" message.
@bkhouri
Improved the XML escaping and address some comments
152c651
@bkhouri bkhouri force-pushed the t/main/improve_xunit_failure_message branch from c7c13a1 to 152c651 Compare January 13, 2025 16:10
@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 13, 2025

@swift-ci please test

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 13, 2025

@swift-ci please test

grynspan
grynspan reviewed Jan 13, 2025
View reviewed changes
Sources/Commands/SwiftTestCommand.swift
.map { "&#\($0);" }
.joined()
default:
String(character)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: tab went missing

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 15, 2025

@swift-ci please test macOS

@bkhouri
Copy link
Contributor Author

bkhouri commented Jan 15, 2025

@swift-ci please test windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grynspan grynspan grynspan left review comments

@bnbarham bnbarham Awaiting requested review from bnbarham bnbarham is a code owner

@MaxDesiatov MaxDesiatov Awaiting requested review from MaxDesiatov MaxDesiatov is a code owner

@jakepetroules jakepetroules Awaiting requested review from jakepetroules jakepetroules is a code owner

@francescomikulis francescomikulis Awaiting requested review from francescomikulis francescomikulis is a code owner

@xedin xedin Awaiting requested review from xedin xedin is a code owner

@dschaefer2 dschaefer2 Awaiting requested review from dschaefer2 dschaefer2 is a code owner

@shawnhyam shawnhyam Awaiting requested review from shawnhyam shawnhyam is a code owner

@bripeticca bripeticca Awaiting requested review from bripeticca bripeticca is a code owner

@owenv owenv Awaiting requested review from owenv owenv is a code owner

@plemarquand plemarquand Awaiting requested review from plemarquand plemarquand is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@bkhouri @grynspan @dschaefer2 @plemarquand