-
Notifications
You must be signed in to change notification settings - Fork 211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for Xcode 9 attachments. #86
Conversation
@@ -181,6 +182,23 @@ - (BOOL)saveFailedReferenceImage:(UIImage *)referenceImage | |||
identifier:(NSString *)identifier | |||
error:(NSError **)errorPtr | |||
{ | |||
UIImage *diffImage = [referenceImage fb_diffWithImage:testImage]; | |||
|
|||
[XCTContext runActivityNamed:identifier ?: NSStringFromSelector(selector) block:^(id<XCTActivity> _Nonnull activity) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like the user should be able to turn this off or on. If you're running a truly massive amount of snapshot tests, you may not want to do this, as it might make Xcode unusable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm that's a good point. I could easily inflate some of the tests we have locally to fail a couple hundred times if we want but perhaps using the Uber codebase would be a better litmus test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK so I added a loop around a single test that was failing which ran 15000 times and caused 30000 failures which resulted in 90000 attachments.
I will admit that Xcode really chugged when the tests finished. It beach balled and was non-responsive for like 5-10 minutes while it processed all of the attachments. Navigating to test files as well as the test pane in the report navigator does take a bit of time to load but nothing that I would consider blocking.
If I change the loop to run 5000 times, causing 10000 failures and 30000 attachments there is nearly no issues at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I created an issue to track adding the ability to turn this feature off #89
There are four separate PRs (#83, #71, #84, #19) that have attempted to add this feature. Rather than pick a single PR and ask the contributor to update it to sync all of the functionality we decided we would do it ourselves to get this feature out as soon as possible. Thank you @mdiep, @niilohlin, @danieltmbr and @delebedev for your hard work and again I apologize for how long it has taken us to address this.
Across the four PRs there were three unique pieces of functionality that had to be added:
XCTActivity
so the attachments can be grouped together for easy inspection.Below are some screenshots to show what Xcode looks like for these scenarios:
Failed Tests
![failed](https://user-images.githubusercontent.com/535202/59882921-614f9c80-9368-11e9-90a6-0bc76313030b.png)
![recorded](https://user-images.githubusercontent.com/535202/59882922-614f9c80-9368-11e9-82b0-dc8b0f5ecca1.png)
Record Mode