-
Notifications
You must be signed in to change notification settings - Fork 223
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NUnit 3 / UITest failing with error: Exception has been thrown by the target of an invocation #421
Comments
I'm also having this issue. Is there a known workaround for it ? NUnit 3 is increasingly becoming a blocking feature, as NUnit 2 test runner support is very unstable in VS 2019. |
@jeromelaban @jpc1963 Are you getting the same error if you disable the local screenshots? |
The behavior is the same with or without local screenshots. Also, it might be of interest that it seems that this exception appers on the unit tests side:
|
@jeromelaban Are you using Xamarin.TestCloud.Agent latest version 0.21.8 in your iOS app? |
Yes, exactly this version, and Xamarin.UITest 3.0.0. |
Thanks for confirming. I'll try to reproduce this behavior on my end. Meanwhile, can you try this sample app & UITest project?https://github.com/King-of-Spades/AppCenter-Samples/tree/master/Xamarin.UITest/UITestDemo/iOS - |
The code I'm using is open source, so I can also provide you with a repro. I'll prepare that and test with that. Thanks! |
I've having the exact same issue with your sources. The simulator starts multiple times, then the uitests fail while the app starts. How can I help you troubleshoot that further ? I'm using VS4mac 8.1 Preview (build 2460) |
I get the This will be investigated and someone will let you know when it's been solved but for now I would suggest using VS for Mac stable if possible. |
@JonStoneman Thanks for the followup! Unfortunately, I cannot do this, as there are specifics in the way VS4Mac has improved (particularly with regards to nuget support for cross-targeted projects) that enable key scenarios. It was previously a very complex task for us to test applications that are built inside of a cross-targeted framework (those needed to be built with VS4Win) and those VS4Mac updates enable that. This is the last key piece that will make this UI Testing scenario possible for us. Thanks! |
I am currently running VS for Mac 8.1.5 (Stable Channel) and am experiencing this as well. Is there a fix and/or workaround? Thanks! |
I've been poking at this issue again, and the latest 3.0.1-dev2 along with VS4Mac 8.2 (build 1090) is getting farther along, but is now giving errors such as this one:
|
Also, now that VS 8.2 is out, it'll become critical for this to work. |
@jeromelaban - we're aware of the |
@jeromelaban: UITest 3.0.2-dev1 has been released which should fix the |
@JonStoneman thanks! Actually, the latest I mentioned uses 3.0.1-dev2, so the issue seems to still be present. Preview bits (dev1 included) do not work at all for me (connection issues), dev2 is the only one that goes farther along but fails for other reasons. |
I am experiencing a similar issue, including on Xamarin.UITest 3.0.2-dev1. This exception only appears on Xamarin.iOS; Xamarin.Android is working fine. Reproduction Samplehttps://github.com/brminnick/InvestmentDataSampleApp Stack Trace
Environment=== Visual Studio Enterprise 2019 for Mac === Version 8.2 (build 1105)
=== Mono Framework MDK === Runtime: === NuGet === Version: 5.1.0.6013 === .NET Core === Runtime: /usr/local/share/dotnet/dotnet === Xamarin.Profiler === Version: 1.6.11 === Updater === Version: 11 === Apple Developer Tools === Xcode 10.3 (14492.2) === Xamarin.Mac === Xamarin.Mac not installed. Can't find /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Version. === Xamarin.iOS === Version: 12.14.0.110 (Visual Studio Enterprise) === Xamarin Designer === Version: 16.2.0.354 === Xamarin.Android === Version: 9.4.0.51 (Visual Studio Enterprise) SDK Tools Version: 26.1.1 Build Information: === Microsoft Mobile OpenJDK === Java SDK: /Users/bramin/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25 === Android Device Manager === Version: 1.2.0.58 === Xamarin Inspector === Version: 1.4.3 === Build Information === Release ID: 802001105 === Operating System === Mac OS X 10.14.6 === Enabled user installed extensions === NuGet Package Management Extensions 0.18 |
@briminnick - our current working theory is that this is an issue with |
Thanks @owenniblock! Reverting to Mono v5.18.1.28 fixed it 👍 |
Thanks for letting us know. I've filed an issue on the |
actually they've just closed it on Mono side with this message |
@cheles - I'm on it. My local dev set-up seems unhappy after I installed Xcode 11 so might be Monday before we get a |
Please try the latest |
@owenniblock - I am seeing some improvement using 3.0.3-dev1 (I can now run the REPL on iOS) however I am still seeing the ThreadAbortException. |
Thanks @owenniblock! The |
@adamzucchi I'm not sure if this will resolve your error, but I saw this work-around in the Release Notes for Xamarin.UITest v3.0.3-dev1:
|
@brminnick - Thanks for the suggestion (which I had not previously seen) : ) Unfortunately nothing seems glaringly obvious when looking at the logs. I will report back if I come across anything that seems to have been the culprit. Thanks again! |
@adamzucchi - I'm just looking at this with fresh eyes (I think it's been muddied slightly by additional issues we've seen lately). The bit of code which is failing here is running a command like this:
Do you have access to another development system you could test this on? If we can get a small project which replicates this consistently it would really help us work out what's going on here. If it works on another system, let's try to discount some Xcode weirdness (maybe removing and re-installing Xcode will make a difference). |
I no longer see any "Connection refused" in 3.0.3-dev1 . Thanks @owenniblock & @brminnick |
I am currently running my tests using NUnit 3.12, Xamarin.UITest 3.0.3 |
Hang on. I don't think I tested this properly the last time.
I am using VS2019 for mac (version 8.2.5 build 5). I have mono 6.0.0.327 but I have set 5.18.1.28 as the default (as this was proposed as a workaround). When I run my test (on ios simulator) I get the error: Exception has been thrown by the target of an invocation. This is the reason I created the post originally. Have I missed some fix? |
Hi @jpc1963 - UITest 3.0.3 should be compatible with the latest mono. Can I ask what version of Xcode you're using? And also whether you have Xcode 11 installed (regardless of whether you are using it for this simulator). |
I am using xcode 10.3. xcode 11 is not installed. |
One more update... |
Thanks @jpc1963 - let me record this over here and get someone to try and replicate. Is it possible to provide a sample project using SpecFlow which replicates this for our engineers? |
@owenniblock here is a simple app & ui test. |
Thanks - I'll add this to the ticket. |
Hi, @jpc1963 Thank you for taking the time to help diagnose this issue. I agree that it appears to be a problem with the interaction between Visual Studio and SpecFlow. However, we don't own Specflow and can't fix the issue. I would recommend taking a look at the SpecFlow bug tracker and see if there are similar issues there regarding Visual Studio as well as any potential workarounds. |
Another update.. |
@jpc1963 Your link is set as this: Corrected target: SpecFlowOSS/SpecFlow#1726 From that discussion:
I'm not sure the best way to resolve this "we say ____, they say _____" back and forth with SpecFlow, @Oddj0b or @owenniblock can we confirm if we still feel this is out of our scope and highlight data relevant to that? |
Just to clarify the quote above: The Mac integration for SpecFlow is not maintained by the SpecFlow team, but is a third-party solution. We only maintain the Windows version. So if the issue is in the SpecFlow Visual Studio extension for Mac, there's nothing the SpecFlow team can do about it. That would need to be addressed to the maintainer of the Mac Visual Studio extension project. If the issue is in SpecFlow itself, then we can address it. I hope that makes it clear that Andi was just trying to clarify ownership and why he is unable to comment on the Mac extension. |
@Oddj0b is this problem still being looked at? |
Sorry, to comment on such an old issue but I just seem unable to get SpecFlow working with Xamarin.UITest to work. I have some working UITests that I've set up without SpecFlow but as soon as I add the SpecFlow and SpecFlow.NUnit packages all my tests fail with |
So it turned out to be a compatibility issue between SpecFlow and Xamarin.UITest. Downgrading the SpecFlow nugets solved my issue. A working combination for me right now is:
|
@Jeanjean Thank you so much for the info. I encountered the same issue, but with your help, I now have a working test pipeline, yeah! |
@Jeanjean What version of the Xamarin.UITest nuget are you using with NUnit 3.13.1 and SpecFlow 3.5.14 ? |
@barrymculhane I'm currently on 3.0.15. I've edited my post to also include the Xamarin.UITest version. |
@Jeanjean This solved it for me. Thank you so much. FYI, I also had to downgrade BoDi to version 1.4.1. |
One other point - @Jeanjean solution worked but I also had to change that target framework for the test project to 4.7.2 Note that as of me writing this, I am using the latest versions of Nunit3TestAdapter, Nunit and Xamarin.UITest. The issue is with the version of SpecFlow and SpecFlow.Nunit. It has to be 3.5.14 for both SpecFlow and SpecFlow.Nunit. Bumping it up to anything higher than 3.5.14 causes issue that @Jeanjean reported. Note that there issue 2263 specific to this issue @Jeanjean reported |
The release notes for Xamarin.UITest 3.2.6 say - [SpecFlow] Fixed "VTable setup of type Xamarin.UITest.SpecFlow.Shared.TestTracerDecorator" exception for SpecFlow 3.* And I can confirm my tests are generated and running correctly after updating my nugets to: NUnit 3.13.2 |
What App Center service does this affect?
testing on ios simulators / android emulators
Describe the bug
Hi I have a Xamarin UI test project with tests apps on iOS and Android simulators. I have recently updated nunit & xamarin.uitest to the latest versions. Now when I start my app on a simulator I get the error: Exception has been thrown by the target of an invocation
To Reproduce
I start the app as follows: (the app is started on the simulator - but then the test crashes)
Expected behavior
With the app now visible and running I expect to be able to test it. With the variable myApp I should be able to run my tests. But the error occurs in the ConfigureApp and the variable myApp remains null.
Screenshots
none
Desktop (please complete the following information):
Additional context
Nuget packages:
The following is the log output after trying to start an ios app:
The text was updated successfully, but these errors were encountered: