Detox is flaky on Travis CI #452
Comments
|
We will regard this as a Detox bug for now until we can determine what the cause is. Since it only reproduces on Travis, it is very hard to debug. Thanks |
|
@rotemmiz Now that we have a visual indicator, we could use a video recording feature to see what is going on. But since Travis doesn't have storage support, so it will be hard. @DanielZlotin Were you able to reproduce locally? |
|
No. Even throttling cpu to the process (1% of total CPU) although made the
tests run locally extremely slow, one minute test, did not reproduce the
flakiness locally. On Travis about 60% of builds fail on a random test,
with the same error message of invalid constraint (something related to
interaction)
…On Dec 6, 2017 19:31, "Leo Natan" ***@***.***> wrote:
@rotemmiz <https://github.com/rotemmiz> Now that we have a visual
indicator, we could use a video recording feature to see what is going on.
But since Travis doesn't have storage support, so it will be hard.
@DanielZlotin <https://github.com/danielzlotin> Were you able to
reproduce locally?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#452 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AGWgj8EChwzseIggrYSH-NuWHOWamhryks5s9s9ngaJpZM4Q38cW>
.
|
|
Sorry @yogevbd, I am unsure have you tried on Circle CI as well? The issue is it most of the time pass, on both CI and local machine. But then Sometimes it just give me the issue with <NSMutableURLRequest...> @LeoNatan I remember you have input in the thread #263 as well. But then I am not sure if this is already solved? Because the thread there is already a month late. thanks. |
|
@LeoNatan |
|
We had a very similar set of errors (failures due to system alerts popping over the app) with our react-native project, except running inside a corporate environment behind a proxy. Something in the app was causing access to the internet (we couldn't work it out at the time, but in retrospect it was possibly an image loaded from a public URL). This was triggering a modal proxy user/pass input system alert view in the simulator which caused interactivity to be lost and the tests to fail. We worked around it by getting a proxy auth exemption as we couldn't work out which network requests we were making, but should have dug deeper probably. Sometimes we wouldn't get the pop-up - not sure whether the proxy was causing the intermittent issues, or something we were doing. Not sure on the nature of the internet access available on Travis CI, or whether all the tests here are correctly mocked, but thought it might help someone dealing with flakey Detox tests caused by system alerts in a constrained CI environment. |
|
I am in the process of making detox work on travis. I constantly get I have got that error locally as well. Maybe because I'm using jest but not mocha. |
|
@yogevbd We had the same issue with travis, ended up using bitrise which is 99% success rate and much faster. |
|
I had some time the : With the help of some screenshots, it appears that this error tell us that the simulator / app is not ready yet, when we try to launch the app. In fact, it seems that detox timeout / resolves I don't know how I can change the default timeout for detox.launchApp(), but after a few tests, here is my current workaround is to try/catch the launchApp() and relaunchApp() after a long sleep. before(async () => {
await detox.init(config, { launchApp: false });
try {
await device.launchApp({ permissions: { notifications: 'YES', calendar: 'YES' } });
} catch (e) {
console.log('It will certainly throw an exception, but lets sleep for 60000ms', e);
await sleep(60000);
await device.launchApp({ permissions: { notifications: 'YES', calendar: 'YES' } });
console.log('hey, sleep is over. Crossed fingers');
}
})
after(async () => {
await deviceUtils.takeScreenshot('after_test');
await detox.cleanup();
})Now my tests are green and here is my log: |
|
@craSH— That’s AppleSimUtils issue. Please open an issue there. |
Description
Tests fail to pass 80% of the time on Travis CI
Does not reproduce locally.
Device and verbose Detox logs
The text was updated successfully, but these errors were encountered: