Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Fix flaky Android emulators #232

Open
ben-xD opened this issue Nov 18, 2021 · 3 comments
Open

CI: Fix flaky Android emulators #232

ben-xD opened this issue Nov 18, 2021 · 3 comments
Assignees
Labels
testing Includes all kinds of automated tests, the way that we run them and the infrastructure around them.

Comments

@ben-xD
Copy link
Contributor

ben-xD commented Nov 18, 2021

CI jobs are randomly failing with Original error: ext.flutter.driver: (112) Service has disappeared. We need to investigate further.

For an example, see https://github.com/ably/ably-flutter/runs/4241734082?check_suite_focus=true

┆Issue is synchronized with this Jira Bug by Unito

@ben-xD ben-xD added the bug Something isn't working. It's clear that this does need to be fixed. label Nov 18, 2021
@ben-xD ben-xD changed the title Fix flaky Android emulators CI: Fix flaky Android emulators Nov 18, 2021
@QuintinWillison QuintinWillison removed the bug Something isn't working. It's clear that this does need to be fixed. label Jan 14, 2022
@QuintinWillison
Copy link
Contributor

Also seen here (Android 21) and here (Android 29).

I notice that we have a comment in our integration testing workflow:

ubuntu-latest cannot be used as it is only a docker container, and unfortunately running hardware acceleration in container is not allowed by android emulator.

Which immediately makes me ask the question "do we need hardware acceleration?". Perhaps we can opt out of that in order to have tests run more reliably... macOS hosts are always less available also, due to GitHub limits.

@ikurek
Copy link
Contributor

ikurek commented Feb 10, 2022

@QuintinWillison It's a matter of performance - without hardware acceleration the emulator is very slow, so the tests will take longer co complete. I've seen this issue raised multiple times on different Ci platforms and usually it's best to stick with MacOS, even the CI step developer has an open issue about this: ReactiveCircus/android-emulator-runner#46

@QuintinWillison
Copy link
Contributor

@ikurek Sorry, I'm only just catching up with some notifications so seen your comment. We've been successfully using ubuntu-latest for just over a month in ably-java's emulation workflow, and my preference (at the current time) is for us to switch all Android virtual device emulation runs to this configuration in order to reduce contention for limited availability of macOS hosts.

To be really clear:

  • Our priority is not performance or speed.
  • Our priority has to be test stability.

There is a possible plan for us to do something to upgrade our capacity to support more simultaneous macOS hosts, but it's not come to anything yet. FYI, @jaley. So, in the meantime, let's switch to Ubuntu and get some more predictability for workflow runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Includes all kinds of automated tests, the way that we run them and the infrastructure around them.
Development

No branches or pull requests

3 participants