Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac tool_integration_tests_4_4 is 2.00% flaky #97525

Closed
fluttergithubbot opened this issue Jan 31, 2022 · 6 comments
Closed

Mac tool_integration_tests_4_4 is 2.00% flaky #97525

fluttergithubbot opened this issue Jan 31, 2022 · 6 comments
Assignees
Labels
c: flake Tests that sometimes, but not always, incorrectly pass P0 Critical issues such as a build break or regression team-infra Owned by Infrastructure team tool Affects the "flutter" command-line tool. See also t: labels.

Comments

@fluttergithubbot
Copy link
Contributor

The post-submit test builder Mac tool_integration_tests_4_4 had a flaky ratio 2.00% for the past (up to) 100 commits, which is above our 20.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_4_4/3036
Commit: 1c673b2

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_4_4/3036
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_integration_tests_4_4/3021

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac%20tool_integration_tests_4_4

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

@fluttergithubbot fluttergithubbot added P1 c: flake Tests that sometimes, but not always, incorrectly pass tool Affects the "flutter" command-line tool. See also t: labels. labels Jan 31, 2022
@zanderso
Copy link
Member

Both 3036 and 3021 are timeouts. Looking at the history, succeeding runs are just below the timeout.

Going entirely by eyeball, it looks like this shard started taking 27+ minutes following the FFI plugin PR landing: #96225.

@dcharkes did your PR add more integration tests, or more work to existing integration tests?

@christopherfujino Can you tell if any new work added by the FFI plugin PR was added to this integration test shard?

@dcharkes
Copy link
Contributor

Yes, I did add more tests in integration.shard (among other places):

  • packages/flutter_tools/test/integration.shard/android_plugin_example_app_build_test.dart
  • packages/flutter_tools/test/integration.shard/android_plugin_ndkversion_mismatch_test.dart

@zanderso
Copy link
Member

@keyonghan since we know the cause of the timeout, and it is not a flake, can we bump the timeout for this shard while the work to re-shard is scheduled?

@keyonghan
Copy link
Contributor

Seems @christopherfujino has extended the timeout to 45 min in https://github.com/flutter/flutter/pull/97552/files.

@keyonghan
Copy link
Contributor

The builders have been passing consistently with 45 min timeout.
Closing here in favor of #97539 for better sharding.

@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 16, 2022
@keyonghan keyonghan moved this to Done in Infra Ticket Queue May 1, 2023
@ricardoamador ricardoamador moved this from Verified to To be archived in Infra Ticket Queue Jun 26, 2023
@flutter-triage-bot flutter-triage-bot bot added P0 Critical issues such as a build break or regression and removed P1 labels Jun 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
c: flake Tests that sometimes, but not always, incorrectly pass P0 Critical issues such as a build break or regression team-infra Owned by Infrastructure team tool Affects the "flutter" command-line tool. See also t: labels.
Projects
Archived in project
Development

No branches or pull requests

4 participants