Detecting flaky tests with sbt

Introduction

This is a fork of sbt-flaky plugin that’s compatible with sbt versions >=1.4.6 This version removes features like sending reports to Slack and keeps only the basic functionality. It adds however the possibility to run a single test suite. The original project seems to be abandoned by now and this fork removes most of its functionality which is why I decided to keep the fork instead of trying to get this change merged.

Instalation

Add this plugin to either project or global configuration

addSbtPlugin("io.github.piotrbosak" % "sbt-flaky-test" % "0.4.6")

Enable the plugin in `build.sbt`

lazy val myModule = (project in file("."))
  .enablePlugins(FlakyPlugin)

Usage

Run all tests 5 times:

sbt
project myProjectWithTests
flaky times=5 all

Run all tests for 50 minutes:

sbt
project myProjectWithTests
flaky duration=50 all

Run a single test suite until failure:

sbt
project myProjectWithTests
flaky firstFail name=MyTestSpec

Note

• The plugin needs to be enabled for specific sbt project. Enabling it only for the root project is not sufficient.
• flaky command won’t find tests that are not directly in the project it’s being run from e.g. running it in the root project will not find tests present in its children.