Skip to content

Commit

Permalink
[SPARK-49523][CONNECT] Increase maximum wait time for connect server …
Browse files Browse the repository at this point in the history
…to come up for testing

### What changes were proposed in this pull request?
This PR increases the max time we wait for a connect server to come up for testing. The current threshold is too low, and is causing flakyness.

### Why are the changes needed?
It makes connect tests less flaky.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
It is test infra code.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#47994 from WweiL/deflake-will-it-work.

Authored-by: Wei Liu <wei.liu@databricks.com>
Signed-off-by: Herman van Hovell <herman@databricks.com>
  • Loading branch information
WweiL authored and hvanhovell committed Sep 5, 2024
1 parent 9a7b6e5 commit 182353d
Showing 1 changed file with 7 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ import java.util.concurrent.TimeUnit
import scala.concurrent.duration.FiniteDuration

import org.scalatest.{BeforeAndAfterAll, Suite}
import org.scalatest.concurrent.Eventually.eventually
import org.scalatest.concurrent.Futures.timeout
import org.scalatest.time.SpanSugar._

import org.apache.spark.SparkBuildInfo
import org.apache.spark.sql.SparkSession
Expand Down Expand Up @@ -184,12 +187,14 @@ object SparkConnectServerUtils {
.port(port)
.retryPolicy(RetryPolicy
.defaultPolicy()
.copy(maxRetries = Some(7), maxBackoff = Some(FiniteDuration(10, "s"))))
.copy(maxRetries = Some(10), maxBackoff = Some(FiniteDuration(30, "s"))))
.build())
.create()

// Execute an RPC which will get retried until the server is up.
assert(spark.version == SparkBuildInfo.spark_version)
eventually(timeout(1.minute)) {
assert(spark.version == SparkBuildInfo.spark_version)
}

// Auto-sync dependencies.
SparkConnectServerUtils.syncTestDependencies(spark)
Expand Down

0 comments on commit 182353d

Please sign in to comment.