[Failing Test]: Spark runner broken logging overflow

[Abacn]

What happened?

Since Jun 1, 2023 all spark tests failing due to logger Overflow:

java.lang.StackOverflowError
	at org.slf4j.impl.JDK14LoggerFactory.getLogger(JDK14LoggerFactory.java:58)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:363)
	at org.slf4j.bridge.SLF4JBridgeHandler.getSLF4JLogger(SLF4JBridgeHandler.java:202)
	at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:293)
	at java.util.logging.Logger.log(Logger.java:738)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:582)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:632)
	at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:221)
	at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:303)
	at java.util.logging.Logger.log(Logger.java:738)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:582)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:632)
	at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:221)
	at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:303)
	at java.util.logging.Logger.log(Logger.java:738)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:582)
	at org.slf4j.impl.JDK14LoggerAdapter.log(JDK14LoggerAdapter.java:632)
	at org.slf4j.bridge.SLF4JBridgeHandler.callLocationAwareLogger(SLF4JBridgeHandler.java:221)
	at org.slf4j.bridge.SLF4JBridgeHandler.publish(SLF4JBridgeHandler.java:303)
	at java.util.logging.Logger.log(Logger.java:738)

check the history of https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Java_Examples_Spark/

all 4 commits between the last green and first red run seems unrelated:

f5e17e3
3dd239e
538c7bd
e1b4ed3

Is it due to transitive dependency change?

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Abacn]

A bisect shows the cause is #26834

git checkout e1b4ed355da518c49aa2971ba5ea4d079055b24e
./gradlew :runners:spark:3:examplesIntegrationTest --tests org.apache.beam.examples.WordCountIT.testE2EWordCount

org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
    java.lang.StackOverflowError at JDK14LoggerFactory.java:58

git checkout 69c58a9e5ee972c0c49c86dfe974657e9eec70a5
./gradlew :runners:spark:3:examplesIntegrationTest --tests org.apache.beam.examples.WordCountIT.testE2EWordCount

BUILD SUCCESSFUL in 23s

[Abacn]

CC: @chamikaramj

[chamikaramj]

Looking

[chamikaramj]

Revert: #26990

[chamikaramj]

Actually, have a proper fix now: #27001