Skip to content

Commit

Permalink
[SPARK-49747][CONNECT] Migrate connect/ files to structured logging
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

We are moving one missing piece in SparkConnect to MDC-based logging.

### Why are the changes needed?

As part of the greater migration to structured logging.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Compilation/existing tests.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#48195 from nemanja-boric-databricks/mdc-connect.

Authored-by: Nemanja Boric <nemanja.boric@databricks.com>
Signed-off-by: Herman van Hovell <herman@databricks.com>
  • Loading branch information
nemanja-boric-databricks authored and hvanhovell committed Sep 23, 2024
1 parent fec1562 commit 3b5c1d6
Showing 1 changed file with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import org.apache.commons.lang3.StringUtils

import org.apache.spark.SparkSQLException
import org.apache.spark.connect.proto
import org.apache.spark.internal.Logging
import org.apache.spark.internal.{Logging, LogKeys, MDC}
import org.apache.spark.sql.connect.common.ProtoUtils
import org.apache.spark.sql.connect.planner.SparkConnectPlanner
import org.apache.spark.sql.connect.service.{ExecuteHolder, ExecuteSessionTag, SparkConnectService}
Expand Down Expand Up @@ -113,7 +113,7 @@ private[connect] class ExecuteThreadRunner(executeHolder: ExecuteHolder) extends
} catch {
// Need to catch throwable instead of NonFatal, because e.g. InterruptedException is fatal.
case e: Throwable =>
logDebug(s"Exception in execute: $e")
logDebug(log"Exception in execute: ${MDC(LogKeys.EXCEPTION, e)}")
// Always cancel all remaining execution after error.
executeHolder.sessionHolder.session.sparkContext.cancelJobsWithTag(
executeHolder.jobTag,
Expand Down Expand Up @@ -298,7 +298,7 @@ private[connect] class ExecuteThreadRunner(executeHolder: ExecuteHolder) extends
ProtoUtils.abbreviate(request, maxLevel = 8).toString)
} catch {
case NonFatal(e) =>
logWarning("Fail to extract debug information", e)
logWarning(log"Fail to extract debug information: ${MDC(LogKeys.EXCEPTION, e)}")
"UNKNOWN"
}
}
Expand Down

0 comments on commit 3b5c1d6

Please sign in to comment.