Skip to content

Commit

Permalink
[SPARK-49142][FOLLOWUP][CONNECT][PYTHON] Use PySparkLogger for Spark …
Browse files Browse the repository at this point in the history
…Connect client log

### What changes were proposed in this pull request?

This followups for #47647 to switch the standard logger into PySparkLogger for Spark Connect client log

### Why are the changes needed?

Use PySpark specific logger

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?
CI

### Was this patch authored or co-authored using generative AI tooling?

No

Closes #47822 from itholic/update_pyspark_logger.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
itholic authored and HyukjinKwon committed Aug 21, 2024
1 parent ae05852 commit 7f1a0c6
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion python/pyspark/sql/connect/client/logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@


import logging
from pyspark.logger import PySparkLogger
import os
from typing import Optional

Expand All @@ -27,7 +28,7 @@

def _configure_logging() -> logging.Logger:
"""Configure logging for the Spark Connect clients."""
logger = logging.getLogger(__name__)
logger = PySparkLogger.getLogger(__name__)
handler = logging.StreamHandler()
handler.setFormatter(
logging.Formatter(fmt="%(asctime)s %(process)d %(levelname)s %(funcName)s %(message)s")
Expand Down

0 comments on commit 7f1a0c6

Please sign in to comment.