[SPARK-48560][SS][PYTHON] Make StreamingQueryListener.spark settable

[HyukjinKwon]

What changes were proposed in this pull request?

This PR proposes to make StreamingQueryListener.spark settable

Why are the changes needed?

from pyspark.sql.streaming.listener import StreamingQueryListener

class MyListener(StreamingQueryListener):
  def __init__(self, spark):
    self.spark = spark

  def onQueryStarted(self, event):
    pass

  def onQueryProgress(self, event):
    pass

  def onQueryTerminated(self, event):
    pass

MyListener(spark)

is broken from 3.5.0 after SPARK-42941.

Does this PR introduce any user-facing change?

Yes, end users who implement StreamingQueryListener can add spark attribute in their implementation.

How was this patch tested?

Manually tested, and added a unittest.

Was this patch authored or co-authored using generative AI tooling?

No.

[WweiL]

Thanks for the fix!

[WweiL]

Pending fixing lint

[HyukjinKwon]

Will fix up the test

[HyukjinKwon]

Merged to master.

[rangadi]

This is awesome. Thanks @HyukjinKwon .