[SPARK-45742][CORE][CONNECT][MLLIB][PYTHON] Introduce an implicit function for Scala Array to wrap into immutable.ArraySeq.
#43607
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
LuciferYang
changed the title
immutable.ArraySeq.immutable.ArraySeq.
LuciferYang
changed the title
immutable.ArraySeq.immutable.ArraySeq.
LuciferYang
commented
Nov 1, 2023
| * Wraps an Array[T] as an immutable.ArraySeq[T] without copying. | ||
| */ | ||
| def toImmutableArraySeq: immutable.ArraySeq[T] = | ||
| if (xs eq null) null |
There was a problem hiding this comment.
Welcome to Scala 2.13.12 (OpenJDK 64-Bit Server VM, Java 17.0.8).
Type in expressions for evaluation. Or try :help.
scala> val a: Array[Int] = null
val a: Array[Int] = null
scala> scala.collection.immutable.ArraySeq.unsafeWrapArray(a)
val res0: scala.collection.immutable.ArraySeq[Int] = null
LuciferYang
commented
Nov 1, 2023
| import org.apache.spark.SparkFunSuite | ||
| import org.apache.spark.util.ArrayImplicits._ | ||
|
|
||
| class ArrayImplicitsSuite extends SparkFunSuite { |
There was a problem hiding this comment.
Although this pr has modified the utils module, looking at the test parameters -Phadoop-3 repl/test avro/test examples/test, it did not execute the tests of the utils module, so it seems that the test of the utils module has not been actually verified on GA, put ArrayImplicitsSuite in the core module for now.
The issue with the utils module test will be followed up separately.
|
Could you help review this pr if you have time. @dongjoon-hyun Thanks ~ |
dongjoon-hyun
approved these changes
Nov 2, 2023
|
Merged to master. |
|
Thanks @dongjoon-hyun ~ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Currently, we need to use
immutable.ArraySeq.unsafeWrapArray(array)to wrap an Array into animmutable.ArraySeq, which makes the code look bloated.So this PR introduces an implicit function
toImmutableArraySeqto make it easier for Scala Array to be wrapped intoimmutable.ArraySeq.After this pr, we can use the following way to wrap an array into an
immutable.ArraySeq:At the same time, this pr replaces the existing use of
immutable.ArraySeq.unsafeWrapArray(array)with the new method.On the other hand, this implicit function will be conducive to the progress of work SPARK-45686 and SPARK-45687.
Why are the changes needed?
Makes the code for wrapping a Scala Array into an
immutable.ArraySeqlook less bloated.Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass GitHub Actions
Was this patch authored or co-authored using generative AI tooling?
No