[SPARK-7383][ML] Feature Parity in PySpark for ml.features

[brkyvz]

Implemented python wrappers for Scala functions that don't exist in ml.features

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32156 has started for PR 5991 at commit bd39fd2.

[SparkQA]

Test build #32156 has finished for PR 5991 at commit bd39fd2.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class Binarizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class IDF(JavaEstimator, HasInputCol, HasOutputCol):
  • class IDFModel(JavaModel):
  • class Normalizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol):
  • class PolynomialExpansion(JavaTransformer, HasInputCol, HasOutputCol):
  • class StandardScaler(JavaEstimator, HasInputCol, HasOutputCol):
  • class StandardScalerModel(JavaModel):
  • class StringIndexer(JavaEstimator, HasInputCol, HasOutputCol):
  • class StringIndexerModel(JavaModel):
  • class Tokenizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol):
  • class Word2Vec(JavaEstimator, HasStepSize, HasMaxIter, HasSeed, HasInputCol, HasOutputCol):
  • class Word2VecModel(JavaModel):
  • class HasSeed(Params):
  • class HasTol(Params):
  • class HasStepSize(Params):

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32156/
Test PASSed.

[mengxr]

minor: I'm not sure which one is the recommended approach to create a DataFrame. @rxin

df = sc.parallelize([Row(values=0.5)]).toDF()

vs.

df = sqlContext.createDataFrame([(0.5,)], ["values"]) # don't need to import Row

[rxin]

I prefer the 2nd approach

[mengxr]

@brkyvz Thanks for working on this! It looks good except the variable naming in the doctests. It seems that RegexTokenizer is missing from the list. Could you add it as well?

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32240 has started for PR 5991 at commit adcca55.

[mengxr]

LGTM pending Jenkins.

[SparkQA]

Test build #32240 has finished for PR 5991 at commit adcca55.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class Binarizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class IDF(JavaEstimator, HasInputCol, HasOutputCol):
  • class IDFModel(JavaModel):
  • class Normalizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class OneHotEncoder(JavaTransformer, HasInputCol, HasOutputCol):
  • class PolynomialExpansion(JavaTransformer, HasInputCol, HasOutputCol):
  • class RegexTokenizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class StandardScaler(JavaEstimator, HasInputCol, HasOutputCol):
  • class StandardScalerModel(JavaModel):
  • class StringIndexer(JavaEstimator, HasInputCol, HasOutputCol):
  • class StringIndexerModel(JavaModel):
  • class Tokenizer(JavaTransformer, HasInputCol, HasOutputCol):
  • class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol):
  • class Word2Vec(JavaEstimator, HasStepSize, HasMaxIter, HasSeed, HasInputCol, HasOutputCol):
  • class Word2VecModel(JavaModel):
  • class HasSeed(Params):
  • class HasTol(Params):
  • class HasStepSize(Params):

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32240/
Test PASSed.

[mengxr]

Merged into master and branch-1.4. Thanks!