-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-13686][MLLIB][STREAMING] Add a constructor parameter reqParam
to (Streaming)LinearRegressionWithSGD
#11527
Conversation
@@ -141,7 +143,7 @@ object LinearRegressionWithSGD { | |||
stepSize: Double, | |||
miniBatchFraction: Double, | |||
initialWeights: Vector): LinearRegressionModel = { | |||
new LinearRegressionWithSGD(stepSize, numIterations, miniBatchFraction) | |||
new LinearRegressionWithSGD(stepSize, numIterations, 0.0, miniBatchFraction) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the same way with LogisticRegressionWithSGD
. The signature of train
function is not changed.
I think that's good, but would like @jkbradley or @mengxr to approve |
Jenkins test this please |
Test build #52514 has finished for PR 11527 at commit
|
Hi, @srowen . Jenkins, retest this please. |
Oh, sorry. I found that. Thanks. :) |
retest this please |
2 similar comments
retest this please |
retest this please |
Jenkins add to whitelist |
Jenkins test this please |
Thank you, @srowen ! |
Test build #52530 has finished for PR 11527 at commit
|
Hi, @mengxr and @jkbradley . |
Rebased. |
Test build #52870 has finished for PR 11527 at commit
|
@mengxr @jkbradley was the intent that you just set the reg param on the optimizer object? or do you agree this should be exposed as a setter? I am not sure enough to know this should be added. |
Right, as you said, we can use optimizer object directly. I thought that's the main reason why this PR doesn't get any response from ML part committers. At the first time, what I focused was the consistency of API among algorithms. Other algorithms having But, what I'm not sure here is Spark MLLib's direction. As you said, the use of optimizer is more recommended way now and in the future. I think this PR should be closed. If I can get some clear advice for this before closing this PR, I'll really be happy. :) |
Test build #53030 has finished for PR 11527 at commit
|
…` to (Streaming)LinearRegressionWithSGD `LinearRegressionWithSGD` and `StreamingLinearRegressionWithSGD` does not have `regParam` as their constructor arguments. They just depends on GradientDescent's default reqParam values. To be consistent with other algorithms, we had better add them.
Test build #53031 has finished for PR 11527 at commit
|
We discussed this during the pipeline API design and agreed that we shouldn't separate model hyper-parameters from training parameters. So this is the right place to set |
@@ -318,6 +318,9 @@ object MimaExcludes { | |||
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.mllib.evaluation.MultilabelMetrics.this"), | |||
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.predictions"), | |||
ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.predictions") | |||
) ++ Seq( | |||
// [SPARK-13686][MLLIB][STREAMING] Add a constructor parameter `reqParam` to (Streaming)LinearRegressionWithSGD |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor: reg
instead of req
.
Merged into master. Thanks! Btw, it should be |
Oh, thank you so much for your attention/reviewing/merging, @mengxr ! |
…` to (Streaming)LinearRegressionWithSGD ## What changes were proposed in this pull request? `LinearRegressionWithSGD` and `StreamingLinearRegressionWithSGD` does not have `regParam` as their constructor arguments. They just depends on GradientDescent's default reqParam values. To be consistent with other algorithms, we had better add them. The same default value is used. ## How was this patch tested? Pass the existing unit test. Author: Dongjoon Hyun <dongjoon@apache.org> Closes apache#11527 from dongjoon-hyun/SPARK-13686.
…` to (Streaming)LinearRegressionWithSGD ## What changes were proposed in this pull request? `LinearRegressionWithSGD` and `StreamingLinearRegressionWithSGD` does not have `regParam` as their constructor arguments. They just depends on GradientDescent's default reqParam values. To be consistent with other algorithms, we had better add them. The same default value is used. ## How was this patch tested? Pass the existing unit test. Author: Dongjoon Hyun <dongjoon@apache.org> Closes apache#11527 from dongjoon-hyun/SPARK-13686.
What changes were proposed in this pull request?
LinearRegressionWithSGD
andStreamingLinearRegressionWithSGD
does not haveregParam
as their constructor arguments. They just depends on GradientDescent's default reqParam values.To be consistent with other algorithms, we had better add them. The same default value is used.
How was this patch tested?
Pass the existing unit test.