Skip to content

Commit

Permalink
[SPARK-5099][Mllib] Simplify logistic loss function
Browse files Browse the repository at this point in the history
This is a minor pr where I think that we can simply take minus of `margin`, instead of subtracting  `margin`.

Mathematically, they are equal. But the modified equation is the common form of logistic loss function and so more readable. It also computes more accurate value as some quick tests show.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #3899 from viirya/logit_func and squashes the following commits:

91a3860 [Liang-Chi Hsieh] Modified for comment.
0aa51e4 [Liang-Chi Hsieh] Further simplified.
72a295e [Liang-Chi Hsieh] Revert LogLoss back and add more considerations in Logistic Loss.
a3f83ca [Liang-Chi Hsieh] Fix a bug.
2bc5712 [Liang-Chi Hsieh] Simplify loss function.
  • Loading branch information
viirya authored and mengxr committed Jan 7, 2015
1 parent bb38ebb commit e21acc1
Showing 1 changed file with 9 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,17 @@ class LogisticGradient extends Gradient {
val gradientMultiplier = (1.0 / (1.0 + math.exp(margin))) - label
val gradient = data.copy
scal(gradientMultiplier, gradient)
val minusYP = if (label > 0) margin else -margin

// log1p is log(1+p) but more accurate for small p
// Following two equations are the same analytically but not numerically, e.g.,
// math.log1p(math.exp(1000)) == Infinity
// 1000 + math.log1p(math.exp(-1000)) == 1000.0
val loss =
if (label > 0) {
math.log1p(math.exp(margin)) // log1p is log(1+p) but more accurate for small p
if (minusYP < 0) {
math.log1p(math.exp(minusYP))
} else {
math.log1p(math.exp(margin)) - margin
math.log1p(math.exp(-minusYP)) + minusYP
}

(gradient, loss)
Expand Down

0 comments on commit e21acc1

Please sign in to comment.