make fuzz vary by rating and over the season

[nicidob]

still has some issues

1. Doesn't disable itself for draft prospects
2. is a binary flag for playoffs. would love for it to be a smooth curve but I couldn't figure out how to do a fast 'games played' query without async/await.
3. can reverse engineer it. Check in playoffs for prior season (no fuzz) -> look at player in new season for that same prior season.
4. Not sure if it's well balanced as is. RAPM is std dev of 2 in +/- terms. Could use that as guideline.

[dumbmatter]

Would need to accept season as a parameter too, right? Otherwise fuzz for past season ratings would be constantly oscillating.

[dumbmatter]

It could default to g.get("season") since in many places it's only the current season, probably.

[nicidob]

But you don't store fuzz per season, do you? That'd need to be added? At which point, better remove the storage for fuzz entirely and make it something you'd want to do deterministically on the fly then (hash pid, season and a per-league salt as the seed to an RNG).

[dumbmatter]

Should just be deleted rather than commented out

[nicidob]

My thinking was I should add tests. Check for decent bounds. Check that the +/- functions behave correctly. I don't think losing tests make sense, especially when the code just got more complicated. I should just write some additional replacement test.

[dumbmatter]

1. Doesn't disable itself for draft prospects

I can do that.

2. is a binary flag for playoffs. would love for it to be a smooth curve but I couldn't figure out how to do a fast 'games played' query without async/await.

Assuming 82 games is enough to get as accurate as we're going to get is fine, I think.

3. can reverse engineer it. Check in playoffs for prior season (no fuzz) -> look at player in new season for that same prior season.

What about this...

Fuzz here is a random number. So if your 35 year old player goes up, you'd assume that 99% of the time it's just fuzz.

What if instead fuzz was a random number picked from a distribution centered around the expected change in ratings (given a player's age)? Factoring in the actual change in ratings. So like if a 35 year old is expected to decline by 5 points, but actually declined by 3, his fuzz would be drawn from a distribution centered around -2.

Another complication - ideally that would be per rating, since different ratings decline differently.

Is this too complicated? It sounds kinda cool though...

[dumbmatter]

Can you sell me on fuzzing on the +/- scale rather than the ovr scale? Is it worth adding that complexity, and having to do something else for football?

[nicidob]

Can you sell me on fuzzing on the +/- scale rather than the ovr scale? Is it worth adding that complexity, and having to do something else for football?

So there's a handful of reasons

1. I think we're more confident in higher quality players, in general. Fuzz should be lower for higher overall players. Giannis and LeBron and Curry are probably more 'certain' in their ratings than Jamal Murray & DeRozen, who are more certain than some bench/role players like Jordan Clarkson or Austin Rivers. I think this is true for ratings as a whole -- it's easier to be confident in a 'great' 3P shooter than knowing exactly how middling someone is. A kind of crappy analogy: it's easier to grade tests that are nearly correct; it's a lot harder to grade the tests from the middling students because of partial credit and the like.
2. The non-linear sigmoid does a lot of this work for you. With some noise -- Great players stay great and players in the middle move around. One way to see the sigmoid is really just a transformation from standard deviations to rank order (logistic is roughly gaussian). So you're taking Curry and saying he's a +3σ player, and then fuzzing that to +2.75σ or +3.25σ) -- which won't change that he's in the top 5% (94% to 96.3%). While taking the mean player (Otto Porter?) and fuzzing him from -0.25σ to +0.25σ will change his result more (43.8% to 56.2%). The fact that I got this sigmoid from fitting a +/- result doesn't really matter for this effect. You can store a sigma and a mean and you'll get very similar results.
3. This +/- sigmoid is actually linear near 70 ovr and nearly flat below 40 overall since its mean is 73ovr. And it came from trying to compute team MOV as a function of the top 10 overalls, and it seems coherent with NBA +/- ideas (minimum is -2. high 50s is 0, below that is negative impact. 70 ovr is around a +5, an 80 would be +8).
4. the fitting results of RAPM do obtain a standard deviation of around 2. Likewise, box score +/- also has a variance in +/- terms. So any estimates of +/- that we see about NBA players have a fuzz/uncertainty in that same scale. We're as sure that LeBron is +5 to +7 as Willie Cauley-Stein is +1 to -1 or Dragan Bender is -1 to -3.
5. You could in theory pick a different sigmoid (like mean = 50, std_dev = 10). Or even a linear function where f(100) = 0 and f(0) = 1 and you scale the fuzz linearly. Using +/- just seemed in line with major stats, but honestly any of those curves would probably work.

[nicidob]

What if instead fuzz was a random number picked from a distribution centered around the expected change in ratings (given a player's age)? Factoring in the actual change in ratings. So like if a 35 year old is expected to decline by 5 points, but actually declined by 3, his fuzz would be drawn from a distribution centered around -2.

That's also be pretty interesting. I like the idea that the uncertainty is drawn from progs distribution. That handles the age factor as well. And it has the property that "known" guys (like Harden) are going to have less uncertainty than unknown guys, although there the variance would be coming from age, not overall quality, it'd have similar gameplay effect of non-uniform fuzz.

[dumbmatter]

After thinking more about it, I think it would ultimately be too confusing to have ratings change over time, but only due to fuzz and nothing else. People would think the actual underlying ratings are changing.

Do you think this PR would still be worthwhile without that part?

[nicidob]

Yeah the idea to have fuzz vary during the season was actually your proposal in discord. I've always been more interested in non-uniform fuzz (or at least uniform in something like "on court impact", where I used +/- as a surrogate)