-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix major bug in v0.3.1: negative counts #347
Conversation
After running simulated data through v0.3.0 and the master branch, I see that the posterior generated in each case is identical. So despite what I initially thought, it seems like the bug must have been introduced in (It is not a CPU / GPU issue, as a run without |
It came down to a small typo in an integer datatype. I had put Changing to Before After |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice job expanding the tests! Just a small comment to address. Also, while you're at it, you may want to search the rest of the codebase for uint8 and other explicit dtypes and refactor accordingly.
@@ -92,11 +93,12 @@ def test_mean_massive_m(log_prob_coo): | |||
new_shape = (coo.shape[0] + greater_than_max_int32, coo.shape[1]) | |||
new_coo = sp.coo_matrix((coo.data, (new_row, coo.col)), | |||
shape=new_shape) | |||
print(new_coo) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Either remove the print statement, or convert it to something more informative-- e.g. old matrix shape = ..., new matrix shape = ...
Alright @mbabadi , what do you think now? I added a warning to the README, and I added a command-line argument to enable people to re-run their old checkpointed v0.3.1 runs to produce a valid output. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
* Fix major bug in v0.3.1: negative counts (#347)
Closes #306
Closes #348
Something about the update from v0.3.0 to v0.3.1 induced a bug (I'm not yet sure of the scope, but potentially affecting all datasets) where negative counts could be produced in the output. It is assumed that these outputs are totally compromised. v0.3.1 has already been redacted, but the master branch is still compromised.
I think the bug probably lies in the updated posterior generation code, which was the target of memory-saving updates in v0.3.1.
Expand tests covering that code.