You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Ramp Fitting step calculates incorrect slopes and uncertainties when an integration has more than 256 integrations. The code uses an 8 bit counter to track the segments between jumps. When the length of the jump-free region is longer than 256 the counter rolls over to zero. The net effect is that the number of groups used for the ramp fitting is mod 256 of the number of groups.
The fix requires changing the data type of three variables from 8 bit to 16 bit.
The text was updated successfully, but these errors were encountered:
Issue JP-3583 was created on JIRA by Michael Regan:
The Ramp Fitting step calculates incorrect slopes and uncertainties when an integration has more than 256 integrations. The code uses an 8 bit counter to track the segments between jumps. When the length of the jump-free region is longer than 256 the counter rolls over to zero. The net effect is that the number of groups used for the ramp fitting is mod 256 of the number of groups.
The fix requires changing the data type of three variables from 8 bit to 16 bit.
The text was updated successfully, but these errors were encountered: