Consider removing err from 4D RampModel

[stscijgbot-jp]

Issue JP-3827 was created on JIRA by Timothy Brandt:

The error array has no meaning for a 4D ramp; only the 2D read noise, the 2D gain, and the 4D DQ flag.  An extra float32 array is being allocated in detector1, filled with zeros, and never used (aside from, currently, a purely time-wasting call in jump detection that is itself never used).  If this attribute were removed, it would save a significant fraction of the Stage 1 memory demand, and I believe there would be no impact to data processing.

A very hacky workaround—forcing the (unused) error array to type uint8 in the stdatamodels .yaml file and commenting out a few lines of Kenneth MacDonald's C code—reduces memory usage substantially throughout Stage 1.  For jw01283001001_03101_00001_mirimage, memory usage in the ramp fitting step drops from 30GB to 21GB with this change; similar reductions are seen throughout Stage 1.  Setting ndim: 1 for the err array in the yaml file and commenting out some additional lines in the ramp fitting code further reduces the steady memory footprint to 18GB in the ramp step.  In all cases, the _rate and _rateints files are identical.  Summary: I think Stage 1 memory usage can be reduced by 30%-40%, runtime can be reduced slightly, and code cleaned up with no effect on results, since the 4D error array isn't a useful concept anyway.

[stscijgbot-jp]

Comment by Timothy Brandt on JIRA:

Quick update: I was able to, I think, find everywhere that err was mentioned or allocated.  I commented them all out and deleted the err entry from the ramp_schema.yaml file.  The Stage 1 pipeline runs and, again, produces identical results.  The code changes are minor: a few dozen lines and a couple of function arguments get deleted.  Unfortunately, these small changes are scattered through three repositories: jwst, stcal, and stdatamodels.  I think that, in order to properly address this, we need to be careful about the order of the changes, and I defer to the judgment of others on the best way forward.  The reductions in memory requirements help open Stage 1 to the typical laptop in a way that has often not been possible, and the amount of work to remove the references to err is modest, so I do think this is a valuable thing to do.  

Edited to add: one more option is to add a line

input.err = np.zeros((2, 2, 2, 2), dtype=np.float32)```
below

<https://github.com/spacetelescope/jwst/blob/ec3304909d54f7c72ad36ed74366267fe8839ed4/jwst/pipeline/calwebb_detector1.py#L80>

It's not very elegant but I think it does realize the memory savings with one line, and it drives home the fact that this array is never used.