Change dtype of drs4 baseline values to int16 to make it work with pre-corrected EVBv6 data. #1182
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I repeat here what was discussed via email:
In EVBv5, similarly to what is done now, data was shifted up by 400 to allow for negative values after the pre-calibration by EVB.
But in the EVBv5 data, this offset wasn't specified, so it wasn't subtracted automatically. Instead, we computed the relative DRS4 offset coefficients on this shifted data, which resulted in drs4 coefficients around 400, subtracting this offset and
also resulting in drs4 coefficients being positive (we store them as uint16 then).
With the new data format, the offset and scale are included and always applied by the source immediately when reading.
This results in values around 0, so the drs4 coefficient script also computes drs4 offsets around 0.
This would all be fine, if we wouldn't store them as uint16, clipping the values below 0.
So my proposal would be to change to int16 for the drs4 offsets. Since we have a 12 bit ADC, that's not an issue, we still can store all possible values, also for not-yet applied pre-calibration by EVB. The binary representation also won't change, since all values up to now have been positive and < 2**15.
So in short, I think we should adapt the computation script to store values as int16 and issue the warning only in case of non-precalibrated drs4 values.