Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix problem of dropping bad_times in spec file and y-scale divergence #130

Merged
merged 6 commits into from
Oct 5, 2022

Conversation

taldcroft
Copy link
Member

@taldcroft taldcroft commented Oct 4, 2022

Description

This fixes the issue noted on slack that after #123, any bad times in a model spec file that were not covered by the fitting range in xija_gui_fit would be dropped from the spec file.

This PR makes self.bad_times be an unchanged copy of the original spec file value. Then after computing the indices (into the model times) of the intersecting bad times, only the indices get carried around on the object. Where the times are needed they are regenerated by indexing into model.times.

This also fixes (3f45047) an unrelated problem where each new bad time region that was plotted was expanding the y-axis limit by a multiplicative factor, leading to an exponential divergence in the y-scale with many bad regions. It isn't clear why this wasn't showing up a long time ago.

Interface impacts

None.

Testing

Unit tests

  • Mac

Functional tests

Dropped bad times

I made a copy of the flight aca_spec.json file and added a new bad time interval (not strictly necessary but I did it):

        [
            "2020:165:14:10:00",
            "2020:170:00:00:00"
        ]

Then using both master branch and this branch, I did:

python -m xija.gui_fit.app aca_spec.json --stop 2020:190 --days 60

In both cases:

  • Confirm seeing the two expected bad time intervals marked in light blue on the plot.
  • Confirm that masked values are hidden in the residual plot.
  • Freeze all parameters and thaw a couple of solar heat P parameters
  • Fit and confirm things look OK with no obvious influence of the bad time intervals.
  • Save spec file as JSON

Using the master branch I saw that all the saved bad times outside of the fit range were missing.
Using this branch I saw that the saved bad times matched exactly the original bad times.

Y-axis scaling

Using the MUPS model with thousands of bad regions with the master branch shows the divergence in y-scale with each new region that gets added to the plot. Running the same with this branch now shows the expected plot with every bad region having the same Y size.

python -m xija.gui_fit.app --stop 2022:050 --days 20 pm2thv1t_spec.json --set-data="mups0=100"

I also tested manually adding an "ignore" filter and this worked OK.

@taldcroft taldcroft requested a review from jzuhone October 4, 2022 18:55
@taldcroft taldcroft changed the title Fix problem of dropping bad_times in spec file Fix problem of dropping bad_times in spec file and y-scale divergence Oct 5, 2022
Copy link
Contributor

@matthewdahmer matthewdahmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested via the MUPS2A model (pm2thv1t_spec_matlab.json) by performing a model fit for only the last 100 days, writing out the resulting model spec to a file, and ensuring the bad times for dates prior to the last 100 days persist.

@jzuhone jzuhone merged commit fe30216 into master Oct 5, 2022
@taldcroft taldcroft deleted the fix-dropping-bad-times branch October 5, 2022 12:55
This was referenced Oct 5, 2022
@javierggt javierggt mentioned this pull request Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants