Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermediate files in outlier_detection clobber each other for MultiSlitModel inputs #8829

Closed
stscijgbot-jp opened this issue Sep 24, 2024 · 6 comments · Fixed by #8782
Closed

Comments

@stscijgbot-jp
Copy link
Collaborator

Issue JP-3762 was created on JIRA by Ned Molter:

When outlier detection runs in during calwebb_spec3 with save_intermediate_results=True and the input to the pipeline is a MultiSlitModel, the filenames for the intermediate (_median, _blot, _outlier_s2d) files do not have any indication of the slit they came from, which means that they clobber one another and only the last slit that was processed is saved.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Ned Molter on JIRA:

Fixed by #8782 

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Ned Molter on JIRA:

David Law it seems to me this is reasonable for INS to test, but I'm happy to close it if you think the testing on the PR was sufficient

@stscijgbot-jp
Copy link
Collaborator Author

Comment by David Law on JIRA:

Tested with 11.2 and confirmed that each source from NIRSpec MSA is now producing its own set of intermediate files.  Spec3 seems to be taking an enormously long time to run though; that's unrelated to this ticket but will file a new one.

Tagging Christian Hayes in case you want to test this too, though I don't think necessary.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Christian Hayes on JIRA:

Closing it is fine with me, I had missed this ticket and thought the name collision issue was related to one of the other outlier detection output issues, but noticed that the naming was fixed when testing the other outlier detection improvements in 11.2.

@emolter
Copy link
Collaborator

emolter commented Jan 14, 2025

@drlaw1558 how were you calling spec3 and on what data? I'm happy to take a look if you link me the ticket you made

@stscijgbot-jp
Copy link
Collaborator Author

Comment by David Law on JIRA:

Ned Molter See https://jira.stsci.edu/browse/JP-3849  My guess is that it's part of a general refactor that spec3 may need for MSA efficiency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants