WFSS background subtraction step cannot use user-defined background files

[stscijgbot-jp]

Issue JP-3809 was created on JIRA by Gael Noirot:

During the WFSS background subtraction step of calwebb_spec2, the pipeline is using the WFSS background models found in the crds_cache. For non-WFSS modes, user-defined background model files can be specified in the association files as 'background' members. For WFSS modes, specifying a specific background model file in the association (as 'background' member) doesn’t work because in the code the chosen WFSS background files are hardcoded to be the crds background files, and any 'background' member entry is overwritten. It should be made possible for WFSS modes to specify background files in the association files, otherwise the only way around for a user to use their own background file is to overwrite the files in the crds_cache, which is not ideal.

[stscijgbot-jp]

Comment by David Law on JIRA:

Gael Noirot In this case since WFSS uses backgrounds from a reference file I'd expect that it needs to be changed via the CRDS 'override' parameter instead of via the associations file.  Does it work to specify spec2.background.override_wfssbkg='newfile.fits' instead?

[stscijgbot-jp]

Comment by Rachel Plesha on JIRA:

David Law I thought about that too when Gaël first mentioned this, but I actually still think that even if you are able to specify it directly like that, shouldn't the associations override what's in CRDS? If not, should the capability to specify them in the associations be removed to not confuse folks? I think this is an interesting order of operations thing that is likely more a feature than a bug, but still worth talking about if I understand correctly.

[stscijgbot-jp]

Comment by David Law on JIRA:

I believe it's effectively two different input streams, with no established way for the association input to override the reference file.  It may be simple though to log an error message if a 'BACKGROUND' type exposure is provided in a WFSS association file.  Thoughts Tyler Pauly ?

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

Apologies, I must have deleted this tag notification. Yes, these are two separate data "delivery streams"; for all non-WFSS backgrounds, those files are generated by the observatory and associated through the rules for background matching. For WFSS data, the backgrounds are models stored in reference files, fetched from CRDS by default. Users wishing to supply a custom reference file will need to utilize the override parameters described here: https://jwst-pipeline.readthedocs.io/en/latest/jwst/user_documentation/parameters.html#override-reference-file

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

I don't believe this ticket has any work required - I'm going to close it, but feel free to comment here if you think otherwise.

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

No fix necessary - quirk of WFSS backgrounds being models stored in reference files, rather than exposures gathered by observatory and added as background members to an association.