Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to override obs_id in SimTelEventSource #2411

Merged
merged 1 commit into from
Oct 25, 2023
Merged

Conversation

maxnoe
Copy link
Member

@maxnoe maxnoe commented Oct 19, 2023

Simulations were performed that re-use the same CORSIKA run numbers, e.g. different pointing positions for the same particles in prod5 and also different particles for the same pointing.

To still enable unique obs_ids in merged files for these productions, an "override_obs_id" option is added to the SimTelEventSource that can be used to assign new, globally unique ids.

The original run number is added to the SimulationConfigContainer.

We have discussed before to decouple simulation obs_ids from CORSIKA / simtel run numbers.

This is a first step in enabling this. This was triggered also because the LST MC production reuses run numbers in a way that completely prevented merging files for model training.

@maxnoe maxnoe marked this pull request as ready for review October 19, 2023 13:17
@maxnoe maxnoe force-pushed the override_obs_id branch 3 times, most recently from 17adb8f to 2f72795 Compare October 19, 2023 13:28
Copy link
Contributor

@kosack kosack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this fits well with the external discussion you mentioned that obs_ids should be decoupled from simulation run_ids.

is a config option enough? It means computing the unique obs_id outside the tool, in a script for example. Another option is to have the obs_id by default be in the agreed ACADA format (run_number + 4000000 or whatever), and then override_obs_id can be used only in special cases where you don't want that behavior

@maxnoe
Copy link
Member Author

maxnoe commented Oct 19, 2023

I need this now for really the case where I need to assign custom obs_ids. The LST simulations restart run numbers for each pointing and I need to merge ~10 pointing directions for model training.

I wouldn't just add the 4, since ctapipe is used on not just SDMC-DPPS produced simtel files, in fact, no such files really exist as of now.

Simulations were performed that re-use the same CORSIKA run numbers,
e.g. different pointing positions for the same particles in prod5
and also different particles for the same pointing.

To still enable unique obs_ids in merged files for these productions,
an "override_obs_id" option is added to the ``SimTelEventSource``
that can be used to assign new, globally unique ids.

The original run number is added to the ``SimulationConfigContainer``.
@maxnoe maxnoe merged commit cbd41da into main Oct 25, 2023
15 checks passed
@maxnoe maxnoe deleted the override_obs_id branch October 25, 2023 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants