-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature Request: Run a single (or a set of) task(s) and any dependent tasks #98
Comments
Perhaps you should just separate out into two separate pippin files and
have one of them point to external directories for the sims that are common
to both pipelines.
…On Wed, Aug 10, 2022 at 12:18 PM Benjamin Rose ***@***.***> wrote:
I want to run only a single task, and all the tasks it depends on.
Similar features in Pippin
I know you can run starting (-s) at, stopping (-f) at, or skipping (-i) a
step, but I don't need all the tasks from a step, but rather just one or
two tasks.
Example
Below is a (truncated) example output from pippin.sh -c where you can see
that Pippin already knows all the tasks and their dependencies. The updated
code would just need to track what task should be run/skipped, similar to
the code that keeps track of skipped tasks.
ROMAN_DATA_LCFIT_ROMAN_SN_CC Listing tasks:
SNANASimulation ROMAN_PLAsTiCC task (1 jobs, deps [])
SNANASimulation ROMAN_SN_Ia task (1 jobs, deps [])
SNANASimulation ROMAN_SN_CC task (1 jobs, deps [])
SNANASimulation ROMAN_BIASCOR task (1 jobs, deps [])
SNANASimulation LOWZ_DATA task (1 jobs, deps [])
SNANASimulation LOWZ_BIASCOR task (1 jobs, deps [])
SNANALightCurveFit ROMAN_DATA_LCFIT_ROMAN_SN_Ia task (12 jobs, deps ['ROMAN_SN_Ia'])
SNANALightCurveFit ROMAN_DATA_LCFIT_ROMAN_SN_CC task (12 jobs, deps ['ROMAN_SN_CC'])
SNANALightCurveFit ROMAN_BIAS_LCFIT_ROMAN_BIASCOR task (36 jobs, deps ['ROMAN_BIASCOR'])
SNANALightCurveFit LOWZ_DATA_LCFIT_LOWZ_DATA task (10 jobs, deps ['LOWZ_DATA'])
SNANALightCurveFit LOWZ_BIAS_LCFIT_LOWZ_BIASCOR task (10 jobs, deps ['LOWZ_BIASCOR'])
I would like to run something like pippin.sh --task ROMAN_PLAsTiCC or --task
ROMAN_DATA_LCFIT_ROMAN_SN_Ia (this will run both
ROMAN_DATA_LCFIT_ROMAN_SN_Ia and ROMAN_SN_Ia).
Reason
Really what I am building two Pippin-pipelines in one file since there are
a few overlapping simulations. I want one pipeline to make a full set of
simulations of about a dozen transient types, and another pipeline that
performs a Roman photometrically classified cosmology analysis. These are
mostly unique tasks, but overlap in the SNIa and CCSN data sims.
—
Reply to this email directly, view it on GitHub
<#98>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJHI2JDI4TLA2OHNH7N7JDVYPI4VANCNFSM56FEHYYA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
That sort of works, but then would require three pippin files: one for each task and one for building the external directories. My goal is to make it easy to get someone access to an SNANA simulation of the Roman reference survey. They can then take that fiducial survey but tweak the one thing of interest to them (say kcor or fitops) and then run Pippin on only the relevant science (say cosmology or transient detection). I can take the first stab at trying to implement this feature. |
ok, well if you tweak one thing in pippin its designed to only run the
dependencies. so you dont need any flags (no -s -f)... just tweak what you
want to change and it will only run the things that depend on that by
default.
…On Wed, Aug 10, 2022 at 1:20 PM Benjamin Rose ***@***.***> wrote:
That sort of works, but then would require three pippin files: one for
each task and one for building the external directories. My goal is to make
it easy to get someone access to an SNANA simulation of the Roman reference
survey. They can then take that fiducial survey but tweak the one thing of
interest to them (say kcor or fitops) and then run Pippin on only the
relevant science (say cosmology or transient detection).
I can take the first stab at trying to implement this feature.
—
Reply to this email directly, view it on GitHub
<#98 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJHI2I75EOLETQXJRGBJUDVYPQG7ANCNFSM56FEHYYA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I want to run only a single task, and all the tasks it depends on.
Similar features in Pippin
I know you can run starting (
-s
) at, stopping (-f
) at, or skipping (-i
) a step, but I don't need all the tasks from a step, but rather just one or two tasks.Example
Below is a (truncated) example output from
pippin.sh -c
where you can see that Pippin already knows all the tasks and their dependencies. The updated code would just need to track what task should be run/skipped, similar to the code that keeps track of skipped tasks.I would like to run something like
pippin.sh --task ROMAN_PLAsTiCC
or--task ROMAN_DATA_LCFIT_ROMAN_SN_Ia
(this will run bothROMAN_DATA_LCFIT_ROMAN_SN_Ia
andROMAN_SN_Ia
).Reason
Really what I am building two Pippin-pipelines in one file since there are a few overlapping simulations. I want one pipeline to make a full set of simulations of about a dozen transient types, and another pipeline that performs a Roman photometrically classified cosmology analysis. These are mostly unique tasks, but overlap in the SNIa and CCSN data sims.
The text was updated successfully, but these errors were encountered: