Directions & production limits for starting Corsika production

[moralejo]

Here I propose some settings for starting the Corsika production.
See also the following issues:
#1 #2 #3

The idea is to start with a training set produced along the Crab Nebula's path on the sky, so that we can validate the whole procedure against real Crab observations. All azimuths below are astronomical ones, i.e. measured from geographic North clockwise (N-E-S-W). They need to be adapted to Corsika's system. In all cases spectral index should be -2, and 10 re-uses of the showers (through the CSCAT parameter) have to be used.

Training proton set:

 zen(deg)  az(deg)  cone_min(deg) cone_max(deg) max imp(m) Emin(TeV) Emax(TeV)

  66.446    75.983    0.000         5.057        2372.8      0.099    200.000
  59.342    79.180    0.000         5.713        2100.6      0.054    200.000
  52.162    82.384    0.000         6.266        1915.2      0.034    200.000
  44.927    85.716    0.000         6.731        1782.7      0.024    200.000
  37.661    89.359    0.000         7.118        1685.9      0.018    179.339
  30.390    93.640    0.000         7.430        1615.0      0.014    144.700
  23.161    99.261    0.000         7.671        1564.4      0.012    123.378
  16.087   108.090    0.000         7.842        1530.3      0.011    110.502
   9.579   126.888    0.000         7.944        1510.6      0.010    103.573
   6.000   180.000    0.000         7.978        1504.1      0.010    101.383
   9.579   233.112    0.000         7.944        1510.6      0.010    103.573
  16.087   251.910    0.000         7.842        1530.3      0.011    110.502
  23.161   260.739    0.000         7.671        1564.4      0.012    123.378
  30.390   266.360    0.000         7.430        1615.0      0.014    144.700
  37.661   270.641    0.000         7.118        1685.9      0.018    179.339
  44.927   274.284    0.000         6.731        1782.7      0.024    200.000
  52.162   277.616    0.000         6.266        1915.2      0.034    200.000
  59.342   280.820    0.000         5.713        2100.6      0.054    200.000
  66.446   284.017    0.000         5.057        2372.8      0.099    200.000

Training gamma set:

zen(deg)  az(deg)  cone_min(deg) cone_max(deg) max imp(m) Emin(TeV) Emax(TeV)

 66.446    75.983    0.000         2.500        1423.7      0.050    200.000
 59.342    79.180    0.000         2.500        1260.4      0.027    200.000
 52.162    82.384    0.000         2.500        1149.1      0.017    169.650
 44.927    85.716    0.000         2.500        1069.6      0.012    118.545
 37.661    89.359    0.000         2.500        1011.5      0.009     89.670
 30.390    93.640    0.000         2.500         969.0      0.007     72.350
 23.161    99.261    0.000         2.500         938.6      0.006     61.689
 16.087   108.090    0.000         2.500         918.2      0.006     55.251
  9.579   126.888    0.000         2.500         906.3      0.005     51.786
  6.000   180.000    0.000         2.500         902.5      0.005     50.691
  9.579   233.112    0.000         2.500         906.3      0.005     51.786
 16.087   251.910    0.000         2.500         918.2      0.006     55.251
 23.161   260.739    0.000         2.500         938.6      0.006     61.689
 30.390   266.360    0.000         2.500         969.0      0.007     72.350
 37.661   270.641    0.000         2.500        1011.5      0.009     89.670
 44.927   274.284    0.000         2.500        1069.6      0.012    118.545
 52.162   277.616    0.000         2.500        1149.1      0.017    169.650
 59.342   280.820    0.000         2.500        1260.4      0.027    200.000
 66.446   284.017    0.000         2.500        1423.7      0.050    200.000

Test pointlike gamma set (points in the test grid which would be needed along the Crab path):

zen(deg)  az(deg)  cone_min(deg) cone_max(deg) max imp(m) Emin(TeV) Emax(TeV)
 10.000   248.117     0.3999       0.4001       710.8      0.005     51.951
 10.000   102.199     0.3999       0.4001       710.8      0.005     51.951
 14.984   175.158     0.3999       0.4001       724.6      0.005     54.516
 32.059   248.099     0.3999       0.4001       826.0      0.008     75.617
 32.059   102.217     0.3999       0.4001       826.0      0.008     75.617
 43.197   262.712     0.3999       0.4001       960.2      0.011    110.192
 43.197    87.604     0.3999       0.4001       960.2      0.011    110.192
 52.374   301.217     0.3999       0.4001      1146.6      0.017    171.694
 52.374    49.119     0.3999       0.4001      1146.6      0.017    171.694
 60.528    99.126     0.3999       0.4001      1422.8      0.029    200.000
 68.068   283.075     0.3999       0.4001      1874.2      0.059    200.000
 68.068    67.271     0.3999       0.4001      1874.2      0.059    200.000

[Voutsi]

Thanks @moralejo
What about the statistics? I see at the previous production, for example for gamma diffuse, Yoshiki produced 1000 jobs of 100000 showers each. If we target for similar statistics per node, taking into account that launching more than 1000 jobs will probably clog the cluster (once I received a number of complaints when launched 5000 jobs), this will take quite some time...

[moralejo]

I think you should be able to launch whatever number of jobs, it does not make sense that you have to manually check when you can submit more etc. I will ask the admins and Cc you. As for stats (training), I would indeed aim at having the same stats per grid point as we had in the previous production e.g. at zd=20, az=180. And let's see how it goes.

Correction: for training I meant having as much statistics up to ~30 degrees of zenith (9 grid points in total), as we had at zd=20, az=180 in the previous production.

[maxnoe]

@moralejo no, that would be great but is unfortunately not how most of these cluster systems work. usually another layer around keeping ~hundreds of queued jobs, starting new jobs when previous jobs finished is the way to go.