PyCBC Live O3 development Old

PyCBC Live Development

Multi IFO Plan Outline

• Calculate all possible double coinc combinations
• Pick "best" one
• For all remaining IFOS calculate a p-value for the on-source time with the same template
• combine double coinc FAR with pvalues from other ifos

Methods and options for improvements

• handling of singles

  • improvement: handle case where one detector is non-vetoed by not the only "active" one.

• Preparation of strain

  • improvement: add in autogating

• Calculation of the pvalue

  • improvement: Better check of DQ in the background time

• How to pick the "best" double coinc?

  • Simple implementation: best far, largest SNR
    • Downside: Larger trials factor
  • Possibly better: Choose based on best instrument combination, past noise, sensitivity

• How to combine FAR with pvalues from additional detectors?

  • Simple implementation: Fisher's method, using fiducial "foreground time" (.01 * IFAR).
  • Possibly better: Take trials factor if additional ifos will reduce significance (might expect low snr)
  • Possibly better: Do a weighted combinations of p-values based on additional information (detector sensitivity, background).

• Additional Followup (Extra Credit)

  • plots of snr time series auto uploaded
  • plots of background, etc

• Add ability to use SG chisq

• Save more metadata to the output HDF files: channel names of strain, status and DQ, status and DQ flags, PyCBC version, values of thresholds, maybe the entire command line.

• Start reading frames at a time after the template waveforms have been generated. This should reduce the amount of lookback frames that need to be present. Sometimes this results in a file not found error if the bank takes a long time to generate.

Testing Plan

1. Verify FAR + pvalue combination gives self-consistent results (satisfies expected distribution)
   • pvalue combining tests
2. Verify production of coincidences, singles, and triples using Gaussian noise
3. Verify that coinc files are understood and processed by gracedb and produce sensible results (Bhooshan, need reverification due to xml format change)
4. Verify coincidences on Gaussian noise + software injection. (Bhooshan: Ongoing, Some issues)
5. Repeat (4) with real noise and real events
6. Run over long duration for stability + check distribution of triggers (Khun Sang: Will contribute to items 4,5 and 6 in pre-ER14, ER14)
7. Data fetching and preprocessing to head node of MPI and then distribute (Bhooshan: Under development)

Write methods paper

1. Where?
2. By when? (Before first publication on O3 data would be nice!)
3. Paper with 5 detector network and for BNS and NSBH (q < 5) (may be)

(Don't need answers here yet, but this needs to be on the radar)