Ideas for automated characterization of lightcurves

[bourque]

Now that we have loads of output individual and composite lightcurves, we can analyze them in an automated fashion in hope to detect/characterize interesting features. This issue can be used as a hub to discuss ideas on how to do this.

[bourque]

One idea is to look at which lightcurves are simply just noise and which have features outside of the noise. One possible quick way to distinguish such light curves is by looking at the distribution of poisson statistics.

Below are plots showing the distribution of what I am calling the 'poisson factor' (not sure if there is a proper name for this), which is defined by:

f = sqrt(mean(counts)) / stdev(counts) = mu / stdev(counts)

For all lightcurves:

For composite-only lightcurves:

If I am understanding correctly, a lightcurve that has a poisson factor around 1.0 means that it consists only of poisson noise(?), and that the lightcurves that exist in the wings of this distribution perhaps have interesting features(?).

@justincely Am I thinking about this correctly? Do you have any thoughts on this? Do the distributions show a clear threshold beyond which we can 'throw out' lightcurves that are just noise?

[bourque]

Things to do and try: DFTs, finding flares, finding transits

[bourque]

Here is the distribution if we define the 'poisson factor' as stdev / mu instead of mu / stdev:

There are some significant outliers. It will be interesting to see what these lightcurves look like.

[bourque]

I would like to try making periodigrams using various methods found online:

https://jakevdp.github.io/blog/2015/06/13/lomb-scargle-in-python/
http://docs.scipy.org/doc/scipy-dev/reference/generated/scipy.signal.lombscargle.html
http://docs.scipy.org/doc/scipy-dev/reference/tutorial/fftpack.html#two-and-n-dimensional-discrete-fourier-transforms

[bourque]

Another thing to try is to use the parameter space offered by the various stats in the stats table to come up with a classification algorithm.

[justincely]

Another thing to try would be an auto-correlation. Something with periodicity will find a lot of correlation peaks at lags larger than the bin-size of the lightcurve

[justincely]

I'm also splitting apart the dataset lists as "interesting" and "uninteresting". The current criteria for "interesting" is a poisson factor > 1.2, which was determined as a rough threshold by looking over the datasets by eye. As our understanding/characterization grows, we can add/improve the constraints.

[bourque]

I explored using lomb-scargle periodograms to characterize periodic lightcurves (see notebook)