Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parser for TESS DVT files #164

Merged
merged 13 commits into from
Jan 14, 2025
2 changes: 2 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
1.1.0 (unreleased)
------------------

* Add support for loading TESS DVT files. [#164]


1.0.0 (12-02-2024)
------------------
Expand Down
17 changes: 16 additions & 1 deletion lcviz/helper.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from astropy.io.fits import getheader
import astropy.units as u
import ipyvue
import os
Expand Down Expand Up @@ -144,7 +145,7 @@
# already been initialized
plugin._obj.vdocs = self.app.vdocs

def load_data(self, data, data_label=None):
def load_data(self, data, data_label=None, extname=None):
"""
Load data into LCviz.

Expand All @@ -161,7 +162,21 @@
automatically determined from filename or randomly generated.
The final label shown in LCviz may have additional information
appended for clarity.
extname : str or `None`
Used for DVT parsing if only a single TCE from a multi-TCE file should be
loaded. Formatted as 'TCE_1', 'TCE_2', etc.
"""
# Determine if we're loading a DVT file, which has a separate parser
if isinstance(data, str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be just as easy to support HDUList too, which would be helpful for writing tests and for files we make/edit on the fly. Can we add that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but handling HDULists in general is probably a separate ticket/PR.

header = getheader(data)
if (header['TELESCOP'] == 'TESS' and 'CREATOR' in header and
'DvTimeSeriesExporter' in header['CREATOR']):
super().load_data(data=data,

Check warning on line 174 in lcviz/helper.py

View check run for this annotation

Codecov / codecov/patch

lcviz/helper.py#L174

Added line #L174 was not covered by tests
parser_reference='tess_dvt_parser',
data_label=data_label,
extname=extname)
return

Check warning on line 178 in lcviz/helper.py

View check run for this annotation

Codecov / codecov/patch

lcviz/helper.py#L178

Added line #L178 was not covered by tests

super().load_data(
data=data,
parser_reference='light_curve_parser',
Expand Down
40 changes: 39 additions & 1 deletion lcviz/parsers.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import os

from astropy.io import fits
from glue.config import data_translator
from jdaviz.core.registries import data_parser_registry
import lightkurve
Expand All @@ -12,9 +14,46 @@
'kepler': {'prefix': 'Q', 'card': 'QUARTER'},
'k2': {'prefix': 'C', 'card': 'CAMPAIGN'},
'tess': {'prefix': 'S', 'card': 'SECTOR'},
'tess dvt': {'prefix': '', 'card': 'EXTNAME'}
}


@data_parser_registry("tess_dvt_parser")
def tess_dvt_parser(app, file_obj, data_label=None, show_in_viewer=True, **kwargs):
'''
Read a TESS DVT file and create a lightkurve object
'''
hdulist = fits.open(file_obj)
ephem_plugin = app._jdaviz_helper.plugins.get('Ephemeris', None)
extname = kwargs.pop('extname')

Check warning on line 28 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L26-L28

Added lines #L26 - L28 were not covered by tests

# Loop through the TCEs in the file. If we only want one (specified by
# `extname` keyword) then only load that one into the viewers and ephemeris.
for i in range(1, len(hdulist)-1):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just iterate over hdulist directly (for hdulitem in hdulist[1:]) or is the iterator i needed anywhere else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at some point I was going to accept an integer rather than string name for extension, I can change this to what you suggested.

data = hdulist[i].data
header = hdulist[i].header
lc = lightkurve.LightCurve(data=data,

Check warning on line 35 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L32-L35

Added lines #L32 - L35 were not covered by tests
flux=data['LC_INIT'],
flux_err=data['LC_INIT_ERR'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some work to do on the units here. We're currently expecting the time unit to be in JD (I think?), but these files have the units:

TTYPE1  = 'TIME    '           / column title: data time stamps                 
TFORM1  = 'D       '           / column format: 64-bit floating point   
TUNIT1  = 'BJD - 2457000, days' / column units: Barycenter corrected TESS Julian

but the offset of 2457000 isn't being reapplied in this parser yet, so the time axis is prehistoric:

Screenshot 2025-01-02 at 10 05 51

We might also need to intentionally not load the DVT's phase column, or mark its caveats. The phase units are not the (0, 1) orbital phase that we usually use:

TTYPE4  = 'PHASE   '           / column title: Phase using period and epoch     
TFORM4  = 'E       '           / column format: 32-bit floating point           
TUNIT4  = 'days    '           / column units: [-0.25*period, 0.75*period]     

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, we probably shouldn't load the phase column but could load the ephemeris and let lcviz automatically phase-fold (if that's easy - otherwise can be a follow-up ticket)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, we probably shouldn't load the phase column but could load the ephemeris and let lcviz automatically phase-fold (if that's easy - otherwise can be a follow-up ticket)

I already made a pass at doing that at L53-54.

lc.meta = hdulist[0].header
lc.meta['MISSION'] = 'TESS DVT'
lc.meta['FLUX_ORIGIN'] = "LC_INIT"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the (near?) future, we should either add flux column validation or work to ensure that non-standard flux columns are selectable.

By default, all of the table columns will be options in the Flux Column plugin. So far, we haven't written any intentional support for plotting models, but there are pre-computed transit light curve models in the DVT files, stored under the "MODEL_INIT" and "MODEL_WHITE" columns. This causes a hiccup if you try to set "MODEL_WHITE" as the flux column, since we look for a corresponding _err column, and none exists for the model:

self.app._jdaviz_helper._set_data_component(dc_item, 'flux', dc_item[self.selected])
self.app._jdaviz_helper._set_data_component(dc_item, 'flux_err', dc_item[self.selected+"_err"]) # noqa

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using .get instead might fix L225, but we should check that's the only issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Down the road, it'll be great to change the plot options defaults for "MODEL_*" columns.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I fixed this in components.py but it needs more testing to make sure something downstream isn't broken by an all-nan uncertainty array.

lc.meta['EXTNAME'] = header['EXTNAME']

Check warning on line 41 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L38-L41

Added lines #L38 - L41 were not covered by tests

if extname is not None and header['EXTNAME'] != extname:
show_ext_in_viewer = False

Check warning on line 44 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L43-L44

Added lines #L43 - L44 were not covered by tests
else:
show_ext_in_viewer = show_in_viewer

Check warning on line 46 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L46

Added line #L46 was not covered by tests

light_curve_parser(app, lc, data_label=data_label,

Check warning on line 48 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L48

Added line #L48 was not covered by tests
show_in_viewer=show_ext_in_viewer, **kwargs)

# add ephemeris information from the DVT extension
if ephem_plugin is not None and show_ext_in_viewer:
ephem_plugin.period = header['TPERIOD']
ephem_plugin.t0 = header['TEPOCH']

Check warning on line 54 in lcviz/parsers.py

View check run for this annotation

Codecov / codecov/patch

lcviz/parsers.py#L52-L54

Added lines #L52 - L54 were not covered by tests


@data_parser_registry("light_curve_parser")
def light_curve_parser(app, file_obj, data_label=None, show_in_viewer=True, **kwargs):
# load a LightCurve or TargetPixelFile object:
Expand All @@ -29,7 +68,6 @@
if data_label is None:
data_label = os.path.splitext(os.path.basename(file_obj))[0]

# read the light curve:
light_curve = lightkurve.read(file_obj)

elif isinstance(file_obj, cls_with_translator):
Expand Down
Loading