| jupyter | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
- Description: A quick introduction on using HEASARC data on Sciserver.
- Level: Beginner
- Data: We will use 4 observations of
Cyg X-1from NuSTAR as an example. - Requirements: Run in the (heasoft) conda environment on Sciserver.
- Credit: Abdu Zoghbi (May 2022).
- Support: Contact the HEASARC helpdesk.
- Last verified to run: 02/28/2024
In this notebook, we present a brief overview of a typical analysis flow. It can be used as a quick reference. We will go through an example of finding, accessing and then analyzing some x-ray data data.
We will be using 4 NuSTAR observation of Cyg-X1.
When running this notebook inside Sciserver, make sure the HEASARC data drive is mounted when initializing the Sciserver compute container. See details here.
Running Outside Sciserver:
This notebook runs in the (heasoft) conda environment on Sciserver. If running outside Sciserver, some changes will be needed, including:
• Make sure heasoftpy and heasoft are correctly installed (Download and Install heasoft).
• Unlike on Sciserver, where the data is available locally, you will need to download the data to your machine.
We need the following python modules:
import glob
# for finding data
import pyvo
# import nupipeline from heasoftpy
# for heasoftpy version >= 1.4, it is under heasoftpy.nustar.nupipeline
# for heasoftpy version < 1.4, it is under heasoftpy.nupipeline
try:
from heasoftpy.nustar import nupipeline
except ModuleNotFoundError:
from heasoftpy import nupipelineThe Heasarc data holdings can be searched and explored in different ways:
-
Using Xamin Web Interface.
-
Using a virtual observatory (VO) client such as pyVO (see below) or Topcat.
-
Using the classical Browse Mission Interface.
In Section 3.1 below, we give an example on how to use pyVO to search for NuSTAR data on a specific object. Alternatively, Section 3.2 assumes you can use Xamin to obtain a list of observations you are interested in. For more details on finding and accessing data, see the notebook on finding and downloading data.
The outcome of sections 3.1 and 3.2 is the same, so you can follow either of them.
We first search the Virtual Observatory (VO) registry for data provided by heasarc. The registry provides an index of all data providers that allow access using VO standards.
In the following example (heasarc_service), we search for all entries in the registry that have the keyword heasarc. This can a large and general set. The search can be filtered for more specfic datasets.
import pyvo as vo
heasarc_service = vo.regsearch(keywords='heasarc')
print(f'The search returned {len(heasarc_service)} entries. Examples include:\n')
# ivoid is the unique identifier for the dataset
heasarc_service.to_table()[['ivoid', 'res_title']][-5:]We can be more specific by selecting only the master catalogs and the services that provide a cone search capability (i.e. search by providing RA, DEC and a search radius).
master = []
for srv in heasarc_service:
if 'master' in srv.ivoid and 'conesearch' in srv.access_modes():
master.append(srv)
print(f'{srv.ivoid}:\t {srv.res_title}')Lets focus on numaster, the master catalog for NuSTAR, and search for data on some object, say the X-ray binary Cyg X-1.
We use astropy to resolve the name into positional coordinate.
We specify the service we want to use as conesearch.
import astropy.coordinates as coord
pos = coord.SkyCoord.from_name("cyg x-1")
nu_master = [m.get_service('conesearch') for m in master if 'numaster' in m.ivoid][0]
result = nu_master.search(pos=pos, radius=0.5)# display the result as an astropy table.
result.to_table()Say we are interested in the first 4 datasets. We use another feature of the VO: datalinks.
For each row of interest, we request the related links, and select those that point to a data directory.
They provide access_url columns. Here, we collect the paths to the directory containing the event files starting with FTP.
paths = []
for i in range(4):
datalink = result[i].getdatalink().to_table()
link_to_dirs = datalink[datalink['content_type'] == 'directory']
link = link_to_dirs['access_url'].value[0]
path = '/FTP/' + link.split('FTP')[1]
paths.append(path)
print(path)Here, we use Xamin to find the data. We again use numaster, the master catalog for NuSTAR, and search for data the X-ray binary Cyg X-1.
When using Xamin to find the data, there is an option in the Data Products Cart to select FTP Paths, which, when selecting the first 4 datasets, provides a text similar to the following:
Note that for the purpose of this tutorual, you can choose any observations
paths_txt = """
/FTP/nustar/data/obs/00/3//30001011002/
/FTP/nustar/data/obs/03/3//30302019002/
/FTP/nustar/data/obs/01/1//10102001002/
/FTP/nustar/data/obs/05/8//80502335006/
"""
paths = paths_txt.split('\n')[1:-1]All the heasarc data is mounted into the compute under /FTP/, so once we have the path to the data (though pyVO or Xamin), we can directly access it without the need to download it.
So to check the content of the observational folder for the first observations of cyg x-1 for example, we can do:
glob.glob(f'{paths[0]}/*')To Analyze the data within the notebook, we use heasoftpy. In the NuSTAR example, we can call the nupipeline tool to produce the cleaned event files.
We focus on the first observation.
# set some input
indir = paths[0]
obsid = indir.split('/')[-2]
outdir = obsid + '_reproc'
stem = 'nu' + obsid
# call the tasks; verbose=20 logs the output to nupipeline.log
out = nupipeline(indir=indir, outdir=outdir, steminputs=stem, instrument='FPMA',
clobber='yes', noprompt=True, verbose=20)Once the task finishes running, we see the new cleaned event files in the local ./30001011002_reproc/ directory
For subsequent analysis, you can use heasoftpy which provides a python access to all tools in heasoft, as well as pyxspec to spectral modeling.