serverlessextract

Astronomica-interferometry

In this pipeline we perform radio interferomic data processing carrying out all the phases: rebinning, calibrationa and imaging. It is computed using the serverless architecture Lithops.

Prerequisites

To execute this notebook you need:

• An AWS Account.
• Setup Lithops to work with AWS Lambda.

Setup

1. Download the data and extract it in a directory similar to /home/user/Downloads/entire_ms/SB205.MS/ .

2. Setup Lithops for AWS backend.

3. Build the runtime in the docker directory :

   $ lithops runtime build -f Dockerfile serverless-extract:1

4. Configure Lithops to use the built runtime (e.g. serverless-extract:1).

5. Create an S3 bucket named aymanb-serverless-genomics to upload the data.

6. Run partition.py located in partition directory. This will create and upload the .ms files to the S3 bucket divided in 70 partition by default.

   $ cd ./partition/
   $ python3 partition.py

7. Run the pipeline.py file. This file performs all the phases of the pipeline [rebinning, calibration, imaging]:

   $ python3 pipeline.py

   More information on how it works in this link.

8. The results obtained should look similar to the images in /stats/stats/ .

NOTE: you can change the names of the S3 bucket and the number of partitions editing the pipeline.pyand partition.pyfiles.