🥇 Challenge : compute H2O on Radis-Lab

[erwanp]

In the latest commits to RADIS-Lab 567b025 I added the auto-download of HITEMP H2O lines.

Challenge is to be able to run a large range, high-temperature H2O computation directly on RADIS-Lab ! (no install needed !) 🥇 🥇

Currently, we are limited because :

1. the .h5 cache files are not generated during the Docker PostBuild phase (it crashed) ; probably because of Memory error while unpacking them.
2. you can still start computing HITEMP H2O from Radis-Lab, but it takes ages to generate the .h5 files from the .par files.
3. Then, it will be a challenge to know if 8 GB RAM provided by GESIS are enough to compute an H2O spectrum.

Fix :

1. could be helped a lot by using Vaex to stream the .par files into HDF5 files (a small change of the Pandas syntax used in HITEMP-Hdf5 improvement radis#203 and Auto fetch hitemp for CO2 & H2O radis#281). If .h5 are cached during Docker build; the Docker build will likely take 2 hrs ; but user computations will be instantaneous then ! ( kind reminder that we need Improve launch time : separate environment (+ databases) from content (notebooks) #13 to be able to add Example Notebooks here without restarting the Docker !)
2. May work after @gagan-aryan 's Reduce Memory Usage Reduce Memory usage radis#118 project

[Springder]

I failed to compute H2O on Radis-Lab(990-1000 cm-1), even if 8 GB RAM was provided by GESIS. It's really necessary to unzip all par files when I just need 990-1000 cm-1, but in fact, that's what it does, converting all files into h5 files. This may be the reason for the failure？

[erwanp]

Yes! Good news : in 0.9.30, we only download, unzip & parse the files needed. This is already implemented in develop version (radis/radis#281) and will be available on RADIS-Lab as soon as 0.9.30 is released

[Springder]

Any independent file conversion from .par to h5 will not exceed 8 GB RAM on Radis-Lab.
So, I try to compute H2O from 2260 to 2270 cm-1, then .par files were convertiing one by one until the last file (01_2250-2500_HITEMP2010) was unzipped and parsed. In current version, this is an ingenious way, because the interval we calculate is just in the last file, so we can ensure that there is enough RAM to unzip & parse the files.

[erwanp]

Hello ! RADIS-Lab was updated with the newest RADIS 0.10.x #25

I also managed to get it to pre-parse the full H2O database #26

Now, a 2260 - 2270 cm-1 works very well :

from radis import calc_spectrum
from astropy import units as u
s = calc_spectrum(wavenum_min = 2260 / u.cm, 
                  wavenum_max = 2270 / u.cm,
                  molecule = 'H2O',
                  isotope = '1,2,3',
                  pressure = 1.01325 * u.bar,
                  mole_fraction = 1,
                  path_length = 1 * u.cm,
                  Tgas=3000, 
                  databank='hitemp',
                  verbose=3,        # adding some extra info for the first computation
                  )
s.plot()
# without verbose=False this will show all the input parameters. 
# With verbose=2,3,etc... we get increasing number of details about the calculation. 

A larger spectrum (ex : 2000 - 4000 cm-1 , 1 isotope) is still out of reach with the limited 8 GB of GESIS, and I run out-of-memory at loading

Generated dataframe from /home/jovyan/.radisdb/hitemp/H2O-01_02000-02250_HITEMP2010.h5 in 31.88s (3283544 rows, 31 columns, 2068.63 MB)
Generated dataframe from /home/jovyan/.radisdb/hitemp/H2O-01_02250-02500_HITEMP2010.h5 in 29.26s (2971420 rows, 31 columns, 1871.99 MB)

But 31 columns are loaded, most of them useless for equilibrium calculations, so it could quickly be optimized further.

Index(['id', 'iso', 'wav', 'int', 'A', 'airbrd', 'selbrd', 'El', 'Tdpair',
       'Pshft', 'ierr', 'iref', 'lmix', 'gp', 'gpp', 'ju', 'Kau', 'Kcu', 'Fu',
       'symu', 'jl', 'Kal', 'Kcl', 'Fl', 'syml', 'v1u', 'v2u', 'v3u', 'v1l',
       'v2l', 'v3l'],
      dtype='object')

[erwanp]

@gagan-aryan with Radis version 0.12 the algorithm is even better: you can already compute much larger ranges. I tried the code above with the 2000 - 4000 cm-1 range and it worked. You can try to push it further.