-
Notifications
You must be signed in to change notification settings - Fork 145
Project Ideas
DART processes the updates from an observation in parallel. Ideally, for observations we want each task to have the same amount of work. Distributing data round-robin is a 'pretty good' load balance for DART since the goal is to work with various models and observations. However, for projects such as operational weather forecasting where time to solution is of utmost importance there may be efficiency gains from load balancing specifically for the forecast model and particular set of observations. The goal of this project is to create a simulator for DART which can be used to look at the impact of various data distributions, computation, and communication costs without having to code directly in DART.
DART combines observations with an ensemble of model forecasts to produce an improved estimate of what is actually happening. The DART assimilation algorithm is that for each observation, you update model state within the localization radius, and any not-yet-assimilated observation forward operators within the localization radius. DART does this one observation at time, but the algorithmic constraint is actually less strict: If you know that the observations are far enough apart (beyond the localization radius) you can assimilate the observations at the same time. That is, unrelated observations can be assimilated simultaneously. Proof of concept work on graph partitioning for the mutual exclusion scheduling problem, https://dl.acm.org/doi/10.1145/2535753.2535760 has been demonstrated. The goal of this project would be to implement simultaneous assimilation of unrelated observations in DART.
DART is used with 25+ models and many more observation sources. As a community facility, DART accepts contributions from data assimilation scientists and practitioners across the world: new models, new algorithms, new observations. It is important to be able to accept these contributions and be confident that the core DART code can deal with any weird and wonderful new science. Currently to trigger error/exception events such as ‘failed vertical conversion’ we have to hack existing working model_mods. The goal of this project is to create a pathological model for testing DART. The pathological model should be able to trigger a variety of obvious and more devious failures.
As earth system science models become higher and higher resolution, metadata that was once trivial, such as grid information and physical data which is static across the ensemble, is both cumbersome and a barrier to high resolution data assimilation. The goal of this project is to profile DART to identify memory usage patterns and reduce the memory footprint of DART using MPI one-sided communication, co-arrays, or other technique. The eventual goal being to run DART while the model is in pause-resume mode, greatly reducing the computational cost of ensemble data assimilation. Any significant memory reduction would be immediately impactful for several upcoming Accelerated Scientific Discovery Projects.
Satellite observations are a vital source of data for weather forecasting. Previous work to successfully scale DART to 100,000 processors has focused on dealing with high resolution model forecasts, and relatively small observation counts. Satellite data has disrupted this assumption: billions of observations per day are expected and will become the norm for data assimilation. This project will involve redesigning the DART observation sequence structure with a focus on parallelism, memory storage, and IO.
A forward operator - what the model predicts an observation should be - must be calculated for any data assimilation. Observations are irregular in space and time, and the model is distributed in memory across many compute nodes. DART makes use of one-sided MPI communication to deal with this irregularity and calculate the forward operator. The one-sided implementation in DART is currently written for MPI-2. This project is to update the communication to MPI-3 and verify the code across various compilers and NSF machines.
Initial work on compressing out unwanted model state for the Red Sea Data Assimilation project was completed by a summer internships in parallel computing student. The work was specific to the MITgcm ocean model interface for DART and provided huge efficiency gains for the project. This GSoC project would be to generalize this approach and allow general masking of the state, which is applicable for land models and ice models where the ensemble may not be complete at every 3D grid location. Currently the check for incomplete ensembles happens within the assimilation code, which is both costly and error prone. DART already distributes and manages model state across nodes. The goal of this project is to add an abstraction layer to separate the logical representation of the model state from the compressed/masked version in memory.
Initial work on porting part of the DART algorithm to GPUs using openACC was done as part of a NERSC hackathon. The goal project is to produce the first GPU-resident model and data assimilation using a chaotic model such as lorenz_96 with DART. The project world target NCAR’s current supercomputer Cheyenne, and upcoming machine Derecho depending on the timeline of the new machine. Depending on the expertise and interest of the contributor this project could extend to multi-gpu work.
The Weather Research and Forecasting (WRF) Model is a state of the art mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications. The DART-WRF interface is one of the longest running interfaces in DART and is actively developed for problems such as hurricane flood prediction. This project would be to refactor the wrf-dart interface to enable easier development of upcoming dust, chemistry, and wildfire assimilation.