-
Notifications
You must be signed in to change notification settings - Fork 50
Submitting Computational Results
This page provides instructions for MaCFP participants to submit their computational results. August 31, 2023 is the deadline to submit modeling results (pool fires, gaseous burners, fire growth cases); late results may be included on individual participant posters, but we cannot guarantee that these results will be directly included in the discussion talks. Following the guidelines below will help the organizers collect and organize data from a broad community in a timely manner. We appreciate your help in this effort.
A video discussing the submission process can be found here.
NEW THIS YEAR ALL PARTICIPANTS MUST:
- Submit results via GitHub Pull Request (PR)
- Submit a poster with model summary and results
- Add a python script to generate plots
- Add a GitHub Actions workflow to automatically run the plotting script and archive the results
Below we will go over how to generate the scripts and Actions entries.
IMPORTANT: Please do not submit any specially formatted files. This includes Word documents (.doc), Excel spreadsheets (.xls), or Matlab (.mat) workbooks.
Contributions must be made via pull request (more on this below), please make sure you start from an up-to-date version of the repository. Your work cannot be merged into the current repo if your repository is out-of-date.
To help make reasonable comparisons between modeling results, we ask modelers to fill out the modeling questionnaire for each case they run. This questionnaire should be completed and included as a README.md file as part of your pull request. Please also include the requested information in the poster (discussed below). This will be helpful if someone views the poster after the MaCFP workshop.
All participants who wish to participate in MaCFP-3 by submitting modeling results are asked to submit a one-page poster abstract to the IAFSS Poster Session with a summary of your formulation (submodel choices, etc.) and images that visually describe your mesh setup and boundary conditions.
Please email your Final Poster (.pdf) including a written model summary and an image of the mesh and computational domain (email to: randall.mcdermott@nist.gov). In addition to the regularly scheduled IAFSS poster session, MaCFP posters should be made available for discussion during the MaCFP-3 workshop (time for discussion will be allotted during the lunch break). Posters will also be saved and included as a Github Release to be viewed after the MaCFP workshop.
The results should be organized in simple ASCII comma-delimited files (*.csv files) with clear, unique header names. Almost all the results can be presented in simple X (independent variable) Y (dependent variable) plots.
An example is provided here.
You may have multiple Y columns in a single computational data file. As discussed below, this can be done using a configuration file that must be submitted along with your results (see Generating Plots).
This may seem like a minor issue, but obviously we must plot the results from all groups in the same units. The units we will use are those given by the Experimental_Data files for the given problem. See, for example, the units spelled out in the Extinction/UMD_Line_Burner/Experimental_Data readme page. If there is ambiguity in the units or you otherwise have questions along these lines, please submit an Issue.
- Files will be stored, for a given problem, in
Computational_Results/YEAR/INSTITUTION/. When submitting your pull request, please add your INSTITUTION as a separate directory under the YEAR for the workshop, put all your *.csv files there, and submit a pull request. - Please make sure you tag your files with your institution or group name. For example,
ugent_casename_1.csv, etc. While the subdirectory structure handles the tag, in theory, adding the tag to the files is an added safeguard in case your files get misplaced. As mentioned above, make sure you add the data file names to your institute_config.csv file so that your data can be linked to the correct experimental data file. - Do not include spaces or special characters in your filenames. Use an underscore or hyphen to represent a space.
- Do not start your filenames with numbers.
- If you submit multiple grid resolutions for the same case, please be sure to tag the file names with a reference for the grid resolution.
Line endings are a perennial problem with integrating text files from multiple sources. Windows adds ^M to the end of all lines (CRLF), whereas Unix does not. Our standard for the MaCFP repo will be Unix. Usually, a smart editor (Notepad++, Sublime Text, etc.) will allow you to set the line endings to Unix. If you can do this, it would be helpful. If not, we will deal with it. Additional notes on handling line endings can be found here. The command used to handle line endings from the top level of the repository is:
for i in $(grep -IURl --color --exclude={"*.pdf","*.png","*.jpg","*.zip","*.ppt*",".DS_Store","*.mat","*.pyc"} --exclude-dir=".git" "^M" .); do echo $i; dos2unix -o $i; done
To manage the large volume of data and plots required for the MaCFP experimental and simulation comparisons, we require a common plotting module that takes as input a comma-delimited (.csv) configuration file. Each row of the configuration file provides the parameters for a singe "x vs y" line (or scatter) plot. An example configuration file may be found here. For multiple lines on the same plot, use the same plot filename (last column in the example configuration file). Further details are provided in Plotting Scripts.
Use the macfp.py python module to read your configuration file and generate the plots. We ask that you store your configuration file and python script together in the Computational_Results/YEAR/INSTITUTION/ directory. An example is provided for you here and shown below.
#!/usr/bin/env python3
# Author
# Date
import sys
# path below should be correct if your script is in the location discussed in "File Names and Location"
sys.path.append('../../../../../../macfp-db/Utilities/')
import macfp
# Create a directory to store your computational results, such as "Preliminary_Results"
# The `pltdir` will be created if not already present
macfp.dataplot(config_filename='NIST_Pool_Fires_cmp_config.csv',
institute='NIST',
revision='MaCFP-3, Tsukuba, 2023',
expdir='../../../Experimental_Data/',
cmpdir='./Preliminary_Results/',
pltdir='./Preliminary_Results/Plots/',
close_figs=True,
verbose=True,
plot_list=['all'])
New to python? Not a problem. Use this as an opportunity to create your first script. Just cut and paste the above into a simple text file (no Word documents!) and name it something like <TEST CASE>_plot_cmp.py, where <TEST CASE> is the case you are working on, and save it in your Computational_Results/YEAR/INSTITUTION/ directory. In the example above we are working on the NIST_Pool_Fires test case. There should be a separate script for each case: Waterloo_Methanol, FM_Burner, etc.
If you cannot run python locally, this is also not a problem. In the next section, we discuss how to run the scripts automatically using GitHub Actions.
In an effort to build some long-term maintenance into the repository, we have implemented GitHub Actions, which is the GitHub continuous integration feature. All you need to do is append your scripts to the current Actions workflow files. You can simply cut and paste from the example and rename the obvious path and script names. Note that this Actions file addendum will be required for acceptance of your PR.
- Poster including model summary and an image of mesh and computational domain (email to: randall.mcdermott@nist.gov)?
- Institution tagged to all data files?
- Files are ASCII comma-delimited (.csv) with clear header names?
- Modeling Questionnaire completed and included as README.md file?
- Units are the same as Experimental_Data?
- Convert line endings to Unix?
- Create python script to generate plots?
- Add your INSTITUTE to GitHub Actions workflow?
You are ready to submit!
Send a pull request to the MaCFP/macfp-db main branch. The PR will be visually inspected to be sure that a GitHub Actions entry has been added to the respective workflow (see example above). If this has been done, then the PR should automatically run your plotting scripts. If this passes, then the PR will be merged and the plots will be compressed and posted to the SCRIPT_FIGURES Releases page.