fenicsR13: A Tensorial Mixed Finite Element Solver for the Linear R13 Equations Using the FEniCS Computing Platform
Paper | Lambert Theisen, Manuel Torrilhon. 2021. fenicsR13: A Tensorial Mixed Finite Element Solver for theLinear R13 Equations Using the FEniCS Computing Platform. ACM Trans. Math. Softw. 47, 2, Article 17 (April 2021), 29 pages, 10.1145/3442378. Preprint @ arXiv (Free PDF). |
- Solving Steady 2D Linear R13 Equations in Normalized Form
- Stabilization with: Continous Interior Penalty (CIP) or Galerkin Least Squares (GLS)
- Option for convergence study with input for exact solution
- Output in XDMF/HDF5 and PDF
- Arbitray finite element combinations (thanks to FEniCS)
- Easy setup of parameter studies
- Structured/Documented YAML input file format
- Rarefied gas flow effects predictable: Knudsen paradox, Knudsen pump, ...
- Interface for Gmsh meshes
- Full exposure of the PETSc preconditioned iterative solver library
- Install Git (if not shipped with you OS).
- Install Git LFS (needed to download the meshes from the repository).
- Clone and open the repository:
# Clone Repository and open main folder
git clone git@git.rwth-aachen.de:lamBOO/fenicsR13.git
cd fenicsR13
Alternatively: download the repository, unpack and open it.
It is recommended to use the program with Docker, a virtualization software available (with free docker account) for `download`_ for most operating systems. Our repository provides a Docker container based on the official FEniCS Docker image which contains all necessities for FEniCS and has been adjusted to solve the R13 equations. The program docker-compose is also required.
Make sure you installed Docker and it is running on your system. You can start the fenicsR13 compute environment from the root-directory of the repository using the command-line
## 1) FOR USERS:
## - Pulls and runs "fenicsr13_release" service
## - Uses preinstalled latest version of fenicsR13
docker compose pull fenicsr13_release
docker compose run --rm fenicsr13_release
## 2) FOR DEVELOPERS (uncommend the last 3 lines of this block):
## - Builds the "fenicsr13_debug"
## - Install an editable version of fenicsR13
## - The "pip"-command install an editable version of fenicsR13
## - This allows a modification of the source files which can be directly
## executed within the debug container
# docker compose build fenicsr13_debug
# docker compose run fenicsr13_debug
# sudo pip install -e .
When you run this for the first time, docker will pull (download and extract) the container image from our repository which is roughly 800MB and the download may require some patience. After the initial download the docker image will be stored (2-3 GB) on your system and any new run will start the container immediately.
The above docker command will start a RedHat-based bash environment. The repository is mounted as a volume under /home/fenics/shared
in the container and should be the default folder on startup. What ever is changed in this folder or its sub-folders will change in the folder on your original system as well. However, any changes outside this folder will be gone after you exit the docker environment.
To execute a simulation case, go to to the case folder (e.g. examples/lid_driven_cavity
)
# [It might be required to install the package inside the Docker container]
pip install --user -e .
# Move to folder:
cd examples/lid_driven_cavity
We provide a script to utilize gmsh and generate a H5 mesh-file from a local geometry file by
# Create mesh:
./create_mesh.sh
To run a simulation execute the solver main program fenicsR13.py
(which is located in the src
-directory in the top level) while specifying an input file as first command line argument.
# Run program with given input file:
fenicsR13 input.yml
Output files will be written to a folder which is named after the output_folder
keyword of the input.yml
. For immediate inspection the output folder contains simple visualizations in PDF files for each of the fields (temperature, pressure,...).
The numerical results for each field is ouput into h5
-files, including mesh data and with corresponding xdmf
-file. The XDMF-files can be opened in Paraview to perform visualization, e.g., with Paraview > File > Open > u_0.xdmf > Apply filters
# Leave directory:
cd ../..
Channel Flow Example
We provide a simple example of a flow through a finite-length channel in 2D.
# Move to folder:
cd examples/channel_flow_force
# Create mesh:
./create_mesh.sh
# Run program with given input file:
fenicsR13 input.yml
In the output folder the results can be post-processed to demonstrate the Knudsen paradox in a simple table.
# Go to folder with simulation results (=casename in input.yml)
cd channel_flow_force
# Generate correlation data between Knudsen number and massflow
bash postprocessing.sh
cat table.csv
# Leave directory:
cd ../..
Convergence Study
We can test the convergence of the R13 discretization on a simple double-cylindrical geometry.
# Move to folder:
cd tests/2d_r13
# Meshes are already in Git:
ls ../2d_mesh
# Run program with given input file:
fenicsR13 inputs/r13_1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab.yml
# Go to folder with simulation results (=casename in input.yml)
cd r13_1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab
# Open errors:
cat errors.csv
FEniCS allows simple parallelization using MPI
# Parallel execution ("-u" to flash stdout)
# Usage: mpirun -n <numberOfProcesses> <serialCommand>
# E.g.: mpirun -n 4 fenicsR13 input.yml
The main folder of this repository contains a Dockerfile
defining the used environment. Here, we used the optimized and official FEniCS Docker image and include Gmsh
and install some requirements from the requirements.txt
. This can take a while, especially the Gmsh
mirror can be quite slow. To avoid very long execution commands (docker run <..> -v <volume share> <etc..>
), a docker-compose.yml
is used to store all these parameters. docker compose
acts as an wrapper for the Docker execution.
The fenics
environment (also called service in the docker-compose.yml
) first has to be build and can be executed afterwards. The command to build the container is
# build fenics service
docker compose build fenicsr13_release
It is possible to use a Jupyter sever or a X11 forwarding but this is not recommended anymore. All relevant plots are now written by default without the need for the tricky X11 forwarding or interactive usage with Jupyter.
Documentation using Sphinx is available.
Visit the hosted version on Gitlab Pages or download the artifacts from Gitlab's CI pages
-pipeline.
# cat .gitlab-ci.yml
cd docs
sphinx-apidoc -o source/src ../src
sphinx-apidoc -o source/tests/2d_heat ../tests/2d_heat
sphinx-apidoc -o source/tests/2d_stress ../tests/2d_stress
sphinx-apidoc -o source/tests/2d_r13 ../tests/2d_r13
sphinx-apidoc -o source/tests/3d_heat ../tests/3d_heat
sphinx-apidoc -o source/tests/3d_stress ../tests/3d_stress
sphinx-apidoc -o source/tests/3d_r13 ../tests/3d_r13
sphinx-apidoc -o source/examples ../examples
make html
make latex
Monitor the performance of the program with e.g.:
htop -p `{ fenicsR13 inputs/1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab.yml > /dev/null & } && echo $!`
Use doctest with
python3 -m doctest -v src/meshes.py
Run
pydocstyle
once in a while- Matplotbib fails when having wrong backend on macOS
- Fix: Add
backend: TkAgg
to~/.matplotlib/matplotlibrc
file
- Fix: Add
Performance in Docker is way better than conda build, especially JIT compilation is faster
Get C++ inlcude paths:
echo | gcc -E -Wp,-v -
- Bessel functions in DOLFIN:
C++17 functions cannpot be used. Boost functions also not per default.
Expression("boost::math::cyl_bessel_i(0,atan2(x[1], x[0]))", degree=2)
is allowed if one changes in file/usr/local/lib/python3.6/dist-packages/dolfin/jit/jit.py
_math_header = """ // cmath functions #include <boost/math/special_functions/bessel.hpp> // Added %s """
Get current work directory:
import os cwd = os.getcwd() print(cwd)
Latex font for matplotlib:
# LaTeX text fonts: # Use with raw strings: r"$\mathcal{O}(h^1)$" plt.rc('text', usetex=True) plt.rc('font', family='serif')
Get system path where modules are searched:
import sys print(sys.path)
- Add CHANGELOG entry
- Adapt version in conf.py for docs and setup.py for package
- Change badge in
README.rst
- Change version in program information printing
- Build new Docker container
- The
build
stage has to be triggered manually when something in the setup changes. This is because it takes a fair amount of time. - In
~/.gitlab-runner/config.toml
(for the runner): - change priviliges to true
- Use local images:
pull_policy = "if-not-present"
- To
[[runners]]
addenvironment = ["DOCKER_TLS_CERTDIR="]
(See https://gitlab.com/gitlab-org/gitlab-ce/issues/64959)
- In
- Run local:
gitlab-runner exec docker --docker-privileged build
or withbuild
replaced by job name - maybe local vars have to be change to use local Docker images because
CI_REGISTRY
,... are not set
- maybe local vars have to be change to use local Docker images because
- Run local:
An example gitlab runner config/toml
in ~/.gitlab-runner
can look like:
concurrent = 1
check_interval = 0
[[runners]]
name = "190716-macbookpro"
url = "https://git.rwth-aachen.de/"
token = "<PRIVATE_TOKEN>"
executor = "docker"
environment = ["DOCKER_TLS_CERTDIR="]
[runners.docker]
tls_verify = false
image = "docker:stable"
privileged = true
disable_cache = false
volumes = ["/cache"]
shm_size = 0
pull_policy = "if-not-present"
[runners.cache]
- Install
miniconda
from here - If using
zsh
, add miniconda bins to PATH:export PATH="$HOME/ miniconda3/bin:$PATH"
to~/.zshrc
- Maybe, activation has to be done with executing
<path to miniconda>/bin/activate
- Optional: Create separate coda environment:
conda creafenics-env
- If using
- Install
- Install FEniCS using conda:
conda install -c conda-forge fenics
- Optional: Install
matplobib
:conda install -c conda-forge matplotlib
- Optional: Install
meshio
:conda install -c mrossi meshio
- Optional (for linting):
conda install pylint
- Install mshr with
conda install -c conda-forge mshr
- Fix macOS bug in matplotbib:
mkdir -p ~/.matplotlib; echo "backend: TkAgg" > ~/.matplotlib/matplotlibrc
- XCode and command line developer tools msut be installed!
- Optional: Install Jupyter:
conda install -c anaconda jupyter
- Optional: Install documentation system:
conda install -c anaconda sphinx
- Optional:
conda install -c anaconda sympy
- Optional: Install
- Install FEniCS using conda:
Interactive Jupyter Notebooks with Microsoft's Visual Studio Code
This is may be a convenient solution.
Run a file with %run ../../fenicsr13/fenicsr13.py
X11 Window Forwarding on OSX
See guide for the programs to install. Then source the open-macos-gui-tunnel.sh
with . open-macos-gui-tunnel
. Afterwards, start the container and run the change-matplotbib-backend-tkagg.sh
script to set the right matplotlib
's output.
X11 Window Forwarding on Windows
A nice guide can be found here on Dev.to.
The steps can be summarized as:
Install the package manager Chocolatey.
REM comment: open cmd.exe as admin @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
Open
cmd.exe
as admin and install VcXsrv Windows X Server.choco install vcxsrv
Open a X11 server and set the
ip
variable (that is used in thedocker-compose.yml
when starting the Docker container to setexport DISPLAY=${ip}:0
).# home of this repo source sripts/open-windows-gui-tunnel.sh
Author: | Lambert Theisen
|
---|---|
Supervisor: | Prof. Dr. Manuel Torrilhon
Lehrstuhl für Mathematik (MathCCES)
RWTH Aachen University
|