Threaded tests should use OPENMP=true builds

[sbrus89]

The OPENMP=true make option is required in order to test threading, otherwise I believe the !$omp directives are treated as comments and the OMP_NUM_THREADS environment variable is ignored.

[xylar]

Thanks, I don't think I've been building with this when I run the pr test suite and clearly I need to be.

I'm trying to think through how to make OPENMP=true become kind of the default (at least for regression testing) and OPENMP=false be something you only do if you know you're running tests that don't use threads. That might fit into #346.

[xylar]

I think we should set OPENMP=true in the load script unless you supply --no_openmp when you run ./conda/configure_compass_env.py. I think it is too dangerous not to have OpenMP on in general.

[xylar]

@mark-petersen and @matthewhoffman, what are your thoughts?

[mark-petersen]

I don't understand. I always compile with openmp at the command line:

make gfortran  OPENMP=true DEBUG=true GEN_F90=true

I thought that was the only way to do it. @xylar are you saying that we could set openmp to be true in the configure_compass_env as an environmental variable?

[xylar]

Yes, you can just set the environment variable OPENMP=true (or DEBUG=true) as environment variables as far as I understand it. This could be part of the load_*.sh scripts.

You could purposefully omit this with ./conda/configure_compass_env.py --no_openmp ....

[matthewhoffman]

We don't have any openmp code in MALI (not on the MPAS side anyway), so this issues doesn't effect me. But from what I understand, I would agree that for MPAS-Ocean it makes sense to be running with it on by default. And enforcing that via an env variable sounds logical to me.

[xylar]

@mark-petersen, I verified that setting:

export USE_PIO2=true
export OPENMP=true
export DEBUG=true

has the desired effect. I see:

$ make intel-mpi
( make all \
"FC_PARALLEL = mpiifort" \
"CC_PARALLEL = mpiicc" \
"CXX_PARALLEL = mpiicpc" \
"FC_SERIAL = ifort" \
"CC_SERIAL = icc" \
"CXX_SERIAL = icpc" \
"FFLAGS_FPIEEE = -fp-model=precise" \
"FFLAGS_PROMOTION = -real-size 64" \
"FFLAGS_OPT = -O3 -convert big_endian -free -align array64byte" \
"CFLAGS_OPT = -O3" \
"CXXFLAGS_OPT = -O3" \
"LDFLAGS_OPT = -O3" \
"FFLAGS_DEBUG = -g -convert big_endian -free -CU -CB -check all -fpe0 -traceback" \
"CFLAGS_DEBUG = -g -traceback" \
"CXXFLAGS_DEBUG = -g -traceback" \
"LDFLAGS_DEBUG = -g -fpe0 -traceback" \
"FFLAGS_OMP = -qopenmp" \
"CFLAGS_OMP = -qopenmp" \
"PICFLAG = -fpic" \
"BUILD_TARGET = intel-mpi" \
"CORE = ocean" \
"DEBUG = true" \
"USE_PAPI = " \
"OPENMP = true" \
"CPPFLAGS =  -D_MPI" )
make[1]: Entering directory `/gpfs/fs1/home/ac.xylar/mpas-work/compass/compass/E3SM-Project/components/mpas-ocean'
Testing compiler for OpenMP support
Checking for a usable PIO library...
=> PIO 2 detected
...