Skip to content
This repository has been archived by the owner on Feb 8, 2022. It is now read-only.

Installation From Source Code

Jump to bottom
Philip Maechling edited this page Sep 5, 2021 · 44 revisions

Required Software Stack

UCVMC is developed for computers that use recent Linux distributions. We define a reference target computing environment. It uses GNU automake, C and Fortran compilers, and Python 3.8. We use the reference environment for development and testing. This is the supported computing environment. Our development environment is available on the USC Center for Advanced Research Computing (CARC) cluster computing environment, and should be readily available through most cloud providers. It may be possible to build and run UCVM in other computing environments, like MacOS, but we currently support only Linux distributions with specific compiler and software versions.

  • Linux operating system (e.g. Centos-8) x86_64-linux
  • vi or emacs
  • Automake, Autoconf build tools for Linux
  • Git client
  • GNU gcc/gfortran compilers version 8.3+
  • Python 3.8 (Anaconda 4.10.0 (64-bit))
  • Autotools build software for Linux

External Libraries Installed by UCVMC:

Optional Software for building MPI binaries:

  • openmpi 4.0.1

The following California velocity models packages are included as part of a standard UCVMC installation. Each of these models can be downloaded during the installation process:

  • SCEC CVM-S4.26: 'cvms5':'1.2G'
  • SCEC CCA06: 'cca':'9.2G'
  • SCEC CS17.3: 'cs173':'72G'
  • SCEC CS17.3H: 'cs173h':'72G'
  • SCEC CVM-S4: 'cvms4':'326M'
  • SCEC CVM-S4.26.M01: 'cvmsi':'1.6G'
  • USGS BayArea Velocity Model 0.8.3: 'cencal080':'21G'
  • SCEC CVM-H v15.1": 'cvmh':'1.6G'
  • Albacore: 'albacore':'2.3M'
  • Coachela Valley: 'cvlsu':'1M'
  • Imperial Valley: 'ivlsu':'1M'

Two 1D models are defined in the ucvm codebase:

  • Southern California 1D Velocity Model
  • Northridge Region 1D Velocity Model

Standard UCVMC installation requires the following considerations:

Installation will require two directories (1) the source directory, and (2) the installation directory. The source directory requires at least 23Gb of storage (231Gb with both CS17.3 and CS17.3H). The installation directory requires at least 21Gb of storage (165G with both CS17.3 and CS17.3H). While preparing to install UCVMC, users should identify filesystems on their computer that have these amounts of free disk space.

As part of the installation process, users download the UCVMC code from the git repository into the source directory. Once the git distribution has been downloaded, the user will need to retrieve "largefiles" into a largefiles subdirectory in the source directory tree. Once the largefiles have been retrieved, and the user has to run the stage_ucvm_files.py script that moves the files into the proper subdirectories for installation, the source directory tree will require at least 231Gb of storage, assuming all standard CVM models are installed.

In general use, once the UCVMC software has been installed, all required files are copied into the installation directory tree and the source directory tree can be deleted. Advanced users that want to modify the UCVMC codebase, or might want to change installation location, or that are building their own customized etrees, will need the source directory tree, in which case, it should not be removed. The installation directory should not be moved once installed because of the model configuration. If a new location is desired, ucvm_setup.py in the source directory should be rerun with the new location.

The UCVMC application is built using gnu automake tools. These automake tools extend the standard Make system. While these tools add another layer of complexity into the system, they automatically construct the required Makefiles, and users do not need to edit Makefiles by hand. When the automake tools work properly, they will build the UCVMC software automatically, with warnings, but no errors, on a variety of Linux and GNU software stacks. The automake tools, and GNU C and FORTRAN compilers, and a Python 2.7 distribution with extensions, must be installed on the target system for the UCVMC installation scripts to work.

The basic steps to building the application are:

  1. Identify the directories on the installation computer that have sufficient storage for the source directory, the installation directory, and the largefiles collection. Users should also confirm that the installation system has the required compilers (GNU C and Fortran) and Python 2.7 (we recommend the Anaconda distribution, a free commercial python distribution that includes many optional python packages) installed.

  2. Move to the UCVM source directory. Check the UCVMC distribution out from github with:

git clone https://github.com/SCECCode/UCVMC.git
  1. Move to the UCVMC/largefiles subdirectory, the user will run:
./get_large_files.py

This script runs in a terminal window and asks user which large file is to be retrieved and will display the message below when all requested large files are downloaded:

...
Completed all required downloads. This computer is now ready to build and install the UCVMC programs and scripts.
  1. To check that the downloaded files are intact, the user will run:
./check_largefiles_md5.py

The script will display a message similar to below. If the script reports and errors, please re-download that file, and try again. If you get repeated errors, please contact us for help.

...
Checked 10 UCVM large files with: 10 OK, and: 0 errors

The required md5sum utility does not exist on Mac. If you are installing on a Mac, skip this check_largefiles_md5.py step, and proceed, assuming that the downloaded largefiles are intact.

  1. Still in the UCVMC/largefiles subdirectory, to move the largefiles to the required directories for installation, the user will run:
./stage_large_files.py

The script will display the message below:

All required library and model files staged in ucvmc work directories.
  1. Move back to the UCVMC source directory and run:
./ucvm_setup.py

  1. It will ask you questions about the installation directory, and which retrieved models you wish to install. The default installation is to install all the models. The installation script will try to create the specified installation directory. The specified directory does not need to exist.

  2. The automake tools will run, configure, make, install, which will build the system, and install it in the installation directory. The config, make, install process takes about 10 minutes.

When the script completes successfully, it will display the following instructions:

Installing UCVM software...

Running aclocal

Running automake

Running autoconf

Running ./configure for UCVM

Making UCVM

Installing UCVM

Done installing UCVM!

Thank you for installing UCVM. Please export the following library paths (note this is in Bash format):
	LD_LIBRARY_PATH=/home/scec-00/maechlin/t1/lib/euclid3/lib:$LD_LIBRARY_PATH
	LD_LIBRARY_PATH=/home/scec-00/maechlin/t1/lib/proj-5/lib:$LD_LIBRARY_PATH
	LD_LIBRARY_PATH=/home/scec-00/maechlin/t1/model/cvms426/lib:$LD_LIBRARY_PATH
	LD_LIBRARY_PATH=/home/scec-00/maechlin/t1/model/cencal/lib:$LD_LIBRARY_PATH
	export LD_LIBRARY_PATH        
        UCVM_INSTALL_PATH=/home/scec-00/maechlin/t1
        export UCVM_INSTALL_PATH

We recommend adding the above lines to the end of your ~/.bashrc file so that they are preserved for the next time you login.

Once you have set these environment variables, return to the UCVMC source directory and type

make check

This will run the UCVMC unit and acceptance tests. If all tests pass. UCVMC is correctly installed and ready to use on your computer.

A successful make check includes following messages. The model unittest tests only the models you have installed.

...
./unittest
Test: UCVM lib initialization
PASS
Test: UCVM lib add model 1D
PASS
Test: UCVM lib query 1D
PASS
Test: UCVM lib get model label 1D
PASS
Test: UCVM lib setparam querymode geo-depth 1D
PASS
Test: UCVM lib setparam querymode geo-elev 1D
PASS
Test: UCVM lib model version 1D
PASS
Test: UCVM lib add model USGS CenCal
PASS
Test: UCVM lib add model SCEC CVM-H
PASS
Test: UCVM lib add model SCEC CVM-S
PASS
Test: UCVM lib add model SCEC CVM-SI
PASS
./accepttest
Test: ucvm library 1d model w/ large grid
PASS
./accept_test.py /home/scec-00/maechlin/t1/tests
Runnning test test_ssh_generate
[SUCCESS]
Runnning test test_vs30_query
[SUCCESS]

...

To try out ucvm, once the tests pass, move to the UCVMC installation directory, and run an example query.

As an example:

./bin/ucvm_query -f ./conf/ucvm.conf -m cvms < ./tests/inputs/test_latlons.txt

You will then see the following output:

Using Geo Depth coordinates as default mode.
 -118.0000    34.0000      0.000    280.896    468.400       cvms    696.491    213.000   1974.976       none      0.000      0.000      0.000      crust    696.491    213.000   1974.976
 -118.0000    34.0000     50.000    280.896    468.400       cvms   1669.540    548.000   2128.620       none      0.000      0.000      0.000      crust   1669.540    548.000   2128.620
 -118.0000    34.0000    100.000    280.896    468.400       cvms   1683.174    603.470   2130.773       none      0.000      0.000      0.000      crust   1683.174    603.470   2130.773
 -118.0000    34.0000    500.000    280.896    468.400       cvms   3097.562   1656.495   2354.105       none      0.000      0.000      0.000      crust   3097.562   1656.495   2354.105
 -118.0000    34.0000   1000.000    280.896    468.400       cvms   3660.809   2056.628   2443.042       none      0.000      0.000      0.000      crust   3660.809   2056.628   2443.042

Using a User Response File

If you are performing repeated installations, you may want to put the required inputs into a file and run the installation as a script, something like the following:

  1. Create install_inputs.txt containing these lines:
/home/scec-00/username/ucvm194
yes
yes
yes
yes
yes
yes
yes
yes
  1. In UCVMC source directory
./ucvm_setup.py < install_inputs.txt &> install_results.log &
  1. The automake, configure, install process takes about 30 minutes on a recent laptop level system.

ucvm_setup.py messages

As the ucvm_setup.py script, it prints user instructions to the screen. These can be hard to read or capture while the script is running. These two messages are information messages. If the user follows the installation process as described, these messages can be ignored and UCVM will still be usable. These messages are potentially relevant for advanced users that want to make use of libraries used by UCVM that installed during UCVM process. These message are shown below:

The following message in associated with the fast-courier transfer library the UCVM installs and uses:

----------------------------------------------------------------------
Libraries have been installed in:
   /home/scec-00/maechlin/t1/lib/fftw/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

The following message is associated with the USGS Bay Area model which is installed as port of the UCVM platform.

----------------------------------------------------------------------
Libraries have been installed in:
   /home/scec-00/maechlin/t1/model/cencal/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

Setting Required Environment Variables

Default environment uses a bash shell. Edit the .bash_profile, adding these entries:

# UCVM Install Cmds
LD_LIBRARY_PATH=/home/scec-00/username/t1/lib/euclid3/lib:$LD_LIBRARY_PATH
LD_LIBRARY_PATH=/home/scec-00/username/t1/lib/proj-5/lib:$LD_LIBRARY_PATH
LD_LIBRARY_PATH=/home/scec-00/username/t1/model/cvms426/lib:$LD_LIBRARY_PATH
LD_LIBRARY_PATH=/home/scec-00/username/t1/model/cencal/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
UCVM_INSTALL_PATH=/home/scec-00/maechlin/t1
export UCVM_INSTALL_PATH
#
PYTHONPATH=${PYTHONPATH}:/home/scec-00/username/t3/utilities:/home/scec-00/username/t1/utilities/pycvm
export PYTHONPATH

Then make the setting take effect by typing: %source .bashrc

Confirm UCVMC MPI utilities (optional)

UCVMC contains MPI codes for building large meshes and for retrieving basin depths using parallel codes and You can confirm the MPI utilities were correctly built and installed using examples in examples/programs. They are designed to run on USC's hpc cluster.

UCVMC is ready for use

A description of the basic UCVMC commands can be found in the UCVMC wiki: UCVM Command Reference

This wiki is licensed by University of Southern California (USC) to the public under a Creative Commons Attribution 4.0 license.

Documentation

  1. Introduction
  2. System Requirements
  3. Installation on Linux
  4. Registered CVMs
    1. GTL
  5. List of Commands
    1. Manual Pages
    2. Basic UCVM Commands
  6. Query by Elevation or Depth
  7. Plotting Examples
    1. Colorscale
    2. Poisson Datatype
  8. Tests
    1. Test Cases 1D Models
    2. Test Cases CVM S4.26
    3. Test Cases CVM-H
    4. Test Cases Basin Query
    5. Test Cases ucvm2mesh
    6. Test Cases ucvm2mesh mpi
    7. Test Cases Adding Elygtl:ifunc
    8. Test Cases Tiling
    9. Test Cases ucvm_query
    10. Test Cases Vs30 Map
    11. Test Cases Mesh and Grid
    12. Test Cases Invalid Points
    13. Test Cases External CVMs
  9. Troubleshooting
Clone this wiki locally