Some minor refactoring of the general structure, and a new memory eff…

…icient active subspace algorithm (#11)

* try / excepting krylov solution failure for nonlinear problems

* checking in on tom_dev

* updating drivers and so on to be compatible with hessianlearn

* rearranging things to sort better by paper

* updating markdown readmes

* small readme edit

* more efficient saving procedure

* not sure what changed here

* updating

* updating data saving procedure

* updating some work on improving active subspace memory footprint

* working on memory efficient active subspace slowly

* a lot more refactoring of the E[JTJ] operator and so on for memory issues

* updating

* updating and working towards continuous integration

* typo

* updating unit tests

* missing python instruction

* updating travis yml

* updating yaml for travis

* updating travis yaml again trying to fix build issues related to cloning hippylib

* issue importing hippyflow in the test

* more testing stuff

* more things being resolved

* more ci issues

* updating readme and playing around more with unit tests

* updating readme and unit tests

* updating

* updating

* updating

* struggling to get the unit tests to work properly

* ci issues

* updating again

* updating again

* updating again

* updating please work thanks

* one last try

* continuing issues with travis build

* library linking error is driving me crazy

* library linking error is driving me crazy

* library linking error is driving me crazy

* library linking error is driving me crazy

* exploring all these different ways travis doesnt make any sense

* exploring all these different ways travis doesnt make any sense

* exploring all these different ways travis doesnt make any sense

* exploring all these different ways travis doesnt make any sense

* exploring all these different ways travis doesnt make any sense

* finally got ci working

* adding citation

.travis.yml

@@ -0,0 +1,23 @@
+language: python
+python:
+  - "3.7"
+install:
+  - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
+  - bash miniconda.sh -b -p $HOME/miniconda
+  - source "$HOME/miniconda/etc/profile.d/conda.sh"
+  - hash -r
+  - conda config --set always_yes yes --set changeps1 no
+  - conda update -q conda
+  # Useful for debugging any issues with conda
+  # - conda info -a
+  - conda create -q -n fenics19 -c conda-forge python=$TRAVIS_PYTHON_VERSION fenics==2019.1.0
+  - conda activate fenics19
+  - git clone --depth 1 --branch matmvmult https://github.com/hippylib/hippylib.git
+  - cd hippylib
+  - python setup.py install
+  - cd ../
+  - export HIPPYLIB_PATH=$(pwd)/hippylib/
+  - export HIPPYFLOW_PATH=$(pwd)
+
+script:
+  - python hippyflow/test/test_derivativeSubspace.py

README.md

@@ -27,6 +27,7 @@
 			                                                      
                                                       
 
+[![Build Status](https://travis-ci.com/hippylib/hippyflow.svg?branch=master)](https://travis-ci.com/github/hippylib/hippyflow)
 [![DOI](https://zenodo.org/badge/301823282.svg)](https://zenodo.org/badge/latestdoi/301823282)
 [![License](https://img.shields.io/github/license/hippylib/hippyflow)](./LICENSE.md)
 [![Top language](https://img.shields.io/github/languages/top/hippylib/hippyflow)](https://www.python.org)
@@ -44,13 +45,106 @@
 * [`hessianlearn`](https://github.com/tomoleary/hessianlearn) is used for second order optimization of keras neural network models.
 
 
-## Model Based Projectors:
-
-* `hIPPYflow` implements software infrastructure for input and output dimension reduction strategies for parametric mappings governed by PDEs. Given a parametric PDE Variational Problem implemented in `hIPPYlib` (using `FEniCS` for finite element representation), and a PDE observable, this code automates the construction of dominant subspaces of the input and output for these mappings. `hIPPYflow` implements both active subspace (AS) and Karhunen Loeve expansion (KLE) for input dimension reduction. `hIPPYflow` implements proper orthogonal decomposition (POD) for output dimension reduction.
-
-* These constructs also implement the generation of training data to be used in surrogate construction, as well as projection error tests that exemplify how good the different model projectors are at capturing key information, and help to detect the "intrinsic dimensionality" of the mappings from inputs to outputs.
-
-## Dimension Reduced Neural Network Strategies
+# Model Based Projectors:
+
+`hIPPYflow` implements software infrastructure for input and output dimension reduction strategies for parametric mappings governed by PDEs. Given a parametric PDE Variational Problem implemented in `hIPPYlib` (using `FEniCS` for finite element representation), and a PDE observable, this code automates the construction of dominant subspaces of the input and output for these mappings. 
+
+<p align="center">
+	<img src="https://latex.codecogs.com/gif.latex?\underbrace{q(m) = q(u(m))}_{\text{Implicit dependence}} \quad \text{where $u$ depends on $m$ through } \quad \underbrace{R(u,m) = 0}_{\text{Forward Model}}" /> 
+</p>
+
+
+`hIPPYflow` implements both active subspace (AS) and Karhunen Loeve expansion (KLE) for input dimension reduction. `hIPPYflow` implements proper orthogonal decomposition (POD) for output dimension reduction.
+
+AS computes the dominant eigenvalues of the following operator:
+<p align="center">
+	<img src="https://latex.codecogs.com/gif.latex? \int_{\mathbb{R}^{d_M}} \nabla q(m)^T \nabla q(m) d \nu(m) \in \mathbb{R}^{d_M \times d_M}" /> 
+</p>
+KLE computes the dominant eigenvectors of the covariance of the  parameter distribution 
+<p align="center">
+	<img src="https://latex.codecogs.com/gif.latex? \text{Cov}(\nu(m))" /> 
+</p>
+POD computes the dominant eigenvalues of the following operator:
+<p align="center">
+	<img src="https://latex.codecogs.com/gif.latex? \int_{\mathbb{R}^{d_M}}  q(m) q(m)^T d \nu(m) \in \mathbb{R}^{d_Q \times d_Q}" /> 
+</p>
+
+These constructs also implement the generation of training data to be used in surrogate construction, as well as projection error tests that exemplify how good the different model projectors are at capturing key information, and help to detect the "intrinsic dimensionality" of the mappings from inputs to outputs.
+
+## Example Usage (reduced basis construction)
+
+* Install [`hIPPYlib`](https://github.com/hippylib/hippylib), set `HIPPYLIB_PATH`, `HIPPYFLOW_PATH` environmental variables.
+
+```python
+import dolfin as dl
+import ufl
+import numpy as np
+import sys, os
+sys.path.append(os.environ.get('HIPPYLIB_PATH'))
+from hippylib import *
+
+sys.path.append(os.environ.get('HIPPYFLOW_PATH'))
+from hippyflow import *
+
+# Set up PDE Variational Problem and observable using a function
+def build_observable(mesh, **kwargs):
+	# Set up the PDE problem in hIPPYlib
+	rank = dl.MPI.rank(mesh.mpi_comm())			
+	Vh2 = dl.FunctionSpace(mesh, 'Lagrange', 2)
+	Vh1 = dl.FunctionSpace(mesh, 'Lagrange', 1)
+	Vh = [Vh2, Vh1, Vh2]
+	# Initialize Expressions
+	f = dl.Constant(0.0)
+
+	u_bdr = dl.Expression("x[1]", degree=1)
+	u_bdr0 = dl.Constant(0.0)
+	bc = dl.DirichletBC(Vh[STATE], u_bdr, u_boundary)
+	bc0 = dl.DirichletBC(Vh[STATE], u_bdr0, u_boundary)
+
+	def pde_varf(u,m,p):
+		return ufl.exp(m)*ufl.inner(ufl.grad(u), ufl.grad(p))*ufl.dx - f*p*ufl.dx
+
+	pde = PDEVariationalProblem(Vh, pde_varf, bc, bc0, is_fwd_linear=True)
+
+	# Instance observable operator (in this case pointwise observation of state)
+	x_targets = np.linspace(0.1,0.9,10)
+	y_targets = np.linspace(0.1,0.9,10)
+	targets = []
+	for xi in x_targets:
+		for yi in y_targets:
+			targets.append((xi,yi))
+	targets = np.array(targets)
+
+	B = assemblePointwiseObservation(Vh[STATE], targets)
+	return LinearStateObservable(pde,B)
+
+# Set up mesh
+ndim = 2
+nx = 10
+ny = 10
+mesh = dl.UnitSquareMesh(nx, ny)
+# Instance observable
+observable_kwargs = {} # No kwargs given in this example
+observable = build_observable(mesh,**observable_kwargs)
+
+# Instance probability distribution for the parameter
+prior = BiLaplacian2D(Vh[PARAMETER],gamma = 0.1, delta = 1.0)
+
+# Instance Active Subspace Operator
+AS = ActiveSubspaceProjector(observable,prior)
+# Compute and save input reduced basis to file:
+AS.construct_input_subspace()
+
+# Instance POD Operator to compute POD basis and training data
+POD = PODProjector(observable,prior)
+POD.construct_subspace()
+output_directory = 'location/for/training/data/'
+POD.generate_training_data(output_directory)
+
+```
+
+
+# Dimension Reduced Neural Network Strategies
 
 * Given information about dominant subspaces of the input and output spaces for the parametric mappings, `hIPPYflow` implements dimension reduced neural network surrogates. These surrogates allow for parsimonious representations of input-output mappings that can achieve good accuracy for very few training data. Few data is a key feature of many high dimensional PDE based inference problems. 
 
@@ -63,13 +157,13 @@ These publications use the hippyflow library
 
 - \[1\] O'Leary-Roseberry, T., Villa, U., Chen P., Ghattas O.,
 [**Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs**](https://arxiv.org/abs/2011.15110).
-arXiv:2011.15110.
+Computer Methods in Applied Mechanics and Engineering (accepted).
 ([Download](https://arxiv.org/pdf/2011.15110.pdf))<details><summary>BibTeX</summary><pre>
 @article{o2020derivative,
   title={Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs},
   author={O'Leary-Roseberry, Thomas and Villa, Umberto and Chen, Peng and Ghattas, Omar},
-  journal={arXiv preprint arXiv:2011.15110},
-  year={2020}
+  journal={Computer Methods in Applied Mechanics and Engineering},
+  year={2021}
 }
 }</pre></details>
 

applications/confusion/confusion_linear_observable.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/confusion/confusion_problem_setup.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/confusion/confusion_utilities.py

@@ -1,4 +1,5 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.
@@ -21,27 +22,41 @@ def load_confusion_data(data_dir,rescale = False,derivatives = False,n_data = np
 
 	m_files = []
 	q_files = []
+	mq_files = []
 	for file in data_files:
 		if 'ms_on_rank_' in file:
 			m_files.append(file)
 		if 'qs_on_rank_' in file:
 			q_files.append(file)
+		if 'mq_on_rank_' in file:
+			mq_files.append(file)
 
-	ranks = [int(file.split(data_dir+'ms_on_rank_')[-1].split('.npy')[0]) for file in m_files]
+	if len(mq_files) == 0:
+		ranks = [int(file.split(data_dir+'ms_on_rank_')[-1].split('.npy')[0]) for file in m_files]
+	else:
+		ranks = [int(file.split(data_dir+'mq_on_rank_')[-1].split('.npz')[0]) for file in m_files]
 	max_rank = max(ranks)
 
 	# Serially concatenate data
-	m_data = np.load(data_dir+'ms_on_rank_0.npy')
-	q_data = np.load(data_dir+'qs_on_rank_0.npy')
-	for i in range(1,max_rank+1):
-			appendage_m = np.load(data_dir+'ms_on_rank_'+str(i)+'.npy')
+	if len(mq_files) == 0:
+		m_data = np.load(data_dir+'ms_on_rank_0.npy')
+		q_data = np.load(data_dir+'qs_on_rank_0.npy')
+		for i in range(1,max_rank+1):
+				appendage_m = np.load(data_dir+'ms_on_rank_'+str(i)+'.npy')
+				m_data = np.concatenate((m_data,appendage_m))
+				appendage_q = np.load(data_dir+'qs_on_rank_'+str(i)+'.npy')
+				q_data = np.concatenate((q_data,appendage_q))
+	else:
+		npz_data = np.load(data_dir+'mq_on_rank_0.npz')
+		m_data = npz_data['m_data']
+		q_data = npz_data['q_data']
+		for i in range(1,max_rank+1):
+			npz_data = np.load(data_dir+'mq_on_rank_'+str(i)+'.npz')
+			appendage_m = npz_data['m_data']
+			appendage_q = npz_data['q_data']
 			m_data = np.concatenate((m_data,appendage_m))
-			appendage_q = np.load(data_dir+'qs_on_rank_'+str(i)+'.npy')
 			q_data = np.concatenate((q_data,appendage_q))
-
-	# print('m_data.shape = ',m_data.shape)
-	# print('q_data.shape = ',q_data.shape)
-
+
 	if n_data < np.inf:
 		assert type(n_data) is int
 		m_data = m_data[:n_data]

applications/confusion/dipnet_paper/confusion_multirun.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/confusion/dipnet_paper/confusion_training.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/confusion/dipnet_paper/neuralNetworks.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/confusion/generate_confusion.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/helmholtz_2d/HelmholtzProblem.py

@@ -1,7 +1,7 @@
 
 # Copyright (c) 2016-2018, The University of Texas at Austin 
 # & University of California--Merced.
-# Copyright (c) 2019-2020, The University of Texas at Austin 
+# Copyright (c) 2019-2021, The University of Texas at Austin 
 # University of California--Merced, Washington University in St. Louis.
 #
 # All Rights reserved.

applications/helmholtz_2d/dipnet_paper/helmholtz_multirun.py

@@ -1,16 +1,15 @@
-# This file is part of the hIPPYflow package
+# Copyright (c) 2020-2021, The University of Texas at Austin 
+# & Washington University in St. Louis.
 #
-# hIPPYflow is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or any later version.
+# All Rights reserved.
+# See file COPYRIGHT for details.
 #
-# hIPPYflow is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
+# This file is part of the hIPPYflow package. For more information see
+# https://github.com/hippylib/hippyflow/
 #
-# You should have received a copy of the GNU General Public License
-# If not, see <http://www.gnu.org/licenses/>.
+# hIPPYflow is free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License (as published by the Free
+# Software Foundation) version 2.0 dated June 1991.
 #
 # Author: Tom O'Leary-Roseberry
 # Contact: tom.olearyroseberry@utexas.edu

applications/helmholtz_2d/dipnet_paper/helmholtz_training.py

@@ -1,16 +1,15 @@
-# This file is part of the hIPPYflow package
+# Copyright (c) 2020-2021, The University of Texas at Austin 
+# & Washington University in St. Louis.
 #
-# hIPPYflow is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or any later version.
+# All Rights reserved.
+# See file COPYRIGHT for details.
 #
-# hIPPYflow is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
+# This file is part of the hIPPYflow package. For more information see
+# https://github.com/hippylib/hippyflow/
 #
-# You should have received a copy of the GNU General Public License
-# If not, see <http://www.gnu.org/licenses/>.
+# hIPPYflow is free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License (as published by the Free
+# Software Foundation) version 2.0 dated June 1991.
 #
 # Author: Tom O'Leary-Roseberry
 # Contact: tom.olearyroseberry@utexas.edu

applications/helmholtz_2d/dipnet_paper/neuralNetworks.py

@@ -1,16 +1,15 @@
-# This file is part of the hIPPYflow package
+# Copyright (c) 2020-2021, The University of Texas at Austin 
+# & Washington University in St. Louis.
 #
-# hIPPYflow is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or any later version.
+# All Rights reserved.
+# See file COPYRIGHT for details.
 #
-# hIPPYflow is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
+# This file is part of the hIPPYflow package. For more information see
+# https://github.com/hippylib/hippyflow/
 #
-# You should have received a copy of the GNU General Public License
-# If not, see <http://www.gnu.org/licenses/>.
+# hIPPYflow is free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License (as published by the Free
+# Software Foundation) version 2.0 dated June 1991.
 #
 # Author: Tom O'Leary-Roseberry
 # Contact: tom.olearyroseberry@utexas.edu

applications/helmholtz_2d/generate_helmholtz.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/helmholtz_2d/helmholtz_linear_observable.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.

applications/helmholtz_2d/helmholtz_problem_setup.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020, The University of Texas at Austin 
+# Copyright (c) 2020-2021, The University of Texas at Austin 
 # & Washington University in St. Louis.
 #
 # All Rights reserved.