Upgrade ensmallen to 2.14.1 (#31)

* Upgrade ensmallen to 2.14.1

* Fix NEWS.md entry.
Add GitHub Action development note.

* Add ChangeLog Update

* Switch to HTTPS for ensmallen and mlpack websites

Co-authored-by: coatless <coatless@users.noreply.github.com>
Co-authored-by: James Balamuta <balamut2@illinois.edu>

ChangeLog

@@ -1,10 +1,21 @@
 2020-08-22  James Balamuta  <balamut2@illinois.edu>
 
+	* DESCRIPTION (Version): Release 2.14.1
+	* NEWS.md: Update for Ensmallen release 2.14.1
+	* inst/include/ensmallen_bits: Upgraded to Ensmallen 2.14.1
+	* inst/include/ensmallen.hpp: ditto
+
 	* .github/workflows/R-CMD-check.yaml: Re-enable devel build check by
-	using Linux instead of macOS to perform the check.
+	using Linux instead of macOS to perform the check. Disables release
+  testing on Linux.
 
 	* .github/workflows/upstream-refresh.yaml: Fix NEWS.md entry formatting
 	issues.
+
+2020-08-21  James Balamuta  <balamut2@illinois.edu>
+
+	* .github/workflows/upstream-refresh.yaml: Add GitHub Action to
+	  automatically retrieve and apply a new upstream release.
 
 2020-07-24  James Balamuta  <balamut2@illinois.edu>
 

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: RcppEnsmallen
 Title: Header-Only C++ Mathematical Optimization Library for 'Armadillo'
-Version: 0.2.13.0.2
+Version: 0.2.14.1.1
 Authors@R: c(
     person("James Joseph", "Balamuta", email = "balamut2@illinois.edu", 
            role = c("aut", "cre", "cph"), 

NEWS.md

@@ -1,9 +1,20 @@
-# RcppEnsmallen 0.2.13.0.2
-
+# RcppEnsmallen 0.2.14.1.1
+
+- Upgraded to ensmallen 2.14.1: "No Direction Home" (2020-08-21)
+  - Fix release script (remove hardcoded information, trim leading
+    whitespaces introduced by `wc -l` in MacOS)
+    ([#216](https://github.com/mlpack/ensmallen/pull/216), [#220](https://github.com/mlpack/ensmallen/pull/220)).
+  - Adjust tolerance for AugLagrangian convergence based on element type ([#217](https://github.com/mlpack/ensmallen/pull/217)). 
+  - Add NSGA2 optimizer for multi-objective functions ([#149](https://github.com/mlpack/ensmallen/pull/149)). 
+  - Update automatic website update release script ([#207](https://github.com/mlpack/ensmallen/pull/207)). 
+  - Clarify and fix documentation for constrained optimizers ([#201](https://github.com/mlpack/ensmallen/pull/201)). 
+  - Fix L-BFGS convergence when starting from a minimum ([#201](https://github.com/mlpack/ensmallen/pull/201)).
 - Switch GitHub Actions to use reference tags to always be up-to-date with 
-  CRAN's check grid (`oldrel`, `release`, `devel`).
+  CRAN's check grid (`oldrel`, `release`, `devel`). ([#29](https://github.com/coatless/rcppensmallen/pull/29), 
+  [#32](https://github.com/coatless/rcppensmallen/pull/32))
+- Added a GitHub Action to automatically create a PR with the new version of
+  Ensmallen when a new release is detected. ([#30](https://github.com/coatless/rcppensmallen/pull/30), [#33](https://github.com/coatless/rcppensmallen/pull/33))
 
-
 # RcppEnsmallen 0.2.13.0.1
 
 - Upgraded to ensmallen 2.13.0: "Automatically Automated Automation" (2020-07-15)
@@ -177,4 +188,3 @@
 - The crux of this package has largely been developed by 
   Ensmallen development team that consists of Conrad Sanderson, Ryan Curtin, 
   and the MLPACK developers
-

README.md

@@ -9,8 +9,8 @@
 
 ## Overview
 
-[Ensmallen](http://ensmallen.org/) is a C++ header-only mathematical optimization
-library written by developers behind [MLPACK](http://www.mlpack.org/) that 
+[Ensmallen](https://ensmallen.org/) is a C++ header-only mathematical optimization
+library written by developers behind [MLPACK](https://www.mlpack.org/) that 
 provides a simple set of abstractions for writing an objective function to 
 optimize. Provided within are various standard and cutting-edge optimizers that
 include full-batch gradient descent techniques, small-batch techniques, 

inst/include/ensmallen.hpp

@@ -35,18 +35,22 @@
 
 #include <cctype>
 #include <cfloat>
+#include <climits>
 #include <cmath>
 #include <cstdint>
 #include <cstdio>
 #include <cstdlib>
 #include <cstring>
 #include <iostream>
+#include <map>
+#include <set>
 #include <limits>
 #include <sstream>
 #include <stdexcept>
 #include <string>
 #include <tuple>
 #include <utility>
+#include <vector>
 
 // On Visual Studio, disable C4519 (default arguments for function templates)
 // since it's by default an error, which doesn't even make any sense because
@@ -94,6 +98,7 @@
 #include "ensmallen_bits/katyusha/katyusha.hpp"
 #include "ensmallen_bits/lbfgs/lbfgs.hpp"
 #include "ensmallen_bits/lookahead/lookahead.hpp"
+#include "ensmallen_bits/nsga2/nsga2.hpp"
 #include "ensmallen_bits/padam/padam.hpp"
 #include "ensmallen_bits/parallel_sgd/parallel_sgd.hpp"
 #include "ensmallen_bits/pso/pso.hpp"

inst/include/ensmallen_bits/aug_lagrangian/aug_lagrangian_impl.hpp

@@ -27,7 +27,8 @@ inline AugLagrangian::AugLagrangian(const size_t maxIterations,
     penaltyThresholdFactor(penaltyThresholdFactor),
     sigmaUpdateFactor(sigmaUpdateFactor),
     lbfgs(lbfgs),
-    terminate(false)
+    terminate(false),
+    sigma(0.0)
 {
 }
 
@@ -107,6 +108,10 @@ AugLagrangian::Optimize(
   // Track the last objective to compare for convergence.
   ElemType lastObjective = function.Evaluate(coordinates);
 
+  // Convergence tolerance---depends on the epsilon of the type we are using for
+  // optimization.
+  ElemType tolerance = 1e3 * std::numeric_limits<ElemType>::epsilon();
+
   // Then, calculate the current penalty.
   ElemType penalty = 0;
   for (size_t i = 0; i < function.NumConstraints(); i++)
@@ -134,6 +139,7 @@ AugLagrangian::Optimize(
     if (!lbfgs.Optimize(augfunc, coordinates, callbacks...))
       Info << "L-BFGS reported an error during optimization."
           << std::endl;
+    Info << "Done with L-BFGS: " << coordinates << "\n";
 
     const ElemType objective = function.Evaluate(coordinates);
 
@@ -142,7 +148,7 @@ AugLagrangian::Optimize(
 
     // Check if we are done with the entire optimization (the threshold we are
     // comparing with is arbitrary).
-    if (std::abs(lastObjective - objective) < 1e-10 &&
+    if (std::abs(lastObjective - objective) < tolerance &&
         augfunc.Sigma() > 500000)
     {
       lambda = std::move(augfunc.Lambda());
@@ -196,6 +202,13 @@ AugLagrangian::Optimize(
       // We multiply sigma by a constant value.
       augfunc.Sigma() *= sigmaUpdateFactor;
       Info << "Updated sigma to " << augfunc.Sigma() << "." << std::endl;
+      if (augfunc.Sigma() >= std::numeric_limits<ElemType>::max() / 2.0)
+      {
+        Warn << "AugLagrangian::Optimize(): sigma too large for element type; "
+            << "terminating." << std::endl;
+        Callback::EndOptimization(*this, function, coordinates, callbacks...);
+        return false;
+      }
     }
 
     terminate |= Callback::StepTaken(*this, function, coordinates,

inst/include/ensmallen_bits/ens_version.hpp

@@ -15,13 +15,13 @@
 #define ENS_VERSION_MAJOR 2
 // The minor version is two digits so regular numerical comparisons of versions
 // work right.  The first minor version of a release is always 10.
-#define ENS_VERSION_MINOR 13
+#define ENS_VERSION_MINOR 14
 #define ENS_VERSION_PATCH 0
 // If this is a release candidate, it will be reflected in the version name
 // (i.e. the version name will be "RC1", "RC2", etc.).  Otherwise the version
 // name will typically be a seemingly arbitrary set of words that does not
 // contain the capitalized string "RC".
-#define ENS_VERSION_NAME "Automatically Automated Automation"
+#define ENS_VERSION_NAME "No Direction Home"
 
 namespace ens {
 

inst/include/ensmallen_bits/lbfgs/lbfgs_impl.hpp

@@ -391,7 +391,7 @@ L_BFGS::Optimize(FunctionType& function,
     //
     // But don't do this on the first iteration to ensure we always take at
     // least one descent step.
-    if (itNum > 0 && (arma::norm(gradient, 2) < minGradientNorm))
+    if (arma::norm(gradient, 2) < minGradientNorm)
     {
       Info << "L-BFGS gradient norm too small (terminating successfully)."
           << std::endl;