From 95404e48b51f7717a4cafdb304c7fbbf1bae7b8e Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Tue, 23 Jan 2024 04:19:55 +0000 Subject: [PATCH] build based on fa9f6d0 --- dev/.documenter-siteinfo.json | 2 +- dev/algorithms/algorithms/index.html | 2 +- dev/algorithms/auglag/index.html | 2 +- dev/algorithms/hyperopt/index.html | 2 +- dev/algorithms/ipopt/index.html | 2 +- dev/algorithms/metaheuristics/index.html | 2 +- dev/algorithms/minlp/index.html | 2 +- dev/algorithms/mma/index.html | 2 +- dev/algorithms/mts/index.html | 2 +- dev/algorithms/nlopt/index.html | 2 +- dev/algorithms/nomad/index.html | 2 +- dev/algorithms/sdp/index.html | 2 +- dev/algorithms/surrogate/index.html | 2 +- dev/algorithms/tobs/index.html | 2 +- dev/gradients/chainrules_fd/index.html | 2 +- dev/gradients/gradients/index.html | 2 +- dev/gradients/history/index.html | 2 +- dev/gradients/implicit/index.html | 2 +- dev/gradients/other_ad/index.html | 2 +- dev/gradients/sparse/index.html | 2 +- dev/gradients/symbolic/index.html | 2 +- dev/gradients/user_defined/index.html | 2 +- dev/index.html | 2 +- dev/problem/dict_model/index.html | 2 +- dev/problem/model/index.html | 2 +- dev/problem/problem/index.html | 2 +- dev/problem/queries/index.html | 2 +- dev/result/index.html | 2 +- 28 files changed, 28 insertions(+), 28 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 765ca78..74d0460 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.9.4","generation_timestamp":"2023-12-10T07:49:51","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-01-23T04:19:51","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/dev/algorithms/algorithms/index.html b/dev/algorithms/algorithms/index.html index 496438d..008f37c 100644 --- a/dev/algorithms/algorithms/index.html +++ b/dev/algorithms/algorithms/index.html @@ -1,2 +1,2 @@ -Overview · Nonconvex.jl
GitHub

Algorithms

Overview of algorithms

A summary of all the algorithms available in Nonconvex through different packages is shown in the table below. Scroll right to see more columns and see a description of the columns below the table.

Algorithm nameIs meta-algorithm?Algorithm packageOrderFinite boundsInfinite boundsInequality constraintsEquality constraintsSemidefinite constraintsInteger variables
Method of moving asymptotes (MMA)❌NonconvexMMA.jl (pure Julia) or NLopt.jl1✅✅✅❌❌❌
Primal dual interior point method❌Ipopt.jl1 or 2✅✅✅✅❌❌
DIviding RECTangles algorithm (DIRECT)❌NLopt.jl0✅❌✅❌❌❌
Controlled random search (CRS)❌NLopt.jl0✅✅❌❌❌❌
Multi-Level Single-Linkage (MLSL)LimitedNLopt.jlDepends on sub-solver✅✅❌❌❌❌
StoGo❌NLopt.jl1✅❌❌❌❌❌
AGS❌NLopt.jl0✅❌✅❌❌❌
Improved Stochastic Ranking Evolution Strategy (ISRES)❌NLopt.jl0✅✅✅✅❌❌
ESCH❌NLopt.jl0✅✅❌❌❌❌
COBYLA❌NLopt.jl0✅✅✅✅❌❌
BOBYQA❌NLopt.jl0✅✅❌❌❌❌
NEWUOA❌NLopt.jl0✅✅❌❌❌❌
Principal AXIS (PRAXIS)❌NLopt.jl0✅✅❌❌❌❌
Nelder Mead❌NLopt.jl0✅✅❌❌❌❌
Subplex❌NLopt.jl0✅✅❌❌❌❌
CCSAQ❌NLopt.jl1✅✅✅❌❌❌
SLSQP❌NLopt.jl1✅✅✅✅❌❌
TNewton❌NLopt.jl1❌✅❌❌❌❌
Shifted limited-memory variable-metric❌NLopt.jl1❌✅❌❌❌❌
Augmented Lagrangian in NLoptLimitedNLopt.jlDepends on sub-solver✅✅✅✅❌❌
Augmented Lagrangian in Percival❌Percival.jl1 or 2✅✅✅✅❌❌
Multiple trajectory search❌NonconvexSearch.jl0✅❌❌❌❌❌
Branch and bound for mixed integer nonlinear programming❌Juniper.jl1 or 2✅✅✅✅❌✅
Sequential polyhedral outer-approximations for mixed integer nonlinear programming❌Pavito.jl1 or 2✅✅✅✅❌✅
Evolutionary centers algorithm (ECA)❌Metaheuristics.jl0✅✅✅✅❌❌
Differential evolution (DE)❌Metaheuristics.jl0✅✅✅✅❌❌
Particle swarm optimization (PSO)❌Metaheuristics.jl0✅✅✅✅❌❌
Artificial bee colony (ABC)❌Metaheuristics.jl0✅✅✅✅❌❌
Gravitational search algorithm (GSA)❌Metaheuristics.jl0✅✅✅✅❌❌
Simulated annealing (SA)❌Metaheuristics.jl0✅✅✅✅❌❌
Whale optimization algorithm (WOA)❌Metaheuristics.jl0✅✅✅✅❌❌
Machine-coded compact genetic algorithm (MCCGA)❌Metaheuristics.jl0✅✅✅✅❌❌
Genetic algorithm (GA)❌Metaheuristics.jl0✅✅✅✅❌❌
Nonlinear optimization with the MADS algorithm (NOMAD)❌NOMAD.jl0✅✅✅Limited❌✅
Topology optimization of binary structures (TOBS)❌NonconvexTOBS.jl1Binary❌✅❌❌Binary
Hyperband✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Random search✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Latin hypercube search✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Surrogate assisted optimization✅NonconvexBayesian.jlDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Log barrier method for nonlinear semidefinite constraint handling✅NonconvexSemidefinite.jlDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver✅Depends on sub-solver

The following is an explanation of all the columns in the table:

  • Algorithm name. This is the name of the algorithm and/or its acronym. Some algorithms have multiple variants implemented in their respective packages. When that's the case, the whole family of algorithms is mentioned only once.
  • Is meta-algorithm? Some algorithms are meta-algorithms that call a sub-algorithm to do the optimization after transforming the problem. In this case, a lot of the properties of the meta-algorithm are inherited from the sub-algorithm. So if the sub-algorithm requires gradients or Hessians of functions in the model, the meta-algorithm will also require gradients and Hessians of functions in the model. Fields where the property of the meta-algorithm is inherited from the sub-solver are indicated using the "Depends on sub-solver" entry. Some algorithms in NLopt have a "Limited" meta-algorithm status because they can only be used to wrap algorithms from NLopt.
  • Algorithm package. This is the Julia package that either implements the algorithm or calls it from another programming language. Nonconvex wraps all these packages using a consistent API while allowing each algorithm to be customized where possible and have its own set of options.
  • Order. This is the order of the algorithm. Zero-order algorithms only require the evaluation of the objective and constraint functions, they don't require any gradients or Hessians of objective and constraint functions. First-order algorithms require both the value and gradients of objective and/or constraint functions. Second-order algorithms require the value, gradients and Hessians of objective and/or constraint functions.
  • Finite bounds. This is true if the algorithm supports finite lower and upper bound constraints on the decision variables. One special case is the TOBS algorithm which only supports binary decision variables so an entry of "Binary" is used instead of true/false.
  • Infinite bounds. This is true if the algorithm supports unbounded decision variables either from below, above or both.
  • Inequality constraints. This is true if the algorithm supports nonlinear inequality constraints.
  • Equality constraints. This is true if the algorithm supports nonlinear equality constraints. Algorithms that only support linear equality constraints are given an entry of "Limited".
  • Semidefinite constraints. This is true if the algorithm supports nonlinear semidefinite constraints.
  • Integer variables. This is true if the algorithm supports integer/discrete/binary decision variables, not just continuous. One special case is the TOBS algorithm which only supports binary decision variables so an entry of "Binary" is used instead of true/false.

Wrapper packages

The JuliaNonconvex organization hosts a number of packages which wrap other optimization packages in Julia or implement their algorithms. The correct wrapper package is loaded using the Nonconvex.@load macro with the algorithm or package name. The following is a summary of all the wrapper packages in the JuliaNonconvex organization. To view the documentation of each package, click on the blue docs badge in the last column.

PackageDescriptionTestsCoverageDocs
NonconvexMMA.jlMethod of moving asymptotes implementation in pure JuliaBuild StatusCoverage
NonconvexIpopt.jlIpopt.jl wrapperBuild StatusCoverage
NonconvexNLopt.jlNLopt.jl wrapperBuild StatusCoverage
NonconvexPercival.jlPercival.jl wrapper (an augmented Lagrangian algorithm implementation)Build StatusCoverage
NonconvexJuniper.jlJuniper.jl wrapperBuild StatusCoverage
NonconvexPavito.jlPavito.jl wrapperBuild StatusCoverage
NonconvexSemidefinite.jlNonlinear semi-definite programming algorithmBuild StatusCoverage
NonconvexMultistart.jlMulti-start optimization algorithmsBuild StatusCoverage
NonconvexBayesian.jlConstrained Bayesian optimization implementationBuild StatusCoverage
NonconvexSearch.jlMulti-trajectory and local search methodsBuild StatusCoverage
NonconvexTOBS.jlBinary optimization algorithm called "topology optimization of binary structures" (TOBS) which was originally developed in the context of optimal distribution of material in mechanical components.Build StatusCoverage
NonconvexMetaheuristics.jlMetaheuristic gradient-free optimization algorithms as implemented in Metaheuristics.jl.Build StatusCoverage
NonconvexNOMAD.jlNOMAD algorithm as wrapped in the NOMAD.jl.Build StatusCoverage
+Overview · Nonconvex.jl
GitHub

Algorithms

Overview of algorithms

A summary of all the algorithms available in Nonconvex through different packages is shown in the table below. Scroll right to see more columns and see a description of the columns below the table.

Algorithm nameIs meta-algorithm?Algorithm packageOrderFinite boundsInfinite boundsInequality constraintsEquality constraintsSemidefinite constraintsInteger variables
Method of moving asymptotes (MMA)❌NonconvexMMA.jl (pure Julia) or NLopt.jl1✅✅✅❌❌❌
Primal dual interior point method❌Ipopt.jl1 or 2✅✅✅✅❌❌
DIviding RECTangles algorithm (DIRECT)❌NLopt.jl0✅❌✅❌❌❌
Controlled random search (CRS)❌NLopt.jl0✅✅❌❌❌❌
Multi-Level Single-Linkage (MLSL)LimitedNLopt.jlDepends on sub-solver✅✅❌❌❌❌
StoGo❌NLopt.jl1✅❌❌❌❌❌
AGS❌NLopt.jl0✅❌✅❌❌❌
Improved Stochastic Ranking Evolution Strategy (ISRES)❌NLopt.jl0✅✅✅✅❌❌
ESCH❌NLopt.jl0✅✅❌❌❌❌
COBYLA❌NLopt.jl0✅✅✅✅❌❌
BOBYQA❌NLopt.jl0✅✅❌❌❌❌
NEWUOA❌NLopt.jl0✅✅❌❌❌❌
Principal AXIS (PRAXIS)❌NLopt.jl0✅✅❌❌❌❌
Nelder Mead❌NLopt.jl0✅✅❌❌❌❌
Subplex❌NLopt.jl0✅✅❌❌❌❌
CCSAQ❌NLopt.jl1✅✅✅❌❌❌
SLSQP❌NLopt.jl1✅✅✅✅❌❌
TNewton❌NLopt.jl1❌✅❌❌❌❌
Shifted limited-memory variable-metric❌NLopt.jl1❌✅❌❌❌❌
Augmented Lagrangian in NLoptLimitedNLopt.jlDepends on sub-solver✅✅✅✅❌❌
Augmented Lagrangian in Percival❌Percival.jl1 or 2✅✅✅✅❌❌
Multiple trajectory search❌NonconvexSearch.jl0✅❌❌❌❌❌
Branch and bound for mixed integer nonlinear programming❌Juniper.jl1 or 2✅✅✅✅❌✅
Sequential polyhedral outer-approximations for mixed integer nonlinear programming❌Pavito.jl1 or 2✅✅✅✅❌✅
Evolutionary centers algorithm (ECA)❌Metaheuristics.jl0✅✅✅✅❌❌
Differential evolution (DE)❌Metaheuristics.jl0✅✅✅✅❌❌
Particle swarm optimization (PSO)❌Metaheuristics.jl0✅✅✅✅❌❌
Artificial bee colony (ABC)❌Metaheuristics.jl0✅✅✅✅❌❌
Gravitational search algorithm (GSA)❌Metaheuristics.jl0✅✅✅✅❌❌
Simulated annealing (SA)❌Metaheuristics.jl0✅✅✅✅❌❌
Whale optimization algorithm (WOA)❌Metaheuristics.jl0✅✅✅✅❌❌
Machine-coded compact genetic algorithm (MCCGA)❌Metaheuristics.jl0✅✅✅✅❌❌
Genetic algorithm (GA)❌Metaheuristics.jl0✅✅✅✅❌❌
Nonlinear optimization with the MADS algorithm (NOMAD)❌NOMAD.jl0✅✅✅Limited❌✅
Topology optimization of binary structures (TOBS)❌NonconvexTOBS.jl1Binary❌✅❌❌Binary
Hyperband✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Random search✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Latin hypercube search✅Hyperopt.jlDepends on sub-solver✅❌Depends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Surrogate assisted optimization✅NonconvexBayesian.jlDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver
Log barrier method for nonlinear semidefinite constraint handling✅NonconvexSemidefinite.jlDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solverDepends on sub-solver✅Depends on sub-solver

The following is an explanation of all the columns in the table:

  • Algorithm name. This is the name of the algorithm and/or its acronym. Some algorithms have multiple variants implemented in their respective packages. When that's the case, the whole family of algorithms is mentioned only once.
  • Is meta-algorithm? Some algorithms are meta-algorithms that call a sub-algorithm to do the optimization after transforming the problem. In this case, a lot of the properties of the meta-algorithm are inherited from the sub-algorithm. So if the sub-algorithm requires gradients or Hessians of functions in the model, the meta-algorithm will also require gradients and Hessians of functions in the model. Fields where the property of the meta-algorithm is inherited from the sub-solver are indicated using the "Depends on sub-solver" entry. Some algorithms in NLopt have a "Limited" meta-algorithm status because they can only be used to wrap algorithms from NLopt.
  • Algorithm package. This is the Julia package that either implements the algorithm or calls it from another programming language. Nonconvex wraps all these packages using a consistent API while allowing each algorithm to be customized where possible and have its own set of options.
  • Order. This is the order of the algorithm. Zero-order algorithms only require the evaluation of the objective and constraint functions, they don't require any gradients or Hessians of objective and constraint functions. First-order algorithms require both the value and gradients of objective and/or constraint functions. Second-order algorithms require the value, gradients and Hessians of objective and/or constraint functions.
  • Finite bounds. This is true if the algorithm supports finite lower and upper bound constraints on the decision variables. One special case is the TOBS algorithm which only supports binary decision variables so an entry of "Binary" is used instead of true/false.
  • Infinite bounds. This is true if the algorithm supports unbounded decision variables either from below, above or both.
  • Inequality constraints. This is true if the algorithm supports nonlinear inequality constraints.
  • Equality constraints. This is true if the algorithm supports nonlinear equality constraints. Algorithms that only support linear equality constraints are given an entry of "Limited".
  • Semidefinite constraints. This is true if the algorithm supports nonlinear semidefinite constraints.
  • Integer variables. This is true if the algorithm supports integer/discrete/binary decision variables, not just continuous. One special case is the TOBS algorithm which only supports binary decision variables so an entry of "Binary" is used instead of true/false.

Wrapper packages

The JuliaNonconvex organization hosts a number of packages which wrap other optimization packages in Julia or implement their algorithms. The correct wrapper package is loaded using the Nonconvex.@load macro with the algorithm or package name. The following is a summary of all the wrapper packages in the JuliaNonconvex organization. To view the documentation of each package, click on the blue docs badge in the last column.

PackageDescriptionTestsCoverageDocs
NonconvexMMA.jlMethod of moving asymptotes implementation in pure JuliaBuild StatusCoverage
NonconvexIpopt.jlIpopt.jl wrapperBuild StatusCoverage
NonconvexNLopt.jlNLopt.jl wrapperBuild StatusCoverage
NonconvexPercival.jlPercival.jl wrapper (an augmented Lagrangian algorithm implementation)Build StatusCoverage
NonconvexJuniper.jlJuniper.jl wrapperBuild StatusCoverage
NonconvexPavito.jlPavito.jl wrapperBuild StatusCoverage
NonconvexSemidefinite.jlNonlinear semi-definite programming algorithmBuild StatusCoverage
NonconvexMultistart.jlMulti-start optimization algorithmsBuild StatusCoverage
NonconvexBayesian.jlConstrained Bayesian optimization implementationBuild StatusCoverage
NonconvexSearch.jlMulti-trajectory and local search methodsBuild StatusCoverage
NonconvexTOBS.jlBinary optimization algorithm called "topology optimization of binary structures" (TOBS) which was originally developed in the context of optimal distribution of material in mechanical components.Build StatusCoverage
NonconvexMetaheuristics.jlMetaheuristic gradient-free optimization algorithms as implemented in Metaheuristics.jl.Build StatusCoverage
NonconvexNOMAD.jlNOMAD algorithm as wrapped in the NOMAD.jl.Build StatusCoverage
diff --git a/dev/algorithms/auglag/index.html b/dev/algorithms/auglag/index.html index 7319c11..d6dc7e6 100644 --- a/dev/algorithms/auglag/index.html +++ b/dev/algorithms/auglag/index.html @@ -4,4 +4,4 @@ alg = AugLag() options = AugLagOptions() -result = optimize(model, alg, x0, options = options)

Percival is an optional dependency of Nonconvex so you need to import it in order to use it.

Construct an instance

To construct an instance of the Ipopt algorithm, use:

alg = AugAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of the AugLagOptions struct when the algorihm is an AugLag. To specify options use keyword arguments in the constructor of AugLagOptions, e.g:

options = AugLagOptions(first_order = false, rtol = 1e-4)

The most important option is first_order which is true by default. When first_order is true, the first order augmented Lagrangian algorithm will be used. And when it is false, the second order augmented Lagrangian algorithm will be used. Other arguments include:

When using the first order augmented Lagrangian and a block constraint (i.e. a constraint function that returns a vector), the use of reverse-mode AD will only require calling the adjoint operator of the block constraint function in order to compute the gradient of the augmented Lagrangian. This is particularly suitable for constraint functions whose Jacobians are expensive but the adjoint operator is relatively inexpensive.

+result = optimize(model, alg, x0, options = options)

Percival is an optional dependency of Nonconvex so you need to import it in order to use it.

Construct an instance

To construct an instance of the Ipopt algorithm, use:

alg = AugAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of the AugLagOptions struct when the algorihm is an AugLag. To specify options use keyword arguments in the constructor of AugLagOptions, e.g:

options = AugLagOptions(first_order = false, rtol = 1e-4)

The most important option is first_order which is true by default. When first_order is true, the first order augmented Lagrangian algorithm will be used. And when it is false, the second order augmented Lagrangian algorithm will be used. Other arguments include:

When using the first order augmented Lagrangian and a block constraint (i.e. a constraint function that returns a vector), the use of reverse-mode AD will only require calling the adjoint operator of the block constraint function in order to compute the gradient of the augmented Lagrangian. This is particularly suitable for constraint functions whose Jacobians are expensive but the adjoint operator is relatively inexpensive.

diff --git a/dev/algorithms/hyperopt/index.html b/dev/algorithms/hyperopt/index.html index 089a21c..c3f330e 100644 --- a/dev/algorithms/hyperopt/index.html +++ b/dev/algorithms/hyperopt/index.html @@ -7,4 +7,4 @@ result = optimize(model, alg, x0, options = options)

Hyperopt is an optional dependency of Nonconvex so you need to import it in order to use it. HyperoptAlg can wrap any other algorithm in Nonconvex, e.g. IpoptAlg(). When the algorithm is a HyperoptAlg, the options keyword argument must of type HyperoptOptions. For more on the options available see below.

Construct an instance

To construct an instance of the Hyperopt + Ipopt algorithm, use:

alg = HyperoptAlg(IpoptAlg())

HyperoptAlg can wrap any other algorithm in Nonconvex, e.g. NLoptAlg(:LD_MMA) or AugLag().

Options

The options keyword argument to the optimize function shown above must be an instance of the HyperoptOptions struct when the algorihm is a HyperoptAlg. To specify options, use keyword arguments in the constructor of HyperoptOptions. The sampler keyword argument determines the sampling algorithm used to propose new starting points in the multi-start procedure. The sub_options keyword argument can be used to pass in the options for the sub-optimizer. There are 2 different ways to pass the sub-options depending on the sampler type.

The sampler argument can be of type:

  1. RandomSampler
  2. LHSampler
  3. CLHSampler
  4. GPSampler
  5. Hyperband

When optimizing the starting point, the upper and lower bounds on the initial solution must be finite, or finite bounds must be passed in to the options constructor. All the options that can be passed to the HyperoptOptions constructor are listed below:

NonconvexMultistart.HyperoptOptions — Type
HyperoptOptions: options performing starting point optimization using Hyperopt.jl
  • sub_options: options for the sub-optimizer.
  • lb: Lower bound of starting point, if don't specify it, the default value will be nothing, then will end up be replaced by the lower bound of optimization problem.
  • ub: Upper bound of starting point, same as above.
  • searchspace_size::Integer: How many potential starting points we generate.
  • iters::Integer: Among all generated potential starting points, how many of them will be evaluated.
  • sampler::Hyperopt.Sampler: An instance of 'Hyperopt.Sampler', which decides search algorithm.
  • ctol: infeasibility tolerance for accepting a solution as feasible
  • keep_all: if true, all the solutions of the sub-problems will be saved
source

Sampler choice

RandomSampler, LHSampler, CLHSampler and GPSampler

All the sampler constructors are functions defined in Nonconvex wrapping the Hyperopt alternatives to define defaults. For GPSampler, Hyperopt.Min is always used by default in Nonconvex so you should not pass this argument. All the other arguments that can be passed to the sampler constructor can be found in the Hyperopt documentation. Example:

options = HyperoptOptions(sub_options = IpoptOptions(), sampler = GPSampler())

Hyperband

The Hyperband algorithm in Hyperopt requires a different way to pass in the sub-options. The Hyperband algorithm tries to optimize the allocation of resources. The sub_options argument must be a function with input as the "resources" and output as the sub-solver options. The Hyperband constructor accepts 3 arguments:

  1. The maximum resources R
  2. η which roughly determines the proportion of trials discarded between each round of successive halving
  3. inner which specifies an inner sampler of type RandomSampler, LHSampler or CLHSampler.

Example:

options = HyperoptOptions(
     sub_options = max_iter -> IpoptOptions(max_iter = max_iter), 
     sampler = Hyperband(R=100, η=3, inner=RandomSampler()),
-)
+) diff --git a/dev/algorithms/ipopt/index.html b/dev/algorithms/ipopt/index.html index 3b9dddd..1e1378c 100644 --- a/dev/algorithms/ipopt/index.html +++ b/dev/algorithms/ipopt/index.html @@ -4,4 +4,4 @@ alg = IpoptAlg() options = IpoptOptions() -result = optimize(model, alg, x0, options = options)

Construct an instance

To construct an instance of the Ipopt algorithm, use:

alg = IpoptAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of the IpoptOptions struct when the algorihm is an IpoptAlg. To specify options use keyword arguments in the constructor of IpoptOptions, e.g:

options = IpoptOptions(first_order = false, tol = 1e-4, sparse = false, symbolic = false)

There are 4 important and special options:

Note that there is no need to use sparsify or symbolify on the model or functions before optimizing it with an IpoptAlg. Setting the sparse and symbolic options above are enough to trigger the symbolic differentiation and/or sparsity exploitation.

All the other options that can be set can be found on the Ipopt options section of Ipopt's documentation.

+result = optimize(model, alg, x0, options = options)

Construct an instance

To construct an instance of the Ipopt algorithm, use:

alg = IpoptAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of the IpoptOptions struct when the algorihm is an IpoptAlg. To specify options use keyword arguments in the constructor of IpoptOptions, e.g:

options = IpoptOptions(first_order = false, tol = 1e-4, sparse = false, symbolic = false)

There are 4 important and special options:

Note that there is no need to use sparsify or symbolify on the model or functions before optimizing it with an IpoptAlg. Setting the sparse and symbolic options above are enough to trigger the symbolic differentiation and/or sparsity exploitation.

All the other options that can be set can be found on the Ipopt options section of Ipopt's documentation.

diff --git a/dev/algorithms/metaheuristics/index.html b/dev/algorithms/metaheuristics/index.html index 1ec3a97..0a501a4 100644 --- a/dev/algorithms/metaheuristics/index.html +++ b/dev/algorithms/metaheuristics/index.html @@ -5,4 +5,4 @@ alg = MetaheuristicsAlg(ECA) options = MetaheuristicsOptions(N = 1000) # population size result = optimize(model, alg, x0, options = options)

Metaheuristics is an optional dependency of Nonconvex so you need to load the package to be able to use it.

Options

The options keyword argument to the optimize function shown above must be an instance of the MetaheuristicsOptions struct when the algorihm is a MetaheuristicsAlg. To specify options use keyword arguments in the constructor of MetaheuristicsOptions, e.g:

options = MetaheuristicsOptions(N = 1000)

All the other options that can be set for each algorithm can be found in the algorithms section of the documentation of Metaheuristics.jl. Note that one notable difference between using Metaheuristics directly and using it through Nonconvex is that in Nonconvex, all the options must be passed in through the options struct and only the algorithm type is part of the alg struct.

Variable bounds

When using Metaheuristics algorithms, finite variables bounds are necessary. This is because the initial population is sampled randomly in the finite interval of each variable. Use of Inf as an upper bound or -Inf is therefore not acceptable.

Initialization

Most metaheuristic algorithms are population algorithms which can accept multiple initial solutions to be part of the initial population. In Nonconvex, you can specify multiple initial solutions by making x0 a vector of solutions. However, since Nonconvex models support arbitrary collections as decision variables, you must specify that the x0 passed in is indeed a population of solutions rather than a single solution that's a vector of vectors for instance. To specify that x0 is a vector of solutions, you can set the multiple_initial_solutions option to true in the options struct, e.g:

options = MetaheuristicsOptions(N = 1000, multiple_initial_solutions = true)
-x0 = [[1.0, 1.0], [0.0, 0.0]]

When fewer solutions are passed in x0 compared to the population size, random initial solutions between the lower and upper bounds are sampled to complete the initial population.

+x0 = [[1.0, 1.0], [0.0, 0.0]]

When fewer solutions are passed in x0 compared to the population size, random initial solutions between the lower and upper bounds are sampled to complete the initial population.

diff --git a/dev/algorithms/minlp/index.html b/dev/algorithms/minlp/index.html index 4e5ac52..8a22eb4 100644 --- a/dev/algorithms/minlp/index.html +++ b/dev/algorithms/minlp/index.html @@ -9,4 +9,4 @@ alg = PavitoIpoptCbcAlg() options = PavitoIpoptCbcOptions() -result = optimize(model, alg, x0, options = options)

Pavito is an optional dependency of Nonconvex, so you need to load it in order to use it. Note that the integer constraints must be specified when defining variables. See the problem definition documentation for more details.

Construct an instance

To construct an instance of the Pavito + Ipopt + Cbc algorithm, use:

alg = PavitoIpoptCbcAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of PavitoIpoptCbcOptions struct when the algorithm is a PavitoIpoptCbcAlg. To specify options, use keyword arguments in the constructor of JuniperIpoptOptions or PavitoIpoptCbcOptions, e.g:

options = PavitoIpoptCbcOptions(first_order = false, subsolver_options = IpoptOptions(), timeout = 120.0)

There are 2 important and special options you can pass to the optimizer:

All the other options to Pavito can be found in the Pavito documentation.

+result = optimize(model, alg, x0, options = options)

Pavito is an optional dependency of Nonconvex, so you need to load it in order to use it. Note that the integer constraints must be specified when defining variables. See the problem definition documentation for more details.

Construct an instance

To construct an instance of the Pavito + Ipopt + Cbc algorithm, use:

alg = PavitoIpoptCbcAlg()

Options

The options keyword argument to the optimize function shown above must be an instance of PavitoIpoptCbcOptions struct when the algorithm is a PavitoIpoptCbcAlg. To specify options, use keyword arguments in the constructor of JuniperIpoptOptions or PavitoIpoptCbcOptions, e.g:

options = PavitoIpoptCbcOptions(first_order = false, subsolver_options = IpoptOptions(), timeout = 120.0)

There are 2 important and special options you can pass to the optimizer:

All the other options to Pavito can be found in the Pavito documentation.

diff --git a/dev/algorithms/mma/index.html b/dev/algorithms/mma/index.html index 50b6bf6..1689dba 100644 --- a/dev/algorithms/mma/index.html +++ b/dev/algorithms/mma/index.html @@ -11,4 +11,4 @@ criteria::ConvergenceCriteria, verbose::Bool, iter::Int -)

Evaluates the convergence state solution.convstate given the current solution, solution, the tolerance, tol, and the convergence criteria criteria. solution.convstate.converged is then updated.

If criteria is an instance of GenericCriteria, converged = (x_converged || f_converged) && infeas_converged. x_converged, f_converged and infeas_converged are explained in Tolerance. If criteria is an instance of KKTCriteria or ScaledKKTCriteria, converged = kkt_converged && infeas_converged. kkt_converged and infeas_converged are explained in Tolerance. If criteria is an instance of IpoptCriteria, converged = ipopt_converged && infeas_converged. ipopt_converged and infeas_converged are explained in Tolerance.

source

To specify the convergence criteria, use:

converiteria = GenericCriteria()

replacing GenericCriteria() by KKTCriteria(), ScaledKKTCriteria() or IpoptCriteria().

+)

Evaluates the convergence state solution.convstate given the current solution, solution, the tolerance, tol, and the convergence criteria criteria. solution.convstate.converged is then updated.

If criteria is an instance of GenericCriteria, converged = (x_converged || f_converged) && infeas_converged. x_converged, f_converged and infeas_converged are explained in Tolerance. If criteria is an instance of KKTCriteria or ScaledKKTCriteria, converged = kkt_converged && infeas_converged. kkt_converged and infeas_converged are explained in Tolerance. If criteria is an instance of IpoptCriteria, converged = ipopt_converged && infeas_converged. ipopt_converged and infeas_converged are explained in Tolerance.

source

To specify the convergence criteria, use:

converiteria = GenericCriteria()

replacing GenericCriteria() by KKTCriteria(), ScaledKKTCriteria() or IpoptCriteria().

diff --git a/dev/algorithms/mts/index.html b/dev/algorithms/mts/index.html index a96e95d..7e38bb6 100644 --- a/dev/algorithms/mts/index.html +++ b/dev/algorithms/mts/index.html @@ -8,4 +8,4 @@ lb = [0, 0] ub = [5, 5] addvar!(m, lb, ub) -result = optimize(model, alg, x0, options = options) +result = optimize(model, alg, x0, options = options) diff --git a/dev/algorithms/nlopt/index.html b/dev/algorithms/nlopt/index.html index 3f01cd7..87974ca 100644 --- a/dev/algorithms/nlopt/index.html +++ b/dev/algorithms/nlopt/index.html @@ -4,4 +4,4 @@ alg = NLoptAlg(:LD_SLSQP) options = NLoptOptions() -result = optimize(model, alg, x0, options = options)

NLopt is an optional dependency of Nonconvex so you need to load the package to be able to use it.

Construct an instance

To construct an instance of NLopt's NLOPT_LD_SLSQP algorithm, use:

alg = NLoptAlg(:LD_SLSQP)

All the algorithms available in NLopt are:

For a description of the above algorithms, please refer to the algorithms section of NLopt's documentation.

Disclaimer:

Not all the algorithms have been tested with Nonconvex. So if you try one and it doesn't work, please open an issue.

Options

The options keyword argument to the optimize function shown above must be an instance of the NLoptOptions struct when the algorihm is an NLoptAlg. To specify options use keyword arguments in the constructor of NLoptOptions, e.g:

options = NLoptOptions(ftol_rel = 1e-4)

All the other options that can be set for each algorithm can be found in the algorithms section section of NLopt's documentation.

+result = optimize(model, alg, x0, options = options)

NLopt is an optional dependency of Nonconvex so you need to load the package to be able to use it.

Construct an instance

To construct an instance of NLopt's NLOPT_LD_SLSQP algorithm, use:

alg = NLoptAlg(:LD_SLSQP)

All the algorithms available in NLopt are:

For a description of the above algorithms, please refer to the algorithms section of NLopt's documentation.

Disclaimer:

Not all the algorithms have been tested with Nonconvex. So if you try one and it doesn't work, please open an issue.

Options

The options keyword argument to the optimize function shown above must be an instance of the NLoptOptions struct when the algorihm is an NLoptAlg. To specify options use keyword arguments in the constructor of NLoptOptions, e.g:

options = NLoptOptions(ftol_rel = 1e-4)

All the other options that can be set for each algorithm can be found in the algorithms section section of NLopt's documentation.

diff --git a/dev/algorithms/nomad/index.html b/dev/algorithms/nomad/index.html index 5e4481d..b0c5062 100644 --- a/dev/algorithms/nomad/index.html +++ b/dev/algorithms/nomad/index.html @@ -5,4 +5,4 @@ alg = NOMADAlg() options = NOMADOptions() result = optimize(model, alg, x0, options = options)

NOMAD is an optional dependency of Nonconvex so you need to load the package to be able to use it.

Algorithm types

There are 3 different variants of the NOMADAlg struct:

The explicit algorithm ensures all the constraints are satisfied at all times removing any infeasible point from the population. The progressive algorithm allows infeasible points to be part of the population but enforces feasibility in a progressive manner. The custom variant allows the use of flags on each constraint to declare it as :explicit or :progressive. For instance, assume model is the Nonconvex model and g1 and g2 are 2 constraint functions.

add_ineq_constraint!(model, g1, flags = [:explicit])
-add_ineq_constraint!(m, g2, flags = [:progressive])

The above code declares the first constraint as explicit and the second as progressive. In other words, every point violating the first constraint will be removed from the population but the second constraint will be more progressively enforced.

Options

The options keyword argument to the optimize function shown above must be an instance of the NOMADOptions struct when the algorihm is a NOMADAlg. To specify options use keyword arguments in the constructor of NOMADOptions, e.g:

options = NOMADOptions()

All the options that can be set can be found in the NOMAD.jl documentation.

+add_ineq_constraint!(m, g2, flags = [:progressive])

The above code declares the first constraint as explicit and the second as progressive. In other words, every point violating the first constraint will be removed from the population but the second constraint will be more progressively enforced.

Options

The options keyword argument to the optimize function shown above must be an instance of the NOMADOptions struct when the algorihm is a NOMADAlg. To specify options use keyword arguments in the constructor of NOMADOptions, e.g:

options = NOMADOptions()

All the options that can be set can be found in the NOMAD.jl documentation.

diff --git a/dev/algorithms/sdp/index.html b/dev/algorithms/sdp/index.html index c22511f..7424af6 100644 --- a/dev/algorithms/sdp/index.html +++ b/dev/algorithms/sdp/index.html @@ -38,4 +38,4 @@ result = optimize(model, alg, x0, options = options) # Relative error norm -norm(sd_constraint(result.minimizer) - Σ) / norm(Σ)

Options

NonconvexSemidefinite.SDPBarrierOptions — Type
SDPBarrierOptions(; kwargs...)

The keyword arguments which can be specified are:

  • c_init: (default 1.0) initial value for the coefficient c that is multiplied by the barrier term, could be a real number or vector in the case of multiple semidefinite constraints.
  • c_decr: (default 0.1) decreasing rate (< 1) that multiplies the barrier term in every iteration, could be either a real number or a vector in the case of multiple semidefinite constraints.
  • n_iter: (default 20) number of sub-problems to solve in the barrier method.
  • sub_options: options for the sub-problem's solver
  • keep_all: (default falue) if set to true, SDPBarrierResult stores the results from all the iterations
source

Optimizer

NonconvexSemidefinite.SDPBarrierAlg — Type
SDPBarrierAlg(sub_alg)

A meta-algorithm that handles semidefinite constraints on nonlinear functions using a barrier approach. The coefficient of the barrier term is exponentially decayed and the sub-problems are solved using sub_alg. sub_alg can be any other compatible solver from Nonconvex.jl. The solver must be able to solve the sub-problem after removing the semidefinite constraints. The options to the solver should be pased to the SDPBarrierOptions struct and passed in as the options to the optimize function. Call ? SDPBarrierOptions to check all the different options that can be set.

source

Matrix interfaces

For every n*n real positive semidefinite matrix that optimization objective contains, please have two inputs x_L and x_D representing the lower-triangular and the diagonal part of it. In the function, call decompress_symmetric(x_L, x_d) to represent that matrix, which will be handled by Nonocnvex automatically

NonconvexSemidefinite.decompress_symmetric — Function
decompress_symmetric

For example: a 3*3 positive semidefinite matrix: [ a b d; b c e; d e f ] represents by: [ x_D[1] x_L[3] x_L[2]; x_L[1] x_D[2] x_L[1]; x_L[2] x_L[3] x_D[3] ]

  • x_L::AbstractArray: representing lower triangular part of a n*n matrix, length should be (n^2-n)÷2
  • x_D::AbstractArray: representing diagonal part of a n*n matrix, length should be n
source
+norm(sd_constraint(result.minimizer) - Σ) / norm(Σ)

Options

NonconvexSemidefinite.SDPBarrierOptions — Type
SDPBarrierOptions(; kwargs...)

The keyword arguments which can be specified are:

  • c_init: (default 1.0) initial value for the coefficient c that is multiplied by the barrier term, could be a real number or vector in the case of multiple semidefinite constraints.
  • c_decr: (default 0.1) decreasing rate (< 1) that multiplies the barrier term in every iteration, could be either a real number or a vector in the case of multiple semidefinite constraints.
  • n_iter: (default 20) number of sub-problems to solve in the barrier method.
  • sub_options: options for the sub-problem's solver
  • keep_all: (default falue) if set to true, SDPBarrierResult stores the results from all the iterations
source

Optimizer

NonconvexSemidefinite.SDPBarrierAlg — Type
SDPBarrierAlg(sub_alg)

A meta-algorithm that handles semidefinite constraints on nonlinear functions using a barrier approach. The coefficient of the barrier term is exponentially decayed and the sub-problems are solved using sub_alg. sub_alg can be any other compatible solver from Nonconvex.jl. The solver must be able to solve the sub-problem after removing the semidefinite constraints. The options to the solver should be pased to the SDPBarrierOptions struct and passed in as the options to the optimize function. Call ? SDPBarrierOptions to check all the different options that can be set.

source

Matrix interfaces

For every n*n real positive semidefinite matrix that optimization objective contains, please have two inputs x_L and x_D representing the lower-triangular and the diagonal part of it. In the function, call decompress_symmetric(x_L, x_d) to represent that matrix, which will be handled by Nonocnvex automatically

NonconvexSemidefinite.decompress_symmetric — Function
decompress_symmetric

For example: a 3*3 positive semidefinite matrix: [ a b d; b c e; d e f ] represents by: [ x_D[1] x_L[3] x_L[2]; x_L[1] x_D[2] x_L[1]; x_L[2] x_L[3] x_D[3] ]

  • x_L::AbstractArray: representing lower triangular part of a n*n matrix, length should be (n^2-n)÷2
  • x_D::AbstractArray: representing diagonal part of a n*n matrix, length should be n
source
diff --git a/dev/algorithms/surrogate/index.html b/dev/algorithms/surrogate/index.html index 52ee865..8fca43a 100644 --- a/dev/algorithms/surrogate/index.html +++ b/dev/algorithms/surrogate/index.html @@ -29,4 +29,4 @@ sub_options = IpoptOptions(print_level = 0), maxiter = 50, ctol = 1e-4, ninit = 2, initialize = true, postoptimize = false, ) -r = optimize(model, alg, x0, options = options, surrogates = [s1, s2]) +r = optimize(model, alg, x0, options = options, surrogates = [s1, s2]) diff --git a/dev/algorithms/tobs/index.html b/dev/algorithms/tobs/index.html index 40a90c0..4519488 100644 --- a/dev/algorithms/tobs/index.html +++ b/dev/algorithms/tobs/index.html @@ -26,4 +26,4 @@ r = optimize(m, TOBSAlg(), x0; options=options) r.minimizer -r.minimum

The following is a visualization of the optimization history using this example.

histories

gif

Options

The following are the options that can be set by passing them to TOBSOptions, e.g. TOBSOptions(movelimit = 0.1).

+r.minimum

The following is a visualization of the optimization history using this example.

histories

gif

Options

The following are the options that can be set by passing them to TOBSOptions, e.g. TOBSOptions(movelimit = 0.1).

diff --git a/dev/gradients/chainrules_fd/index.html b/dev/gradients/chainrules_fd/index.html index ada9a40..e654015 100644 --- a/dev/gradients/chainrules_fd/index.html +++ b/dev/gradients/chainrules_fd/index.html @@ -1,2 +1,2 @@ -Using ChainRules in ForwardDiff · Nonconvex.jl
GitHub

Using ChainRules in ForwardDiff

ForwardDiff is a forward-mode AD package that pre-dates ChainRules. ForwardDiff therefore does not use the frules defined in ChainRules. In order to force ForwardDiff to use the frule defined for a function, one can use the Nonconvex.NonconvexUtils.@ForwardDiff_frule macro provided in Nonconvex. This is useful in case ForwardDiff is used for the entire function but a component of this function has an efficient frule defined that you want to take advantage of. To force ForwardDiff to use the frule defined for a function f(x::AbstractVector), you can use:

Nonconvex.NonconvexUtils.@ForwardDiff_frule f(x::AbstractVector{<:ForwardDiff.Dual})

The signature of the function specifies the method that will be re-directed to use the frule from ChainRules. Such frule therefore needs to be defined for f to begin with. f with multiple inputs, scalar inputs and other input collection types are also supported.

+Using ChainRules in ForwardDiff · Nonconvex.jl
GitHub

Using ChainRules in ForwardDiff

ForwardDiff is a forward-mode AD package that pre-dates ChainRules. ForwardDiff therefore does not use the frules defined in ChainRules. In order to force ForwardDiff to use the frule defined for a function, one can use the Nonconvex.NonconvexUtils.@ForwardDiff_frule macro provided in Nonconvex. This is useful in case ForwardDiff is used for the entire function but a component of this function has an efficient frule defined that you want to take advantage of. To force ForwardDiff to use the frule defined for a function f(x::AbstractVector), you can use:

Nonconvex.NonconvexUtils.@ForwardDiff_frule f(x::AbstractVector{<:ForwardDiff.Dual})

The signature of the function specifies the method that will be re-directed to use the frule from ChainRules. Such frule therefore needs to be defined for f to begin with. f with multiple inputs, scalar inputs and other input collection types are also supported.

diff --git a/dev/gradients/gradients/index.html b/dev/gradients/gradients/index.html index 8a48d3b..71efb56 100644 --- a/dev/gradients/gradients/index.html +++ b/dev/gradients/gradients/index.html @@ -1,2 +1,2 @@ -Overview · Nonconvex.jl
GitHub

Gradients, Jacobians and Hessians

By default, Nonconvex uses:

  • The reverse-mode automatic differentiation (AD) package, Zygote.jl, for computing gradients and Jacobians of functions, and
  • The forward-mode AD package, ForwardDiff.jl, over Zygote.jl for computing Hessians.

However, one can force Nonconvex to use other AD packages or even user-defined gradients and Hessians using special function modifiers. Those special function modifiers customize the behaviour of functions without enforcing the same behaviour on other functions. For instance:

  • A specific AD package can be used for one constraint function while the default AD packages are used for other functions in the optimization problem.
  • The history of gradients of a specific function can be stored without storing all the gradients of all the functions.
  • For functions with a sparse Jacobian or Hessian, the sparsity can be used to speedup the AD using sparse, forward-mode AD for these functions.

In some cases, function modifiers can even be composed on top of each other to create more complex behaviours.

In Nonconvex, function modifiers modify the behaviour of a function when differentiated once or twice using either ForwardDiff or any ChainRules-compatible AD package, such as Zygote.jl. The following features are all implemented in NonconvexUtils.jl and re-exported from Nonconvex.

Table of contents

+Overview · Nonconvex.jl
GitHub

Gradients, Jacobians and Hessians

By default, Nonconvex uses:

  • The reverse-mode automatic differentiation (AD) package, Zygote.jl, for computing gradients and Jacobians of functions, and
  • The forward-mode AD package, ForwardDiff.jl, over Zygote.jl for computing Hessians.

However, one can force Nonconvex to use other AD packages or even user-defined gradients and Hessians using special function modifiers. Those special function modifiers customize the behaviour of functions without enforcing the same behaviour on other functions. For instance:

  • A specific AD package can be used for one constraint function while the default AD packages are used for other functions in the optimization problem.
  • The history of gradients of a specific function can be stored without storing all the gradients of all the functions.
  • For functions with a sparse Jacobian or Hessian, the sparsity can be used to speedup the AD using sparse, forward-mode AD for these functions.

In some cases, function modifiers can even be composed on top of each other to create more complex behaviours.

In Nonconvex, function modifiers modify the behaviour of a function when differentiated once or twice using either ForwardDiff or any ChainRules-compatible AD package, such as Zygote.jl. The following features are all implemented in NonconvexUtils.jl and re-exported from Nonconvex.

Table of contents

diff --git a/dev/gradients/history/index.html b/dev/gradients/history/index.html index 5fd4498..6ab1bed 100644 --- a/dev/gradients/history/index.html +++ b/dev/gradients/history/index.html @@ -1,2 +1,2 @@ -Storing history of gradients · Nonconvex.jl
GitHub

Storing history of gradients

Often one may want to store intermediate solutions, function values and gradients for visualisation or post-processing. This is currently not possible with Nonconvex.jl as not all solvers support a callback mechanism. To workround this, the TraceFunction modifier can be used to store input, output and optionally gradient values during the optimization:

F = TraceFunction(f; on_call = false, on_grad = true)

F can now be used inplace of f in objective and/or constraint functions in a Nonconvex model. If the on_call keyword argument is set to true (default is true), the input and output values are stored every time the function F is called. If the on_grad keyword argument is set to true (default is true), the input, output and gradient values are stored every time the function F is differentiated with either ForwardDiff or any ChainRules-compatible AD package such as Zygote.jl. The history is stored in F.trace. The TraceFunction modifier can be compsed with other AD-centric function modifiers in Nonconvex, e.g. the sparsify or symbolify function modifiers.

+Storing history of gradients · Nonconvex.jl
GitHub

Storing history of gradients

Often one may want to store intermediate solutions, function values and gradients for visualisation or post-processing. This is currently not possible with Nonconvex.jl as not all solvers support a callback mechanism. To workround this, the TraceFunction modifier can be used to store input, output and optionally gradient values during the optimization:

F = TraceFunction(f; on_call = false, on_grad = true)

F can now be used inplace of f in objective and/or constraint functions in a Nonconvex model. If the on_call keyword argument is set to true (default is true), the input and output values are stored every time the function F is called. If the on_grad keyword argument is set to true (default is true), the input, output and gradient values are stored every time the function F is differentiated with either ForwardDiff or any ChainRules-compatible AD package such as Zygote.jl. The history is stored in F.trace. The TraceFunction modifier can be compsed with other AD-centric function modifiers in Nonconvex, e.g. the sparsify or symbolify function modifiers.

diff --git a/dev/gradients/implicit/index.html b/dev/gradients/implicit/index.html index 57dc85e..273527e 100644 --- a/dev/gradients/implicit/index.html +++ b/dev/gradients/implicit/index.html @@ -25,4 +25,4 @@ imf = ImplicitFunction(forward, f) return sum(imf()) end -g = Zygote.gradient(obj, p0)[1]

Notice that p was not an explicit argument to f or forward in the above example and that the implicit function is called using imf(). Using some explicit parameters and some implicit parameters is also supported.

Matrix-free linear solver in the adjoint

In the adjoint definition of implicit functions, a linear system:

(df/dy) * x = v

is solved to find the adjoint vector. To solve the system using a matrix-free iterative solver (GMRES by default) that avoids constructing the Jacobian df/dy, you can set the matrixfree keyword argument to true (default is false). When set to false, the entrie Jacobian matrix is formed and the linear system is solved using LU factorization.

Arbitrary data structures

Both p and x above can be arbitrary data structures, not just arrays of numbers.

Tolerance

The implicit function theorem assumes that some conditions f(p, x) == 0 is satisfied. In practice, this will only be approximately satisfied. When this condition is violated, the gradient reported by the implicit function theorem cannot be trusted since its assumption is violated. The maximum tolerance allowed to "accept" the solution x(p) and the gradient is given by the keyword argument tol (default value is 1e-5). When the norm of the residual function f(p, x) is greater than this tolerance, NaNs are returned for the gradient instead of the value computed via the implicit function theorem. If additionally, the keyword argument error_on_tol_violation is set to true (default value is false), an error is thrown if the norm of the residual exceeds the specified tolerance tol.

+g = Zygote.gradient(obj, p0)[1]

Notice that p was not an explicit argument to f or forward in the above example and that the implicit function is called using imf(). Using some explicit parameters and some implicit parameters is also supported.

Matrix-free linear solver in the adjoint

In the adjoint definition of implicit functions, a linear system:

(df/dy) * x = v

is solved to find the adjoint vector. To solve the system using a matrix-free iterative solver (GMRES by default) that avoids constructing the Jacobian df/dy, you can set the matrixfree keyword argument to true (default is false). When set to false, the entrie Jacobian matrix is formed and the linear system is solved using LU factorization.

Arbitrary data structures

Both p and x above can be arbitrary data structures, not just arrays of numbers.

Tolerance

The implicit function theorem assumes that some conditions f(p, x) == 0 is satisfied. In practice, this will only be approximately satisfied. When this condition is violated, the gradient reported by the implicit function theorem cannot be trusted since its assumption is violated. The maximum tolerance allowed to "accept" the solution x(p) and the gradient is given by the keyword argument tol (default value is 1e-5). When the norm of the residual function f(p, x) is greater than this tolerance, NaNs are returned for the gradient instead of the value computed via the implicit function theorem. If additionally, the keyword argument error_on_tol_violation is set to true (default value is false), an error is thrown if the norm of the residual exceeds the specified tolerance tol.

diff --git a/dev/gradients/other_ad/index.html b/dev/gradients/other_ad/index.html index dda70f4..c2024e2 100644 --- a/dev/gradients/other_ad/index.html +++ b/dev/gradients/other_ad/index.html @@ -4,4 +4,4 @@ backend = AbstractDifferentiation.ReverseDiffBackend()

Having defined F like this, whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F, the AD package corresponding to the chosen backend will be used instead.

To use ForwardDiff as the backend of choice, a shortcut is also available using the forwarddiffy function modifier instead of the more general abstractdiffy:

F = forwarddiffy(f, x...)
 F(x...)

which is short for:

backend = AbstractDifferentiation.ForwardDiffBackend()
-F = abstractdiffy(f, backend, x...)

abstractdiffying a model

Instead of abstractdiffying or forwarddiffying one function at a time, the user can instead abstractdiffy or forwarddiffy an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

ad_model = abstractdiffy(model, backend)

where model is of type Model or DictModel. ad_model can now be optimized using any of the Nonconvex algorithms compatible with the model. Similarly, forwarddiffy can be used on an entire model:

fd_model = forwarddiffy(model)

By default, the objective and all the constraint functions will be modified with abstractdiffy/forwarddiffy. To prevent the modification of some component of the model, any of the following keyword arguments can be set to false (default is true):

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the modification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

+F = abstractdiffy(f, backend, x...)

abstractdiffying a model

Instead of abstractdiffying or forwarddiffying one function at a time, the user can instead abstractdiffy or forwarddiffy an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

ad_model = abstractdiffy(model, backend)

where model is of type Model or DictModel. ad_model can now be optimized using any of the Nonconvex algorithms compatible with the model. Similarly, forwarddiffy can be used on an entire model:

fd_model = forwarddiffy(model)

By default, the objective and all the constraint functions will be modified with abstractdiffy/forwarddiffy. To prevent the modification of some component of the model, any of the following keyword arguments can be set to false (default is true):

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the modification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

diff --git a/dev/gradients/sparse/index.html b/dev/gradients/sparse/index.html index 313fd80..f8b4051 100644 --- a/dev/gradients/sparse/index.html +++ b/dev/gradients/sparse/index.html @@ -1,3 +1,3 @@ Sparse Jacobian or Hessian · Nonconvex.jl
GitHub

Sparse Jacobian or Hessian

Background

For functions with a sparse Jacobian or Hessian, it can sometimes be useful to exploit such sparsity to speedup the computation of the Jacobian. This can be done using the SparseDiffTools.jl package.

SparseDiffTools can compute multiple columns of the Jacobian matrix of a vector-valued function y = f(x) simulatenously using a single Jacobian-vector product operation. Such columns corresponding to a subset of the input variables, e.g. (x[1], x[3]), however need not overlap in the output variables they influence. For instance, assume

  • y[1] and y[2] are a function of x[1] and x[2] only, and
  • y[3] and y[4] are a function of x[3] and x[4] only.

The Jacobian dy/dx will therefore have a block diagonal structure. Additionally, since x[1] and x[3] do not affect the same output variables, their corresponding columns in the block-diagonal Jacobian can be computed simulatenously using a single Jacobian-vector block. The same thing for x[2] and x[4]. Finding such subsets of input variables such that no 2 input variables in a subset affect the same output is done using SparseDiffTools. In the diagonal Jacobian case, all the input variables do not overlap so all the columns of the Jacobian can be obtained using a single Jacobian-vector product.

The problem of finding the optimal splitting of input variables to require the least number of Jacobian-vector products when computing the full Jacobian can be formulated as a graph coloring problem in computer science, which is an NP-hard problem. SparseDiffTools uses a tractable heuristic to find reasonable splittings for different Jacobian or Hessian sparsity patterns. The sparsity pattern of the Jacobian or Hessian can either be user-provided or it will be automatically uncovered using Symbolics.jl.

In Nonconvex, you can enforce the use of SparseDiffTools for specific functions using the sparsify function modifier. In particular, the rrule and frule of the modified function will be using SparseDiffTools to find the full Jacobian first and then doing either a Jacobian-vector product in the frule or a vector-Jacobian product in the rrule. Such frule will also be used by ForwardDiff if used to differentiate the modified function. For more on frules, rrules, Jacobian-vector products and vector-Jacobian products, refer to the following video on Understanding autmoatic differentiation (in Julia).

Sparsifying a function

First order derivatives

In order to force Nonconvex to use SparseDiffTools when differentiating a function f(x...) once, the sparsify function modifier can be used:

F = sparsify(f, x...; hessian = false)
-F(x...)

where x is some sample input arguments to f. F(x...) can now be used inplace of f(x...) in objectives and/or constraints to be differentiated. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F once, SparseDiffTools will now be used.

Second order derivatives

When hessian = false (the default value), only the Jacobian/gradient of F will be treated as sparse. In order to use SparseDiffTools to compute sparse second order derivatives as well, you can set hessian = true. This is recommended for scalar-valued functions with sparse Hessian matrices. Setting hessian = true will also work for vector-valued functions f or for functions f that return multiple, non-vector outputs. The sparsity of the third order derivative tensor will be used to compute the third order tensor efficiently.

User-defined sparsity patterns

Using sparsify as shown above will make use of Symbolics to uncover the sparsity of the Jacobian and Hessian matrices of f. In some cases, the function f may not be Symbolics-compatible or it may have a known sparsity pattern. The user can therefore use the jac_pattern or hess_pattern keyword arguments to set the pattern manually.

The jac_pattern is expected to be a SparseMatrixCSC of element type Bool with true where there is a structural non-zero, and false where there is a structural zero in the Jacobian matrix. The size of jac_pattern should be noutputs x ninputs where noutputs is the number of outputs of f and ninputs is the number of inputs to f. When the inputs and/or outputs are multiple and/or non-vector, they are assumed to be flattened to vectors and noutputs/ninputs is the length of the flat vector.

Passing the Hessian sparsity pattern is also possible using the hess_pattern keyword argument. For scalar-valued functions, hess_pattern should have size ninputs x ninputs where ninputs is the number of input variables in the flattened input arguments. For vector-valued functions, the sparsity pattern will be the sparsity pattern of the Jacobian of the linearized Jacobian of f. Assume f takes a single vector input x and returns a single output vector y. The sparsity pattern will be that of d(vec(dy/dx))/dx. hess_pattern should therefore have size (noutputs * ninputs) x ninputs. For example, assume y is a vector of length 2 and x is a vector of length 3. The Jacobian dy/dx will be a matrix of size 2 x 3. vec(dy/dx) will be a vector of length 6. d(vec(dy/dx))/dx will be a matrix of size 6 x 3. hess_pattern should therefore be a SparseMatrixCSC with element type Bool and size 6 x 3.

For general functions f with multiple or non-vector inputs or outputs, noutputs and ninputs are the lengths of the flattened outputs and inputs respectively.

Sparsifying a model

Instead of sparsifying one function at a time, the user can instead sparsify an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

sp_model = sparsify(model, hessian = true)

where model is of type Model or DictModel and hessian has the same intepretation from the function sparisfication above. sp_model can now be optimized using any of the Nonconvex algorithms compatible with the model.

By default, the objective and all the constraint functions will be sparsified. To prevent the sparisfication of some component of the model, any of the following keyword arguments can be set to false (default is true):

  • objective = false
  • ineq_constraints = false
  • eq_constraints = false
  • sd_constraints = false

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the sparisification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

+F(x...)

where x is some sample input arguments to f. F(x...) can now be used inplace of f(x...) in objectives and/or constraints to be differentiated. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F once, SparseDiffTools will now be used.

Second order derivatives

When hessian = false (the default value), only the Jacobian/gradient of F will be treated as sparse. In order to use SparseDiffTools to compute sparse second order derivatives as well, you can set hessian = true. This is recommended for scalar-valued functions with sparse Hessian matrices. Setting hessian = true will also work for vector-valued functions f or for functions f that return multiple, non-vector outputs. The sparsity of the third order derivative tensor will be used to compute the third order tensor efficiently.

User-defined sparsity patterns

Using sparsify as shown above will make use of Symbolics to uncover the sparsity of the Jacobian and Hessian matrices of f. In some cases, the function f may not be Symbolics-compatible or it may have a known sparsity pattern. The user can therefore use the jac_pattern or hess_pattern keyword arguments to set the pattern manually.

The jac_pattern is expected to be a SparseMatrixCSC of element type Bool with true where there is a structural non-zero, and false where there is a structural zero in the Jacobian matrix. The size of jac_pattern should be noutputs x ninputs where noutputs is the number of outputs of f and ninputs is the number of inputs to f. When the inputs and/or outputs are multiple and/or non-vector, they are assumed to be flattened to vectors and noutputs/ninputs is the length of the flat vector.

Passing the Hessian sparsity pattern is also possible using the hess_pattern keyword argument. For scalar-valued functions, hess_pattern should have size ninputs x ninputs where ninputs is the number of input variables in the flattened input arguments. For vector-valued functions, the sparsity pattern will be the sparsity pattern of the Jacobian of the linearized Jacobian of f. Assume f takes a single vector input x and returns a single output vector y. The sparsity pattern will be that of d(vec(dy/dx))/dx. hess_pattern should therefore have size (noutputs * ninputs) x ninputs. For example, assume y is a vector of length 2 and x is a vector of length 3. The Jacobian dy/dx will be a matrix of size 2 x 3. vec(dy/dx) will be a vector of length 6. d(vec(dy/dx))/dx will be a matrix of size 6 x 3. hess_pattern should therefore be a SparseMatrixCSC with element type Bool and size 6 x 3.

For general functions f with multiple or non-vector inputs or outputs, noutputs and ninputs are the lengths of the flattened outputs and inputs respectively.

Sparsifying a model

Instead of sparsifying one function at a time, the user can instead sparsify an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

sp_model = sparsify(model, hessian = true)

where model is of type Model or DictModel and hessian has the same intepretation from the function sparisfication above. sp_model can now be optimized using any of the Nonconvex algorithms compatible with the model.

By default, the objective and all the constraint functions will be sparsified. To prevent the sparisfication of some component of the model, any of the following keyword arguments can be set to false (default is true):

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the sparisification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

diff --git a/dev/gradients/symbolic/index.html b/dev/gradients/symbolic/index.html index e1cd3b9..753b876 100644 --- a/dev/gradients/symbolic/index.html +++ b/dev/gradients/symbolic/index.html @@ -1,3 +1,3 @@ Symbolic differentiation · Nonconvex.jl
GitHub

Symbolic differentiation

Background

For functions, a tractable symbolic gradient/Jacobian/Hessian may exist. Symbolics.jl is a symbolic mathematics package in Julia that can uncover the mathematical expression from Julia functions and then symbolically differentiate the resulting expression. Symbolic simplifications and cancellations can sometimes lead to computational savings compared to algorithmic differentiation. Symbolic differentiation can further exploit the sparsity of the graident, Jacobian and/Hessian if one exists.

In Nonconvex, you can enforce the use of Symbolics to symbolically differentiate specific functions using the symbolify function modifier. In particular, the Symbolics-derived gradient/Jacobian/Hessian functions will be used whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate the modified function.

Symbolifying a function

First order derivatives

In order to force Nonconvex to use Symbolics when differentiating a function f(x...) once, the symbolify function modifier can be used:

F = symbolify(f, x...; hessian = false, sparse = false, simplify = false)
-F(x...)

where x is some sample input arguments to f. F(x...) can now be used inplace of f(x...) in objectives and/or constraints to be differentiated. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F once, the Symbolics-derived gradient/Jacobian will now be used.

The sparse keyword argument can be set to true (default is false) to tell Symbolics to return a sparse gradient/Jacobian for the function F. The simplify keyword argument can be set to true (default is false) to tell Symbolics to simplify the mathematical expressions for the gradient/Jacobian functions.

Second order derivatives

When hessian = false (the default value), only the Jacobian/gradient of F will be computed with Symbolics. In order to use Symbolics to differentiate the function F twice, you can set hessian = true. Setting hessian = true will also work for vector-valued functions f or for functions f that return multiple, non-vector outputs. The sparse and simplify keyword arguments work the same way when hessian is set to true.

Symbolifying a model

Instead of symbolifying one function at a time, the user can instead symbolify an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

sym_model = symbolify(model, hessian = true, simplify = true, sparse = true)

where model is of type Model or DictModel and hessian, simplify and sparse have the same intepretation from the function symbolification above. sym_model can now be optimized using any of the Nonconvex algorithms compatible with the model.

By default, the objective and all the constraint functions will be symbolified. To prevent the symbolification of some component of the model, any of the following keyword arguments can be set to false (default is true):

  • objective = false
  • ineq_constraints = false
  • eq_constraints = false
  • sd_constraints = false

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the symbolification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

+F(x...)

where x is some sample input arguments to f. F(x...) can now be used inplace of f(x...) in objectives and/or constraints to be differentiated. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F once, the Symbolics-derived gradient/Jacobian will now be used.

The sparse keyword argument can be set to true (default is false) to tell Symbolics to return a sparse gradient/Jacobian for the function F. The simplify keyword argument can be set to true (default is false) to tell Symbolics to simplify the mathematical expressions for the gradient/Jacobian functions.

Second order derivatives

When hessian = false (the default value), only the Jacobian/gradient of F will be computed with Symbolics. In order to use Symbolics to differentiate the function F twice, you can set hessian = true. Setting hessian = true will also work for vector-valued functions f or for functions f that return multiple, non-vector outputs. The sparse and simplify keyword arguments work the same way when hessian is set to true.

Symbolifying a model

Instead of symbolifying one function at a time, the user can instead symbolify an entire Nonconvex model including the objective, all the inequality constraint functions, all the equality constraint functions and all the semidefinite constraint functions.

sym_model = symbolify(model, hessian = true, simplify = true, sparse = true)

where model is of type Model or DictModel and hessian, simplify and sparse have the same intepretation from the function symbolification above. sym_model can now be optimized using any of the Nonconvex algorithms compatible with the model.

By default, the objective and all the constraint functions will be symbolified. To prevent the symbolification of some component of the model, any of the following keyword arguments can be set to false (default is true):

Setting the objective, ineq_constraints, eq_constraints, and/or sd_constraints keyword arguments to false (default is true) will prevent the symbolification of the objective, all the inequality constraint functions, all the equality constraint functions, and/or all the semidefinite constraint functions respectively.

diff --git a/dev/gradients/user_defined/index.html b/dev/gradients/user_defined/index.html index 5782f6d..ae28250 100644 --- a/dev/gradients/user_defined/index.html +++ b/dev/gradients/user_defined/index.html @@ -2,4 +2,4 @@ User-defined gradient, Jacobian or Hessian · Nonconvex.jl
GitHub

User-defined gradient, Jacobian or Hessian

Gradients and Jacobians

To use a user-defined gradient/Jacobian function g(x) for a function f(x), you can use the CustomGradFunction modifier:

F = CustomGradFunction(f, g)
 F(x)

F can be then used in place of f as an objective function, as a constraint function or as part of any such function. When f is scalar-valued, g is expected to return a gradient vector. When f is vector-valued, g is expected to return a Jacobian matrix. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F, the custom gradient/Jacobian function g will be used.

Hessian or Hessian vector product

For second-order optimization algorithms, a user-defined Hessian function h(x) can be used for any scalar-valued function f(x) with gradient g(x). To use a user-defined Hessian function h(x), you can use the CustomHessianFunction modifier:

F = CustomHessianFunction(f, g, h)
 F(x)

F can be then used in place of f as an objective function, as a constraint function or as part of any such function. f is expected to return a scalar, g is expected to return a gradient vector and h is expected to return a symmetric Hessian matrix. Whenever ForwardDiff or any ChainRules-compatible AD package such as Zygote is used to differentiate F, the custom gradient and Hessian functions will be used.

Instead of a Hessian function h, alternatively a Hessian-vector product operator h(x, v) can be used, which multiplies the Hessian of f at x by the vector v. To use a Hessian-vector product operator hvp(x, v) instead of computing the full Hessian, you can pass the hvp function as the third argument to CustomHessianFunction and set the hvp keyword argument to true:

F = CustomHessianFunction(f, g, hvp; hvp = true)
-F(x)
+F(x) diff --git a/dev/index.html b/dev/index.html index 898d5f8..329b054 100644 --- a/dev/index.html +++ b/dev/index.html @@ -14,4 +14,4 @@ options = NLoptOptions() r = optimize(model, alg, [1.0, 1.0], options = options) r.minimum # objective value -r.minimizer # decision variables

Table of contents

+r.minimizer # decision variables

Table of contents

diff --git a/dev/problem/dict_model/index.html b/dev/problem/dict_model/index.html index 0645193..ef6fc81 100644 --- a/dev/problem/dict_model/index.html +++ b/dev/problem/dict_model/index.html @@ -2,4 +2,4 @@ DictModel definition · Nonconvex.jl
GitHub

DictModel definition

There are 2 ways to define a DictModel. The direct method is:

model = DictModel()

To pass an objective function while constructing the model, use:

model = DictModel(obj)

where obj is a function that takes a single OrderedDict argument.

JuMP model to Nonconvex DictModel

JuMP.jl has an excellent API for defining variables and linear constraints. Using JuMP makes it straightforward to copy a set of linear constraints and variable definitions from a paper. In Nonconvex, you can start with a JuMP model, define variables and constraints using JuMP's API then convert it to a DictModel. For example:

jump_model = JuMP.Model()
 @variable jump_model 0 <= x[i=1:3] <= 1
 @constraint jump_model sum(x) <= 1
-model = DictModel(jump_model)

The objective can also be defined either using JuMP or Nonconvex. Once you convert the JuMP model to a Nonconvex model, you can go ahead and define more variables and constraints and/or set the objective in Nonconvex.

Variable definition

Add a single variable

Each variable in a DictModel has a name which can be a symbol or string. Each named variable can have an arbitrary type, e.g:

  1. Vectors or arrays in general
  2. Dictionaries
  3. Structs
  4. Tuples
  5. NamedTuples
  6. Nested data structures

Vectorization and de-vectorization are handled automatically by Nonconvex.

To add a new named variable to a DictModel with a name :a and lower and upper bounds lb and ub respectively, use:

addvar!(model, :a, lb, ub)

Similar to Model, optional keyword arguments init and integer can be set, and the types of the initial value, lb and ub must be the same.

Add multiple variables

There is no way to add multiple variables simultaneously to a DictModel however a single named variable that's a vector can be added.

Objective definition

To specify an objective function after creating the model, use:

set_objective!(model, obj)

where obj is a function that takes a single OrderedDict argument. The OrderedDict input to obj will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds.

Inequality constraint definition

To define an inequality constraint f(x) <= 0, where f is a Julia function that accepts a single OrderedDict input, use:

add_ineq_constraint!(model, f)

The OrderedDict input to f will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds. The function f can return:

  1. A number, in which case the constraint will be f(x) <= 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .<= 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Equality constraint definition

To define an inequality constraint f(x) == 0, where f is a Julia function that accepts a single OrderedDict input, use:

add_eq_constraint!(model, f)

The OrderedDict input to f will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds. The function f can return:

  1. A number, in which case the constraint will be f(x) == 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .== 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.
+model = DictModel(jump_model)

The objective can also be defined either using JuMP or Nonconvex. Once you convert the JuMP model to a Nonconvex model, you can go ahead and define more variables and constraints and/or set the objective in Nonconvex.

Variable definition

Add a single variable

Each variable in a DictModel has a name which can be a symbol or string. Each named variable can have an arbitrary type, e.g:

  1. Vectors or arrays in general
  2. Dictionaries
  3. Structs
  4. Tuples
  5. NamedTuples
  6. Nested data structures

Vectorization and de-vectorization are handled automatically by Nonconvex.

To add a new named variable to a DictModel with a name :a and lower and upper bounds lb and ub respectively, use:

addvar!(model, :a, lb, ub)

Similar to Model, optional keyword arguments init and integer can be set, and the types of the initial value, lb and ub must be the same.

Add multiple variables

There is no way to add multiple variables simultaneously to a DictModel however a single named variable that's a vector can be added.

Objective definition

To specify an objective function after creating the model, use:

set_objective!(model, obj)

where obj is a function that takes a single OrderedDict argument. The OrderedDict input to obj will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds.

Inequality constraint definition

To define an inequality constraint f(x) <= 0, where f is a Julia function that accepts a single OrderedDict input, use:

add_ineq_constraint!(model, f)

The OrderedDict input to f will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds. The function f can return:

  1. A number, in which case the constraint will be f(x) <= 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .<= 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Equality constraint definition

To define an inequality constraint f(x) == 0, where f is a Julia function that accepts a single OrderedDict input, use:

add_eq_constraint!(model, f)

The OrderedDict input to f will be of the same structure, shape and types as the OrderedDict initial solution, lower bounds and upper bounds. The function f can return:

  1. A number, in which case the constraint will be f(x) == 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .== 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.
diff --git a/dev/problem/model/index.html b/dev/problem/model/index.html index 528455a..de4aab8 100644 --- a/dev/problem/model/index.html +++ b/dev/problem/model/index.html @@ -1,2 +1,2 @@ -Model definition · Nonconvex.jl
GitHub

Model definition

To define an empty model, run:

model =  Model()

To specify an objective function obj when creating the model, run:

model = Model(obj)

where obj is a function that takes a single vector argument.

Variable definition

Add a single variable

To add a new variable to a Model with lower and upper bounds lb and ub respectively, use:

addvar!(model, lb, ub)

The variables added will be stacked on top of each other with a linear integer index. The lower and upper bounds for each variable don't have to be numbers, they can be:

  1. Dictionaries
  2. Structs
  3. Tuples
  4. NamedTuples
  5. Nested data structures

However, the values input cannot be vectors. A vector input has a different interpretation. See the next section.

The types of the lower and upper bounds of each variable must be the same and this type will be assumed to be the type of the decision variable. Different variables can have different types though. Vectorization and de-vectorization are handled automatically by Nonconvex.

To specify an initial value, use the init keyword argument. To add an integer constraint on the variable, use the integer keyword argument. For example:

addvar!(model, 0.0, 10.0, init = 1.0, integer = true)

init must have the same type as the lower and upper bounds.

Add multiple variables

To add multiple variables simultaneously, pass in a vector of values for the bounds and optionally for the init and integer keyword arguments.

addvar!(model, [0.0, 0.0], [10.0, 10.0], init = [1.0, 1.0], integer = [true, false])

The elements of the vector can be:

  1. Vectors or arrays in general
  2. Dictionaries
  3. Structs
  4. Tuples
  5. NamedTuples
  6. Nested data structures

Note that the use of vectors as elements is allowed. Similarly, the types of the lower and upper bounds and the initial values must be the same.

Objective definition

To specify an objective function after creating the model, use:

set_objective!(model, obj)

where obj is a function that takes a single vector argument. The vector input to obj will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector.

Inequality constraint definition

To define an inequality constraint f(x) <= 0, where f is a Julia function that accepts a single input vector, use:

add_ineq_constraint!(model, f)

The vector input to f will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector. The function f can return:

  1. A number, in which case the constraint will be f(x) <= 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .<= 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Equality constraint definition

To define an inequality constraint f(x) == 0, where f is a Julia function that accepts a single input vector, use:

add_eq_constraint!(model, f)

The vector input to f will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector. The function f can return:

  1. A number, in which case the constraint will be f(x) == 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .== 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Changing variable bounds

After defining the variables, it is possible to set the minimum and maximum variable bounds to different variables. This is useful for example in iterative procedures where the bounds are updated and the problem is resolved.

To update the entire vector of minimum bounds, you can use:

setmin!(model, newmin)

where newmin is a vector of bounds. -Inf is allowed in the bounds.

To set a new minimum bound for the ith variable only, you can use:

setmin!(model, i, newmin)

instead where newmin is a minimum bound of the appropriate type depending on the type of the ith variable.

Similarly, to update the entire vector of maximum bounds, you can use:

setmax!(model, newmax)

where newmax is a vector of bounds. Inf is allowed in the bounds.

To set a new maximum bound for the ith variable only, you can use:

setmax!(model, i, newmax)

instead where newmax is a maximum bound of the appropriate type depending on the type of the ith variable.

Changing integrality constraints

To constrain a variable to be integer or relax the integrality constraint on the ith variable, you can use:

setinteger!(model, i, integer)

where integer is true to constrain the variable or false to relax the constraint.

+Model definition · Nonconvex.jl
GitHub

Model definition

To define an empty model, run:

model =  Model()

To specify an objective function obj when creating the model, run:

model = Model(obj)

where obj is a function that takes a single vector argument.

Variable definition

Add a single variable

To add a new variable to a Model with lower and upper bounds lb and ub respectively, use:

addvar!(model, lb, ub)

The variables added will be stacked on top of each other with a linear integer index. The lower and upper bounds for each variable don't have to be numbers, they can be:

  1. Dictionaries
  2. Structs
  3. Tuples
  4. NamedTuples
  5. Nested data structures

However, the values input cannot be vectors. A vector input has a different interpretation. See the next section.

The types of the lower and upper bounds of each variable must be the same and this type will be assumed to be the type of the decision variable. Different variables can have different types though. Vectorization and de-vectorization are handled automatically by Nonconvex.

To specify an initial value, use the init keyword argument. To add an integer constraint on the variable, use the integer keyword argument. For example:

addvar!(model, 0.0, 10.0, init = 1.0, integer = true)

init must have the same type as the lower and upper bounds.

Add multiple variables

To add multiple variables simultaneously, pass in a vector of values for the bounds and optionally for the init and integer keyword arguments.

addvar!(model, [0.0, 0.0], [10.0, 10.0], init = [1.0, 1.0], integer = [true, false])

The elements of the vector can be:

  1. Vectors or arrays in general
  2. Dictionaries
  3. Structs
  4. Tuples
  5. NamedTuples
  6. Nested data structures

Note that the use of vectors as elements is allowed. Similarly, the types of the lower and upper bounds and the initial values must be the same.

Objective definition

To specify an objective function after creating the model, use:

set_objective!(model, obj)

where obj is a function that takes a single vector argument. The vector input to obj will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector.

Inequality constraint definition

To define an inequality constraint f(x) <= 0, where f is a Julia function that accepts a single input vector, use:

add_ineq_constraint!(model, f)

The vector input to f will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector. The function f can return:

  1. A number, in which case the constraint will be f(x) <= 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .<= 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Equality constraint definition

To define an inequality constraint f(x) == 0, where f is a Julia function that accepts a single input vector, use:

add_eq_constraint!(model, f)

The vector input to f will be of the same structure, shape and types as the initial solution, lower bound and upper bound vector. The function f can return:

  1. A number, in which case the constraint will be f(x) == 0
  2. A vector or array of numbers, in which case the constraint will be applied element-wise f(x) .== 0.
  3. An arbitrary container or data structure, in which case the output will be vectorized first and the constraint will be applied element-wise on the vectorized output.

Changing variable bounds

After defining the variables, it is possible to set the minimum and maximum variable bounds to different variables. This is useful for example in iterative procedures where the bounds are updated and the problem is resolved.

To update the entire vector of minimum bounds, you can use:

setmin!(model, newmin)

where newmin is a vector of bounds. -Inf is allowed in the bounds.

To set a new minimum bound for the ith variable only, you can use:

setmin!(model, i, newmin)

instead where newmin is a minimum bound of the appropriate type depending on the type of the ith variable.

Similarly, to update the entire vector of maximum bounds, you can use:

setmax!(model, newmax)

where newmax is a vector of bounds. Inf is allowed in the bounds.

To set a new maximum bound for the ith variable only, you can use:

setmax!(model, i, newmax)

instead where newmax is a maximum bound of the appropriate type depending on the type of the ith variable.

Changing integrality constraints

To constrain a variable to be integer or relax the integrality constraint on the ith variable, you can use:

setinteger!(model, i, integer)

where integer is true to constrain the variable or false to relax the constraint.

diff --git a/dev/problem/problem/index.html b/dev/problem/problem/index.html index bb50adf..e06774c 100644 --- a/dev/problem/problem/index.html +++ b/dev/problem/problem/index.html @@ -1,2 +1,2 @@ -Overview · Nonconvex.jl
GitHub

Problem definition

There are 3 ways to define a model in Nonconvex.jl:

  1. Model which assumes all the variables are indexed by an integer index starting from 1. The decision variables are therefore a vector.
  2. DictModel which assumes each variable has a name. The decision variables are stored in an OrderedDict, an ordered dictionary data structure.
  3. Start from JuMP.Model and convert it to DictModel. This is convenient to make use of JuMP's user-friendly macros for variable and linear expression, objective or constraint definitions.

Table of contents

+Overview · Nonconvex.jl
GitHub

Problem definition

There are 3 ways to define a model in Nonconvex.jl:

  1. Model which assumes all the variables are indexed by an integer index starting from 1. The decision variables are therefore a vector.
  2. DictModel which assumes each variable has a name. The decision variables are stored in an OrderedDict, an ordered dictionary data structure.
  3. Start from JuMP.Model and convert it to DictModel. This is convenient to make use of JuMP's user-friendly macros for variable and linear expression, objective or constraint definitions.

Table of contents

diff --git a/dev/problem/queries/index.html b/dev/problem/queries/index.html index dd9c46b..02b7135 100644 --- a/dev/problem/queries/index.html +++ b/dev/problem/queries/index.html @@ -1,2 +1,2 @@ -Querying models · Nonconvex.jl
GitHub

Model queries

There are a number of information you can query about the model after constructing it. These can be useful to check that the model was defined correctly or in the post-processing step after the model has been optimized.

Number of decision variables

To query the number of decision variables in a model, use:

NonconvexCore.getnvars(model)

Number of constraints

To query the number of inequality constraints in a model, you can use:

NonconvexCore.getnineqconstraints(model)

A vector-valued constraint will be counted only once.

To query the number of equality constraints, you can use:

NonconvexCore.getneqconstraints(model)

To query the number of semidefinite constraints, you can use:

NonconvexCore.getnsdconstraints(model)

To query the total number of constraints in a model, you can use:

NonconvexCore.getnconstraints(model)

This is the sum of all the previous queries.

Problem dimension

To get a quick overview of the number of constraints and variables in the model, you can use:

NonconvexCore.getdim(model)

which is short for:

(NonconvexCore.getnconstraints(model), NonconvexCore.getnvars(model))

Objective and constraint functions

You can get the objective and constraint functions as regular Julia functions which can be evaluated. To get the objective function, you can use:

obj = NonconvexCore.getobjective(model)

To get a function for all the inequality constraints, you can use:

ineq = NonconvexCore.getineqconstraints(model)

To get a function for all the equality constraints, you can use:

ineq = NonconvexCore.geteqconstraints(model)

To get a function for all the semideifnite constraint functions, you can use:

ineq = NonconvexCore.getsdconstraints(model)

Initial solution

You can the initial solution using:

x0 = NonconvexCore.getinit(model)

Variables bounds

You can query the maximum variable bounds for all the variables using:

NonconvexCore.getmax(model)

Similarly, you can query the minimum variable bounds for all the variables using:

NonconvexCore.getmin(model)

Integrality constraints

To get the vector indiciting which variables are integer, you can use:

model.integer

which will be a BitVector (similar to a vector of Bool) with true corresponding to an integer constraint and false corresponding to a continuous variable.

+Querying models · Nonconvex.jl
GitHub

Model queries

There are a number of information you can query about the model after constructing it. These can be useful to check that the model was defined correctly or in the post-processing step after the model has been optimized.

Number of decision variables

To query the number of decision variables in a model, use:

NonconvexCore.getnvars(model)

Number of constraints

To query the number of inequality constraints in a model, you can use:

NonconvexCore.getnineqconstraints(model)

A vector-valued constraint will be counted only once.

To query the number of equality constraints, you can use:

NonconvexCore.getneqconstraints(model)

To query the number of semidefinite constraints, you can use:

NonconvexCore.getnsdconstraints(model)

To query the total number of constraints in a model, you can use:

NonconvexCore.getnconstraints(model)

This is the sum of all the previous queries.

Problem dimension

To get a quick overview of the number of constraints and variables in the model, you can use:

NonconvexCore.getdim(model)

which is short for:

(NonconvexCore.getnconstraints(model), NonconvexCore.getnvars(model))

Objective and constraint functions

You can get the objective and constraint functions as regular Julia functions which can be evaluated. To get the objective function, you can use:

obj = NonconvexCore.getobjective(model)

To get a function for all the inequality constraints, you can use:

ineq = NonconvexCore.getineqconstraints(model)

To get a function for all the equality constraints, you can use:

ineq = NonconvexCore.geteqconstraints(model)

To get a function for all the semideifnite constraint functions, you can use:

ineq = NonconvexCore.getsdconstraints(model)

Initial solution

You can the initial solution using:

x0 = NonconvexCore.getinit(model)

Variables bounds

You can query the maximum variable bounds for all the variables using:

NonconvexCore.getmax(model)

Similarly, you can query the minimum variable bounds for all the variables using:

NonconvexCore.getmin(model)

Integrality constraints

To get the vector indiciting which variables are integer, you can use:

model.integer

which will be a BitVector (similar to a vector of Bool) with true corresponding to an integer constraint and false corresponding to a continuous variable.

diff --git a/dev/result/index.html b/dev/result/index.html index 10af484..bc73eeb 100644 --- a/dev/result/index.html +++ b/dev/result/index.html @@ -1,2 +1,2 @@ -Optimization result · Nonconvex.jl
GitHub

Optimization result

Each algorithm is free to return a different result type from the optimize function. However, all the result types have 2 fields:

  • result.minimum: stores the minimum objective value reached in the optimization
  • result.minimizer: stores the optimal decision variables reached during optimization

Some result types store additional information returned by the solver, e.g. the convergence status. Please explore the fields of the result output from optimize and/or check the documentation of the individual algorithms in the algorithms section of the documentation. If you have further questions, feel free to open issues in the Nonconvex.jl repository.

+Optimization result · Nonconvex.jl
GitHub

Optimization result

Each algorithm is free to return a different result type from the optimize function. However, all the result types have 2 fields:

  • result.minimum: stores the minimum objective value reached in the optimization
  • result.minimizer: stores the optimal decision variables reached during optimization

Some result types store additional information returned by the solver, e.g. the convergence status. Please explore the fields of the result output from optimize and/or check the documentation of the individual algorithms in the algorithms section of the documentation. If you have further questions, feel free to open issues in the Nonconvex.jl repository.