Flow differentiation

[jbcaillau]

@joseph-gergaud @ocots A brief survey:

• our context: we want first and (at least) second order derivatives wrt. initial condition, initial / final time, parameters of (a priori Hamiltonian) flows; low dimension (< 1e2)
• https://docs.sciml.ai/SciMLSensitivity/stable/getting_started/#Forward-Mode-Automatic-Differentiation-with-ForwardDiff.jl
• https://docs.sciml.ai/SciMLSensitivity/stable/manual/differential_equation_sensitivities/#More-Details-on-Sensitivity-Algorithm-Choices
• some tests (mostly ML inspired, rather big dim ODE's then, but not only) in ¹
• "4. ForwardSensitivity", p. 33: what we want to retrieve in Julia: it is implemented in SciML by the algorithm SciMLSensitivity.ForwardSensitivity, see https://docs.sciml.ai/SciMLSensitivity/stable/manual/differential_equation_sensitivities/#Sensitivity-Algorithms
• "5. ForwardDiffSensitivity", p. 34: "discretize-then-optimize", so I guess the (variable step size) ODE solver is passed through AD; not desirable, although ¹ states (p. 36)

Previous research has shown that the discrete adjoint approach is more stable than continuous adjoints in some cases [53, 47, 94, 95, 96, 97] while continuous adjoints have been demonstrated to be more stable in others [98, 95] and can reduce spurious oscillations [99, 100, 101]. This trade-off between discrete and continuous adjoint approaches has been demonstrated on some equations as a trade-off between stability and computational efficiency

• "Continuous vs. discrete adjoints", p. 36
• on the same topic, see also https://docs.sciml.ai/SciMLSensitivity/stable/manual/differential_equation_sensitivities/#More-Details-on-Sensitivity-Algorithm-Choices

Footnotes

1. https://arxiv.org/pdf/2001.04385 ↩ ↩²