-
Notifications
You must be signed in to change notification settings - Fork 517
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ampl_repn
option to Incidence Analysis
#3069
Conversation
As written, this PR introduces a large performance bottleneck. For example: import os
import examples.dae.distill_DAE as distill_module
import pyomo.environ as pyo
from pyomo.contrib.incidence_analysis import IncidenceGraphInterface
from pyomo.contrib.incidence_analysis.config import IncidenceMethod
from pyomo.common.timing import TicTocTimer
datfile = os.path.join(
os.path.dirname(distill_module.__file__),
"distill.dat",
)
instance = distill_module.model.create_instance(datfile)
pyo.TransformationFactory("dae.finite_difference").apply_to(
instance, wrt=instance.t, nfe=50,
)
timer = TicTocTimer()
timer.tic()
igraph = IncidenceGraphInterface(instance)
timer.toc("default (standard_repn)")
igraph = IncidenceGraphInterface(instance, method=IncidenceMethod.ampl_repn)
timer.toc("ampl_repn") gives the output: [ 0.00] Resetting the tic/toc delta timer
[+ 0.56] default (standard_repn)
[+ 12.97] ampl_repn |
We can make this a little better by re-using the visitor (and [ 0.00] Resetting the tic/toc delta timer
[+ 0.57] default (standard_repn)
[+ 6.68] ampl_repn |
ampl_repn
option to Incidence Analysisampl_repn
option to Incidence Analysis
Changing to WIP until this performance hit is figured out. |
Can you add an option to use / time Some notes on the NL writer:
|
This is only the case if |
I will try this. In the meantime, these tests timer = TicTocTimer()
timer.tic()
igraph = IncidenceGraphInterface(instance)
timer.toc("default (standard_repn)")
igraph = IncidenceGraphInterface(instance, method=IncidenceMethod.standard_repn_compute_va
timer.toc("standard_repn_compute_values")
igraph = IncidenceGraphInterface(instance, method=IncidenceMethod.ampl_repn)
timer.toc("ampl_repn")
pyo.SolverFactory("ipopt").solve(instance)
timer.toc("solve with Ipopt") give these results [ 0.00] Resetting the tic/toc delta timer
[+ 0.57] default (standard_repn)
[+ 0.56] standard_repn_compute_values
[+ 6.65] ampl_repn
[+ 0.21] solve with Ipopt We can write the nl file and solve with Ipopt in a fraction of a second, so the extremely long compilation time in |
I had missed passing the visitor to another spot where [ 0.00] Resetting the tic/toc delta timer
[+ 0.58] default (standard_repn)
[+ 0.58] standard_repn_compute_values
[+ 0.60] ampl_repn
[+ 0.21] solve with Ipopt This couples Given that |
It's surprising to me that the visitor's ramp-up is so significant, but I could see it being an issue. You could consider making Separately, if you are going to walk / collect a bunch of expressions, it might make sense to to it all within a |
I did not notice any difference when using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just pushed a change that moves construction of the AMPLRepnVisitor
to the config system. The motivation for this is that I don't want to have code in the main interface class and functions handle a particular IncidenceMethod
specially. However, this involves some hacking of ConfigDict
, so I may revert this change. The alternative would be to call a method like kwds = _construct_visitor_if_necessary(kwds)
before passing kwds
to IncidenceConfig
.
class _ReconstructVisitor: | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At some point I needed a singleton to trigger re-construction of IncidenceConfig
's visitor. I'm not sure if this is still necessary, and will re-visit.
_ampl_repn_visitor = ConfigValue( | ||
default=_ReconstructVisitor, | ||
domain=_amplrepnvisitor_validator, | ||
description="Visitor used to generate AMPLRepn of each constraint", | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By default, _ReconstructVisitor
is passed to the validator, and we construct a new visitor. However, as written, cloning this ConfigValue (via __call__
) will create a new value pointing to the same visitor. This effectively means that, unless a new visitor is explicitly received, the same default visitor is used for all new ConfigValues. I.e. we re-use the visitor for multiple models, or even worse, for the same model before and after modification. The code below implements a hack around ConfigDict
to re-construct the visitor instead of using the default.
class _IncidenceConfigDict(ConfigDict): | ||
def __call__( | ||
self, | ||
value=NOTSET, | ||
default=NOTSET, | ||
domain=NOTSET, | ||
description=NOTSET, | ||
doc=NOTSET, | ||
visibility=NOTSET, | ||
implicit=NOTSET, | ||
implicit_domain=NOTSET, | ||
preserve_implicit=False, | ||
): | ||
init_value = value | ||
new = super().__call__( | ||
value=value, | ||
default=default, | ||
domain=domain, | ||
description=description, | ||
doc=doc, | ||
visibility=visibility, | ||
implicit=implicit, | ||
implicit_domain=implicit_domain, | ||
preserve_implicit=preserve_implicit, | ||
) | ||
|
||
if ( | ||
new.method == IncidenceMethod.ampl_repn | ||
and "ampl_repn_visitor" not in init_value | ||
): | ||
new.ampl_repn_visitor = _ReconstructVisitor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we are using ampl_repn
and did not provide a visitor, explicitly trigger reconstruction of the visitor.
The most recent changes still give the timing results that I expect: [ 0.00] Resetting the tic/toc delta timer
[+ 0.64] default (standard_repn)
[+ 0.66] standard_repn_compute_values
[+ 0.60] ampl_repn
[+ 0.97] solve with Ipopt |
ampl_repn
option to Incidence Analysisampl_repn
option to Incidence Analysis
I've updated this to use a I think this PR is ready for review. |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #3069 +/- ##
=======================================
Coverage 88.29% 88.30%
=======================================
Files 832 832
Lines 92234 92293 +59
=======================================
+ Hits 81438 81495 +57
- Misses 10796 10798 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
…into incidence-amplrepn
Following our conversation at the dev meeting today, I tested a [ 0.00] Resetting the tic/toc delta timer
[+ 4.19] default (standard_repn)
[+ 4.24] standard_repn_compute_values
[+ 4.02] ampl_repn
[+ 4.29] linear_repn
[+ 6.25] solve with Ipopt It seems |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noted a few imports that can be cleaned up but otherwise I think this looks good!
Fixes #3068
Summary/Motivation:
The main motivation is to add an option that greedily removes
0*x
terms from nonlinear subexpressions. Secondarily,AMPLRepnVisitor
likely has performance benefits compared togenerate_standard_repn
and is consistent with the problem structure sent to nl files.Changes proposed in this PR:
IncidenceMethod.ampl_repn
option inpyomo.contrib.incidence_analysis.config
Legal Acknowledgement
By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution: