[contrib] Refactor incidence_analysis to cache a graph instead of a matrix

[Robbybp]

EDIT: Dependence of #2715, #2716, #2723, and #2727

• [contrib] Add plot method to IncidenceGraphInterface #2716 incorporates [contrib] Refactor incidence_analysis to cache a graph instead of a matrix #2715
• [contrib] Document Incidence Analysis and update API #2727 incorporates [contrib] Add plot method to IncidenceGraphInterface #2716 and Adding basic diagnostics output to solve_strongly_connected_components #2723
• Adding basic diagnostics output to solve_strongly_connected_components #2723 is small and NFC
• [contrib] Document Incidence Analysis and update API #2727 changes behavior

In summary, merging #2727 will incorporate all the others, but they are broken up to be easier to review.

Summary/Motivation:

IncidenceGraphInterface accepts a model and caches the incidence matrix (in COO format) of variables and constraints. This allows calling dulmage_mendelsohn, etc. without re-constructing the matrix, but does not allow fast look-ups of "adjacent" constraints and variables, i.e. getting all constraints that contain a variable or all variables that participate in a constraint. This would require storing the incidence matrix in both CSR and CSC formats, or alternatively storing the bipartite incidence graph as an adjacency list. This PR refactors incidence_analysis to cache the bipartite incidence graph as a NetworkX Graph instead of caching a SciPy COO matrix, allowing a get_adjacent_to method that quickly looks up adjacent variables/constraints.

Changes proposed in this PR:

• incidence_graph attribute on IncidenceGraphInterface
• incidence_matrix attribute is now a "property" that computes the matrix from the cached graph. This means that this attribute still works, but is now an expensive operation.
• Refactor "algorithm methods", e.g. dulmage_mendelsohn, and the underlying implementations so we do not convert the cached graph to a COO matrix, just to convert it back to a graph
• Add get_adjacent_to method

This PR also runs Black on incidence_analysis, for good measure.

Next steps:

This PR sets the stage for several improvements I'd like to make to incidence_analysis, which will be subsequent PRs:

1. Add a plotting method such as [WIP] Pyomo Model Plot #2251 to IncidenceGraphInterface
2. Update the API to remove public attributes from IncidenceGraphInterface. I would also like to update the block_triangularize and get_diagonal_blocks method names and APIs, as I think the maps returned by block_triangularize are confusing, and the name get_diagonal_blocks is an incomplete misnomer.
3. Add documentation

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[Robbybp]

win/3.9 teest failure appears unrelated

[mrmundt]

win/3.9 teest failure appears unrelated

It is unrelated. We're looking into this failure - it's been consistent across all new jobs run today.

[Robbybp]

@mrmundt Thanks!

[mrmundt]

There are some small typos and questions that I have, but otherwise, this is looking great.

[mrmundt]

@Robbybp - There were some immense issues with our Jenkins server over the last 16ish hours. I'm hopeful that they are resolved now. Please ignore previous failures.

[codecov]

Codecov Report

Base: 87.03% // Head: 86.98% // Decreases project coverage by -0.06% ⚠️

Coverage data is based on head (6f1995f) compared to base (8fe21bb).
Patch coverage: 96.62% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2715      +/-   ##
==========================================
- Coverage   87.03%   86.98%   -0.06%     
==========================================
  Files         771      771              
  Lines       86162    86246      +84     
==========================================
+ Hits        74993    75019      +26     
- Misses      11169    11227      +58     

Flag	Coverage Δ
linux	84.14% <96.62%> (+<0.01%)	⬆️
osx	73.61% <96.62%> (+0.02%)	⬆️
other	84.33% <96.62%> (+0.01%)	⬆️
win	81.52% <96.62%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pyomo/contrib/incidence_analysis/connected.py	100.00% <ø> (ø)
...o/contrib/incidence_analysis/dulmage_mendelsohn.py	84.61% <ø> (ø)
pyomo/contrib/incidence_analysis/util.py	100.00% <ø> (ø)
pyomo/contrib/incidence_analysis/matching.py	90.00% <88.00%> (-10.00%)	⬇️
pyomo/contrib/incidence_analysis/interface.py	98.33% <97.50%> (+0.04%)	⬆️
pyomo/contrib/incidence_analysis/__init__.py	100.00% <100.00%> (ø)
...ib/incidence_analysis/common/dulmage_mendelsohn.py	100.00% <100.00%> (ø)
pyomo/contrib/incidence_analysis/triangularize.py	100.00% <100.00%> (ø)
pyomo/util/subsystems.py	89.13% <0.00%> (-8.70%)	⬇️
...ontrib/appsi/utils/collect_vars_and_named_exprs.py	93.75% <0.00%> (-6.25%)	⬇️
... and 10 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.