Implement destructive in-place rewrites for Cholesky and Solve Ops

[ricardoV94]

Description

Reviving accidentally closed #577

All scipy.linalg functions offer an overwrite_a and/or overwrite_b argument that can enhance performance by re-using the input memory for the outputs. This PR implements re-writes that will set these flags to True at compile time.

These rewrites are also a nice example for the in-place docs here, so I'll update them with an example as a later commit to this PR.

We added a general inplace functionality to Blockwise, that automatically makes the core_op inplace. This will translate to the core_op if the Blockwise is later removed due to being useless (no batch dims actually exist).

Related Issue

• Closes ENH: Add destructive rewrites for Cholesky Op #572
• Related to #

Checklist

• Checked that the pre-commit linting/style checks pass
• Included tests that prove the fix is effective or that the new feature works
• Added necessary documentation (docstrings and/or example notebooks)
• If you are a pro: each commit corresponds to a relevant logical change

Type of change

• New feature / enhancement
• Bug fix
• Documentation
• Maintenance
• Other (please specify):

---

📚 Documentation preview 📚: https://pytensor--1028.org.readthedocs.build/en/1028/

[codecov]

Codecov Report

Attention: Patch coverage is 81.72043% with 17 lines in your changes missing coverage. Please review.

Project coverage is 81.88%. Comparing base (b248eba) to head (15384ef).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
pytensor/tensor/slinalg.py	77.96%	7 Missing and 6 partials ⚠️
pytensor/tensor/blockwise.py	50.00%	1 Missing and 1 partial ⚠️
pytensor/tensor/rewriting/blockwise.py	92.85%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1028      +/-   ##
==========================================
- Coverage   81.89%   81.88%   -0.01%     
==========================================
  Files         182      182              
  Lines       47778    47858      +80     
  Branches     8597     8618      +21     
==========================================
+ Hits        39126    39190      +64     
- Misses       6487     6494       +7     
- Partials     2165     2174       +9     

Files with missing lines	Coverage Δ
pytensor/graph/op.py	88.02% <100.00%> (+0.12%)	⬆️
pytensor/scan/rewriting.py	81.52% <ø> (ø)
pytensor/sparse/rewriting.py	76.15% <ø> (ø)
pytensor/tensor/random/rewriting/basic.py	93.19% <ø> (ø)
pytensor/tensor/rewriting/blas.py	88.37% <ø> (ø)
pytensor/tensor/rewriting/blas_scipy.py	82.35% <ø> (ø)
pytensor/tensor/rewriting/elemwise.py	91.14% <ø> (ø)
pytensor/tensor/rewriting/subtensor.py	90.32% <ø> (ø)
pytensor/typed_list/rewriting.py	100.00% <ø> (ø)
pytensor/tensor/blockwise.py	81.21% <50.00%> (-0.65%)	⬇️
... and 2 more

... and 1 file with indirect coverage changes

[jessegrabowski]

Very cool, glad to see this over the finish line (with a bunch of extended functionality!)

I left mostly nitpicks. This could be merged as-is without any changes IMO.

My biggest criticism of the approach, if I had to offer one, would be the use of a list of integers for allowed_inplace_inputs, which you seem to be aware of yoruself because of the extended comment in slinalg.py:L521-523. I don't have a better alternative (what came immediately to mind is to always pass a list of bool of length len(self.inputs), and check the appropriate positions for True. but it's no better). I guess it's frustrating that we sometimes encode informaton about codes with dicts (destroy_map) and other times with lists (allowed_inplace_inputs). It's somewhat difficult as a dev to know what to expect.