-
Notifications
You must be signed in to change notification settings - Fork 149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PETSc 3.20.0 broke the world #963
Comments
Mat.setValuesCSR() got astronomically slower. Compare petsc4py 3.18.4:
to petsc4py 3.20.1
( |
PETSc matrix preallocation has been commented out since the suite was added to FiPy
|
Making a simplistic attempt at preallocation @@ -482,7 +482,7 @@ class _PETScMatrix(_SparseMatrix):
class _PETScMatrixFromShape(_PETScMatrix):
- def __init__(self, rows, cols, bandwidth=0, sizeHint=None, matrix=None, comm=PETSc.COMM_SELF):
+ def __init__(self, rows, cols, bandwidth=1, sizeHint=None, matrix=None, comm=PETSc.COMM_SELF):
"""Instantiates and wraps a PETSc `Mat` matrix
Parameters
@@ -510,13 +510,14 @@ class _PETScMatrixFromShape(_PETScMatrix):
matrix.setSizes([[rows, None], [cols, None]])
matrix.setType('aij') # sparse
matrix.setUp()
-# matrix.setPreallocationNNZ(bandwidth) # FIXME: ??? None, bandwidth
-# matrix.setOption(matrix.Option.NEW_NONZERO_ALLOCATION_ERR, False)
+ if bandwidth > 0:
+ matrix.setPreallocationNNZ(bandwidth)
+ matrix.setOption(matrix.Option.NEW_NONZERO_ALLOCATION_ERR, False) combined with setting
|
Turning on |
Fixes #963 (Actually, PETSC 3.19.0 broke the world.) This PR: - Assembles before `multTranspose` to prevent newly added exception - Renames `bandwidth` to `nonZerosPerRow` and removes `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. - Uses `Term`'s knowledge of own stencil to preallocate more effectively. Still doesn't do a good job with vector equations, but that's a deeper change (the resolution of which might help #920). * Assemble before multTranspose * Rename `bandwidth` to `nonZerosPerRow` and remove `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. * Fix(?) conda/mamba installs * Fix(?) race condition
Fixes #963 (Actually, PETSC 3.19.0 broke the world.) This PR: - Assembles before `multTranspose` to prevent newly added exception - Renames `bandwidth` to `nonZerosPerRow` and removes `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. - Uses `Term`'s knowledge of own stencil to preallocate more effectively. Still doesn't do a good job with vector equations, but that's a deeper change (the resolution of which might help #920). - Fixes(?) conda/mamba installs - Fixes(?) race condition
Breakages with PETSc observed in #946 correspond to PETSc 3.20.0, (released 2023-09-28, conda-forge petsc4py feedstock updated 2023-10-04). Previous PETSc 3.18.4 worked fine.
Assembly error
Really slow
Comparison of the runtimes for FiPy's test suite vs petsc4py version (these are only the 40 slowest tests when using petsc4py 3.20.1):
Ratio of runtimes for FiPy's test suite between petsc4py versions
The text was updated successfully, but these errors were encountered: