PETSc 3.20.0 broke the world #963
Labels
Comments
|
Mat.setValuesCSR() got astronomically slower. Compare petsc4py 3.18.4: to petsc4py 3.20.1 ( |
|
PETSc matrix preallocation has been commented out since the suite was added to FiPy
|
|
Making a simplistic attempt at preallocation @@ -482,7 +482,7 @@ class _PETScMatrix(_SparseMatrix):
class _PETScMatrixFromShape(_PETScMatrix):
- def __init__(self, rows, cols, bandwidth=0, sizeHint=None, matrix=None, comm=PETSc.COMM_SELF):
+ def __init__(self, rows, cols, bandwidth=1, sizeHint=None, matrix=None, comm=PETSc.COMM_SELF):
"""Instantiates and wraps a PETSc `Mat` matrix
Parameters
@@ -510,13 +510,14 @@ class _PETScMatrixFromShape(_PETScMatrix):
matrix.setSizes([[rows, None], [cols, None]])
matrix.setType('aij') # sparse
matrix.setUp()
-# matrix.setPreallocationNNZ(bandwidth) # FIXME: ??? None, bandwidth
-# matrix.setOption(matrix.Option.NEW_NONZERO_ALLOCATION_ERR, False)
+ if bandwidth > 0:
+ matrix.setPreallocationNNZ(bandwidth)
+ matrix.setOption(matrix.Option.NEW_NONZERO_ALLOCATION_ERR, False)combined with setting
|
|
Comparison of the runtimes for FiPy's test suite vs petsc4py version, including case of naïve preallocation (these are only the 40 slowest tests when using petsc4py 3.20.1): Ratio of runtimes for FiPy's test suite between petsc4py versions, when using naïve preallocation Overall, about the same, and worst cases are within a factor of 4. |
|
Turning on |
guyer
added a commit
that referenced
this issue
Dec 4, 2023
Fixes #963 (Actually, PETSC 3.19.0 broke the world.) This PR: - Assembles before `multTranspose` to prevent newly added exception - Renames `bandwidth` to `nonZerosPerRow` and removes `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. - Uses `Term`'s knowledge of own stencil to preallocate more effectively. Still doesn't do a good job with vector equations, but that's a deeper change (the resolution of which might help #920). * Assemble before multTranspose * Rename `bandwidth` to `nonZerosPerRow` and remove `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. * Fix(?) conda/mamba installs * Fix(?) race condition
guyer
added a commit
that referenced
this issue
Dec 4, 2023
Fixes #963 (Actually, PETSC 3.19.0 broke the world.) This PR: - Assembles before `multTranspose` to prevent newly added exception - Renames `bandwidth` to `nonZerosPerRow` and removes `sizeHint` The two were confusingly redundant: - PySparse takes `sizeHint`, the number of non-zeros in the matrix. - PyTrilinos takes `NumEntriesPerRow`. - petsc4py didn't used to be clear what it took, but is now documented as number of non-zeros per row (of the local portion of the matrix, but we'll ignore that part). - scipy doesn't preallocate. - Linear algebra [defines bandwidth](https://en.wikipedia.org/wiki/Band_matrix#Bandwidth) as "number $k$ such that $a_{i,j}=0$ if $|i-j| > k$", which is roughly half the number of non-zeros per row (and only applies to a band-diagonal matrix). Better to be explicit about what we really mean. Now all take same parameter and PySparse adjusts as needed. `sizeHint` was introduced in @a15d696 (in 2006!) to "allow smaller preallocations", but it was never used that way. Now, `nonZerosPerRow` can take an array_like to specify row-by-row preallocations, which are directly supported by PyTrilinos and petsc4py, and can be simulated for PySparse. Added `exactNonZeros`, which may have performance benefits for PyTrilinos and petsc4py. Currently unused. - Uses `Term`'s knowledge of own stencil to preallocate more effectively. Still doesn't do a good job with vector equations, but that's a deeper change (the resolution of which might help #920). - Fixes(?) conda/mamba installs - Fixes(?) race condition
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Breakages with PETSc observed in #946 correspond to PETSc 3.20.0, (released 2023-09-28, conda-forge petsc4py feedstock updated 2023-10-04). Previous PETSc 3.18.4 worked fine.
Assembly error
Really slow
Comparison of the runtimes for FiPy's test suite vs petsc4py version (these are only the 40 slowest tests when using petsc4py 3.20.1):
Ratio of runtimes for FiPy's test suite between petsc4py versions
The text was updated successfully, but these errors were encountered: