hSVD (rank and tolerance, new) #1126

mrfh92 · 2023-03-20T10:45:24Z

replaces old #1042 which I closed due to a problem with the CI

Description

Hiearchical SVD (hSVD) computes an approximate truncated SVD of $A \in \mathbb{R}^{m \times n}$ utilizing a distributed hiearchical algorithm; the truncation rank is given by $maxrank$, i.e. if $A = U diag(\sigma) V^T$, $U \in \mathbb{R}^{m \times m}$, $V \in \mathbb{R}^{n \times n}$, $\sigma \in \mathbb{R}^{\min(m,n)}$ is the true SVD of $A$, this routine computes an approximation for $U[:,:maxrank]$ (and $sigma[:maxrank]$, $V[:,:maxrank]$).

There are three routines:

hsvd: "expert" version with all parameters to be set; potential conflicts are not catched
hsvd_rank: user-friendly version with appropriate default-parameters to compute hSVD with prescribed truncation rank
hsvd_reltol: user-friendly version with appropriate default-parameters to compute hSVD with prescribed relative reconstruction accuracy

Selected References:

[1] Iwen, Ong. A distributed and incremental SVD algorithm for agglomerative data analysis on large networks. SIAM J. Matrix Anal. Appl., 37(4), 2016.
[2] Himpe, Leibner, Rave. Hierarchical approximate proper orthogonal decomposition. SIAM J. Sci. Comput., 40 (5), 2018.

Dependencies

hSVD makes use of the pytorch-SVD and MPI-communication. No dependencies on HeAT linear algebra, except for matmul.

Type of change

New feature: hSVD

Memory requirements

TBD

Performance

TBD

Due Diligence

All split configurations tested
Multiple dtypes tested in relevant functions
Documentation updated (if needed)
Title of PR is suitable for corresponding CHANGELOG entry

Does this change modify the behaviour of other functions? If so, which?

no

ghost · 2023-03-20T10:46:41Z

👇 Click on the image for a new way to code review

Legend

github-actions · 2023-03-20T10:49:41Z

Thank you for the PR!

codecov · 2023-03-20T11:04:40Z

Codecov Report

Merging #1126 (6ddc295) into main (9f6b2eb) will increase coverage by 0.06%.
The diff coverage is 94.89%.

❗ Current head 6ddc295 differs from pull request most recent head 9de4e20. Consider uploading reports for the commit 9de4e20 to get more accurate results

@@            Coverage Diff             @@
##             main    #1126      +/-   ##
==========================================
+ Coverage   91.79%   91.85%   +0.06%     
==========================================
  Files          72       74       +2     
  Lines       10497    10712     +215     
==========================================
+ Hits         9636     9840     +204     
- Misses        861      872      +11

Flag	Coverage Δ
unit	`91.85% <94.89%> (+0.06%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
heat/core/linalg/svdtools.py	`92.72% <92.72%> (ø)`
heat/core/linalg/__init__.py	`100.00% <100.00%> (ø)`
heat/utils/data/__init__.py	`100.00% <100.00%> (ø)`
heat/utils/data/matrixgallery.py	`100.00% <100.00%> (ø)`
heat/utils/data/spherical.py	`100.00% <100.00%> (ø)`

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

…VD of the zero-matrix (or numerically zero-matrix) appears ... added also corresponding tests

github-actions · 2023-03-24T13:29:25Z

Thank you for the PR!

github-actions · 2023-03-24T14:04:15Z

Thank you for the PR!

github-actions · 2023-03-27T14:55:49Z

Thank you for the PR!

github-actions · 2023-03-27T15:21:28Z

Thank you for the PR!

github-actions · 2023-03-29T11:08:44Z

Thank you for the PR!

github-actions · 2023-04-18T08:54:51Z

Thank you for the PR!

…cs/heat into features/1041-hsvd-new

github-actions · 2023-04-21T13:27:56Z

Thank you for the PR!

ClaudiaComito

I'm not done with the review, filing what I have so far so it doesn't get lost. Will get back to it later.

heat/core/linalg/tests/test_svdtools.py

heat/core/linalg/svdtools.py

Co-authored-by: Claudia Comito <39374113+ClaudiaComito@users.noreply.github.com>

…cs/heat into features/1041-hsvd-new

github-actions · 2023-06-02T09:18:02Z

Thank you for the PR!

github-actions · 2023-06-02T09:20:18Z

Thank you for the PR!

github-actions · 2023-06-02T09:23:18Z

Thank you for the PR!

github-actions · 2023-06-02T09:35:44Z

Thank you for the PR!

github-actions · 2023-06-02T10:45:13Z

Thank you for the PR!

github-actions · 2023-06-02T11:05:48Z

Thank you for the PR!

ClaudiaComito

Brilliant @mrfh92 !

…ty with versions torch<=1.11.0

github-actions · 2023-06-05T13:16:10Z

Thank you for the PR!

github-actions · 2023-06-05T14:47:41Z

Thank you for the PR!

…atmul`

github-actions · 2023-06-06T08:03:03Z

Thank you for the PR!

Changes have been addressed

added all stuff from the old features/1041-hsvd branch

3318752

mrfh92 mentioned this pull request Mar 20, 2023

hSVD (prescribed rank or tolerance) #1042

Closed

4 tasks

ClaudiaComito self-assigned this Mar 20, 2023

mrfh92 requested review from Markus-Goetz and ClaudiaComito March 20, 2023 11:05

mrfh92 enabled auto-merge March 20, 2023 11:06

ClaudiaComito added enhancement New feature or request linalg high-level functions High-level machine-learning algorithms labels Mar 20, 2023

ClaudiaComito added this to the 1.3.0 milestone Mar 20, 2023

added handling for the exceptional cases where locally/globally the S…

c9d018e

…VD of the zero-matrix (or numerically zero-matrix) appears ... added also corresponding tests

removed some prints that were for debugging only

801c2cd

tiny change to save memory consumption

b5250e6

one step back...

6247bd7

auto-merge was automatically disabled March 29, 2023 09:41
Merge queue setting changed

Merge branch 'main' into features/1041-hsvd-new

a23feb3

Merge branch 'main' into features/1041-hsvd-new

79aac43

Hoppe added 2 commits April 21, 2023 15:21

Merge branch 'main' into features/1041-hsvd-new

df83e9a

Merge branch 'features/1041-hsvd-new' of github.com:helmholtz-analyti…

3f9eee7

…cs/heat into features/1041-hsvd-new

ClaudiaComito requested changes Apr 24, 2023

View reviewed changes

mrfh92 and others added 3 commits June 2, 2023 11:15

Update heat/utils/data/matrixgallery.py

4a6f858

Co-authored-by: Claudia Comito <39374113+ClaudiaComito@users.noreply.github.com>

further requested changes

eb1ddcf

Merge branch 'features/1041-hsvd-new' of github.com:helmholtz-analyti…

202a96a

…cs/heat into features/1041-hsvd-new

further requested changes, in particular w.r.t. the docs

4186e99

dummy commit since AMD CI had a problem

c19583e

skip tests "partial data set" that does not run on AMD

0686505

ClaudiaComito previously approved these changes Jun 2, 2023

View reviewed changes

replaced torch.argwhere by torch.nonzero for backward compatibili…

92b0812

…ty with versions torch<=1.11.0

mrfh92 dismissed ClaudiaComito’s stale review via 92b0812 June 5, 2023 13:11

dummy commit to rerun the CI

2b09167

mrfh92 requested a review from ClaudiaComito June 5, 2023 15:39

ClaudiaComito previously approved these changes Jun 6, 2023

View reviewed changes

changes for backward compatibility: torch.linalg.matmul -> `torch.m…

9de4e20

…atmul`

mrfh92 dismissed ClaudiaComito’s stale review via 9de4e20 June 6, 2023 07:58

mrfh92 removed the request for review from Markus-Goetz June 6, 2023 07:59

mrfh92 requested a review from ClaudiaComito June 6, 2023 08:57

ClaudiaComito removed the benchmark PR label Jun 6, 2023

ClaudiaComito approved these changes Jun 6, 2023

View reviewed changes

mrfh92 merged commit 1bc1cca into main Jun 6, 2023

mrfh92 deleted the features/1041-hsvd-new branch June 6, 2023 11:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hSVD (rank and tolerance, new) #1126

hSVD (rank and tolerance, new) #1126

mrfh92 commented Mar 20, 2023 •

edited

Loading

ghost commented Mar 20, 2023 •

edited by ghost

Loading

Legend

github-actions bot commented Mar 20, 2023

codecov bot commented Mar 20, 2023 •

edited

Loading

github-actions bot commented Mar 24, 2023

github-actions bot commented Mar 24, 2023

github-actions bot commented Mar 27, 2023

github-actions bot commented Mar 27, 2023

github-actions bot commented Mar 29, 2023

github-actions bot commented Apr 18, 2023

github-actions bot commented Apr 21, 2023

ClaudiaComito left a comment

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

ClaudiaComito left a comment

github-actions bot commented Jun 5, 2023

github-actions bot commented Jun 5, 2023

github-actions bot commented Jun 6, 2023

hSVD (rank and tolerance, new) #1126

hSVD (rank and tolerance, new) #1126

Conversation

mrfh92 commented Mar 20, 2023 • edited Loading

Description

Selected References:

Dependencies

Type of change

Memory requirements

Performance

Due Diligence

Does this change modify the behaviour of other functions? If so, which?

ghost commented Mar 20, 2023 • edited by ghost Loading

Legend

github-actions bot commented Mar 20, 2023

codecov bot commented Mar 20, 2023 • edited Loading

Codecov Report

github-actions bot commented Mar 24, 2023

github-actions bot commented Mar 24, 2023

github-actions bot commented Mar 27, 2023

github-actions bot commented Mar 27, 2023

github-actions bot commented Mar 29, 2023

github-actions bot commented Apr 18, 2023

github-actions bot commented Apr 21, 2023

ClaudiaComito left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

github-actions bot commented Jun 2, 2023

ClaudiaComito left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 5, 2023

github-actions bot commented Jun 5, 2023

github-actions bot commented Jun 6, 2023

mrfh92 commented Mar 20, 2023 •

edited

Loading

ghost commented Mar 20, 2023 •

edited by ghost

Loading

codecov bot commented Mar 20, 2023 •

edited

Loading