Deterministic 2 #162

larsgottesbueren · 2023-09-28T13:49:26Z

This PR contains

some non-determinism fixes in the coarsener
a parallelization of the block-pair-wise move approval algorithm
adds fixed vertex support for the deterministic code
adds the new gain types to the deterministic code

removes some old unused code.

…efix finding

… correctness of the parallel implementation

…if balance == 0 --> also use the revert from back approach

…lization over block-pairs in combination with parallelization per block-pair as well as sequential over block-pairs with parallel per block-pair (for larger balancing ranges)

…graph contraction

…ruction if non-deterministic

…ndent of the machine's number of threads (the algorithms using it don't have non-determinism concerning the distribution of elements to buckets)

kittobi1992

LGTM, only some nitpicks.

Currently, using fixed vertices in deterministic mode will lead to a NonSupportedOperationException. Remove the corresponding restriction in partitioner.cpp. Also make sure that using another gain function in deterministic mode will not silently switch to km1. I have not found the corresponding part in the code where this is enforced. If this is already possible ignore this comment.

mt-kahypar/datastructures/static_hypergraph.cpp

mt-kahypar/partition/coarsening/deterministic_multilevel_coarsener.cpp

mt-kahypar/partition/coarsening/multilevel_coarsener.h

mt-kahypar/partition/refinement/deterministic/deterministic_label_propagation.cpp

…stic mode

larsgottesbueren · 2023-10-02T09:30:10Z

LGTM, only some nitpicks.

Currently, using fixed vertices in deterministic mode will lead to a NonSupportedOperationException. Remove the corresponding restriction in partitioner.cpp. Also make sure that using another gain function in deterministic mode will not silently switch to km1. I have not found the corresponding part in the code where this is enforced. If this is already possible ignore this comment.

There is no silent switch to km1. There is a silent switch to the deterministic refiner, but the refiner names don't pertain to an objective function (as in sequential kahypar).

N-Maas

LGTM, only the issue with GainComputation should be fixed before merging.

mt-kahypar/partition/refinement/deterministic/deterministic_label_propagation.cpp

larsgottesbueren added 30 commits June 3, 2021 22:06

unfinished draft implementation of parallel merge style block pair pr…

7986bfe

…efix finding

fix part weight aggregation (wasn't used before)

134de4a

don't search left range if mid is feasible

ddfb66e

finish up pruning. still to go: symmetric case

a53f549

implement symmetric case

62baad4

implementation seems to be working :) remove debug output

5c86a96

add stub for sequential best prefix implementation. used for checking…

0a1371a

… correctness of the parallel implementation

add implementation of sequential prefix

69e7a8d

previous sequential best prefix implementation had other preferences …

6d6955f

…if balance == 0 --> also use the revert from back approach

extract approve per block-pair into lambda and provide both a paralle…

4e74dc3

…lization over block-pairs in combination with parallelization per block-pair as well as sequential over block-pairs with parallel per block-pair (for larger balancing ranges)

merge master into improved block pair prefix apply

14a8127

shuffle block pairs randomly if handled sequentially

db28c11

fix

6d0921f

fix

03582bc

remove shortcut switch

70aee80

use parallelism over block pairs

19ba1b5

parse recalc gains parameter

94b7903

Merge branch 'deterministic' into improved_block_pair_prefix_apply

5b5ce0e

clean up block pair apply code a little more

e304b69

fix warning

cb18649

fix sign bug that was introduced while cleaning up

d67912e

fix division by unsigned int bug

56617f3

prettier fix

0af81ca

fix assertion that does not consider disabled nodes in IP

959af2c

add assertion

045801d

enabled check

508893c

dont waste tls keys on code thats not performance critical

5fd1d70

add TODO (large he remover breaks in case of multi-pins)

87a271e

fix non-determinism source from high degree special handling in hyper…

9acaabf

…graph contraction

only sort for high degree vertices

b95555e

larsgottesbueren added 6 commits September 29, 2023 12:33

use custom assert

6a788c0

clarify command line message comment

ebff482

remove old TODO that was fixed in the meantime

b5d2933

forgot to disable randomization in new gain computation

5c2dd5e

add deterministic flag to contraction to speed up incident nets const…

59b386f

…ruction if non-deterministic

also add flag to graph

b633aaf

larsgottesbueren marked this pull request as draft September 29, 2023 10:58

larsgottesbueren added 5 commits September 29, 2023 13:06

remove old flag from context test

1623222

remove old flag from context test again

8da6668

graph contraction is already deterministic --> mark parameter as unused

66d7823

Revert change where ConcurrentBucketMap's number of buckets is indepe…

4629319

…ndent of the machine's number of threads (the algorithms using it don't have non-determinism concerning the distribution of elements to buckets)

add a deterministic refinement test with low imbalance

58409c5

kittobi1992 requested changes Oct 2, 2023

View reviewed changes

larsgottesbueren added 4 commits October 2, 2023 11:19

use HyperedgeID cast instead of size_t in static_hypergraph

d8c4743

define gain types at class scopes

33a2921

other comment style/syntax for named parameters

d7968f5

remove NonSupportedOperationException for fixed vertices in determini…

7a91aa0

…stic mode

remove old members

f5e4160

larsgottesbueren marked this pull request as ready for review October 2, 2023 10:25

larsgottesbueren requested review from kittobi1992 and N-Maas October 2, 2023 10:26

fix

d99722c

N-Maas approved these changes Oct 4, 2023

View reviewed changes

mt-kahypar/partition/refinement/deterministic/deterministic_label_propagation.cpp Outdated Show resolved Hide resolved

larsgottesbueren added 3 commits October 4, 2023 11:41

add gain computation as member to deterministic label prop

d01c0db

use current_k instead of phg.k() in deterministic label prop

d0f12ff

fix

beaac7f

larsgottesbueren merged commit bf24f75 into master Oct 4, 2023
15 checks passed

larsgottesbueren deleted the deterministic branch November 16, 2023 20:16

N-Maas mentioned this pull request Dec 4, 2023

Performance improvements for coarsening on (large irregular) graphs #171

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deterministic 2 #162

Deterministic 2 #162

larsgottesbueren commented Sep 28, 2023 •

edited

Loading

kittobi1992 left a comment

larsgottesbueren commented Oct 2, 2023

N-Maas left a comment

Deterministic 2 #162

Deterministic 2 #162

Conversation

larsgottesbueren commented Sep 28, 2023 • edited Loading

kittobi1992 left a comment

Choose a reason for hiding this comment

larsgottesbueren commented Oct 2, 2023

N-Maas left a comment

Choose a reason for hiding this comment

larsgottesbueren commented Sep 28, 2023 •

edited

Loading