[dask] [python] client.rebalance on dask ranker test #3892
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ffineis
changed the title
jameslamb
approved these changes
Feb 2, 2021
There was a problem hiding this comment.
This is awesome! This is one of those fun changes that looks super simple, but I know a ton of good research went into it in #3817 (comment).
Thanks very much 🚀
| @@ -409,7 +418,7 @@ def test_ranker(output, client, listen_port, group): | |||
| # have high rank correlation with scores from serial ranker. | |||
| dcor = spearmanr(rnkvec_dask, y).correlation | |||
| assert dcor > 0.6 | |||
| assert spearmanr(rnkvec_dask, rnkvec_local).correlation > 0.75 | |||
| assert spearmanr(rnkvec_dask, rnkvec_local).correlation > 0.8 | |||
There was a problem hiding this comment.
nice! I'm comfortable with this, given the distributed you saw in #3817 (comment)
| from distributed.utils_test import client, cluster_fixture, gen_cluster, loop | ||
| from scipy.sparse import csr_matrix | ||
| from sklearn.datasets import make_blobs, make_regression | ||
| from sklearn.utils import check_random_state |
There was a problem hiding this comment.
oh thanks for removing this. Didn't realize it was unused.
There was a problem hiding this comment.
all credit goes to pycharm
|
This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Further addresses #3817 by making
test_dask.pya little bit more predictable. Previously thetest_rankertest while usingoutput='array'provided very unevengroupallocations among its two test workers, unlike in the case of dask.dataframe input. Because#datais so small (e.g. 100 rows), when one worker gets a small amount of data relative to the other(s), this can cause rather significant discrepancies between the Dask ranker and the standardLGBMRanker. See comment for more background.Applying
client.rebalancein this case (small data and uneven worker data distributions) helps make the tests a bit more predictable, tightening the distribution of observed spearman correlations withLGBMRanker. Thanks for the patience to the maintainers!