PR1: Add mkmmd loss #168
Merged
PR1: Add mkmmd loss #168
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…additional sampler tests
for more information, see https://pre-commit.ci
…rInstitute/FL4Health into sa_add_cifar10_experiments
for more information, see https://pre-commit.ci
This reverts commit e130077.
…L4Health into add_mkmmd_loss
This reverts commit 1f213c3.
This reverts commit fd13904.
for more information, see https://pre-commit.ci
This reverts commit 053e4fc.
emersodb
approved these changes
Sep 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Type
[Feature ]
Short Description
This is a simplified updated version of MK-MMD loss implementation with additions to only Ditto and MR-MTL models.
One mk-mmd loss is implemented one to minimize distance between local features and aggregated global model. In this setting betas are updated even during training after a certain number of step based on beta_update_interval. Also in order to prevent from feature value boost a l2-norm regularizer has been added which can be controlled via feature_l2_norm parameter.
Also I generalized loss reporting function in fl4health/utils/losses as it was failing to report correctly due to addition of new keys during further steps in MOON setup.
Flamby MK-MMD loss experimental setup has also been added to the research/flamby/fedisic2019 folder.
Tests Added
Describe the tests that have been added to ensure the codes correctness, if applicable.