New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Transformer] fix ORPO loss for MOE models #479

Merged

shivam15s merged 1 commit into linkedin:main from kashif:fix-orpo-moe

Dec 16, 2024

Contributor

kashif commented Dec 15, 2024 •

edited

Loading

Summary

Add missing MOE loss when specified in the trainer.

Hardware Type:
run make test to ensure correctness
run make checkstyle to ensure code style
run make test-convergence to ensure convergence


          fix ORPO loss for MOE models

54c6ce3

austin362667 requested a review from shivam15s

December 16, 2024 16:37

shivam15s approved these changes

View reviewed changes

Collaborator

shivam15s left a comment

lgtm

shivam15s merged commit 21baccc into linkedin:main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet