Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Flax BLOOM implementation + demo #17761

Closed
wants to merge 5 commits into from
Closed

[WIP] Flax BLOOM implementation + demo #17761

wants to merge 5 commits into from

Commits on Jun 17, 2022

  1. initial flax skeleton code

    haileyschoelkopf committed Jun 17, 2022
    Configuration menu
    Copy the full SHA
    1bb7b05 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2022

  1. use self.config for consistency

    haileyschoelkopf committed Jun 22, 2022
    Configuration menu
    Copy the full SHA
    3fc824b View commit details
    Browse the repository at this point in the history

  2. add FlaxBloomScaledSoftmax

    haileyschoelkopf committed Jun 22, 2022
    Configuration menu
    Copy the full SHA
    961cee3 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. fuse query_key_value layers

    haileyschoelkopf committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    faddb8d View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2022

  1. add gradient checkpointing

    haileyschoelkopf committed Jul 5, 2022
    Configuration menu
    Copy the full SHA
    c4ed06a View commit details
    Browse the repository at this point in the history