Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stream weights to the GPU when loading a model #7994

Merged
merged 16 commits into from
Nov 28, 2023
Merged

Stream weights to the GPU when loading a model #7994

merged 16 commits into from
Nov 28, 2023

Commits on Nov 15, 2023

  1. Stream weights

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    9453536 View commit details
    Browse the repository at this point in the history

  2. Fix not passing loadOptions

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    fe23272 View commit details
    Browse the repository at this point in the history

  3. Test graph model weights streaming

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    149460d View commit details
    Browse the repository at this point in the history

  4. Only upload to GPU if it's above the cpu forwarding threshold

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    7e22599 View commit details
    Browse the repository at this point in the history

  5. fix lint

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    ae7bd51 View commit details
    Browse the repository at this point in the history

  6. Refactor decodeWeights

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    c4a0967 View commit details
    Browse the repository at this point in the history

  7. Fix string tensor weight reading

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    cd8ce23 View commit details
    Browse the repository at this point in the history

  8. Test decodeWeightsStreaming

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    12a22ee View commit details
    Browse the repository at this point in the history

  9. Fix lint

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    eac3e2d View commit details
    Browse the repository at this point in the history

  10. Do not check webgpu handoff if not using webgpu

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    03f504c View commit details
    Browse the repository at this point in the history

  11. Remove commented line

    @mattsoulanille
    mattsoulanille committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    ed74c0e View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. Remove commented code

    @mattsoulanille
    mattsoulanille committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    7685f1a View commit details
    Browse the repository at this point in the history

  2. Rename streamWeights to getWeightStream

    @mattsoulanille
    mattsoulanille committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    c036afc View commit details
    Browse the repository at this point in the history

  3. Formatting

    @mattsoulanille
    mattsoulanille committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    a987e88 View commit details
    Browse the repository at this point in the history

  4. Apply streamWeights -> getWeightStream rename to converter

    @mattsoulanille
    mattsoulanille committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    f4baf51 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. Merge branch 'master' into weights_stream

    @Linchenn
    Linchenn committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    c2b6559 View commit details
    Browse the repository at this point in the history