Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ZeRO-3 Slowdown #1170

Closed
wants to merge 40 commits into from
Closed

ZeRO-3 Slowdown #1170

wants to merge 40 commits into from

Commits on Jun 17, 2021

  1. Fix tracing; Add timers; Disable norm()

    @tjruwase
    tjruwase committed Jun 17, 2021
    Configuration menu
    Copy the full SHA
    9a179e4 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jun 17, 2021
    Configuration menu
    Copy the full SHA
    038da9d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into olruwase/zero3_broken_tracing

    @jeffra
    jeffra authored Jun 17, 2021
    Configuration menu
    Copy the full SHA
    8d5c784 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2021

  1. Simplify prefetch code

    @tjruwase
    tjruwase committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    46eb232 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'olruwase/zero3_broken_tracing' of github.com:microsoft/… 

    …DeepSpeed into olruwase/zero3_broken_tracing
    @tjruwase
    tjruwase committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    e93610c View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jun 19, 2021
    Configuration menu
    Copy the full SHA
    6f7c1c2 View commit details
    Browse the repository at this point in the history

  2. Move available parameter numbel book-keeping to param partitioner.

    @tjruwase
    tjruwase committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    af66609 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'olruwase/zero3_broken_tracing' of github.com:microsoft/… 

    …DeepSpeed into olruwase/zero3_broken_tracing
    @tjruwase
    tjruwase committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    7432867 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2021

  1. Merge with master

    @tjruwase
    tjruwase committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    11e6c8f View commit details
    Browse the repository at this point in the history

  2. Format fixes

    @tjruwase
    tjruwase committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    eceb707 View commit details
    Browse the repository at this point in the history

  3. Bug fix

    @tjruwase
    tjruwase committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    bb43da0 View commit details
    Browse the repository at this point in the history

  4. Avoid unneeded synch

    @tjruwase
    tjruwase committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    f3048f2 View commit details
    Browse the repository at this point in the history

  5. Remove assert

    @tjruwase
    tjruwase committed Jun 30, 2021
    Configuration menu
    Copy the full SHA
    0514811 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jul 1, 2021
    Configuration menu
    Copy the full SHA
    c81422b View commit details
    Browse the repository at this point in the history

  2. Remove dead code

    @tjruwase
    tjruwase committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    e09b80a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of github.com:microsoft/DeepSpeed into olruwase… 

    …/zero3_broken_tracing
    @tjruwase
    tjruwase committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    f938a73 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'olruwase/zero3_broken_tracing' of github.com:microsoft/… 

    …DeepSpeed into olruwase/zero3_broken_tracing
    @tjruwase
    tjruwase committed Jul 1, 2021
    Configuration menu
    Copy the full SHA
    34ec7da View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2021

  1. Bug fix

    @tjruwase
    tjruwase committed Jul 2, 2021
    Configuration menu
    Copy the full SHA
    3c4a949 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jul 14, 2021
    Configuration menu
    Copy the full SHA
    a3d92f0 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jul 16, 2021
    Configuration menu
    Copy the full SHA
    bc65ea7 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jul 27, 2021
    Configuration menu
    Copy the full SHA
    43f4bb4 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Jul 29, 2021
    Configuration menu
    Copy the full SHA
    f495465 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Aug 27, 2021
    Configuration menu
    Copy the full SHA
    96c87d7 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Sep 7, 2021
    Configuration menu
    Copy the full SHA
    b2549df View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Sep 8, 2021
    Configuration menu
    Copy the full SHA
    c2e4826 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    fc61a5a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Sep 10, 2021
    Configuration menu
    Copy the full SHA
    0ad2ffe View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 1, 2021
    Configuration menu
    Copy the full SHA
    c648f17 View commit details
    Browse the repository at this point in the history

  2. Remove redudant conditional

    @tjruwase
    tjruwase committed Oct 1, 2021
    Configuration menu
    Copy the full SHA
    f218446 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2021

  1. Avoid multiple zero3 contexts

    @tjruwase
    tjruwase committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    a75e465 View commit details
    Browse the repository at this point in the history

  2. Merge

    @tjruwase
    tjruwase committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    ec3eb56 View commit details
    Browse the repository at this point in the history

  3. Merge with master

    @tjruwase
    tjruwase committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    d5e49d1 View commit details
    Browse the repository at this point in the history

  4. Formatting fixes

    @tjruwase
    tjruwase committed Oct 9, 2021
    Configuration menu
    Copy the full SHA
    5c9b188 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2021

  1. Restore tracing fixes

    @tjruwase
    tjruwase committed Oct 10, 2021
    Configuration menu
    Copy the full SHA
    038993e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 10, 2021
    Configuration menu
    Copy the full SHA
    e7832d6 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 11, 2021
    Configuration menu
    Copy the full SHA
    13c0532 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 12, 2021
    Configuration menu
    Copy the full SHA
    f4cebc0 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 14, 2021
    Configuration menu
    Copy the full SHA
    98fa12f View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 19, 2021
    Configuration menu
    Copy the full SHA
    05acc48 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2021

  1. Merge branch 'master' into olruwase/zero3_broken_tracing

    @tjruwase
    tjruwase authored Oct 21, 2021
    Configuration menu
    Copy the full SHA
    30afb5b View commit details
    Browse the repository at this point in the history