Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DRAFT] [feat] Experimental CPU offloading #279

Closed
wants to merge 18 commits into from
Closed

[DRAFT] [feat] Experimental CPU offloading #279

wants to merge 18 commits into from

Commits on Dec 30, 2020

  1. clean start

    @blefaudeux
    blefaudeux committed Dec 30, 2020
    Configuration menu
    Copy the full SHA
    f166609 View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2021

  1. Merge remote-tracking branch 'upstream/master' into offload_experimental

    @blefaudeux
    blefaudeux committed Jan 4, 2021
    Configuration menu
    Copy the full SHA
    fc1310a View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2021

  1. removing per layer split strategy, probably not that useful indeed

    @blefaudeux
    blefaudeux committed Jan 5, 2021
    Configuration menu
    Copy the full SHA
    26cfd92 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into offload_experimental

    @blefaudeux
    blefaudeux committed Jan 5, 2021
    Configuration menu
    Copy the full SHA
    0363630 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2021

  1. initial transformer benchmark

    @blefaudeux
    blefaudeux committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    ff44ddd View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2021

  1. Merge remote-tracking branch 'upstream/master' into offload_experimental

    @blefaudeux
    blefaudeux committed Jan 11, 2021
    Configuration menu
    Copy the full SHA
    f20e3f8 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2021

  1. hack, enable testing ViT + offload, python3 benchmarks/oss.py --epoch… 

    …s 2 --optim_type oss_offload_ddp --batch_size=32 --model vit_large_patch16_224
    @blefaudeux
    blefaudeux committed Jan 12, 2021
    Configuration menu
    Copy the full SHA
    3bcea0a View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2021

  1. proper cuda streams and device, something off in terms of mems consum… 

    …ption
    @blefaudeux
    blefaudeux committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    62c15e4 View commit details
    Browse the repository at this point in the history

  2. minor, stashing

    @blefaudeux
    blefaudeux committed Jan 13, 2021
    Configuration menu
    Copy the full SHA
    43c56cd View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2021

  1. Merge branch 'master' into offload_experimental

    @blefaudeux
    blefaudeux committed Jan 22, 2021
    Configuration menu
    Copy the full SHA
    042daa3 View commit details
    Browse the repository at this point in the history

  2. unit test fix

    @blefaudeux
    blefaudeux committed Jan 22, 2021
    Configuration menu
    Copy the full SHA
    850c5bf View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2021

  1. Merge branch 'master' into offload_experimental

    @blefaudeux
    blefaudeux committed Feb 1, 2021
    Configuration menu
    Copy the full SHA
    52e0be4 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2021

  1. removing all the distributed parts

    @blefaudeux
    blefaudeux committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    1f6c018 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2021

  1. simpler test, needs debugging

    @blefaudeux
    blefaudeux committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    8490dd8 View commit details
    Browse the repository at this point in the history

  2. working OOP, running a model which does not fit on the gpu memory

    @blefaudeux
    blefaudeux committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    9ec3892 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into offload_experimental

    @blefaudeux
    blefaudeux committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    d4e929d View commit details
    Browse the repository at this point in the history

  4. spring cleaning

    @blefaudeux
    blefaudeux committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    8e92a4c View commit details
    Browse the repository at this point in the history

  5. removing the ill-advised optimizer bits, better keep that orthogonal

    @blefaudeux
    blefaudeux committed Feb 5, 2021
    Configuration menu
    Copy the full SHA
    6bfeaed View commit details
    Browse the repository at this point in the history