Code

Megablocks-based MoE#1197

Code

Closed

DayOfThePenguin wants to merge 7 commits intoEleutherAI:mainEleutherAI/gpt-neox:mainfrom DayOfThePenguin:megablocks_prDayOfThePenguin/gpt-neox-public:megablocks_prCopy head branch name to clipboard

+1,526-147

Commits

Commits on Mar 27, 2024

• feat: megablocks grouped gemm-based implementation

  DayOfThePenguin

  committed
  923afa6

  Copy full SHA for 923afa6

• fix: run pre-commit

  DayOfThePenguin

  committed
  01c9577

  Copy full SHA for 01c9577

Commits on Mar 28, 2024

• Update moe.py

  StellaAthena

  authored
  c2cc9c4

  Copy full SHA for c2cc9c4

• Update moe_mlp.py

  StellaAthena

  authored
  c7bfed0

  Copy full SHA for c7bfed0

• Update moe.py

  StellaAthena

  authored
  bde88e8

  Copy full SHA for bde88e8

• Update router.py

  StellaAthena

  authored
  89aa3f0

  Copy full SHA for 89aa3f0

Commits on May 4, 2024

• fix: router inference time topk -> top_k

  DayOfThePenguin

  authored
  6697b0c

  Copy full SHA for 6697b0c