Code

adding-support-for-mamba2#1009

Code

Open

Goekdeniz-Guelmez wants to merge 60 commits intoml-explore:mainml-explore/mlx-examples:mainfrom Goekdeniz-Guelmez:adding-support-for-mamba2Goekdeniz-Guelmez/mlx-examples:adding-support-for-mamba2Copy head branch name to clipboard

+591-2

Commits

Commits on Oct 2, 2024

• Create mamba2.py

  Goekdeniz-Guelmez

  authored
  49b9fc1

  Copy full SHA for 49b9fc1

• updating ACKNOWLEDGMENTS.md file

  Goekdeniz-Guelmez

  committed
  409ddc4

  Copy full SHA for 409ddc4

• update trainer/lora.py and adding DepthWiseConv1d because mlx 0.18.0 doesnt axepts groups parameter

  Goekdeniz-Guelmez

  committed
  264ba43

  Copy full SHA for 264ba43

Commits on Oct 4, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  52d6ca0

  Copy full SHA for 52d6ca0

Commits on Oct 11, 2024

• fixing loading the model

  Goekdeniz-Guelmez

  committed
  4e1236c

  Copy full SHA for 4e1236c

• Merge branch 'adding-support-for-mamba2' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-support-for-mamba2

  Goekdeniz-Guelmez

  committed
  9c075a7

  Copy full SHA for 9c075a7

• quick clean up and fix

  Goekdeniz-Guelmez

  committed
  6f88dd5

  Copy full SHA for 6f88dd5

• adding debug statements

  Goekdeniz-Guelmez

  committed
  00ba27f

  Copy full SHA for 00ba27f

Commits on Oct 14, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  3f1c1dd

  Copy full SHA for 3f1c1dd

Commits on Oct 16, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  855fcc4

  Copy full SHA for 855fcc4

• adding debug statements (somehiw generating only goes through the fist MambaMixer block pass)

  Goekdeniz-Guelmez

  committed
  8073cb4

  Copy full SHA for 8073cb4

• Merge branch 'adding-support-for-mamba2' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-support-for-mamba2

  Goekdeniz-Guelmez

  committed
  181d6ab

  Copy full SHA for 181d6ab

• fix generation works too (almost)

  Goekdeniz-Guelmez

  committed
  cd036cc

  Copy full SHA for cd036cc

Commits on Oct 20, 2024

• quick save

  Goekdeniz-Guelmez

  committed
  4ab5139

  Copy full SHA for 4ab5139

• generation works but outputs gibberish

  Goekdeniz-Guelmez

  committed
  ab4cf1d

  Copy full SHA for ab4cf1d

• still generating gibberish

  Goekdeniz-Guelmez

  committed
  c1634ce

  Copy full SHA for c1634ce

Commits on Oct 21, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  0ef73f3

  Copy full SHA for 0ef73f3

Commits on Oct 22, 2024

• generation works! trying training now

  Goekdeniz-Guelmez

  committed
  b9c57cd

  Copy full SHA for b9c57cd

• Merge branch 'adding-support-for-mamba2' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-support-for-mamba2

  Goekdeniz-Guelmez

  committed
  5326d93

  Copy full SHA for 5326d93

• adding multi token input and correct cache handling in ssm step

  Goekdeniz-Guelmez

  committed
  758597e

  Copy full SHA for 758597e

• update

  Goekdeniz-Guelmez

  committed
  55485b9

  Copy full SHA for 55485b9

• not working, incorrect handling with cache probably

  Goekdeniz-Guelmez

  committed
  e43a2ab

  Copy full SHA for e43a2ab

• notes

  Goekdeniz-Guelmez

  committed
  9ab581d

  Copy full SHA for 9ab581d

• inference works but is hella slow

  Goekdeniz-Guelmez

  committed
  a677638

  Copy full SHA for a677638

Commits on Oct 24, 2024

• update

  Goekdeniz-Guelmez

  committed
  7c8849e

  Copy full SHA for 7c8849e

Commits on Oct 25, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  3b70708

  Copy full SHA for 3b70708

Commits on Oct 30, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  ffc7ab0

  Copy full SHA for ffc7ab0

• updates

  Goekdeniz-Guelmez

  committed
  58b448d

  Copy full SHA for 58b448d

Commits on Nov 6, 2024

• save push

  Goekdeniz-Guelmez

  committed
  906f972

  Copy full SHA for 906f972

Commits on Nov 10, 2024

• save checkpoint

  Goekdeniz-Guelmez

  committed
  800b602

  Copy full SHA for 800b602

• fixed inference slowness but it cant handle multible Token inputs and is generateing gibberish

  Goekdeniz-Guelmez

  committed
  3a499f9

  Copy full SHA for 3a499f9

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  49d3f18

  Copy full SHA for 49d3f18

• removed the custom Mamba2Cache adn updated the existing MambaCache but still only one input Token and outputs gibberish

  Goekdeniz-Guelmez

  committed
  2f95b36

  Copy full SHA for 2f95b36

• imopemented multi Token inputs, but still generating Gibberish

  Goekdeniz-Guelmez

  committed
  1a66883

  Copy full SHA for 1a66883

• nits

  Goekdeniz-Guelmez

  committed
  1d85106

  Copy full SHA for 1d85106

Commits on Nov 21, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  e4eae97

  Copy full SHA for e4eae97

• Fixed streaming generation and got rid of generating gibberish, but is still a litle slow: 0.222 tokens-per-sec

  Goekdeniz-Guelmez

  committed
  e22b2db

  Copy full SHA for e22b2db

• removing some files

  Goekdeniz-Guelmez

  committed
  117ffd3

  Copy full SHA for 117ffd3

• inference fixed

  Goekdeniz-Guelmez

  committed
  57b1717

  Copy full SHA for 57b1717

• removing last checkpoint file

  Goekdeniz-Guelmez

  committed
  a6ddc27

  Copy full SHA for a6ddc27

Commits on Nov 24, 2024

• loading codestral works but no tinference

  Goekdeniz-Guelmez

  committed
  38e5801

  Copy full SHA for 38e5801

Commits on Dec 10, 2024

• Merge branch 'main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  ddad210

  Copy full SHA for ddad210

• inference on codestral works but is giberish

  Goekdeniz-Guelmez

  committed
  9f8a6a3

  Copy full SHA for 9f8a6a3

• nits

  Goekdeniz-Guelmez

  committed
  b10afe3

  Copy full SHA for b10afe3

• nits

  Goekdeniz-Guelmez

  committed
  80e88b4

  Copy full SHA for 80e88b4

• clean up

  Goekdeniz-Guelmez

  committed
  184d3d3

  Copy full SHA for 184d3d3

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  c1d9ec3

  Copy full SHA for c1d9ec3

Commits on Dec 12, 2024

• optimizing the code for faster inference but still generates giberish

  Goekdeniz-Guelmez

  committed
  a883e39

  Copy full SHA for a883e39

• adding the modelnames in the LORA.md file and removing unused functions from mamba2.py

  Goekdeniz-Guelmez

  committed
  dff4e52

  Copy full SHA for dff4e52

Commits on Dec 17, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  68533e2

  Copy full SHA for 68533e2

Commits on Dec 18, 2024

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  7996a6f

  Copy full SHA for 7996a6f

• update: using einsum on som elines making it faster, but still generates Gibberish on Codestral

  Goekdeniz-Guelmez

  committed
  0ae536c

  Copy full SHA for 0ae536c

Commits on Dec 27, 2024

• update

  Goekdeniz-Guelmez

  committed
  d044db9

  Copy full SHA for d044db9

• new set but still gibberish

  Goekdeniz-Guelmez

  committed
  f4cbe27

  Copy full SHA for f4cbe27

• still gibberish

  Goekdeniz-Guelmez

  committed
  2ed5194

  Copy full SHA for 2ed5194

• nits

  Goekdeniz-Guelmez

  committed
  3384d38

  Copy full SHA for 3384d38

• nits

  Goekdeniz-Guelmez

  committed
  4e94e87

  Copy full SHA for 4e94e87

• optimizations

  Goekdeniz-Guelmez

  committed
  8deada9

  Copy full SHA for 8deada9

Commits on Jan 13, 2025

• Merge branch 'ml-explore:main' into adding-support-for-mamba2

  Goekdeniz-Guelmez

  authored
  5509ef8

  Copy full SHA for 5509ef8

• adding correct initialisation of dt, A and D

  Goekdeniz-Guelmez

  committed
  dd4957f

  Copy full SHA for dd4957f