Code

Cache optimizations#418

Code

Merged

hnyls2002 merged 9 commits intomainsgl-project/sglang:mainfrom fix-cachesgl-project/sglang:fix-cacheCopy head branch name to clipboardMay 13, 2024

+117-96

Commits

Commits on May 10, 2024

• rename "ref_counter" -> "lock_ref"

  hnyls2002

  committed
  da9f0a3

• merge backend config into global config

  hnyls2002

  committed
  acca326

• api change and reduce add_ref overhead

  hnyls2002

  committed
  bf910db

Commits on May 11, 2024

• cache filled batch

  hnyls2002

  committed
  8ebe40c

• fix retract decode

  hnyls2002

  committed
  6ff5ffc

Commits on May 12, 2024

• rename heuristic

  hnyls2002

  committed
  f51824d

• Merge branch 'main' into fix-cache

  hnyls2002

  committed
  f2a8957

Commits on May 13, 2024

• Merge branch 'main' of github.com:sgl-project/sglang into fix-cache

  hnyls2002

  committed
  981727d

• fix

  hnyls2002

  committed
  189a805