Skip to content

v0.0.19: AWS Neuron SDK 2.17.0, training cache system, TGI improved batching

Compare
Choose a tag to compare
@JingyaHuang JingyaHuang released this 19 Feb 15:48
· 149 commits to main since this release

What's Changed

Training

TGI

  • Support higher batch sizes using transformers-neuronx continuous batching by @dacorvo in #488
  • Lift max-concurrent-request limitation usingTGI 1.4.1 by @dacorvo in #488

AMI

Major bugfixes

Other changes

New Contributors

Full Changelog: v0.0.18...v0.0.19