Skip to content

TorchServe v0.6.0 Release Notes
Compare
Choose a tag to compare
@lxning lxning released this 16 May 20:02
· 713 commits to master since this release
v0.6.0
4236a86

This is the release of TorchServe v0.6.0.

New Features

  • Support PyTorch 1.11 and Cuda 11.3 - Added support for PyTorch 1.11 and Cuda 11.3.
  • Universal Auto Benchmark and Dashboard Tool - Added one command line tool for model analyzer to get benchmark report(sample) and dashboard on any device.
  • HuggingFace model parallelism integration - Added example for HuggingFace model parallelism integration.

Build and CI

  • Added nightly benchmark dashboard - Added nightly benchmark dashboard.
  • Migrated CI, nightly binary and docker build to github workflow - Added CI, docker migration.
  • Fixed gpu regression test buildspec.yaml - Added fixing for gpu regression test buildspec.yaml.

Documentation

Deprecations

  • Deprecated old benchmark/automated directory in favor of new Github Action based workflow

Improvements

  • Fixed workflow threads cleanup - Added fixing to clean workflow inference threadpool.
  • Fixed empty model url - Added fixing for empty model url in model archiver.
  • Fixed load model failure - Added support for loading a model directory.
  • HuggingFace text generation example - Added text generation example.
  • Updated metrics json and qlog format log - Added support for metrics json and qlog format log in log4j2.
  • Added cpu, gpu and memory usage - Added cpu, gpu and memory usage in benchmark-ab.py report.
  • Added exception for torch < 1.8.1 - Added exception to notify torch < 1.8.1.
  • Replaced hard code in install_dependencies.py - Added sys.executable in install_dependencies.py.
  • Added default envelope for workflow - Added default envelope in model manager for workflow.
  • Fixed multiple docker build errors - Fixed /home/venv write permission, typo in docker and added common requirements in docker.
  • Fixed snapshot test - Added fixing for snapshot test.
  • Updated model_zoo.md - Added dog breed, mmf and BERT in model zoo.
  • Added nvgpu in common requirements - Added nvgpu in common dependencies.
  • Fixed Inference API ping response - Fixed typo in Inference API ping response.

Platform Support

Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows 10 Pro, Windows Server 2019, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4). TorchServe now requires Python 3.8 and above.

GPU Support

Torch 1.11+ Cuda 10.2, 11.3
Torch 1.9.0 + Cuda 11.1
Torch 1.8.1 + Cuda 9.2

Assets 11
Loading