* Update index.md

Update to fix a broken link in index.md where the trailing .md is cut off from the management_api.md. Added an anchor link to force the .md to show up.

* Update to index.md

Update to index.md to fix several links ending in .md that sphinx is breaking. Added anchor links to each link and a corresponding anchor in the affected doc. Tested locally and seems to be working.

* Update inference_api.md

* Updated typos

Fixed typos and updated wordslist.txt

* Update wordlist.txt

---------

Co-authored-by: sekyonda <7411+sekyonda@users.noreply.ghe.oculus-rep.com>
Co-authored-by: lxning <23464292+lxning@users.noreply.github.com>

* Inf2 nightly benchmark

* fix linter spellcheck error

---------

Co-authored-by: Naman Nandan <namannan@amazon.com>

* fix: kserve fastapi migration issues

* Update index.md

Update to fix a broken link in index.md where the trailing .md is cut off from the management_api.md. Added an anchor link to force the .md to show up.

* Update to index.md

Update to index.md to fix several links ending in .md that sphinx is breaking. Added anchor links to each link and a corresponding anchor in the affected doc. Tested locally and seems to be working.

* Update inference_api.md

* Updated typos

Fixed typos and updated wordslist.txt

* Update wordlist.txt

* FAQs Updates

Updated a couple of broken links on the FAQ site per issue pytorch#2204

* updates to resolve links

* Update some links in index

Updated some links in index.md to go to the pytorch html page instead of github. This is a nicer fix for the .md sphinx issue

---------

Co-authored-by: sekyonda <7411+sekyonda@users.noreply.ghe.oculus-rep.com>
Co-authored-by: lxning <23464292+lxning@users.noreply.github.com>
Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>

* Add unit test for micro batching

* Adds simple implementation for microbatching with coroutines

* Added testing to microbatching

* Created MBHandler

* Added more tests to microbatching handler

* Adds configurable parallelism

* Enables loading of micro batching parameters through config file

* Moved microbatching into example

* Moved micro batching test into test/pytest folder

* Rewrote micro batching to use threading

* Implemented method to update parallelism

* Fix and test spin up spin down fo threads

* Clean up and comments

* More comments

* Adds README to micro batching example

* Refined readme + added config.yaml

* Add config_file

* Fix linting error

* Fix spell check error

* Fix linting error

* Move micro_batching.py into ts.utils and use model_yaml_config for configuration

* Fix links in README

* Moved to handler_utils

* remove __all__

Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>

pytorch#2371)

Co-authored-by: Naman Nandan <namannan@amazon.com>

* Enable ONNX test in CI

* update cpuinfo

* update

* lint

* lint

* lint

* install numactl

---------

Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>
Co-authored-by: Mark Saroufim <marksaroufim@fb.com>

* fix tcp port in model_service_worker.py

* update torchrun parameter name

* fmt

* update cpuinfo

* update

* lint

* lint

* lint

* install numactl

* update num workers

* skip if gpu

---------

Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>
Co-authored-by: Mark Saroufim <marksaroufim@fb.com>

…h#2374)

torch == 2.0.1
torchvision == 0.15.2
torchtext == 0.15.2
torchaudio == 2.0.2

Co-authored-by: Naman Nandan <namannan@amazon.com>
Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>

… server to stop

* adding type annotations to functions

* reflect the model config from properties

* honor model config in config.properties

Bumps [transformers](https://github.com/huggingface/transformers) from 4.28.1 to 4.30.0.
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](huggingface/transformers@v4.28.1...v4.30.0)

---
updated-dependencies:
- dependency-name: transformers
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ytorch#2350)

* add jobQsize

* add jobQueueSize in model_config.yaml for mnist_scripted.mar

* feature job tickets

* rename hasJobTickets to getJobTickets

* fmt doc

* update doc

* fmt doc

* fmt doc

* update doc

…:pytorch/serve into torchserve-23mt-v0.8.0-fix-1