Tensor parallelism for GPT fast example

[mreso]

Description

This PR adds tensor parallelism and speculative decoding to the GPT-fast example

Fixes #(issue)

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

• pytest test/pytest/test_example_gpt_fast.py -k test_handler[false]

============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.10.13, pytest-7.3.1, pluggy-1.3.0
rootdir: /home/ubuntu/serve
plugins: mock-3.12.0, cov-4.1.0
collected 4 items / 3 deselected / 1 selected

test/pytest/test_example_gpt_fast.py .                                                                                                                                                                                                 [100%]

============================================================================================================== warnings summary ==============================================================================================================
test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
    _torch_pytree._register_pytree_node(

test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/ts/torch_handler/base_handler.py:13: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import packaging

test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/pkg_resources/__init__.py:2868: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/pkg_resources/__init__.py:2868: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/pkg_resources/__init__.py:2348: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(parent)

test/pytest/test_example_gpt_fast.py::test_handler[false]
test/pytest/test_example_gpt_fast.py::test_handler[false]
  /home/ubuntu/miniconda3/envs/gpt-fast/lib/python3.10/site-packages/pkg_resources/__init__.py:2868: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('zope')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================================ 1 passed, 3 deselected, 7 warnings in 11.06s ================================================================================================

• pytest test/pytest/test_example_gpt_fast.py -k test_gpt_fast_mar[mar_file_path0]

============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.10.13, pytest-7.3.1, pluggy-1.3.0
rootdir: /home/ubuntu/serve
plugins: mock-3.12.0, cov-4.1.0
collected 4 items / 3 deselected / 1 selected

test/pytest/test_example_gpt_fast.py .                                                                                                                                                                                                 [100%]

====================================================================================================== 1 passed, 3 deselected in 29.09s ======================================================================================================

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[agunapal]

LGTM. Minor comments.