Improve llama models performance

[dacorvo]

What does this PR do?

This modifies the default Neuron configuration when exporting Llama models for inference, setting the attention layout to "BSH" instead of "HSB".

This configuration has almost no impact on the token generation time (a.k.a decode), and significantly reduces the context encoding time (a.k.a prefill) for Llama2-7b and Llama3-8B.

Benchmarks updates:

• Llama 2 benchmarks now only use 8 cores instead of 24. This has a significant impact on Llama 2 13B results.
• new Llama 3 benchmarks,
• updated Mistral benchmarks since they had not been rerun with the switch to pytorch 2.1.

In the process, the TGI router version is bumped to 2.0.2.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[dacorvo]

The TGI tests are failing because I need to remove the test llama neuron model under the optimum org. But if I do that before the pr is merged, it will break the CI for all other pull-requests.

[michaelbenayoun]

LGTM
That looks awesome @dacorvo !