Expand LLM support

[masci]

Haystack can already work with LLM (see GPT-3 usage on answer generation) but it falls short in showing how the feature can actually help users. We can collect LLM related features under two large umbrellas:

• External usage of LLM
  • Haystack is used to communicate with a 3rd party service managing the LLM (e.g. OpenAI)
  • feat: Add CohereEmbeddingEncoder to EmbeddingRetriever #3453
  • feat: Update cohere embedding models #3704
  • feat: Add OpenAIEmbeddingEncoder to EmbeddingRetriever #3356
  • feat: Expand LLM support with PromptModel, PromptNode, and PromptTemplate #3667
  • feat: Enable text-embedding-ada-002 for EmbeddingRetriever #3721
• Internal usage of LLM
  • Haystack is capable of managing a local LLM (e.g. training, evaluation)

Ideally Haystack should not have a preferred way to work with LLM but In Q4 we'll focus on making progress on the first group.

Desired outcome

• Use GPT-3 embeddings support to provide document retrieval
• Support co:here in one use case within Haystack
• Write a tutorial showing how to use Haystack to compare different technologies (e.g. GPT-3 vs. classic transformers)
• Draft a design to generically support an "inference as a service" concept within Haystack

[masci]

Closing as complete