[CORE] Adding support for insertion of soft-tuned prompts

[SwapnilDreams100]

This PR adds support for inserting soft-tuned prompts into the input embeddings (trained using PEFT).

This functionality is required by the IBM team.

Summary of Changes:

• New prompt_adapter folder similar to the lora folder to create a LRU cache management system for multiple prompt adapters
• New prompt_adapter_config engine parameter - for easy extension to more sophisticated prompt-tuning techniques in the future
• New prompt_adapter_request parameter added to the generate functionality of the LLM_Engine
• Current support for several models like bloom, llama, mistral, easily extensible for others
• Simple test demonstrating that prompt adapters work for bloom
• Some parameter documentation still pending, will complete post successful code review

[Yard1]

Some high level comments - if this is not time sensitive, I think it would be good if we could come up with some more generic APIs to promote code reuse between LoRA and prompt adapter

[SwapnilDreams100]

Hey @Yard1,
I have addressed your comments on this soft prompt tuning PR. Some updates:

• New adapter_commons folder with all the common code between LoRA and Prompt Adapters abstracted
• Reduced redundancy of having to add the adapter addition code in all model classes by extending the VocalParallelEmbedding layer
• New parameter enable_prompt_adapter for more consistency w LoRA
• Added testing for multi-adapter inference

After your review, happy to add more extensive tests + docs.

[Yard1]

@SwapnilDreams100 thanks! let me take a look

[SwapnilDreams100]

Hi @Yard1 just a friendly reminder to review this PR when you get a chance, thanks!
Once this design is approved, happy to update this with support for prefix tuning as well, which should be similar in design!

[g-eoj]

Hi @SwapnilDreams100, I have an initial implementation of adapter support for the OpenAI entry points based on https://github.com/SwapnilDreams100/vllm/tree/main. Would you be open to me contributing to your PR?

[Yard1]

@SwapnilDreams100 Actually I just realized this is only applied during prefill. In this case we do not need to do anything special for CUDA graph support. The current code looks fine and it should be decently performant. We should just add a comment to say that this will not work with CUDA graphs.

[Yard1]

Approving pending CI passing

[g-eoj]

@SwapnilDreams100 first of all congrats!

I'd like to add tests for the openai server, would it be best to wait for this to merge so I can open my own PR against main?

[SwapnilDreams100]

Sounds good @g-eoj, big thank you for your help!

[SwapnilDreams100]

Hey @Yard1 are we good to merge?

[njhill]

Thanks for this epic effort @SwapnilDreams100!! And big thanks to @Yard1 for the many detailed reviews.

I'll merge it before any new conflicts pop up!

[SwapnilDreams100]

Big thank you to @Yard1 for your guidance on this, this was a great learning experience!