feat: add pixtral support

pixtral-12b/.bentoignore

@@ -0,0 +1,5 @@
+__pycache__/
+*.py[cod]
+*$py.class
+.ipynb_checkpoints
+venv/

pixtral-12b/README.md

@@ -0,0 +1,76 @@
+<div align="center">
+    <h1 align="center">Self-host Pixtral 12B with vLLM and BentoML</h1>
+</div>
+
+This is a BentoML example project, showing you how to serve and deploy Pixtral 12B using [vLLM](https://vllm.ai), a high-throughput and memory-efficient inference engine.
+
+See [here](https://github.com/bentoml/BentoML?tab=readme-ov-file#%EF%B8%8F-what-you-can-build-with-bentoml) for a full list of BentoML example projects.
+
+💡 This example is served as a basis for advanced code customization, such as custom model, inference logic or vLLM options. For simple LLM hosting with OpenAI compatible endpoint without writing any code, see [OpenLLM](https://github.com/bentoml/OpenLLM).
+
+
+## Prerequisites
+
+If you want to test the Service locally, we recommend you use a Nvidia GPU with at least 16G VRAM.
+
+## Install dependencies
+
+```bash
+git clone https://github.com/bentoml/BentoVLLM.git
+cd BentoVLLM/pixtral-12b
+
+# Recommend Python 3.11
+pip install -r requirements.txt
+```
+
+## Run the BentoML Service
+
+We have defined a BentoML Service in `service.py`. Run `bentoml serve` in your project directory to start the Service.
+
+```python
+$ bentoml serve .
+
+2024-01-18T07:51:30+0800 [INFO] [cli] Starting production HTTP BentoServer from "service:VLLM" listening on http://localhost:3000 (Press CTRL+C to quit)
+INFO 01-18 07:51:40 model_runner.py:501] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
+INFO 01-18 07:51:40 model_runner.py:505] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode.
+INFO 01-18 07:51:46 model_runner.py:547] Graph capturing finished in 6 secs.
+```
+
+The server is now active at [http://localhost:3000](http://localhost:3000/). You can interact with it using the Swagger UI or in other different ways.
+
+<details>
+
+<summary>CURL</summary>
+
+```bash
+curl -X 'POST' \
+  'http://localhost:3000/generate' \
+  -H 'accept: text/event-stream' \
+  -H 'Content-Type: multipart/form-data' \
+  -F 'prompt=Describe this image in detail words.' \
+  -F 'system_prompt=
+You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
+
+If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don'\''t know the answer to a question, please don'\''t share false information.
+' \
+  -F 'image=@demo.jpg' \
+  -F 'max_tokens=1024'
+```
+
+</details>
+
+For detailed explanations of the Service code, see [vLLM inference](https://docs.bentoml.org/en/latest/use-cases/large-language-models/vllm.html).
+
+## Deploy to BentoCloud
+
+After the Service is ready, you can deploy the application to BentoCloud for better management and scalability. [Sign up](https://www.bentoml.com/) if you haven't got a BentoCloud account.
+
+Make sure you have [logged in to BentoCloud](https://docs.bentoml.com/en/latest/bentocloud/how-tos/manage-access-token.html), then run the following command to deploy it.
+
+```bash
+bentoml deploy .
+```
+
+Once the application is up and running on BentoCloud, you can access it via the exposed URL.
+
+**Note**: For custom deployment in your own infrastructure, use [BentoML to generate an OCI-compliant image](https://docs.bentoml.com/en/latest/guides/containerization.html).

pixtral-12b/bentofile.yaml

@@ -0,0 +1,12 @@
+service: 'service:VLLM'
+labels:
+  owner: bentoml-team
+  stage: demo
+include:
+  - '*.py'
+  - 'bentovllm_openai/*.py'
+python:
+  requirements_txt: './requirements.txt'
+  lock_packages: false
+docker:
+  python_version: 3.11