pip install outlines-haystackOutlines is a Python library that allows you to use Large Language Model in a simple and robust way (with structured generation). It is built by .txt.
This library allow you to use outlines generators in your Haystack pipelines!
This library currently supports the following generators:
- JSON: generate a JSON object with a given schema
- Text: simply generate text
- Choices:
⚠️ coming soon - Regex:
⚠️ coming soon - Format:
⚠️ coming soon - Grammar:
⚠️ coming soon
outlines supports a wide range of models and frameworks, we are currently supporting:
Tip
See the Example Notebooks for complete examples.
All below examples only use the transformers models.
>>> from enum import Enum
>>> from pydantic import BaseModel
>>> from outlines_haystack.generators.transformers import TransformersJSONGenerator
>>> class User(BaseModel):
... name: str
... last_name: str
>>> generator = TransformersJSONGenerator(
... model_name="microsoft/Phi-3-mini-4k-instruct",
... schema_object=User,
... device="cuda",
... sampling_algorithm_kwargs={"temperature": 0.5},
... )
>>> generator.warm_up()
>>> generator.run(prompt="Create a user profile with the fields name, last_name")
{'structured_replies': [{'name': 'John', 'last_name': 'Doe'}]}>>> generator = TransformersTextGenerator(
... model_name="microsoft/Phi-3-mini-4k-instruct",
... device="cuda",
... sampling_algorithm_kwargs={"temperature": 0.5},
... )
>>> generator.warm_up()
>>> generator.run(prompt="What is the capital of Italy?")
{'replies': ['\n\n# Answer\nThe capital of Italy is Rome.']}outlines-haystack is distributed under the terms of the MIT license.