Relax dependencies, add Streaming Callback

[dnoliver]

Hi, so I am trying to make a RAG app with fastRAG.
But the dependency definition of fastRAG is causing conflicts with other dependencies.
For example, did pip install with this requirement.txt fails:

haystack-ai==2.1.2
fastrag[openvino]==3.0.2
fastembed-haystack==1.3.0
qdrant-haystack==3.8.1
transformers[torch]==4.46.2
pypdf==5.1.0
fastapi==0.110.1
uvicorn[standard]==0.25.0

It's usually about Pillow and protobuf.
So, forked the lib, relaxed the dependencies, rebuilt it, and installed it in my project.
It is still working, but I only tried the OpenVINOGenerator feature.

So, proposing this change to allow better compatibility with other libs!

[danielfleischer]

Please see if there's still an issue, after updating protobuf.

[dnoliver]

@danielfleischer Solved the Protobuf issue, but issues with Pillow are still there:

CUSTOMBUILD : error : Cannot install fastembed-haystack, fastrag, fastrag[openvino]==3.1.0, transformers and transformers[torch]==4.46.2 because these package versions have conflicting dependencies. 

  The conflict is caused by:
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.4.1 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.4.0 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.6 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.5 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.4 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.3 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.2 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.1 depends on pillow<11.0.0 and >=10.3.0
      fastrag 3.1.0 depends on Pillow==10.1.0
      sentence-transformers 2.3.0 depends on Pillow
      fastrag[openvino] 3.1.0 depends on Pillow==10.1.0
      fastembed 0.3.0 depends on pillow<11.0.0 and >=10.3.0
      transformers 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      transformers[torch] 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      sentence-transformers 2.3.0 depends on huggingface-hub>=0.15.1
      accelerate 0.26.0 depends on huggingface-hub
      fastembed 0.2.7 depends on huggingface-hub<0.21 and >=0.20
      transformers 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      transformers[torch] 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      sentence-transformers 2.3.0 depends on huggingface-hub>=0.15.1
      accelerate 0.26.0 depends on huggingface-hub
      fastembed 0.2.6 depends on huggingface-hub<0.21 and >=0.20
      transformers 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      transformers[torch] 4.46.2 depends on huggingface-hub<1.0 and >=0.23.2
      sentence-transformers 2.3.0 depends on huggingface-hub>=0.15.1
      accelerate 0.26.0 depends on huggingface-hub
      fastembed 0.2.5 depends on huggingface-hub<0.21 and >=0.20

  To fix this you could try to:
  1. loosen the range of package versions you've specified
  2. remove package versions to allow pip to attempt to solve the dependency conflict

CUSTOMBUILD : error : ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/depen
dency-resolution/#dealing-with-dependency-conflicts

[dnoliver]

@danielfleischer last commit added something else I need: passing the streaming callback to the parent.
Parent's constructor definition is at https://github.com/deepset-ai/haystack/blob/v2.1.2/haystack/components/generators/hugging_face_local.py#L61

With that commit, this works:

# Configure Streaming Callback
def streaming_callback(response):
    logger.info(f"Streaming response: {response}")

# Create OpenVINO generator
logger.info("Creating Generator")
generator = OpenVINOGenerator(
    model="microsoft/Phi-3.5-mini-instruct",
    compressed_model_dir="lokinfey/Phi-3.5-mini-instruct-ov-int4",
    device_openvino="GPU",
    task="text-generation",
    generation_kwargs={
        "num_beams": 1,
        "do_sample": True,
        "max_new_tokens": 256,
        "temperature": 0.7,
        "top_k": 50,
        "top_p": 0.95,
    },
    streaming_callback=streaming_callback,
)

And this is the output to a "Hello" prompt:

INFO:     Streaming response: StreamingChunk(content='', meta={})
INFO:     Streaming response: StreamingChunk(content='', meta={})
INFO:     Streaming response: StreamingChunk(content='Hello! ', meta={})
INFO:     Streaming response: StreamingChunk(content='How ', meta={})
INFO:     Streaming response: StreamingChunk(content='can ', meta={})
INFO:     Streaming response: StreamingChunk(content='I ', meta={})
INFO:     Streaming response: StreamingChunk(content='assist ', meta={})
INFO:     Streaming response: StreamingChunk(content='you ', meta={})
INFO:     Streaming response: StreamingChunk(content='', meta={})
INFO:     Streaming response: StreamingChunk(content='', meta={})
INFO:     Streaming response: StreamingChunk(content='today?<|end|>\n', meta={}

The intention here is to get "avg token generation per second" metric and similar
Sorry for re-using the PR for a secondary purpose, but it is something else I need for the app :)

[danielfleischer]

Looks good, waiting on feedback.

[danielfleischer]

Thanks, merging.