Chat Engine returning Empty Response

[andreabrgit]

Hi,

I created a custom prompt and pass to a Chat Engine type Context. But, when I question about something that doesn't return nodes of VectorStore, the stack stops on a event of type "Retrieve" and return a reponse 'Empty Reponse'. If the question return nodes that doesn't have information about the question, the response is correct according what asked on prompt.

My custom prompt:
qa_prompt_str = ("Below is the information where you should look for an answer\n"
"If the answer to the question is not explicitly in the information provided, always respond this way: 'We were unable to find an answer to the question you asked. It may be that the information is not in the documents or it may be necessary to supplement the prompt of the question'\n"

The chat:
chat = index.as_chat_engine(
chat_mode="context",
context_template=qa_prompt_str,
chat_history=legacy_chat_history,
verbose=True,
similarity_top_k=3,
filters=filters,
node_postprocessors=[post_processor],
)

I'd like some help about it.

Thanks in advance.

[dosubot[bot]]

The issue you're experiencing with the Chat Engine returning an 'Empty Response' when no nodes are retrieved from the VectorStore is a known behavior. This happens because the context generation relies on retrieving nodes from the index. If no nodes are retrieved, the context used in the prompt is effectively empty, leading to an empty or non-informative response from the LLM ^[1].

To address this, you might consider implementing a fallback mechanism in your custom prompt or chat engine configuration. For example, you could modify the logic to handle cases where no nodes are retrieved by providing a default response or using previously retrieved nodes if available ^[2].

Additionally, you might find these similar discussions helpful:

1. Facing issues to get the correct response even after generating a proper context related to the user query and then passing it in system prompt, and then using the simple chat engine to get the answers ^[3].
2. How to use Custom Prompts on Chat Engine? ^[4].

These threads discuss similar issues and might offer insights or solutions applicable to your situation.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}