Error when Orchestrating Workflows Involving files and text chatting.

[Copilotes]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

An old bug, said to have been fixed in version 0.10.0, but it seems has not been fixed yet.
refer: #8838 #8824
The general operation is as follows:
Firstly, upload an image file and use the vl model to respond based on the type of sys. files;
Then we chat casually, and the workflow will use text model nodes to answer, and then an error will be reported. Different model services report different errors, such as: Run failed: [xinference] Error: An error occurred during streaming. Or run failed: [siliconflow] Error: API request failed with status code 400: {"code": 20041, "message": "The model is not a VLM (Vision Language Model). Please use text only prompts.", "data": null}

The llm node must use enabling memory, or the error will not occur.

✔️ Expected Behavior

Chat normally without any errors

❌ Actual Behavior

Run failed: [siliconflow] Error: API request failed with status code 400: {"code": 20041, "message": "The model is not a VLM (Vision Language Model). Please use text only prompts.", "data": null}
Run failed: [xinference] Error: An error occurred during streaming

[dosubot]

I found some relevant information that might help with your issue. In Dify version 0.10.1, the new file upload feature supports a wide range of file types, including documents like TXT, MARKDOWN, PDF, HTML, XLSX, XLS, DOCX, CSV, EML, MSG, PPTX, PPT, XML, and EPUB ^[1]. If you're encountering an error stating that the file type is not supported, it might be due to a configuration issue or a bug.

Additionally, enabling memory in the LLM node includes chat history in each input to the intent classifier, which helps with context understanding but might be interacting with file processing in unexpected ways ^[2].

You might want to double-check your configuration settings to ensure that all necessary components are correctly set up. If the issue persists, it could be related to a bug that hasn't been fully resolved. You can also refer to similar discussions or issues, such as the closed issue File chat bug？, which suggested ensuring the document is in a supported format and checking configuration settings ^[3].

To continue talking to Dosu, mention @dosu.

[Copilotes]

@dosu Regardless of the file type, any file can lead to the same result. I suspect that there is a conflict between the processing logic of the memory function and file upload. The closed issue is also useless.

[dosubot]

The issue you're experiencing might indeed be related to a conflict between the memory function and file upload processing. Unfortunately, I couldn't find specific details on how the memory function might conflict with file uploads in Dify. However, it's possible that the memory function's handling of chat history and context could be affecting the file processing logic.

To troubleshoot, you might want to:

1. Review Configuration Settings: Ensure that the memory function and file upload settings are correctly configured. Misconfigurations could lead to unexpected behavior.

2. Check for Updates: Verify if there are any updates or patches available for Dify that address this issue.

3. Debugging: If possible, add logging around the file upload and memory function processes to identify where the conflict might be occurring.

If these steps don't resolve the issue, it might be necessary to delve deeper into the codebase or wait for a fix in a future update. This seems like a complex issue that needs your expertise, @takatost, could you take a closer look?

To continue talking to Dosu, mention @dosu.

[liuquanqing]

Same issue - seems like a frontend bug. No matter what files I select, I can only upload image type files.

[nguyenphan]

@liuquanqing I don;t think it is just frontend, do you use API to invoke the app/workflow? Using "remote_url" in the API also caused issue when sending to the LLM, you can see #9776

[nguyenphan]

@Copilotes @liuquanqing check out the my issue #9776 hopefully you can find your answer there.

[laipz8200]

duplicated with #8824

[laipz8200]

This problem will be fixed in #9790, but please note that using a model that does not support Vision means that the pictures in memory will not be included in the context of the model.

[Copilotes]

@laipz8200 I also think it caused by the files in the historical data during the operation of the no-vl model, but how can I remove files from the historical data now?