Iteration node with input array length>100 always fails saying 'Max submit count 100 of workflow thread pool reached'

[kazuhisa-wada]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.13.2

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

The error 'ValueError: Max submit count 100 of workflow thread pool reached' easily happens if input to the iteration node is long to some extent.

Example condition:

• iteration node has parallel mode with maximum parallelism>= 2
• input to the iteration node >=101
• run the workflow, then terminates with error saying Max submit count 100 of workflow thread pool reached.

Please see captures.

This never happens for single thread mode.

Although I am not sure the role of max_submit_count, this behavior is obviously wired and should be enhanced so that all the thread is created simultaneously.

Attached is the very easy sample workflow.
workflow.zip

✔️ Expected Behavior

Workflows should be successful even if input length is large.

❌ Actual Behavior

Input length to iteration node with parallel mode fails 100%.

[dosubot]

The error you're encountering, "Max submit count 100 of workflow thread pool reached," is due to the GraphEngineThreadPool class having a max_submit_count set to 100. This limits the number of tasks that can be submitted simultaneously in parallel mode. When your input array length exceeds 100, it surpasses this limit, causing the error.

To resolve this, you can increase the max_submit_count when initializing the GraphEngineThreadPool. Modify the initialization to allow more tasks by setting desired_max_submit_count to a value greater than 100:

self.thread_pool = GraphEngineThreadPool(
    max_workers=thread_pool_max_workers, max_submit_count=desired_max_submit_count
)

Additionally, consider that the iteration node in parallel mode supports up to 10 concurrent iterations. If processing more than 10 tasks, the first 10 execute simultaneously, with the rest processed after completion of the initial batch. Adjust your workflow to handle tasks in batches of 10 or increase the max_submit_count if your system supports it ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[yihong0618]

yes max_submit_count is 100

[kazuhisa-wada]

Yes I know and it's too small. Let me change that configurable. I'll push the code change later.

[kazuhisa-wada]

also it's wired for this behavior for small parallelism case like 2. Why is it reasonable to create many of threadpool simultaneously?