[Feature] Due to GIL issues, the overlap mode doesn't actually always bring benefits?

[CSEEduanyu]

Checklist

• 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 2. Please use English, otherwise it will be closed.

Motivation

Workaround python GIL (Work in progress)：
Idea 1: Try python 3.13 which can remove GIL
Idea 2: Use multiple processes, but need make the serialization very fast
In actual testing, I found that overlap does not necessarily bring benefits. I think this may be related to the GIL, since the current version is implemented with multi-threading. I'm wondering if this is expected? And under what circumstances would the GIL issue become more severe? @merrymercy

Related resources

No response