Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sending data will block the working thread, causing the gateway to crash #3651

Open
chgang opened this issue Dec 20, 2024 · 5 comments
Open

Comments

@chgang
Copy link

chgang commented Dec 20, 2024

We have found that a netty thread occasionally sends data for 5 seconds (usually only a few milliseconds), causing the thread to be occupied and unable to process other requests. If all threads (such as 6 threads) are occupied, the gateway will crash.

The following is the debug log information of the reactor net source code. During this 5-second period, the reactor-http-epoll-3 thread has been blocking and not processing any other requests, while the other 5 worker threads are processing requests.

================================================================================================

2024-12-18 19:48:40.857  DEBUG [reactor-http-epoll-3] reactor.netty.resources.DefaultPooledConnectionProvider.debug:259 -[d3704e8f-64, L:/10.71.51.11:14010 - R:/10.70.1.11:443] onStateChange(GET{uri=/v1/around, connection=PooledConnection{channel=[id: 0xd3704e8f, L:/10.71.51.11:14010 - R:/10.70.1.11:443]}}, [request_sent])

2024-12-18 19:48:45.872  DEBUG [reactor-http-epoll-3] reactor.netty.resources.DefaultPooledConnectionProvider.debug:259 -[d3704e8f-64, L:/10.71.51.11:14010 - R:/10.70.1.11:443] onStateChange(GET{uri=/v1/around, connection=PooledConnection{channel=[id: 0xd3704e8f, L:/10.71.51.11:14010 - R:/10.70.1.11:443]}}, [response_received])
@chgang
Copy link
Author

chgang commented Dec 20, 2024

spring cloud gateway 3.1.9

@spencergibb
Copy link
Member

That version is out of open source support. Please upgrade to spring cloud 2023.0.x or 2024.0.0 and let us know if the issue persists. https://spring.io/projects/spring-cloud-gateway#support

@chgang
Copy link
Author

chgang commented Dec 20, 2024

If version 3.1.9 exceeds open source, which version from spring-cloud-gateway3. x.x should I use

@spencergibb
Copy link
Member

4.1.x and 4.2.x are the only open source supported versions

@spring-cloud-issues
Copy link

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants