Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Note: There are bugs within the graceful shutdown process of many HTTP server backends that can cause the process to hang. #2

Open
czy-29 opened this issue May 6, 2024 · 8 comments
Labels
bug Something isn't working tracking-issue Umbrella tracking issue

Comments

@czy-29
Copy link
Contributor

czy-29 commented May 6, 2024

备注:有很多HTTP服务器后端的“优雅停机”过程都存在会使进程卡死的bug。


This issue will list the associated issues or PRs for each buggy backend one by one. They may have been solved and released, or solved but not yet released, or not yet resolved. The main purpose of this issue is to serve as a reminder.

For backends with unfixed or unreleased bugs, we will provide temporary workarounds (through tokio::task::AbortHandle::abort()). But when the bug is finally fixed and released, the workaround should be removed and updated to the official implementation.

This issue will be closed when all backend hang bugs are fixed and released.

Summary of current status:

  1. The warp backend is waiting for fix[1][2] and cannot work properly, and there is no perfect workaround solution, so the backend is temporarily removed[3]. Since other backends have fixed the bug, if warp still has not fixed the bug after one month until July 1st, this issue will be closed and the warp backend will be permanently removed.
  2. The remaining backends (salvo, poem, actix-web, axum, rocket, viz, ntex) may or may not have had bugs in the past, but in short they are currently working fine.

本issue将逐一列举每个存在bug的后端的关联issue或pr,它们可能是已解决并发布的,或已解决但尚未发布的,或尚未解决的。本issue的主要目的是备忘。

对于存在未修复或未发布的bug的后端,我们会对其提供临时性的workaround(通过tokio::task::AbortHandle::abort())。但是当最终bug被修复并发布后,workaround应当被移除并更新为正式的实现。

当所有后端的hang bug都修复并发布后,这个issue将会被关闭。

当前状态总结:

  1. warp后端正在等待修复[1][2],无法正常工作,且没有完美Workaround方案,因此暂时移除了该后端[3]。由于其它后端均已修复该bug,因此等一个月至7月1日,如果warp依然没有修复该bug,该issue将会关闭,且warp后端将被永久移除。
  2. 其余后端(salvopoemactix-webaxumrocketvizntex)可能曾经有bug,可能一直没有,但总之它们目前工作正常。
@czy-29
Copy link
Contributor Author

czy-29 commented May 6, 2024

salvo (fixed and released in v0.67.2 - 已修复并发布于v0.67.2):
issue: salvo-rs/salvo#764
pr: salvo-rs/salvo#765

@czy-29
Copy link
Contributor Author

czy-29 commented May 6, 2024

poem (fixed and released in v3.0.0 - 已修复并发布于v3.0.0):
issue: poem-web/poem#661
pr: poem-web/poem#780

@czy-29
Copy link
Contributor Author

czy-29 commented May 6, 2024

actix-web does not have this bug.
actix-web不存在此bug。

@czy-29
Copy link
Contributor Author

czy-29 commented May 6, 2024

axum (fixed and released in hyper-util v0.1.4):
issue: tokio-rs/axum#2611

This is actually a lower-level hyper-util bug (fixed and released in v0.1.4):
issue: hyperium/hyper#3576
pr: hyperium/hyper-util#101

The following bug has been fixed (24.05.09), and the patch has been returned to the current HEAD version:

But in fact, currently (24.05.08) the HEAD commit of hyper-util is still buggy, so our workaround cannot simply patch hyper-util to the git address, but needs to lock it to a commit without bugs (see: f43236e). For details about this bug, see the following PR comment: hyperium/hyper-util#102 (comment)
And issue: hyperium/hyper#3662

When both axum and hyper-util release updates containing these fixes, we should update the version number of axum and revert the commits: 6f86013 , 3852e4c , f43236e , and aa6fd33


axum(已修复并发布于hyper-util v0.1.4):
issue: tokio-rs/axum#2611

这个其实是更底层的hyper-util的bug(已修复并发布于v0.1.4):
issue: hyperium/hyper#3576
pr: hyperium/hyper-util#101

以下bug已修复(24.05.09),patch已回到当前的HEAD版本上:

但实际上目前(24.05.08)hyper-utilHEAD提交依然是有bug的,因此我们的workaround不能简单地将hyper-util patch到git地址上,而是需要锁定到一个没有bug的提交上(见:f43236e )。对于这个bug的详情,见以下pr评论:hyperium/hyper-util#102 (comment)
以及issue:hyperium/hyper#3662

axumhyper-util都发布了包含这些修复的更新之后,我们应当更新axum的版本号,并撤回提交:6f860133852e4cf43236e ,和 aa6fd33

@czy-29 czy-29 pinned this issue May 6, 2024
@czy-29
Copy link
Contributor Author

czy-29 commented May 7, 2024

rocket does not have this bug.
rocket不存在此bug。

@czy-29
Copy link
Contributor Author

czy-29 commented May 8, 2024

warp (not fixed):
issue: seanmonstar/warp#1100

This is very similar to the Axum bug above, but the solution using the patch hyper-util provided by Axum is invalid. We need to wait for the warp author to reply and fix it.
Because there is no working workaround solution, the backend is temporarily removed until the upstream fix: d4af016
After the fix, we should first patch warp, then test the fix results, and then undo the above commit.
Then when warp is released, we should revert the patch and update the version number of warp.

Latest update:
This bug of warp can be solved by upgrading hyper to 1.x, that is to say, the underlying issue is actually this issue: seanmonstar/warp#1088


warp(未修复):
issue:seanmonstar/warp#1100

这个跟上面Axum的bug非常像,但用Axum提供的patch hyper-util的解决方案无效,需等待warp作者回复并修复。
因为没有可以正常工作的workaround方案,因此上游修复前,该后端被暂时移除:d4af016
修复后,我们应当先patch warp,然后测试修复结果,然后撤回上述提交。
然后当warp发版后,我们应当撤回patch,然后更新warp的版本号。

最新更新:
warp的这个bug可以通过将hyper升级至1.x来解决,也就是说,实际上更底层关联的是此issue:seanmonstar/warp#1088

@czy-29
Copy link
Contributor Author

czy-29 commented May 9, 2024

viz (fixed and released in v0.8.5):
issue: viz-rs/viz#138

This bug is almost 100% the same as the above-mentioned axum bug, so I won’t go into details again.


viz(已修复并发布于v0.8.5):
issue:viz-rs/viz#138

这个bug跟上述axum的bug几乎100%一样,因此不再赘述。

@czy-29
Copy link
Contributor Author

czy-29 commented May 9, 2024

ntex does not have this bug.
ntex不存在此bug。

@czy-29 czy-29 changed the title Note: There are stuck bugs in the graceful shutdown process of many HTTP server backends. Note: There are bugs within the graceful shutdown process of many HTTP server backends that can cause the process to hang. May 9, 2024
@czy-29 czy-29 added bug Something isn't working tracking-issue Umbrella tracking issue labels May 29, 2024
@czy-29 czy-29 unpinned this issue May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working tracking-issue Umbrella tracking issue
Projects
None yet
Development

No branches or pull requests

1 participant