Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace SIGTERM by SIGQUIT in waitshutdown to enable connection draining #8650

Closed
wants to merge 1 commit into from

Conversation

mtparet
Copy link
Contributor

@mtparet mtparet commented May 30, 2022

What this PR does / why we need it:

SIGTERM kill nginx processus without draining connection because it is a "fast shutdown".

As waitshutdown is called in the prestop lifecycle hook, immediately when we ask for pod termination, it means we don't
wait any time for draining existing connections which will be immediately dropped.

Use SIGQUIT which waits connections to end before terminating nginx processus.

References :
http://nginx.org/en/docs/control.html
https://medium.com/codecademy-engineering/kubernetes-nginx-and-zero-downtime-in-production-2c910c6a5ed8

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation only

Which issue/s this PR fixes

How Has This Been Tested?

Start nginx

docker run -p 8080:80 nginx 

Open a telnet ongoing connection

telnet 127.0.0.1 8080

Send a SIGTERM

pkill -SIGTERM -f nginx

Observe that connection is immediately terminated and nginx process are killed.
Restart nginx container and the telnet command.
Send a SIGQUIT

pkill -SIGQUIT -f nginx

Observe that the connection is not terminated and all nginx process are not killed (only idle nginx are terminated)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I've read the CONTRIBUTION guide
  • I have added tests to cover my changes.
  • All new and existing tests passed.

waitshutdown is called in the prestop lifecycle hook, immediately when we ask pod termination, it seems we don't
wait any time for draining existing connection which will be immediately dropped.
Use SIGQUIT which wait connection to end before terminating nginx processus.
http://nginx.org/en/docs/control.html
https://medium.com/codecademy-engineering/kubernetes-nginx-and-zero-downtime-in-production-2c910c6a5ed8

Signed-off-by: Matthieu Paret <matthieu.paret@lifen.fr>
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 30, 2022
@k8s-ci-robot
Copy link
Contributor

@mtparet: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 30, 2022
@k8s-ci-robot
Copy link
Contributor

Hi @mtparet. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-priority size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels May 30, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mtparet
To complete the pull request process, please assign strongjz after the PR has been reviewed.
You can assign the PR to them by writing /assign @strongjz in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mtparet
Copy link
Contributor Author

mtparet commented May 30, 2022

It seems it is already a QUIT #6928 (comment)

@mtparet mtparet closed this May 30, 2022
@mtparet
Copy link
Contributor Author

mtparet commented May 30, 2022

Same issue/solution here #6928 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants