Add a more graceful termination to Allocator #3105

chiayi · 2023-04-17T20:08:44Z

What type of PR is this?

Uncomment only one /kind <> line, press enter to put that in a new line, and remove leading whitespace from that line:

/kind breaking

/kind bug

/kind cleanup
/kind documentation
kind feature
/kind hotfix

What this PR does / Why we need it:
This ensure that we properly lame-duck the allocator pod when disruption occurs. Also make sure that no request comes in while it is being lame-ducked.

Which issue(s) this PR fixes:

Special notes for your reviewer:
Created a helper package for allocator tests and also the pod termination test, most of the functions were originally from allocator_test.go

agones-bot · 2023-04-17T20:15:22Z

Build Failed 😱

Build Id: 76bea120-d576-4b81-ab30-e2eaa0189e45

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-17T21:42:35Z

Build Succeeded 👏

Build Id: 07c515ef-89d5-4890-8bdc-3386fa5b1fba

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-22ca04f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-22ca04f-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-22ca04f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-22ca04f-amd64
Linux C++ SDK (build): agonessdk-1.32.0-22ca04f-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-22ca04f-amd64.zip

A preview of the website (the last 30 builds are retained):

https://22ca04f-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-22ca04f-amd64

agones-bot · 2023-04-19T21:52:12Z

Build Failed 😱

Build Id: 55bad680-7f4c-4201-bd25-4c172594b316

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-19T23:18:09Z

Build Failed 😱

Build Id: e844639a-4bbd-4144-9645-7ea70ed70e0c

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-20T01:01:03Z

Build Succeeded 👏

Build Id: f8864008-d299-4439-bf9d-00431f87e13b

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-6f27c4a-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-6f27c4a-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-6f27c4a-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-6f27c4a-amd64
Linux C++ SDK (build): agonessdk-1.32.0-6f27c4a-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-6f27c4a-amd64.zip

A preview of the website (the last 30 builds are retained):

https://6f27c4a-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-6f27c4a-amd64

agones-bot · 2023-04-20T18:31:58Z

Build Succeeded 👏

Build Id: 99f61297-a607-41a7-b81d-26ae2bbfb120

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-6f27c4a-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-6f27c4a-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-6f27c4a-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-6f27c4a-amd64
Linux C++ SDK (build): agonessdk-1.32.0-6f27c4a-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-6f27c4a-amd64.zip

A preview of the website (the last 30 builds are retained):

https://6f27c4a-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-6f27c4a-amd64

agones-bot · 2023-04-20T19:17:42Z

Build Failed 😱

Build Id: 34b1735d-b7db-4ef4-a6b5-b60923c0eb06

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-20T20:16:19Z

Build Failed 😱

Build Id: c745bd64-c9f0-4801-b234-c58aeb8469e1

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-20T20:41:28Z

Build Failed 😱

Build Id: d343602e-1982-4964-af04-dc152f6b3f59

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-04-20T21:55:03Z

Build Succeeded 👏

Build Id: 5fee5585-394b-4ea8-9548-905bbdb38693

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-0f811d4-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-0f811d4-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-0f811d4-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-0f811d4-amd64
Linux C++ SDK (build): agonessdk-1.32.0-0f811d4-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-0f811d4-amd64.zip

A preview of the website (the last 30 builds are retained):

https://0f811d4-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-0f811d4-amd64

zmerlynn

Driveby comment, but looks like you're on the right track.

zmerlynn · 2023-04-20T23:53:08Z

cmd/allocator/main.go

 		_, err := agonesClient.ServerVersion()
-		return err
+		if err != nil {
+			return errors.New("failed to reach Kubernetes")


you squashed the error from the client here - what was wrong with line 219? If you want to differentiate between these error cases you could do something like fmt.Errorf("failed to reach Kubernetes: %w")

agones-bot · 2023-04-21T18:36:59Z

Build Failed 😱

Build Id: 17f16b22-7608-4938-8e60-023fcf321a74

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

test/e2e/allocator/pod_termination_test.go

agones-bot · 2023-05-03T20:35:12Z

Build Failed 😱

Build Id: b602cdb1-3a19-47b4-b4b2-aaa86a362482

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-03T21:49:40Z

Build Failed 😱

Build Id: 41ad4ef9-3b02-424a-81a7-3101f613bdef

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-03T22:41:06Z

Build Failed 😱

Build Id: 72a02920-a420-4a3c-b770-206a82020cfc

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-04T17:49:38Z

Build Failed 😱

Build Id: 39a67e92-467b-40f4-8099-15a8622a6a17

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-04T18:40:12Z

Build Failed 😱

Build Id: 578ddc20-7939-40b2-af14-a3acd4db2a8a

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-04T23:21:47Z

Build Failed 😱

Build Id: 67a870ca-7190-4c22-b1ba-ccd97b8daa78

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-04T23:44:18Z

Build Failed 😱

Build Id: b3a10034-478a-4142-9e9e-ea7e6c1b6bee

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-05T02:01:12Z

Build Failed 😱

Build Id: 5b107421-aa5f-4949-9f42-0ea8f9460b0b

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-05T19:16:25Z

Build Failed 😱

Build Id: 12e116fe-1cf8-4f65-8873-e94e4e8ac528

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2023-05-06T00:14:14Z

Build Succeeded 👏

Build Id: ab5a4822-c6fd-4664-8975-ee3156b72ff1

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-0a10b1f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-0a10b1f-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-0a10b1f-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-0a10b1f-amd64
Linux C++ SDK (build): agonessdk-1.32.0-0a10b1f-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-0a10b1f-amd64.zip

A preview of the website (the last 30 builds are retained):

https://0a10b1f-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-0a10b1f-amd64

agones-bot · 2023-05-08T18:12:09Z

Build Succeeded 👏

Build Id: ad59641b-4b87-4ecd-bf61-991cb4403289

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.32.0-7dcf146-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.32.0-7dcf146-linux-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.32.0-7dcf146-amd64
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.32.0-7dcf146-amd64
Linux C++ SDK (build): agonessdk-1.32.0-7dcf146-amd64-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.32.0-7dcf146-amd64.zip

A preview of the website (the last 30 builds are retained):

https://7dcf146-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/3105/head:pr_3105 && git checkout pr_3105
helm install agones ./install/helm/agones --namespace agones-system --agones.image.release=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.32.0-7dcf146-amd64

google-oss-prow · 2023-05-08T23:44:49Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chiayi, zmerlynn
Once this PR has been reviewed and has the lgtm label, please assign ericfortin for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow bot added the do-not-merge/work-in-progress label Apr 17, 2023

google-oss-prow bot requested review from cyriltovena and roberthbailey April 17, 2023 20:08

google-oss-prow bot added the size/M label Apr 17, 2023

chiayi requested review from zmerlynn and removed request for cyriltovena and roberthbailey April 17, 2023 20:08

chiayi force-pushed the agoneshabranch branch from 568669c to 22ca04f Compare April 17, 2023 20:52

chiayi force-pushed the agoneshabranch branch from 22ca04f to 60578b8 Compare April 19, 2023 21:46

google-oss-prow bot added size/L and removed size/M labels Apr 19, 2023

chiayi force-pushed the agoneshabranch branch from 60578b8 to f157df7 Compare April 19, 2023 21:55

chiayi force-pushed the agoneshabranch branch from f157df7 to 6f27c4a Compare April 19, 2023 23:52

chiayi force-pushed the agoneshabranch branch from 6f27c4a to 7af369f Compare April 20, 2023 19:07

chiayi force-pushed the agoneshabranch branch from 7af369f to da7f5fb Compare April 20, 2023 20:31

chiayi force-pushed the agoneshabranch branch from da7f5fb to 0f811d4 Compare April 20, 2023 20:48

zmerlynn reviewed Apr 20, 2023

View reviewed changes

chiayi force-pushed the agoneshabranch branch from 0f811d4 to 5bbf0f8 Compare April 21, 2023 17:35

chiayi force-pushed the agoneshabranch branch from 5bbf0f8 to 392d6cb Compare April 21, 2023 21:38

zmerlynn reviewed May 3, 2023

View reviewed changes

test/e2e/allocator/pod_termination_test.go Show resolved Hide resolved

chiayi force-pushed the agoneshabranch branch from d550adb to 1e65837 Compare May 3, 2023 20:29

chiayi force-pushed the agoneshabranch branch from 1e65837 to 1e40ccf Compare May 3, 2023 20:55

chiayi force-pushed the agoneshabranch branch from 1e40ccf to 87613d9 Compare May 4, 2023 17:39

chiayi force-pushed the agoneshabranch branch from 87613d9 to 40f8500 Compare May 4, 2023 22:45

chiayi force-pushed the agoneshabranch branch from 40f8500 to 75d0f4b Compare May 4, 2023 23:38

chiayi force-pushed the agoneshabranch branch from 75d0f4b to a3be24c Compare May 5, 2023 00:39

chiayi force-pushed the agoneshabranch branch from a3be24c to abe07e6 Compare May 5, 2023 18:05

chiayi force-pushed the agoneshabranch branch 2 times, most recently from f409a24 to 0a10b1f Compare May 5, 2023 22:04

Adding a more graceful termination for allocator

7dcf146

chiayi force-pushed the agoneshabranch branch from 0a10b1f to 7dcf146 Compare May 8, 2023 17:13

zmerlynn approved these changes May 8, 2023

View reviewed changes

google-oss-prow bot assigned zmerlynn May 8, 2023

google-oss-prow bot added the lgtm label May 8, 2023

zmerlynn merged commit d7a9e6a into googleforgames:main May 8, 2023

chiayi mentioned this pull request May 9, 2023

Add allocator readiness configurations doc #3142

Merged

Kalaiselvi84 added the kind/bug These are bugs. label May 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a more graceful termination to Allocator #3105

Add a more graceful termination to Allocator #3105

chiayi commented Apr 17, 2023 •

edited

Loading

agones-bot commented Apr 17, 2023

agones-bot commented Apr 17, 2023

agones-bot commented Apr 19, 2023

agones-bot commented Apr 19, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

zmerlynn left a comment

zmerlynn Apr 20, 2023

agones-bot commented Apr 21, 2023

agones-bot commented May 3, 2023

agones-bot commented May 3, 2023

agones-bot commented May 3, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 5, 2023

agones-bot commented May 5, 2023

agones-bot commented May 6, 2023

agones-bot commented May 8, 2023

google-oss-prow bot commented May 8, 2023

Add a more graceful termination to Allocator #3105

Add a more graceful termination to Allocator #3105

Conversation

chiayi commented Apr 17, 2023 • edited Loading

agones-bot commented Apr 17, 2023

agones-bot commented Apr 17, 2023

agones-bot commented Apr 19, 2023

agones-bot commented Apr 19, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

agones-bot commented Apr 20, 2023

zmerlynn left a comment

Choose a reason for hiding this comment

zmerlynn Apr 20, 2023

Choose a reason for hiding this comment

agones-bot commented Apr 21, 2023

agones-bot commented May 3, 2023

agones-bot commented May 3, 2023

agones-bot commented May 3, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 4, 2023

agones-bot commented May 5, 2023

agones-bot commented May 5, 2023

agones-bot commented May 6, 2023

agones-bot commented May 8, 2023

google-oss-prow bot commented May 8, 2023

chiayi commented Apr 17, 2023 •

edited

Loading