Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auto-pause: restart service after configuration #19900

Merged
merged 1 commit into from
Dec 11, 2024

Conversation

ComradeProgrammer
Copy link
Member

@ComradeProgrammer ComradeProgrammer commented Nov 1, 2024

auto-pause: restart service after configuration

FIX #17945

After:
when the cluster is running

./out/minikube addons configure auto-pause
-- Enter interval time of auto-pause-interval (ex. 1m0s): 5s
    ▪ Using image gcr.io/k8s-minikube/auto-pause-hook:v0.0.5
✅  auto-pause was successfully configured

when the cluster is not running

./out/minikube addons configure auto-pause
-- Enter interval time of auto-pause-interval (ex. 1m0s): 8s
✅  auto-pause was successfully configured

./out/minikube ssh -- docker ps           
CONTAINER ID   IMAGE                                 COMMAND                  CREATED         STATUS         PORTS     NAMES
df294fd0ff4f   gcr.io/k8s-minikube/auto-pause-hook   "/auto-pause-hook -t…"   2 minutes ago   Up 2 minutes             k8s_webhook_env-inject-85b8f7f46d-ss9cf_auto-pause_d7ce9b55-bc40-4ff4-8751-d987477099f6_0
766548c03aff   haproxy                               "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes             k8s_auto-pause_auto-pause-proxy-654bcfdc5d-tv8fj_auto-pause_ff871384-1382-4d0f-a0ba-7b22ba9e1083_0
15d49e4bd423   registry.k8s.io/pause:3.10            "/pause"                 2 minutes ago   Up 2 minutes             k8s_POD_env-inject-85b8f7f46d-ss9cf_auto-pause_d7ce9b55-bc40-4ff4-8751-d987477099f6_0
a1f07d7b1a80   registry.k8s.io/pause:3.10            "/pause"                 2 minutes ago   Up 2 minutes             k8s_POD_auto-pause-proxy-654bcfdc5d-tv8fj_auto-pause_ff871384-1382-4d0f-a0ba-7b22ba9e1083_0
0e5b25de08c2   ba04bb24b957                          "/storage-provisioner"   4 minutes ago   Up 4 minutes             k8s_storage-provisioner_storage-provisioner_kube-system_4e8dae27-75e3-489f-87bf-aded35faa7cd_0
a6b24327a39f   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_storage-provisioner_kube-system_4e8dae27-75e3-489f-87bf-aded35faa7cd_0
b4e4f7e2b922   2f6c962e7b83                          "/coredns -conf /etc…"   4 minutes ago   Up 4 minutes             k8s_coredns_coredns-7c65d6cfc9-gtsg8_kube-system_0d99c47d-b290-43fa-b871-82ece5cc681d_0
d78d1c933db1   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_coredns-7c65d6cfc9-gtsg8_kube-system_0d99c47d-b290-43fa-b871-82ece5cc681d_0
fa7753ac24f9   021d24201330                          "/usr/local/bin/kube…"   4 minutes ago   Up 4 minutes             k8s_kube-proxy_kube-proxy-s5hcx_kube-system_018929a6-5d58-4e17-a262-8d6214923b9d_0
523ac32d1637   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_kube-proxy-s5hcx_kube-system_018929a6-5d58-4e17-a262-8d6214923b9d_0
d8fc47af5031   f9c26480f1e7                          "kube-apiserver --ad…"   4 minutes ago   Up 4 minutes             k8s_kube-apiserver_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
885233cb72e8   27e3830e1402                          "etcd --advertise-cl…"   4 minutes ago   Up 4 minutes             k8s_etcd_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0
1b317bb53354   9404aea098d9                          "kube-controller-man…"   4 minutes ago   Up 4 minutes             k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
cf19cff01292   d6b061e73ae4                          "kube-scheduler --au…"   4 minutes ago   Up 4 minutes             k8s_kube-scheduler_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
742ca08e2ee2   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
1deeca2fa823   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
a6cd56e814df   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
cc1aa0b10a6b   registry.k8s.io/pause:3.10            "/pause"                 4 minutes ago   Up 4 minutes             k8s_POD_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0


@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 1, 2024
@ComradeProgrammer ComradeProgrammer removed the request for review from prezha November 1, 2024 22:24
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Nov 1, 2024
@medyagh
Copy link
Member

medyagh commented Nov 7, 2024

@ComradeProgrammer please fix the lint

Copy link
Member

@spowelljr spowelljr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fails if auto-pause has already paused the pods.

$ minikube start --addons=auto-pause
...
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

# Wait for auto-pause to pause the pods
$ minikube ssh -- docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED          STATUS                   PORTS     NAMES
fb89d41715cf   6e38f40d628d                          "/storage-provisioner"   13 seconds ago   Up 12 seconds (Paused)             k8s_storage-provisioner_storage-provisioner_kube-system_d52cbb57-0ca1-41aa-b034-21729f201412_1
d1ee449c1051   haproxy                               "docker-entrypoint.s…"   40 seconds ago   Up 39 seconds                      k8s_auto-pause_auto-pause-proxy-654bcfdc5d-mph86_auto-pause_f4c78ef7-a32b-4258-8ef3-ae091547c1a5_0
00222ed908d8   gcr.io/k8s-minikube/auto-pause-hook   "/auto-pause-hook -t…"   43 seconds ago   Up 42 seconds                      k8s_webhook_env-inject-85b8f7f46d-mjpmm_auto-pause_d6da64ed-c118-4609-a226-c753acfd6357_0
473fbc807da3   c69fa2e9cbf5                          "/coredns -conf /etc…"   44 seconds ago   Up 43 seconds (Paused)             k8s_coredns_coredns-7c65d6cfc9-hvsv2_kube-system_d15615d9-a665-40d8-8679-f965d2ca28fd_0
991a0b0998a9   registry.k8s.io/pause:3.10            "/pause"                 44 seconds ago   Up 44 seconds (Paused)             k8s_POD_storage-provisioner_kube-system_d52cbb57-0ca1-41aa-b034-21729f201412_0
f82cb980bf91   505d571f5fd5                          "/usr/local/bin/kube…"   45 seconds ago   Up 44 seconds (Paused)             k8s_kube-proxy_kube-proxy-55cwz_kube-system_401d0ff4-15ac-4cb4-a8fc-363122bbeee7_0
8cff0eee2d73   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds                      k8s_POD_auto-pause-proxy-654bcfdc5d-mph86_auto-pause_f4c78ef7-a32b-4258-8ef3-ae091547c1a5_0
ba590072db6b   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds                      k8s_POD_env-inject-85b8f7f46d-mjpmm_auto-pause_d6da64ed-c118-4609-a226-c753acfd6357_0
3727f0ea3b2b   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds (Paused)             k8s_POD_coredns-7c65d6cfc9-hvsv2_kube-system_d15615d9-a665-40d8-8679-f965d2ca28fd_0
efd0e08db135   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds (Paused)             k8s_POD_kube-proxy-55cwz_kube-system_401d0ff4-15ac-4cb4-a8fc-363122bbeee7_0
5837fd475fdb   0486b6c53a1b                          "kube-controller-man…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
ad67231ce651   847c7bc1a541                          "kube-scheduler --au…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-scheduler_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
ea677b6ed62d   9499c9960544                          "kube-apiserver --ad…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-apiserver_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
fdc86a9cccff   2e96e5913fc0                          "etcd --advertise-cl…"   55 seconds ago   Up 54 seconds (Paused)             k8s_etcd_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0
8c5601e2edd4   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
2203501708a1   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
b319a80a16be   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
bf729faab5a2   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0

$ minikube addons configure auto-pause
-- Enter interval time of auto-pause-interval (ex. 1m0s): 3m
    ▪ Using image gcr.io/k8s-minikube/auto-pause-hook:v0.0.5
# Hangs here for several minutes
💣  Failed to configure auto-pause minikube
✅  auto-pause was successfully configured

@medyagh
Copy link
Member

medyagh commented Nov 22, 2024

@ComradeProgrammer can you plz take a look ?

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 9, 2024
@ComradeProgrammer
Copy link
Member Author

This fails if auto-pause has already paused the pods.

$ minikube start --addons=auto-pause
...
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

# Wait for auto-pause to pause the pods
$ minikube ssh -- docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED          STATUS                   PORTS     NAMES
fb89d41715cf   6e38f40d628d                          "/storage-provisioner"   13 seconds ago   Up 12 seconds (Paused)             k8s_storage-provisioner_storage-provisioner_kube-system_d52cbb57-0ca1-41aa-b034-21729f201412_1
d1ee449c1051   haproxy                               "docker-entrypoint.s…"   40 seconds ago   Up 39 seconds                      k8s_auto-pause_auto-pause-proxy-654bcfdc5d-mph86_auto-pause_f4c78ef7-a32b-4258-8ef3-ae091547c1a5_0
00222ed908d8   gcr.io/k8s-minikube/auto-pause-hook   "/auto-pause-hook -t…"   43 seconds ago   Up 42 seconds                      k8s_webhook_env-inject-85b8f7f46d-mjpmm_auto-pause_d6da64ed-c118-4609-a226-c753acfd6357_0
473fbc807da3   c69fa2e9cbf5                          "/coredns -conf /etc…"   44 seconds ago   Up 43 seconds (Paused)             k8s_coredns_coredns-7c65d6cfc9-hvsv2_kube-system_d15615d9-a665-40d8-8679-f965d2ca28fd_0
991a0b0998a9   registry.k8s.io/pause:3.10            "/pause"                 44 seconds ago   Up 44 seconds (Paused)             k8s_POD_storage-provisioner_kube-system_d52cbb57-0ca1-41aa-b034-21729f201412_0
f82cb980bf91   505d571f5fd5                          "/usr/local/bin/kube…"   45 seconds ago   Up 44 seconds (Paused)             k8s_kube-proxy_kube-proxy-55cwz_kube-system_401d0ff4-15ac-4cb4-a8fc-363122bbeee7_0
8cff0eee2d73   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds                      k8s_POD_auto-pause-proxy-654bcfdc5d-mph86_auto-pause_f4c78ef7-a32b-4258-8ef3-ae091547c1a5_0
ba590072db6b   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds                      k8s_POD_env-inject-85b8f7f46d-mjpmm_auto-pause_d6da64ed-c118-4609-a226-c753acfd6357_0
3727f0ea3b2b   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds (Paused)             k8s_POD_coredns-7c65d6cfc9-hvsv2_kube-system_d15615d9-a665-40d8-8679-f965d2ca28fd_0
efd0e08db135   registry.k8s.io/pause:3.10            "/pause"                 45 seconds ago   Up 44 seconds (Paused)             k8s_POD_kube-proxy-55cwz_kube-system_401d0ff4-15ac-4cb4-a8fc-363122bbeee7_0
5837fd475fdb   0486b6c53a1b                          "kube-controller-man…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-controller-manager_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
ad67231ce651   847c7bc1a541                          "kube-scheduler --au…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-scheduler_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
ea677b6ed62d   9499c9960544                          "kube-apiserver --ad…"   55 seconds ago   Up 54 seconds (Paused)             k8s_kube-apiserver_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
fdc86a9cccff   2e96e5913fc0                          "etcd --advertise-cl…"   55 seconds ago   Up 54 seconds (Paused)             k8s_etcd_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0
8c5601e2edd4   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-controller-manager-minikube_kube-system_a0f79d40fa1f417afd22780e833eca6a_0
2203501708a1   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-scheduler-minikube_kube-system_b8c9ab2df1ccf7573c3a515e3c781475_0
b319a80a16be   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_kube-apiserver-minikube_kube-system_d2ead1331f69543592886560584ea808_0
bf729faab5a2   registry.k8s.io/pause:3.10            "/pause"                 55 seconds ago   Up 54 seconds (Paused)             k8s_POD_etcd-minikube_kube-system_a5363f4f31e043bdae3c93aca4991903_0

$ minikube addons configure auto-pause
-- Enter interval time of auto-pause-interval (ex. 1m0s): 3m
    ▪ Using image gcr.io/k8s-minikube/auto-pause-hook:v0.0.5
# Hangs here for several minutes
💣  Failed to configure auto-pause minikube
✅  auto-pause was successfully configured

Now it should be fixed. The cluster will be unpaused before restarting auto-pause service

@medyagh
Copy link
Member

medyagh commented Dec 11, 2024

thank you @ComradeProgrammer

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ComradeProgrammer, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 11, 2024
@medyagh medyagh merged commit 52cd866 into kubernetes:master Dec 11, 2024
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix auto-pause interval value not immediately being taking affect
4 participants