Skip to content

Latest commit

 

History

History
72 lines (45 loc) · 2.57 KB

operations.md

File metadata and controls

72 lines (45 loc) · 2.57 KB
Raw

Operations

Metrics

Prometheus formatted metrics for API rquests are exposed on the /metrics endpoint.

Testing

Assuming you've followed the quickstart steps

  • Review Deployment and Pod spec to validate the webhook is working

    $ kubectl get deploy hello-world -n test1 -o yaml
$ kubectl get pods -n test1
$ kubectl get pod <pod_name> -n test1 -o yaml

    NOTE: You should see the swapped image definition instead of the original definition in the test-deploy.yaml manifest.

Cautions

Production Considerations

  • By Default the ImageSwap Mutating Webhook Configuration is set to fail "closed". Meaning if the webhook is unreachable or doesn't return an expected response, requests to the Kubernetes API will be blocked. Please adjust the configuration if this is not something that fits your environment.
  • ImageSwap supports operation with multiple replicas that can increase availability and performance for critical clusters.
  • The certificate generated by the imageswap-init container is valid for 12 months and will be automatically rotated once the Pod restarts within 6 months of expiration. If the certificate expires, calls to the webhook wil fail. Make sure you plan for this certificate rotation.

Break Glass Scenarios

Per Workload

ImageSwap can be disabled on a per workload level by adding the k8s.twr.io/imageswap label with a value of disabled to the pod template.

Refer to this test manifest as an example: ./testing/deployments/test-deploy05.yaml

Per Namespace

ImageSwap can be enabled and disabled on a per namespace basis by utilizing the k8s.twr.io/imageswap label on the namespace resources. In emergency situations the label can be removed from a namespace to disable image swapping in that namespace.

Cluster Wide

If there are cluster-wide issues you can disable ImageSwap completely by removing the imagewap-webhook Mutating Webhook Configuration and deleting the ImageSwap deployment.

Troubleshooting

Run Docker Image Locally

$ docker run -p 5000:5000/tcp -it imageswapwebhook_app bash
$ ./deny-env.py

Access Kubernetes Service without Ingress/LB

$ kubectl get pods # to get the name of the running pod
$ kubectl port-forward <pod_name> 5000:5000

Use Curl to perform HTTP POST to webhook server

$ curl -vX POST https://localhost:5000/ -d @test.json -H "Content-Type: application/json"

Follow logs of the webhook pod

$ kubectl get pods # to get the name of the running pod
$ kubectl logs <pod_name> -f