Deadlocks configuring envoy for IAP #903
Assignees
Comments
jlewi
added a commit
to jlewi/kubeflow
that referenced
this issue
Jun 5, 2018
* We want a single process to update the backend service and IAP.
Right now we do this in the envoy side car which is replicated.
* The script gets a lock to prevent multiple services from trying to update
IAP simultaneously.
* But until the process acquires the lock it can't update the local copy
of the envoy config to do proper JWT validation.
* So we move updating the backend into a separate deployment that has a single
replica 1 (although we still use a lock).
* envoy sidecars now do not modify the backend but just get relevant info
and update the envoy config.
* To support getting the ksonnet app of the bootstrapper add source repository
IAM role to the admin service account and mount the service account
into the bootstrapper.
This will allow use to push the ksonnet app from the bootstrapper to
a cloud source repository.
Related to kubeflow#903
|
/assign @jlewi |
k8s-ci-robot
pushed a commit
that referenced
this issue
Jun 7, 2018
* Prevent deadlocks trying to setup IAP.
* We want a single process to update the backend service and IAP.
Right now we do this in the envoy side car which is replicated.
* The script gets a lock to prevent multiple services from trying to update
IAP simultaneously.
* But until the process acquires the lock it can't update the local copy
of the envoy config to do proper JWT validation.
* So we move updating the backend into a separate deployment that has a single
replica 1 (although we still use a lock).
* envoy sidecars now do not modify the backend but just get relevant info
and update the envoy config.
* To support getting the ksonnet app of the bootstrapper add source repository
IAM role to the admin service account and mount the service account
into the bootstrapper.
This will allow use to push the ksonnet app from the bootstrapper to
a cloud source repository.
Related to #903
* format jsonnet files.
* Address comments; no need to modify the envoy config.
|
Closing this issue because #924 should fix the deadlocks and we have open issues for the other issues. |
saffaalvi
pushed a commit
to StatCan/kubeflow
that referenced
this issue
Feb 11, 2021
* Prevent deadlocks trying to setup IAP.
* We want a single process to update the backend service and IAP.
Right now we do this in the envoy side car which is replicated.
* The script gets a lock to prevent multiple services from trying to update
IAP simultaneously.
* But until the process acquires the lock it can't update the local copy
of the envoy config to do proper JWT validation.
* So we move updating the backend into a separate deployment that has a single
replica 1 (although we still use a lock).
* envoy sidecars now do not modify the backend but just get relevant info
and update the envoy config.
* To support getting the ksonnet app of the bootstrapper add source repository
IAM role to the admin service account and mount the service account
into the bootstrapper.
This will allow use to push the ksonnet app from the bootstrapper to
a cloud source repository.
Related to kubeflow#903
* format jsonnet files.
* Address comments; no need to modify the envoy config.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm noticing deadlocks and other problems configuring envoy using the IAP script. Some of the problems I observe
I think we should make the following changes
There should be a single pod responsible for enabling IAP and updating the envoy-config map as needed
The envoy sidecars are now just responsible for updating envoy config based on the config map
We should provide a default config that will allow envoy to startup but ensure non secure traffic is blocked
The text was updated successfully, but these errors were encountered: