Failed to start Kubernetes runtime of workspace. Cause: Server 'theia' not available #13844
Comments
|
After successfully deploying Multiuser Che,Attempting to create and run a new workspace with the "java maven" stack fails |
|
che 7.0.0 |
|
Hell @hjbbjh |
|
installation via : |
|
Can you be more specific with Che version? We have 7.0.0-rc-4.0-SNAPSHOT in master at this moment. |
|
che-7.0.0-RC-1.0.tar.gz |
|
Is it because there's a timeout？Start image eclipse/che-theia:7.0.0-rc-3.0 up unsuccessfully。 It feels like theia hasn't been fully started yet。 |
|
Im experiencing the exact same issue. I'm using eclipse/che-theia:7.0.0-rc-3.0. I'm using the following config: I'm seeing the exact same log entries as mentioned above, the following line is the last one in the theia pod:
Ingresses are created fine and seem to work. The che server logs show the following: Is there a config option to configure the server check timeout? I've tried tho change the "CHE_INFRA_KUBERNETES_BOOTSTRAPPER_SERVER__CHECK__PERIOD__SEC" to "1000" but this doesn't help. |
|
This is the Devfile I'm using: |
nickboldt
added
area/editor/theia
|
|
|
I tried to start the eclipse/che-theia:7.0.0-rc-3.0 image locally, I get the exact same log as if I start it within a workspace. the last line in the log is always: Is it possible to test an older version of the editor ? I've checked https://che-plugin-registry.openshift.io/v3/plugins/eclipse/che-theia/ but RC-2.0 is not there anymore. Do you have older versions of the plugin registry still published or what is the best way to test different images? |
|
@synax can you try to install che with https://github.com/che-incubator/chectl and try again? |
|
what is the url field of the table che_k8s_server when you start workspace？ |
That is an url of the running server inside of you workspace. Null in url means that something went wrong. I assume some variables or some components may not be set correctly. Can you try to install che with https://github.com/che-incubator/chectl |
|
Can you explain my problem about the null host of the server url please? Is it normal? |
I can't explain it. That is not a normal situation. |
|
@skabashnyuk would be something wrong with rbac ? I just read the source code , It seems che will use this url to check whether theia service is available. |
could be. https://github.com/che-incubator/chectl should handle that. |
|
in my case the query: outputs this: These are the ingresses created correctly in my che namespace. But I guess the server check fails because the endpoint of the ingress never becomes available, due to the container not starting properly. |
|
I'm using Kustomize for my deployment. I've converted the RC-3.0 helm chart using |
|
@skabashnyuk but chectl dependent on minikube，it used for local development and testing, not for production。 |
|
@synax why your url field of table che_k8s_server have no host？ |
|
I've uploaded my kustomization for reference https://github.com/synax-io/echlipse-che-kustomization In case you try to deploy this kustomization, some remarks :)
|
|
@hjbbjh che creates ingresses for each of the services. So the traffic is then forwarded to the services and the pods. I guess you have a different config than I do. If you use the Kustomization I've posted, you should have the same behavior. |
chectl is also working with openshift, k8s, etc |
|
@benoitf in https://github.com/che-incubator/chectl，i can see this sentence：Currently chectl requires minikube and helm to be locally installed ，and in https://www.eclipse.org/che/docs/che-7/che-quick-starts.html，i can see this sentence：-p, --platform=platform [default: minikube] Type of Kubernetes platform. Valid values are "minikube", "minishift", "docker4mac", "ocp", "oso". |
slemeur
added
the
severity/P1
|
@hjbbjh yeah the README file hasn't been updated. |
|
@l0rd how can i replace the file of the directory deploy/kubernetes/helm/che? |
|
Ok, So I've changed the plugin version to I've double checked the services and ingresses, something seems to be off here: The service for theia has all the ports from the plugin spec configured and it seems all endpoints are available: but the ingress for port
I would expect an ingress for all endpoints configured as https://che-plugin-registry.openshift.io/v3/plugins/eclipse/che-theia/next/ Is this behavior expected? |
|
As far as I understand, the readiness checker https://github.com/eclipse/che/blob/7.0.0-rc-3.x/wsmaster/che-core-api-workspace/src/main/java/org/eclipse/che/api/workspace/server/hc/ServersChecker.java just attempts to reach each server for a particular machine. so in my example, the checker iterates thru all The error message states that the server theia is not available, which means the url I've checked the ingress and service. the jwtproxy pod is running, service and ingress are present. the jwtproxy log shows: |
|
Here the config of the jwtproxy I've checked the upstream service and it exists. But now we are back to the theia container not starting up properly. is it possible to get debug output from the server checker? |
|
Try to deploy it via
today，i try again，get the same error: |
|
@hjbbjh I think there is something wrong with your ENV config. Try to use my kustomization values. |
|
Ok, after a bit more testing... I'm not sure if this is actually related to che-theia. I've changed the editor to Eclipse GWT IDE and the behavior is very similar. The only difference is that I actually see an error message during the listener registration. could this issue be related to some issue with the jwtproxy and the keycloak config? is there a way to get more detailed info from the server check? so I can confirm what actually goes wrong? looks like the ?token= is missing... the CHE_MACHINE_TOKEN variable is configured for the pod. If I append the token, I get access to the workspace via the api. So I wonder why is it not using the token? is this configured somewhere else? |
l0rd
removed
the
status/info-needed
l0rd
changed the title
|
@slemeur clearing the milestone for now. The issue is still under investigation. Not sure if we are able to reproduce, if it's a duplicate etc... @skabashnyuk please have a look at latest @hjbbjh and @synax tests results |
l0rd
changed the title
|
@hjbbjh 's issue : we've changed two part of helm chart because In China we can not connect to google service directory, we use aliyun instead |
|
Try to deploy via chectl (we've changed chectl source code to prevent updating requirements.yaml),found errors below: but che pod is working, we can access to dashboard, when we try to start a work space, the same error |
|
thx all @skabashnyuk @l0rd @synax We try to redeploy che base on It seems that something wrong with |
|
@skabashnyuk @l0rd why default-host not work？Users must use our DNS Service in the multi-host mode，i want to avoid it，i want to deploy in default-host mode。 |
|
Here is a brief explanation of why |
|
I suppose it's fixed. Feel free to reopen in case it's reproduced. |
|
Received SIGTERM
The text was updated successfully, but these errors were encountered: