check status of router, registry, metrics, logging, imagestreams in oc cluster status #14436

jim-minter · 2017-06-01T15:20:22Z

@openshift/devex
@csrwng

csrwng · 2017-06-01T17:38:52Z

@jim-minter just a general comment... not sure that a warning is the best way to report that the registry/router/etc. is not yet running. It may lead people to think that there's something wrong with their cluster. I would simply report that it's either running or not running. Maybe a warning is warranted only when the uptime of the container exceeds a certain threshold and the registry or router are still not functional.

csrwng · 2017-06-01T17:42:04Z

pkg/bootstrap/docker/status.go

+
+		resp, _ := insecureCli.Get(config.AssetConfig.MetricsPublicURL + "/status")
+		if resp == nil || resp.StatusCode != http.StatusOK {
+			fmt.Println("Warning: metrics are not yet successfully deployed")


s /are/is/ ... I would tend to think of metrics as a singular component

csrwng · 2017-06-01T17:43:04Z

It would also be helpful to report the status of the persistent volume job (-n default job/persistent-volume-setup)

jim-minter · 2017-06-02T16:31:12Z

@csrwng all done

csrwng · 2017-06-05T14:05:08Z

So I tried this locally ... I get this output:

The OpenShift cluster was started 31 seconds ago

Web console URL: https://127.0.0.1:8443

Config is at host directory /var/lib/origin/openshift.local.config
Volumes are at host directory /var/lib/origin/openshift.local.volumes
Persistent volumes are at host directory /var/lib/origin/openshift.local.pv
Data will be discarded when cluster is destroyed

Router is not yet successfully deployed
Persistent volumes are not yet successfully created
Error: OpenShift cluster additional health checks failed

So a couple of nits ...

the message about persistent volumes could be better... maybe say something to the effect that creation has not completed. Even when the job hasn't completed, you are able to start using PV's since some will have been created.
The last line would sound better if it said something like provisioning tasks are still in progress... which is not necessarily an error.

Also, just a question, why are we doing the logging check that way? (Instead of for example hitting a URL like we do with metrics?)

jim-minter · 2017-06-05T16:20:49Z

Also, just a question, why are we doing the logging check that way? (Instead of for example hitting a URL like we do with metrics?)

Because of the oauth proxy, I couldn't find a suitable URL to hit. If you can find one, do let me know!

csrwng · 2017-06-05T17:09:11Z

@sosiouxme @jcantrill please see above. Is there a reliable check that we can do to ensure that logging is working?

sosiouxme · 2017-06-05T18:41:31Z

"Working" comprises a lot of different things, some more subtle than others. But if we're just looking for "present" then why not hit the public logging URL and check that it returns a 302 status (as opposed to 503 if nothing were listening)? Technically the logging stack doesn't have to be deployed in the logging namespace; and I don't think you want to recreate oc adm diagnostics AggregatedLogging in order to get a thorough test for "working".

jim-minter · 2017-06-06T15:02:34Z

@csrwng updated and repushed:

[root@default origin]# oc cluster status
The OpenShift cluster was started 15 seconds ago

Web console URL: https://127.0.0.1:8443

Config is at host directory /var/lib/origin/openshift.local.config
Volumes are at host directory /var/lib/origin/openshift.local.volumes
Persistent volumes are at host directory /var/lib/origin/openshift.local.pv
Data will be discarded when cluster is destroyed

Notice: Docker registry is not yet ready
Notice: Router is not yet ready
Notice: Persistent volumes are not yet ready
Notice: Imagestreams are not yet ready

Notice: 4 OpenShift component(s) are not yet ready (see above)

csrwng · 2017-06-06T16:58:53Z

LGTM
[merge]

openshift-bot · 2017-06-06T17:05:54Z

[Test]ing while waiting on the merge queue

jim-minter · 2017-06-07T15:15:57Z

re-pushed with fix to failing 'oc cluster status' test, ptal/remerge

csrwng · 2017-06-07T15:19:10Z

[merge]

smarterclayton · 2017-06-09T03:51:09Z

pkg/bootstrap/docker/status.go

+		notReady++
+	}
+
+	stdout, _, _ = eh.Command(strings.Split("oc get dc router -n default -o template --template {{.status.availableReplicas}}", " ")...).Output()


Why are you using split here and not below? Don't use split.

smarterclayton · 2017-06-09T03:51:17Z

pkg/bootstrap/docker/status.go

+
+	eh := exec.NewExecHelper(dockerClient, openshift.OpenShiftContainer)
+
+	stdout, _, _ := eh.Command(strings.Split("oc get dc docker-registry -n default -o template --template {{.status.availableReplicas}}", " ")...).Output()


Don't use split.

What if the registry runs as a daemon set?

I guess this is based on the default set by oc cluster up.

…c cluster status

jim-minter · 2017-06-09T11:18:00Z

@csrwng repushed to take care of @smarterclayton's comments above - please remerge

openshift-bot · 2017-06-09T11:21:54Z

Evaluated for origin test up to fc526b7

openshift-bot · 2017-06-09T13:21:56Z

continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin/2048/) (Base Commit: 2458531)

jim-minter · 2017-06-12T08:21:01Z

@csrwng ping :)

csrwng · 2017-06-12T11:58:07Z

[merge]

csrwng · 2017-06-12T16:06:00Z

Flakes #14385 and #14590
[merge]

csrwng · 2017-06-12T20:45:38Z

#14603
#14208
[merge]

csrwng · 2017-06-13T02:23:00Z

#14523
[merge]

openshift-bot · 2017-06-13T02:29:34Z

Evaluated for origin merge up to fc526b7

openshift-bot · 2017-06-13T03:21:43Z

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_origin/978/) (Base Commit: 1284263) (Image: devenv-rhel7_6347)

jim-minter assigned csrwng Jun 1, 2017

csrwng reviewed Jun 1, 2017

View reviewed changes

jim-minter force-pushed the issue12224 branch from 17367a4 to fa66d47 Compare June 2, 2017 16:31

jim-minter force-pushed the issue12224 branch 3 times, most recently from 4533795 to 19f0fce Compare June 6, 2017 15:02

jim-minter force-pushed the issue12224 branch from 19f0fce to 7c8d374 Compare June 7, 2017 15:14

smarterclayton reviewed Jun 9, 2017

View reviewed changes

check status of router, registry, metrics, logging, imagestreams in o…

fc526b7

…c cluster status

jim-minter force-pushed the issue12224 branch from 7c8d374 to fc526b7 Compare June 9, 2017 11:17

openshift-bot merged commit a014711 into openshift:master Jun 13, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

check status of router, registry, metrics, logging, imagestreams in oc cluster status #14436

check status of router, registry, metrics, logging, imagestreams in oc cluster status #14436

jim-minter commented Jun 1, 2017

csrwng commented Jun 1, 2017

csrwng Jun 1, 2017

csrwng commented Jun 1, 2017

jim-minter commented Jun 2, 2017

csrwng commented Jun 5, 2017

jim-minter commented Jun 5, 2017

csrwng commented Jun 5, 2017

sosiouxme commented Jun 5, 2017

jim-minter commented Jun 6, 2017

csrwng commented Jun 6, 2017

openshift-bot commented Jun 6, 2017

jim-minter commented Jun 7, 2017

csrwng commented Jun 7, 2017

smarterclayton Jun 9, 2017

smarterclayton Jun 9, 2017

0xmichalis Jun 9, 2017

0xmichalis Jun 9, 2017

jim-minter commented Jun 9, 2017

openshift-bot commented Jun 9, 2017

openshift-bot commented Jun 9, 2017

jim-minter commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 13, 2017

openshift-bot commented Jun 13, 2017

openshift-bot commented Jun 13, 2017 •

edited

Loading


		eh := exec.NewExecHelper(dockerClient, openshift.OpenShiftContainer)

		stdout, _, _ := eh.Command(strings.Split("oc get dc docker-registry -n default -o template --template {{.status.availableReplicas}}", " ")...).Output()

check status of router, registry, metrics, logging, imagestreams in oc cluster status #14436

check status of router, registry, metrics, logging, imagestreams in oc cluster status #14436

Conversation

jim-minter commented Jun 1, 2017

csrwng commented Jun 1, 2017

csrwng Jun 1, 2017

Choose a reason for hiding this comment

csrwng commented Jun 1, 2017

jim-minter commented Jun 2, 2017

csrwng commented Jun 5, 2017

jim-minter commented Jun 5, 2017

csrwng commented Jun 5, 2017

sosiouxme commented Jun 5, 2017

jim-minter commented Jun 6, 2017

csrwng commented Jun 6, 2017

openshift-bot commented Jun 6, 2017

jim-minter commented Jun 7, 2017

csrwng commented Jun 7, 2017

smarterclayton Jun 9, 2017

Choose a reason for hiding this comment

smarterclayton Jun 9, 2017

Choose a reason for hiding this comment

0xmichalis Jun 9, 2017

Choose a reason for hiding this comment

0xmichalis Jun 9, 2017

Choose a reason for hiding this comment

jim-minter commented Jun 9, 2017

openshift-bot commented Jun 9, 2017

openshift-bot commented Jun 9, 2017

jim-minter commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 12, 2017

csrwng commented Jun 13, 2017

openshift-bot commented Jun 13, 2017

openshift-bot commented Jun 13, 2017 • edited Loading

openshift-bot commented Jun 13, 2017 •

edited

Loading