Integration test container should have a wait condition

[michael-valdron]

Which area this feature is related to?

/kind bug

Which area this bug is related to?

/area ci
/area registry
/area integration-tests

What versions of software are you using?

Go project

Operating System and version: N/A

Go Pkg Version: 1.18

Bug Summary

Describe the bug:

The acceptance tests after staging is deployed are failing due to the tests running before the registry viewer container is ready (could take up to the timeout time of 20 seconds). The default run command for the integration test container performs the tests without waiting for the tied devfile registry deployment to enter the available state.

To Reproduce:

Any triggering of staging at this point could lead to the acceptance tests executing before the devfile registry deployment is available to be tested on.

Expected behavior

Any logs, error output, screenshots etc? Provide the devfile that sees this bug, if applicable

Summarizing 1 Failure:
[Fail] [Verify index server is working properly] [It] /viewer should serve the registry viewer
/registry-test/pkg/tests/indexserver_tests.go:76
Ran 30 of 41 Specs in 7.646 seconds
FAIL! -- 29 Passed | 1 Failed | 0 Pending | 11 Skipped
--- FAIL: TestDevfileRegistryController (7.65s)
FAIL

Additional context

Any workaround?

No workaround can be done for this issue.

Suggestion on how to fix the bug

The entrypoint for the integration tests Dockerfile needs to be modified to allow the specification of a wait condition and timeout.

Registry health check probing would ensure target registry is ready before running tests.

Target Date: 04-03-2023

[michael-valdron]

The entrypoint for the integration tests Dockerfile needs to be modified to allow the specification of a wait condition and timeout.

Looking into a simpler alternative solution by probing /health with a timeout in the test suite command.

[michael-valdron]

A few more updates on this issue:

• Found a test case for the registry viewer which fails due to an outdated expected value: https://github.com/devfile/registry-support/blob/0f64fa10dd3da827d4ad36371a2e7f73eb0a7956/tests/integration/pkg/tests/indexserver_tests.go#L76
• Was able to repeat a probe in intervals until timeout, currently determining the best way to interval the http requests

[michael-valdron]

Health check probing is now working, updated README.md to instruct how to enable and use it. Fixed additional issues in the test suite:

1. Registry viewer test case has been revised to work with the new registry viewer content
2. Error handling has been added for the getting and setting of environment variables

These changes are under devfile/registry-support#170 and are ready for review.