Explore options to handle faulty configuration gracefully

[yorugac]

Faulty configuration includes insufficient resources, missing service account, incorrect node selector, etc. This is a case of "stuck" test run that currently has to be removed manually.

It appears that adding smth like ttlSecondsAfterFinished to TestRun CRD would allow to mitigate such scenarios. Find out if this option would work as expected or another is required. Additionally, consider how to monitor and report such events occurring in case of cloud test runs.

These two might be partially or fully resolved if TTL approach works out:

• Timeout Implement timeout error #222
• OOM Monitor runners in case of insufficient resources #251