fail live probe if fail-fast mechanism was triggered

[abrbird]

Motivation

As described here , it may be useful to fail live probe if fail-fast mechanism was triggered. Modelmesh responses 503 for Ready probe if some models failed to load on start, even if these models successfully loaded later, mm remains in unrecoverable state.

Modifications

overrided isLive method: return false if startup aborted

[oss-prow-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: abrbird
Once this PR has been reviewed and has the lgtm label, please assign njhill for approval by writing /assign @njhill in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[spolti]

/lgtm

[abrbird]

I triggered the pipeline from forked repo and the build completed successfully
https://github.com/abrbird/modelmesh/actions/runs/9804929528

May I ask you to rerun the build, it seems there are some flaky tests?

[abrbird]

@rafvasq hello! could please check this out - test job failed and the error is the same as occurred on the main - it very seems that the ModelMeshTearDownTest.testDestroyNode test is not stable.
Could we do smth to go further?

[fsatka]

@spolti hi! Can you answer please)

[spolti]

@abrbird you can send an empty commit to retrigger the tests:

git commit -s --allow-empty -m "empty commit"