Configurable startup time

[Isalia20]

Description

Adds feature from #3261 and from #2452

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

• Test for long startup while having small response timeout. Test can be found in test/pytest/test_startup_timeout.py

================================================================================ test session starts =================================================================================
platform linux -- Python 3.10.14, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/isalia/Desktop/torchserve
plugins: mock-3.14.0, timeout-2.3.1, cov-4.1.0, typeguard-4.3.0
collected 1 item

test/pytest/test_startup_timeout.py . [100%]

================================================================================= 1 passed in 15.02s =================================================================================

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[mreso]

Thanks for the contribution. Please see my comment regarding testing. Otherwise this look good to go after adjusting the test.

[Isalia20]

I see some checks are failing but I'm not sure what the issue really is, to fix it

[mreso]

I see some checks are failing but I'm not sure what the issue really is, to fix it

Yeah, these regression tests can be a pain. Taking a look

[Isalia20]

One of the tests was failing due to startup_timeout not being cast to int in test/pytest/test_gRPC_management_apis.py. Added it to the list of params that should be casted to int

[Isalia20]

Seems like tests have passed(required ones). Docker ones are failing due to:
0.811 fatal: Remote branch configurable-startup-time not found in upstream origin
which could be because it can't clone from my branch 🤔

[mreso]

Hi @Isalia20 yes, that should be the docker build script which only supports pytorch/serve as remote. We can proceed with merging this PR. Thanks again for your contribution!

[mreso]

LGTM now