Fix the early terminative issue in httpRequestHandlerChain

[Silas606]

Description

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #1174

Type of change

minor code change.
Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)

Feature/Issue validation/testing

Tested with

curl localhost:8080/ping
curl localhost:8080/models
curl localhost:8080/workflow

Checklist:

• Have you added tests that prove your fix is effective or that this feature works?
• New and existing unit tests pass locally with these changes?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-cpu
• Commit ID: 54e1be6
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-gpu
• Commit ID: 54e1be6
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-win
• Commit ID: 54e1be6
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-cpu
• Commit ID: 15936ff
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-gpu
• Commit ID: 15936ff
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[maaquib]

@Silas606 Thanks fot this. Do you mind also adding the output of the sanity suite run as well (doc)

[Silas606]

Hi @maaquib here is the output of torchserve_sanity.py. The doc in your reply linked to a Team meeting which I don't have access to. I just attach the log here. Thanks.
out.log

[chauhang]

@lxning Can you please clarify why the same port is being used for the Management and Inference APIs? And how is this fix addressing the issue?

[Silas606]

Sorry I think I made a mistake here as mentioned in #1174 (comment)

No changes needed in ManagementRequestHandler.java as it's not blocking anyone. The issue is just with WorkflowInferenceRequestHandler.java

@chauhang The fix in WorkflowInferenceRequestHandler.java simply follows rules in InferenceRequestHandler.java to only throw ResourceNotFoundException when this API request should clearly be handled here by moving the segment length check after confirming segments[1] equals wfpredict. others such as /models will be handed to subsequent handlers.

Currently in 0.4.0, WorkflowInferenceRequestHandler will throw ResourceNotFoundException for any API requests with segments.length<3 without considering subsequent handlers. With different API ports, that's OK for now since WorkflowInferenceRequestHandler itself is the last handler in inference chain. This logic seems a bit weird to me for a handler in chain though it can work properly. However, when sharing the same port between inference and management API, it will terminate the single chain early and localhost:8080/models just falls in this category. Thus those 404 errors.

I think sharing port is working fine for 0.3.1 and it would be reasonable to expect sharing API port to work also in 0.4.0 or later versions ? Do you have plans to deprecate this feature or are there any concerns with sharing API port ? Really appreciate your help.

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-cpu
• Commit ID: 3ef7b7f
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[Silas606]

I have updated the commit to drop changes in ManagementRequestHandler.java and here is the sanity output out_3ef7b7f.log

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-gpu
• Commit ID: 3ef7b7f
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[sagemaker-neo-ci-bot]

AWS CodeBuild CI Report

• CodeBuild project: torch-serve-build-win
• Commit ID: 926216d
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository