Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Init Indexing leads to exception being not outputted on timeout #1496

Open
tspring418 opened this issue May 15, 2024 · 0 comments
Open
Assignees

Comments

@tspring418
Copy link

tspring418 commented May 15, 2024

Investigative information

Please provide the following:
  • Timestamp:
    started around may 10 20:43 UTC
  • Function App name:
    prd-modelo-preditivo(staging)
    hml-modelo-preditivo(staging)

Repro steps

Provide the steps required to reproduce the problem:

No repro at the moment, but basically since the push out of :
PYTHON_ENABLE_INIT_INDEXING

We have seen an issue that has led to the exceptions during init leading to ten second timeout between host and language worker prevents logging of the exception causing the problem during this step. Which leads to confusion on why a failure may be occuring during startup of the language worker.

Indexing times using old indexing

 

percentile_AverageTime_30 | percentile_AverageTime_50 | percentile_AverageTime_75 | percentile_AverageTime_90 | percentile_AverageTime_95 | percentile_AverageTime_99 -- | -- | -- | -- | -- | -- 16.5 | 16.5 | 16.6 | 21.406 | 21.406 | 21.406

 

Expected behavior

Metadata call during Indexing should occur while if a timeout happens, it will return the exception that occurred during this time that leads to the timeout and log that exception as well.

Actual behavior

Only a timeout exception is shown
System.TimeoutException : The operation has timed out. at Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.PendingItem.OnTimeout() at /src/azure-functions-host/src/WebJobs.Script.Grpc/Channel/GrpcWorkerChannel.cs : 1764

While the captured exception isn't logged out during indexing.

Known workarounds

Provide a description of any known workarounds.

PYTHON_ENABLE_INIT_INDEXING = 0 in app settings will allow you to see the real exception of the failure

Contents of the requirements.txt file:

Provide the requirements.txt file to help us find out module related issues.

n/a

Related information

Provide any related information

(https://github.com/Azure/azure-functions-python-worker/blob/5bc5d39efffe86da25cacdf7d612a2bc03b32ec3/azure_functions_worker/constants.py#L68C32-L68C59)

ICM # 503543832

Source
-->
@tspring418 tspring418 changed the title [BUG] Bug Title [BUG] Init Indexing leads to exception being not outputted on timeout May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants