Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Reliability of Dynamic UDF Resolution in Coral Hive #505

Open
1fanwang opened this issue May 20, 2024 · 0 comments · May be fixed by #527
Open

Improve Reliability of Dynamic UDF Resolution in Coral Hive #505

1fanwang opened this issue May 20, 2024 · 0 comments · May be fixed by #527
Assignees
@1fanwang
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@1fanwang
Copy link
Collaborator

1fanwang commented May 20, 2024 •
edited
Loading

Problem

The ArtifactsResolver in Coral Hive is getting stuck for extended periods (15+ minutes) when fetching Ivy artifacts during dynamic UDF resolution. This issue is causing significant delays and requiring manual retries of entire translation operations.

Current Behavior

  • No timeout mechanism for artifact resolution
  • No automatic retry logic

Observed Symptoms

  • Log entries showing repeated failed attempts to acquire locks
  • Extended periods of inactivity (15+ minutes)
  • Eventual failure with "Unable to fetch dependencies" error

Example Log Output

24/05/20 11:32:00 INFO functions.ArtifactsResolver: Reading Ivy settings from: jar:file:/Users/stewang/.gradle/caches/modules-2/files-2.1/com.linkedin.coral/coral-hive/2.2.1/9b95be32614d2247ae7f4b16266bfc514967012b/coral-hive-2.2.1.jar!/ivy.settings.xml
:: problems summary ::
24/05/20 11:48:04 WARN functions.ArtifactsResolver: Unable to fetch dependencies for UDF org.apache.hadoop.hive.ql.udf.generic.GenericUDFReflect: [download failed: com.google.inject.extensions#guice-servlet;3.0!guice-servlet.jar]
:::: ERRORS
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0
impossible to acquire lock for com.google.inject.extensions#guice-servlet;3.0

Impact

Resolving this issue will:

  • Reduce unexpected delays in translation operations
  • Improve overall reliability of the Coral Hive module
  • Enhance developer experience when working with dynamic UDFs

Additional Context

This issue is particularly impactful when developing tools that rely on Coral's translation capabilities, as it requires retrying entire operations at a higher level.
@1fanwang 1fanwang linked a pull request Sep 5, 2024 that will close this issue
Draft
@1fanwang 1fanwang self-assigned this Sep 5, 2024
@1fanwang 1fanwang added enhancement New feature or request good first issue Good for newcomers labels Sep 5, 2024
@1fanwang 1fanwang changed the title Dynamic UDF resolution stuck at ivy artifact resolution in Coral Hive Improve Reliability of Dynamic UDF Resolution in Coral Hive Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@1fanwang 1fanwang

Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Coral-Hive] Add timeout for artifact resolver via ivy settings linkedin/coral
1 participant
@1fanwang