[Bug] Translation example : Connection to UI failed

[ShankarRIntel]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source

Deploy method

• Docker compose
• Docker
• Kubernetes
• Helm

Running nodes

Single Node

What's the version?

1.1

Description

Was not able to bring up the servers but errors are being thrown when testing and connecting to the UI
translation_errors.log

Reproduce steps

As in the documentation

Raw log

No response

[xiguiw]

@ShankarRIntel

From the log, the llm_endpoint is not set correctly.

The GenAIExamples are being refactored recently, the document and environment variables may not be updated.

Let me know how you set environment variables etc.

�[33mtranslation-gaudi-backend-server  | �[0m  File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
�[32mllm-tgi-gaudi-server              | �[0m    return AsyncOpenAI(api_key=OPENAI_API_KEY, base_url=llm_endpoint + "/v1", timeout=600, default_headers=headers)
�[32mllm-tgi-gaudi-server              | �[0m                                                        ~~~~~~~~~~~~~^~~~~~~
�[32mllm-tgi-gaudi-server              | �[0mTypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

[ShankarRIntel]

Can I please know how I can set it? Is it in the documentation?

…

________________________________ From: xiguiw ***@***.***> Sent: Monday, January 13, 2025 1:28 AM To: opea-project/GenAIExamples ***@***.***> Cc: Ratneshwaran, Shankar ***@***.***>; Mention ***@***.***> Subject: Re: [opea-project/GenAIExamples] [Bug] Translation example : Connection to UI failed (Issue #1382) @ShankarRIntel<https://github.com/ShankarRIntel> From the log, the llm_endpoint is not set correctly. The GenAIExamples are being refactored recently, the document and environment variables may not be updated. Let me know how you set environment variables etc. �[33mtranslation-gaudi-backend-server | �[0m File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__ �[32mllm-tgi-gaudi-server | �[0m return AsyncOpenAI(api_key=OPENAI_API_KEY, base_url=llm_endpoint + "/v1", timeout=600, default_headers=headers) �[32mllm-tgi-gaudi-server | �[0m ~~~~~~~~~~~~~^~~~~~~ �[32mllm-tgi-gaudi-server | �[0mTypeError: unsupported operand type(s) for +: 'NoneType' and 'str' — Reply to this email directly, view it on GitHub<#1382 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AYMDOK3PQ3T5J5D3DPC6F2D2KOBK7AVCNFSM6AAAAABU7L4QJ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBWGU4TKMBQGI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[xiguiw]

@ShankarRIntel

I tried the wrong examples (ChatQnA) hours ago.

After tried Translation, I cannot reproduce your issue.
I ran Translate on Gaudi successfully.

Please give you steps to reproduce this issue.

Can I please know how I can set it? Is it in the documentation?

Here are steps to set environment variables:

https://github.com/opea-project/GenAIExamples/tree/main/Translation/docker_compose/intel/hpu/gaudi#setup-environment-variables

Set up other environment variables:

cd ../../../
source set_env.sh

Please check TGI_LLM_ENDPOINT in your environment as following:

echo $TGI_LLM_ENDPOINT 

[xiguiw]

@ShankarRIntel

I unset TGI_LLM_ENDPOINT, did not get the error log as yours.
If I run source set_env.sh, I get the result successfully on Console.

Please

1. update your GenAIExamples and GenAIComps code to latest
2. follow REAME to build docker images and tried it again.

If you still ran into the issue, please list each step at you did, so that I can reproduce it at my side.