Jupyter_ai for Azure OpenAI throws 'InternalServerError' for all chat responses #1208
Comments
eazuman
changed the title
eazuman
changed the title
eazuman
changed the title
|
Also ,I was able to get this working with an older version of jupyter_ai. Here are the package details. Do you have any idea what's causing the issue with the latest versions? 
|
eazuman
changed the title
|
@eazuman "Internal server error" means that OpenAI's servers cannot handle this request right now due to an internal error. In other words, this error is being caused by OpenAI's backend servers, not Jupyter AI. If you retry again today, you should hopefully be able to consistently receive good replies. I would recommend trying again with the latest version of Jupyter AI (v2.29.0). |
|
@dlqqq thanks for your response! As I said above, I got it working with Jupyter AI version 2.11, but not with the latest version. |
|
@eazuman This is very strange. It is possible that this is a bug with the |
|
@eazuman Could you try using the |
|
Thank you @dlqqq for the suggestion! 
|
|
Additionally, I tested other package versions and found that Do you have any thoughts on other factors that could be causing these issues, or any specific package versions? Thanks!! 2.14.0
2.17.0/ 2.18.0 Error, even though I have passed api_version in the settings . Also tried the same as env. variable
2.19.0 - 2.29.0
|
|
I’m still trying a few other things and wanted to check with you to see if there are any package dependencies we should consider that should be pin? As I mentioned, we’re using When we moved to For this specific version of
I just want to make sure I’m trying a few options to troubleshoot the issue and not missing something. Thanks for the help! |
|
@eazuman Thanks so much for investigating this further. I looked into all the code changes made when we updated from Best to do this in a clean new environment: Depending on your environment, you may see some errors but they may not matter. Then run Jupyter Lab with The Settings are in a different location, as shown here:
And you can open a new chat from the top left as shown on the screenshot above. Hope this works or gives some insight into where the issue lies (and also gives a look at v3!). |
Description
Jupyter_ai throwing InternalServerError for the chat response for Azure Openai provider
It works for the
/generatecommand but the chat responds with the below error for all questionsthis is with the latest version of jupyter_ai and its dependencies
Any help or insights on this issue would be greatly appreciated
Traceback (most recent call last): File "/opt/conda/lib/python3.11/site-packages/jupyter_ai/chat_handlers/base.py", line 226, in on_message await self.process_message(message) File "/opt/conda/lib/python3.11/site-packages/jupyter_ai/chat_handlers/default.py", line 72, in process_message await self.stream_reply(inputs, message) File "/opt/conda/lib/python3.11/site-packages/jupyter_ai/chat_handlers/base.py", line 564, in stream_reply async for chunk in chunk_generator: File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5535, in astream async for item in self.bound.astream( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5535, in astream async for item in self.bound.astream( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3430, in astream async for chunk in self.atransform(input_aiter(), config, **kwargs): File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3413, in atransform async for chunk in self._atransform_stream_with_config( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2301, in _atransform_stream_with_config chunk: Output = await asyncio.create_task( # type: ignore[call-arg] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3383, in _atransform async for output in final_pipeline: File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5571, in atransform async for item in self.bound.atransform( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 4941, in atransform async for output in self._atransform_stream_with_config( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2301, in _atransform_stream_with_config chunk: Output = await asyncio.create_task( # type: ignore[call-arg] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 4922, in _atransform async for chunk in output.astream( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5535, in astream async for item in self.bound.astream( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3430, in astream async for chunk in self.atransform(input_aiter(), config, **kwargs): File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3413, in atransform async for chunk in self._atransform_stream_with_config( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2301, in _atransform_stream_with_config chunk: Output = await asyncio.create_task( # type: ignore[call-arg] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3383, in _atransform async for output in final_pipeline: File "/opt/conda/lib/python3.11/site-packages/langchain_core/output_parsers/transform.py", line 84, in atransform async for chunk in self._atransform_stream_with_config( File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2259, in _atransform_stream_with_config final_input: Optional[Input] = await py_anext(input_for_tracing, None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/utils/aiter.py", line 76, in anext_impl return await __anext__(iterator) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/utils/aiter.py", line 125, in tee_peer item = await iterator.__anext__() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1471, in atransform async for output in self.astream(final, config, **kwargs): File "/opt/conda/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 494, in astream raise e File "/opt/conda/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 472, in astream async for chunk in self._astream( File "/opt/conda/lib/python3.11/site-packages/langchain_openai/chat_models/base.py", line 881, in _astream response = await self.async_client.create(**payload) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1720, in create return await self._post( ^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1849, in post return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1543, in request return await self._request( ^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1629, in _request return await self._retry_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1676, in _retry_request return await self._request( ^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1629, in _request return await self._retry_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1676, in _retry_request return await self._request( ^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/lib/python3.11/site-packages/openai/_base_client.py", line 1644, in _request raise self._make_status_error_from_response(err.response) from None openai.InternalServerError: Internal Server ErrorReproduce
Start with the base Docker image for JupyterLab 4.1.8.
Install the jupyter_ai package in the Dockerfile.
Build the Docker Image and run in a container
Verify Jupyter AI Chat in the local host
Expected behavior
The chat should work and provide the correct answer.
Context
Hello,
We are upgrading from JupyterLab 3.6.7, along with other packages, including Jupyter AI. Jupyter AI works fine with the current setup (JupyterLab 3.6.7).
However, after upgrading the packages and Jupyter AI, I am encountering an Internal Server Error for all chat-based queries. Interestingly, some commands, such as /generate a notebook about how to add 5 numbers in Python, work fine and successfully generate the notebook.
other package versions
Troubleshoot Output
Command Line Output
I 2025-01-17 21:29:33.641 ServerApp] al_server_extension | extension was successfully linked. [I 2025-01-17 21:29:33.648 ServerApp] jupyter_ai | extension was successfully linked. [I 2025-01-17 21:29:33.648 ServerApp] jupyter_lsp | extension was successfully linked. [I 2025-01-17 21:29:33.652 ServerApp] jupyter_server_terminals | extension was successfully linked. [I 2025-01-17 21:29:33.657 ServerApp] jupyterlab | extension was successfully linked. [I 2025-01-17 21:29:33.662 ServerApp] nbclassic | extension was successfully linked. [I 2025-01-17 21:29:33.667 ServerApp] notebook | extension was successfully linked. [I 2025-01-17 21:29:33.675 ServerApp] notebook_shim | extension was successfully linked. /opt/conda/lib/python3.11/site-packages/traitlets/traitlets.py:1241: UserWarning: Overriding existing pre_save_hook (custom_pre_save_hook) with a new one (custom_pre_save_hook). return self.func(*args, **kwargs) /opt/conda/lib/python3.11/site-packages/traitlets/traitlets.py:1241: UserWarning: Overriding existing post_save_hook (custom_post_save_hook) with a new one (custom_post_save_hook). return self.func(*args, **kwargs) [I 2025-01-17 21:29:33.698 ServerApp] notebook_shim | extension was successfully loaded. [I 2025-01-17 21:29:33.699 ServerApp] Registered common endpoints server extension [I 2025-01-17 21:29:33.699 ServerApp] al_server_extension | extension was successfully loaded. [I 2025-01-17 21:29:33.699 AiExtension] Configured provider allowlist: ['azure-chat-openai'] [I 2025-01-17 21:29:33.699 AiExtension] Configured provider blocklist: None [I 2025-01-17 21:29:33.699 AiExtension] Configured model allowlist: None [I 2025-01-17 21:29:33.699 AiExtension] Configured model blocklist: None [I 2025-01-17 21:29:33.700 AiExtension] Configured model parameters: {'azure-chat-openai:XXXXXXXXXX': {'azure_endpoint': 'https://XXXXXXXXXXXXXXXX/openai-proxy', 'openai_api_version': '2023-07-01-preview'}} [I 2025-01-17 21:29:33.707 AiExtension] Skipping blocked provider `ai21`. [I 2025-01-17 21:29:33.829 AiExtension] Skipping blocked provider `bedrock`. [I 2025-01-17 21:29:33.829 AiExtension] Skipping blocked provider `bedrock-chat`. [I 2025-01-17 21:29:33.829 AiExtension] Skipping blocked provider `bedrock-custom`. [I 2025-01-17 21:29:33.938 AiExtension] Skipping blocked provider `anthropic-chat`. [I 2025-01-17 21:29:34.119 AiExtension] Registered model provider `azure-chat-openai`. [I 2025-01-17 21:29:34.919 AiExtension] Skipping blocked provider `cohere`. [I 2025-01-17 21:29:35.149 AiExtension] Skipping blocked provider `gemini`. [I 2025-01-17 21:29:35.149 AiExtension] Skipping blocked provider `gpt4all`. [I 2025-01-17 21:29:35.149 AiExtension] Skipping blocked provider `huggingface_hub`. [I 2025-01-17 21:29:35.160 AiExtension] Skipping blocked provider `mistralai`. [I 2025-01-17 21:29:35.178 AiExtension] Skipping blocked provider `nvidia-chat`. [I 2025-01-17 21:29:35.244 AiExtension] Skipping blocked provider `ollama`. [I 2025-01-17 21:29:35.244 AiExtension] Skipping blocked provider `openai`. [I 2025-01-17 21:29:35.244 AiExtension] Skipping blocked provider `openai-chat`. [I 2025-01-17 21:29:35.257 AiExtension] Skipping blocked provider `openrouter`. [I 2025-01-17 21:29:35.257 AiExtension] Skipping blocked provider `qianfan`. [I 2025-01-17 21:29:35.257 AiExtension] Skipping blocked provider `sagemaker-endpoint`. [I 2025-01-17 21:29:35.257 AiExtension] Skipping blocked provider `togetherai`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `azure`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `bedrock`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `cohere`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `gpt4all`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `huggingface_hub`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `mistralai`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `ollama`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `openai`. [I 2025-01-17 21:29:35.265 AiExtension] Skipping blocked provider `qianfan`. [I 2025-01-17 21:29:35.271 AiExtension] Registered providers. [I 2025-01-17 21:29:35.271 AiExtension] Registered jupyter_ai server extension [I 2025-01-17 21:29:35.286 AiExtension] Registered context provider `file`. [I 2025-01-17 21:29:35.287 AiExtension] Initialized Jupyter AI server extension in 1588 ms. [I 2025-01-17 21:29:35.288 ServerApp] jupyter_ai | extension was successfully loaded. [I 2025-01-17 21:29:35.289 ServerApp] jupyter_lsp | extension was successfully loaded. [I 2025-01-17 21:29:35.290 ServerApp] jupyter_server_terminals | extension was successfully loaded. [I 2025-01-17 21:29:35.291 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.11/site-packages/jupyterlab [I 2025-01-17 21:29:35.291 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab [I 2025-01-17 21:29:35.291 LabApp] Extension Manager is 'pypi'. [I 2025-01-17 21:29:35.298 ServerApp] jupyterlab | extension was successfully loaded.
| | | |_ __ | | | | ___
| || | ' /
/ _| / -)_/| ./_,_,|____|
|_|
Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions.
https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html
Please note that updating to Notebook 7 might break some of your extensions.
[I 2025-01-17 21:29:35.300 ServerApp] nbclassic | extension was successfully loaded.
[I 2025-01-17 21:29:35.301 ServerApp] notebook | extension was successfully loaded.
[C 2025-01-17 21:29:35.302 ServerApp] Running as root is not recommended. Use --allow-root to bypass.
The text was updated successfully, but these errors were encountered: