Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update faiss-cpu version range #1097

Merged
merged 1 commit into from
Nov 11, 2024
Merged

Update faiss-cpu version range #1097

merged 1 commit into from
Nov 11, 2024

Conversation

dlqqq
Copy link
Member

@dlqqq dlqqq commented Nov 8, 2024 •
edited
Loading

Description

This PR updates the version range of faiss-cpu to:

  1. Include all versions with Python 3.12 support (Python 3.12 not supported #538)
  2. Exclude versions lacking wheels for all supported platforms (Jupyter AI installation failing on x86 macOS #858)
  3. Allow for newer minor releases of faiss-cpu~=1.0, notably faiss-cpu==1.9.0.

After this PR is merged and released in a patch version, we can update the dependency in the Conda Forge recipe similarly to close #1096.

Testing instructions

  1. Run jlpm dev-uninstall && jlpm dev-install.
  2. Run pip install -U faiss-cpu.
  3. Run pip show faiss-cpu, then verify that the latest release v1.9.0 is installed.
  4. Verify /learn and /ask still work in Jupyter AI.

@dlqqq dlqqq added bug Something isn't working dependency:faiss-cpu Issues pertaining to `faiss-cpu` labels Nov 8, 2024
@krassowski krassowski mentioned this pull request Nov 11, 2024
Closed
@krassowski krassowski linked an issue Nov 11, 2024 that may be closed by this pull request
jupyter-ai2.28 still does not support python3.13.0 #1098
Closed
@krassowski
Copy link
Member

Great! But why is faiss-cpu a required dependency in the first place?

@dlqqq
Copy link
Member Author

dlqqq commented Nov 11, 2024

@krassowski Good callout. The original line of thought was that /learn was a built-in slash command, and therefore it had to work "out-of-the-box". Therefore we listed faiss-cpu as a required dependency.

I agree that we should explore making faiss-cpu an optional dependency. This would likely be considered a breaking change for admins deploying Jupyter AI as a service (since users may not have permissions to install faiss-cpu and restart the server). I'll open an issue for this and add it to the v3.0.0 roadmap.

srdas
srdas approved these changes Nov 11, 2024
View reviewed changes
Copy link
Collaborator

@srdas srdas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested all steps in the PR.

  1. Version = faiss-cpi 1.9.0
  2. /learn works
  3. /ask works

@dlqqq
Copy link
Member Author

dlqqq commented Nov 11, 2024

@srdas Verified locally as well. Thank you for verifying this PR on your machine.

@krassowski I've opened a new issue to track making faiss-cpu optional: #1100

@dlqqq dlqqq merged commit d5d604d into jupyterlab:main Nov 11, 2024
13 checks passed
@dlqqq
Copy link
Member Author

dlqqq commented Nov 11, 2024

@meeseeksdev please backport to v3-dev

meeseeksmachine pushed a commit to meeseeksmachine/jupyter-ai that referenced this pull request Nov 11, 2024
@dlqqq @meeseeksmachine
Backport PR jupyterlab#1097: Update faiss-cpu version range
cefd30d
@meeseeksmachine meeseeksmachine mentioned this pull request Nov 11, 2024
Merged
dlqqq added a commit that referenced this pull request Nov 11, 2024
@meeseeksmachine @dlqqq
Backport PR #1097: Update faiss-cpu version range (#1101)
94ee002 
Co-authored-by: david qiu <david@qiu.dev>
dlqqq added a commit that referenced this pull request Dec 26, 2024
@dlqqq @meeseeksmachine
@srdas @pre-commit-ci @alanmeeson @mcavdar @ctcjab @JanusChoi @brichet @github-actions @divyansshhh
V3: The Beginning (#1169)
899af0a 
* Backport PR #1049: Added new Anthropic Sonnet3.5 v2 models (#1050)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1051: Added Developer documentation for streaming responses (#1058)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1048: Implement streaming for `/fix` (#1059)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1057: [pre-commit.ci] pre-commit autoupdate (#1060)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Backport PR #1064: Added Ollama to the providers table in user docs (#1066)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1056: Add examples of using Fields and EnvAuthStrategy to developer documentation (#1073)

Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>

* Backport PR #1069: Merge Anthropic language model providers (#1076)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1068: Allow `$` to literally denote quantities of USD in chat (#1079)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1075: Fix magic commands when using non-chat providers w/ history (#1080)

Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>

* Backport PR #1077: Fix `/export` by including streamed agent messages (#1081)

Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com>

* Backport PR #1072: Reduced padding in cell around code icons in code toolbar (#1084)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1087: Improve installation documentation and clarify provider dependencies (#1091)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1092: Remove retired models and add new `Haiku-3.5` model in Anthropic (#1093)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1094: Continue to allow `$` symbols to delimit inline math in human messages (#1095)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1097: Update `faiss-cpu` version range (#1101)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1104: Fix rendering of code blocks in JupyterLab 4.3.0+ (#1105)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1106: Catch error on non plaintext files in `@file` and reply gracefully in chat (#1110)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1109: Bump LangChain minimum versions (#1112)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1119: Downgrade spurious 'error' logs (#1124)

Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com>

* Backport PR #1127: Removes outdated OpenAI models and adds new ones (#1130)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1131: [pre-commit.ci] pre-commit autoupdate (#1132)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Backport PR #1125: Update model fields immediately on save (#1133)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1139: Fix install step in CI (#1140)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1129: Fix JSON serialization error in Ollama models (#1141)

Co-authored-by: Mr.W <janus.choy@gmail.com>

* Backport PR #1137: Update completion model fields immediately on save (#1142)

Co-authored-by: david qiu <david@qiu.dev>

* [v3-dev] Initial migration to `jupyterlab-chat` (#1043)

* Very first version of the AI working in jupyterlab_collaborative_chat

* Allows both collaborative and regular chat to work with AI

* handle the help message in the chat too

* Autocompletion (#2)

* Fix handler methods' parameters

* Add slash commands (autocompletion) to the chat input

* Stream messages (#3)

* Allow for stream messages

* update jupyter collaborative chat dependency

* AI settings (#4)

* Add a menu option to open the AI settings

* Remove the input option from the setting widget

* pre-commit

* linting

* Homogeneize typing for optional arguments

* Fix import

* Showing that the bot is writing (answering) (#5)

* Show that the bot is writing (answering)

* Update jupyter chat dependency

* Some typing

* Update extension to jupyterlab_chat (0.6.0) (#8)

* Fix linting

* Remove try/except to import jupyterlab_chat (not optional anymore), and fix typing

* linter

* Python unit tests

* Fix typing

* lint

* Fix lint and mypy all together

* Fix web_app settings accessor

* Fix jupyter_collaboration version

Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com>

* Remove unecessary try/except

* Dedicate one set of chat handlers per room (#9)

* create new set of chat handlers per room

* make YChat an instance attribute on BaseChatHandler

* revert changes to chat handlers

* pre-commit

* use room_id local var

Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>

---------

Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>

---------

Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com>
Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1134: Improve user messaging and documentation for Cross-Region Inference on Amazon Bedrock (#1143)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR #1136: Add base API URL field for Ollama and OpenAI embedding models (#1149)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* [v3-dev] Remove `/export`, `/clear`, and `/fix` (#1148)

* remove /export

* remove /clear

* remove /fix

* Fix CI in `v3-dev` branch (#1154)

* fix check release by bumping to impossible version

* fix types

* Update Playwright Snapshots

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* [v3-dev] Dedicate one LangChain history object per chat (#1151)

* dedicate a separate LangChain history object per chat

* pre-commit

* fix mypy

* Backport PR #1160: Trigger update snapshots based on commenter's role (#1161)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR #1155: Fix code output format in IPython (#1162)

Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>

* Backport PR #1158: Update `/generate` to not split classes & functions across cells (#1164)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Remove v2 frontend components (#1156)

* First pass to remove the front end chat

* Remove code-toolbar by using a simplified markdown renderer in settings

* Remove chat-message-menu (should be ported in jupyter-chat)

* Remove chat handler

* Follow up 'Remove chat-message-menu (should be ported in jupyter-chat)' commit

* Clean package.json

* Remove UI tests

* Remove the generative AI menu

* Remove unused components

* run yarn dedupe

---------

Co-authored-by: David L. Qiu <david@qiu.dev>

* Upgrade to `jupyterlab-chat>=0.7.0` (#1166)

* upgrade to jupyterlab-chat 0.7.0

* pre-commit

* upgrade to @jupyter/chat ^0.7.0 in frontend

* Remove v2 backend components (#1168)

* remove v2 llm memory, implement ReplyStream

* remove v2 websockets & REST handlers

* remove unused v2 data models

* fix slash command autocomplete

* fix unit tests

* remove unused _learned context provider

* fix mypy

* pre-commit

* fix optional k arg in YChatHistory

* bump jupyter chat to 0.7.1 to fix Python 3.9 tests

* revert accidentally breaking /learn

---------

Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com>
Co-authored-by: Sanjiv Das <srdas@scu.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>
Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com>
Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com>
Co-authored-by: Mr.W <janus.choy@gmail.com>
Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>
srdas pushed a commit to srdas/jupyter-ai that referenced this pull request Jan 6, 2025
@dlqqq @srdas
Update faiss-cpu version range (jupyterlab#1097)
3133ce9
srdas added a commit to srdas/jupyter-ai that referenced this pull request Jan 6, 2025
@dlqqq @meeseeksmachine
@srdas @pre-commit-ci @alanmeeson @mcavdar @ctcjab @JanusChoi @brichet @github-actions @divyansshhh
V3: The Beginning (jupyterlab#1169)
4d5bbfc 
* Backport PR jupyterlab#1049: Added new Anthropic Sonnet3.5 v2 models (jupyterlab#1050)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1051: Added Developer documentation for streaming responses (jupyterlab#1058)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1048: Implement streaming for `/fix` (jupyterlab#1059)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1057: [pre-commit.ci] pre-commit autoupdate (jupyterlab#1060)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Backport PR jupyterlab#1064: Added Ollama to the providers table in user docs (jupyterlab#1066)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1056: Add examples of using Fields and EnvAuthStrategy to developer documentation (jupyterlab#1073)

Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>

* Backport PR jupyterlab#1069: Merge Anthropic language model providers (jupyterlab#1076)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1068: Allow `$` to literally denote quantities of USD in chat (jupyterlab#1079)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1075: Fix magic commands when using non-chat providers w/ history (jupyterlab#1080)

Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>

* Backport PR jupyterlab#1077: Fix `/export` by including streamed agent messages (jupyterlab#1081)

Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com>

* Backport PR jupyterlab#1072: Reduced padding in cell around code icons in code toolbar (jupyterlab#1084)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1087: Improve installation documentation and clarify provider dependencies (jupyterlab#1091)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1092: Remove retired models and add new `Haiku-3.5` model in Anthropic (jupyterlab#1093)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1094: Continue to allow `$` symbols to delimit inline math in human messages (jupyterlab#1095)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1097: Update `faiss-cpu` version range (jupyterlab#1101)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1104: Fix rendering of code blocks in JupyterLab 4.3.0+ (jupyterlab#1105)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1106: Catch error on non plaintext files in `@file` and reply gracefully in chat (jupyterlab#1110)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1109: Bump LangChain minimum versions (jupyterlab#1112)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1119: Downgrade spurious 'error' logs (jupyterlab#1124)

Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com>

* Backport PR jupyterlab#1127: Removes outdated OpenAI models and adds new ones (jupyterlab#1130)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1131: [pre-commit.ci] pre-commit autoupdate (jupyterlab#1132)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Backport PR jupyterlab#1125: Update model fields immediately on save (jupyterlab#1133)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1139: Fix install step in CI (jupyterlab#1140)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1129: Fix JSON serialization error in Ollama models (jupyterlab#1141)

Co-authored-by: Mr.W <janus.choy@gmail.com>

* Backport PR jupyterlab#1137: Update completion model fields immediately on save (jupyterlab#1142)

Co-authored-by: david qiu <david@qiu.dev>

* [v3-dev] Initial migration to `jupyterlab-chat` (jupyterlab#1043)

* Very first version of the AI working in jupyterlab_collaborative_chat

* Allows both collaborative and regular chat to work with AI

* handle the help message in the chat too

* Autocompletion (jupyterlab#2)

* Fix handler methods' parameters

* Add slash commands (autocompletion) to the chat input

* Stream messages (jupyterlab#3)

* Allow for stream messages

* update jupyter collaborative chat dependency

* AI settings (jupyterlab#4)

* Add a menu option to open the AI settings

* Remove the input option from the setting widget

* pre-commit

* linting

* Homogeneize typing for optional arguments

* Fix import

* Showing that the bot is writing (answering) (jupyterlab#5)

* Show that the bot is writing (answering)

* Update jupyter chat dependency

* Some typing

* Update extension to jupyterlab_chat (0.6.0) (jupyterlab#8)

* Fix linting

* Remove try/except to import jupyterlab_chat (not optional anymore), and fix typing

* linter

* Python unit tests

* Fix typing

* lint

* Fix lint and mypy all together

* Fix web_app settings accessor

* Fix jupyter_collaboration version

Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com>

* Remove unecessary try/except

* Dedicate one set of chat handlers per room (jupyterlab#9)

* create new set of chat handlers per room

* make YChat an instance attribute on BaseChatHandler

* revert changes to chat handlers

* pre-commit

* use room_id local var

Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>

---------

Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>

---------

Co-authored-by: david qiu <44106031+dlqqq@users.noreply.github.com>
Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1134: Improve user messaging and documentation for Cross-Region Inference on Amazon Bedrock (jupyterlab#1143)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Backport PR jupyterlab#1136: Add base API URL field for Ollama and OpenAI embedding models (jupyterlab#1149)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* [v3-dev] Remove `/export`, `/clear`, and `/fix` (jupyterlab#1148)

* remove /export

* remove /clear

* remove /fix

* Fix CI in `v3-dev` branch (jupyterlab#1154)

* fix check release by bumping to impossible version

* fix types

* Update Playwright Snapshots

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* [v3-dev] Dedicate one LangChain history object per chat (jupyterlab#1151)

* dedicate a separate LangChain history object per chat

* pre-commit

* fix mypy

* Backport PR jupyterlab#1160: Trigger update snapshots based on commenter's role (jupyterlab#1161)

Co-authored-by: david qiu <david@qiu.dev>

* Backport PR jupyterlab#1155: Fix code output format in IPython (jupyterlab#1162)

Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>

* Backport PR jupyterlab#1158: Update `/generate` to not split classes & functions across cells (jupyterlab#1164)

Co-authored-by: Sanjiv Das <srdas@scu.edu>

* Remove v2 frontend components (jupyterlab#1156)

* First pass to remove the front end chat

* Remove code-toolbar by using a simplified markdown renderer in settings

* Remove chat-message-menu (should be ported in jupyter-chat)

* Remove chat handler

* Follow up 'Remove chat-message-menu (should be ported in jupyter-chat)' commit

* Clean package.json

* Remove UI tests

* Remove the generative AI menu

* Remove unused components

* run yarn dedupe

---------

Co-authored-by: David L. Qiu <david@qiu.dev>

* Upgrade to `jupyterlab-chat>=0.7.0` (jupyterlab#1166)

* upgrade to jupyterlab-chat 0.7.0

* pre-commit

* upgrade to @jupyter/chat ^0.7.0 in frontend

* Remove v2 backend components (jupyterlab#1168)

* remove v2 llm memory, implement ReplyStream

* remove v2 websockets & REST handlers

* remove unused v2 data models

* fix slash command autocomplete

* fix unit tests

* remove unused _learned context provider

* fix mypy

* pre-commit

* fix optional k arg in YChatHistory

* bump jupyter chat to 0.7.1 to fix Python 3.9 tests

* revert accidentally breaking /learn

---------

Co-authored-by: Lumberbot (aka Jack) <39504233+meeseeksmachine@users.noreply.github.com>
Co-authored-by: Sanjiv Das <srdas@scu.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Alan Meeson <alanmeeson@users.noreply.github.com>
Co-authored-by: Mahmut CAVDAR <4072246+mcavdar@users.noreply.github.com>
Co-authored-by: ctcjab <joshua.bronson@chicagotrading.com>
Co-authored-by: Mr.W <janus.choy@gmail.com>
Co-authored-by: Nicolas Brichet <32258950+brichet@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Divyansh Choudhary <divyanshchoudhary99@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srdas srdas srdas approved these changes

Assignees
No one assigned
Labels
bug Something isn't working dependency:faiss-cpu Issues pertaining to `faiss-cpu`
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

jupyter-ai2.28 still does not support python3.13.0 conda installing jupyter-ai fails with TypeError
3 participants
@dlqqq @krassowski @srdas