jacob/5890 (#5956)

* feat: add mixedbread ai integration

* mxbai integration

* mxbai integration

* Naming correction

* Docs nits and format

---------

Co-authored-by: Julius Lipp <lipp.julius@gmail.com>
Co-authored-by: Julius Lipp <43986145+juliuslipp@users.noreply.github.com>

docs/core_docs/docs/integrations/document_compressors/mixedbread_ai.mdx

@@ -0,0 +1,54 @@
+# Mixedbread AI reranking
+
+## Overview
+
+This guide will help you integrate and use the [Mixedbread AI](https://mixedbread.ai/) reranking API. The reranking API allows you to reorder a list of documents based on a given query, improving the relevance of search results or any ranked list.
+
+## Installation
+
+To get started, install the `@langchain/mixedbread-ai` package:
+
+import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";
+
+<IntegrationInstallTooltip></IntegrationInstallTooltip>
+
+```bash
+npm install @langchain/mixedbread-ai
+```
+
+## Authentication
+
+Obtain your API key by signing up at [Mixedbread AI](https://mixedbread.ai/). You can then set the `MXBAI_API_KEY` environment variable to your Mixedbread AI API key or pass it directly as the `apiKey` option when constructing the class.
+
+## Using Reranking
+
+The `MixedbreadAIReranker` class provides access to the reranking API. Here’s how to use it:
+
+1. **Import the Class**: First, import the `MixedbreadAIReranker` class from the package.
+
+```typescript
+import { MixedbreadAIReranker } from "@langchain/mixedbread-ai";
+```
+
+2. **Instantiate the Class**: Create an instance of `MixedbreadAIReranker` with your API key.
+
+```typescript
+const reranker = new MixedbreadAIReranker({ apiKey: "your-api-key" });
+```
+
+3. **Rerank Documents**: Use the `rerankDocuments` method to reorder documents based on a query.
+
+```typescript
+const documents = [
+  { pageContent: "To bake bread you need flour" },
+  { pageContent: "To bake bread you need yeast" },
+  { pageContent: "To eat bread you need nothing but good taste" },
+];
+const query = "What do you need to bake bread?";
+const result = await reranker.compressDocuments(documents, query);
+console.log(result);
+```
+
+## Additional Resources
+
+For more information, refer to the [Reranking API documentation](https://mixedbread.ai/docs/reranking).

docs/core_docs/docs/integrations/text_embedding/mixedbread_ai.mdx

@@ -0,0 +1,85 @@
+# Mixedbread AI Embeddings
+
+The `MixedbreadAIEmbeddings` class uses the [Mixedbread AI](https://mixedbread.ai/) API to generate text embeddings. This guide will walk you through setting up and using the `MixedbreadAIEmbeddings` class, helping you integrate it into your project effectively.
+
+## Installation
+
+To install the `@langchain/mixedbread-ai` package, use the following command:
+
+import IntegrationInstallTooltip from "@mdx_components/integration_install_tooltip.mdx";
+
+<IntegrationInstallTooltip></IntegrationInstallTooltip>
+
+```bash npm2yarn
+npm install @langchain/mixedbread-ai
+```
+
+## Initialization
+
+First, sign up on the Mixedbread AI website and get your API key from [here](https://mixedbread.ai/). You can then use this key to initialize the `MixedbreadAIEmbeddings` class.
+
+You can pass the API key directly to the constructor or set it as an environment variable (`MXBAI_API_KEY`).
+
+### Basic Usage
+
+Here’s how to create an instance of `MixedbreadAIEmbeddings`:
+
+```typescript
+import { MixedbreadAIEmbeddings } from "@langchain/mixedbread-ai";
+
+const embeddings = new MixedbreadAIEmbeddings({
+  apiKey: "YOUR_API_KEY",
+  // Optionally specify model
+  // model: "mixedbread-ai/mxbai-embed-large-v1",
+});
+```
+
+If the `apiKey` is not provided, it will be read from the `MXBAI_API_KEY` environment variable.
+
+## Generating Embeddings
+
+### Embedding a Single Query
+
+To generate embeddings for a single text query, use the `embedQuery` method:
+
+```typescript
+const embedding = await embeddings.embedQuery(
+  "Represent this sentence for searching relevant passages: Is baking fun?"
+);
+console.log(embedding);
+```
+
+### Embedding Multiple Documents
+
+To generate embeddings for multiple documents, use the `embedDocuments` method. This method handles batching automatically based on the `batchSize` parameter:
+
+```typescript
+const documents = ["Baking bread is fun", "I love baking"];
+
+const embeddingsArray = await embeddings.embedDocuments(documents);
+console.log(embeddingsArray);
+```
+
+## Customizing Requests
+
+You can customize the SDK by passing additional parameters.
+
+```typescript
+const customEmbeddings = new MixedbreadAIEmbeddings({
+  apiKey: "YOUR_API_KEY",
+  baseUrl: "...",
+  maxRetries: 6,
+});
+```
+
+## Error Handling
+
+If the API key is not provided and cannot be found in the environment variables, an error will be thrown:
+
+```typescript
+try {
+  const embeddings = new MixedbreadAIEmbeddings();
+} catch (error) {
+  console.error(error);
+}
+```

libs/langchain-mixedbread-ai/.eslintrc.cjs

@@ -0,0 +1,75 @@
+module.exports = {
+  extends: [
+    "airbnb-base",
+    "eslint:recommended",
+    "prettier",
+    "plugin:@typescript-eslint/recommended",
+  ],
+  parserOptions: {
+    ecmaVersion: 12,
+    parser: "@typescript-eslint/parser",
+    project: "./tsconfig.json",
+    sourceType: "module",
+  },
+  plugins: ["@typescript-eslint", "no-instanceof", "eslint-plugin-jest"],
+  ignorePatterns: [
+    ".eslintrc.cjs",
+    "scripts",
+    "node_modules",
+    "dist",
+    "dist-cjs",
+    "*.js",
+    "*.cjs",
+    "*.d.ts",
+  ],
+  rules: {
+    "no-process-env": 2,
+    "no-instanceof/no-instanceof": 2,
+    "@typescript-eslint/explicit-module-boundary-types": 0,
+    "@typescript-eslint/no-empty-function": 0,
+    "@typescript-eslint/no-shadow": 0,
+    "@typescript-eslint/no-empty-interface": 0,
+    "@typescript-eslint/no-use-before-define": ["error", "nofunc"],
+    "@typescript-eslint/no-unused-vars": ["warn", { args: "none" }],
+    "@typescript-eslint/no-floating-promises": "error",
+    "@typescript-eslint/no-misused-promises": "error",
+    camelcase: 0,
+    "class-methods-use-this": 0,
+    "import/extensions": [2, "ignorePackages"],
+    "import/no-extraneous-dependencies": [
+      "error",
+      { devDependencies: ["**/*.test.ts"] },
+    ],
+    "import/no-unresolved": 0,
+    "import/prefer-default-export": 0,
+    "keyword-spacing": "error",
+    "max-classes-per-file": 0,
+    "max-len": 0,
+    "no-await-in-loop": 0,
+    "no-bitwise": 0,
+    "no-console": 0,
+    "no-restricted-syntax": 0,
+    "no-shadow": 0,
+    "no-continue": 0,
+    "no-void": 0,
+    "no-underscore-dangle": 0,
+    "no-use-before-define": 0,
+    "no-useless-constructor": 0,
+    "no-return-await": 0,
+    "consistent-return": 0,
+    "no-else-return": 0,
+    "func-names": 0,
+    "no-lonely-if": 0,
+    "prefer-rest-params": 0,
+    'jest/no-focused-tests': 'error',
+    "new-cap": ["error", { properties: false, capIsNew: false }],
+  },
+  overrides: [
+    {
+      files: ['**/*.test.ts'],
+      rules: {
+        '@typescript-eslint/no-unused-vars': 'off'
+      }
+    }
+  ]
+};

libs/langchain-mixedbread-ai/.gitignore

@@ -0,0 +1,7 @@
+index.cjs
+index.js
+index.d.ts
+index.d.cts
+node_modules
+dist
+.yarn

libs/langchain-mixedbread-ai/.prettierrc

@@ -0,0 +1,19 @@
+{
+  "$schema": "https://json.schemastore.org/prettierrc",
+  "printWidth": 80,
+  "tabWidth": 2,
+  "useTabs": false,
+  "semi": true,
+  "singleQuote": false,
+  "quoteProps": "as-needed",
+  "jsxSingleQuote": false,
+  "trailingComma": "es5",
+  "bracketSpacing": true,
+  "arrowParens": "always",
+  "requirePragma": false,
+  "insertPragma": false,
+  "proseWrap": "preserve",
+  "htmlWhitespaceSensitivity": "css",
+  "vueIndentScriptAndStyle": false,
+  "endOfLine": "lf"
+}

libs/langchain-mixedbread-ai/.release-it.json

@@ -0,0 +1,12 @@
+{
+  "github": {
+    "release": true,
+    "autoGenerate": true,
+    "tokenRef": "GITHUB_TOKEN_RELEASE"
+  },
+  "npm": {
+    "versionArgs": [
+      "--workspaces-update=false"
+    ]
+  }
+}

libs/langchain-mixedbread-ai/LICENSE

@@ -0,0 +1,21 @@
+The MIT License
+
+Copyright (c) 2023 LangChain
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

libs/langchain-mixedbread-ai/README.md

@@ -0,0 +1,87 @@
+# @langchain/mixedbread-ai
+
+This package contains the LangChain.js integrations for the [Mixedbread AI API](https://mixedbread.ai/).
+
+## Installation
+
+```bash
+npm install @langchain/mixedbread-ai
+```
+
+This package, along with the main LangChain package, depends on [`@langchain/core`](https://npmjs.com/package/@langchain/core/). If you are using this package with other LangChain packages, you should make sure that all of the packages depend on the same instance of `@langchain/core`.
+
+## Authentication
+
+To use this package, you need a Mixedbread AI API key. You can obtain your API key by signing up at [Mixedbread AI](https://mixedbread.ai).
+
+Either set the `MXBAI_API_KEY` environment variable to your Mixedbread AI API key, or pass it as the `apiKey` option to the constructor of the class you are using.
+
+## Embeddings
+
+This package provides access to the different embedding models provided by the Mixedbread AI API, such as the "mixedbread-ai/mxbai-embed-large-v1" model.
+
+Learn more: [Embeddings API](https://mixedbread.ai/docs/embeddings)
+
+```typescript
+const embeddings = new MixedbreadAIEmbeddings({ apiKey: 'your-api-key' });
+const texts = ["Baking bread is fun", "I love baking"];
+const result = await embeddings.embedDocuments(texts);
+console.log(result);
+```
+
+## Reranking
+
+This package provides access to the reranking API provided by Mixedbread AI. It allows you to rerank a list of documents based on a query. Available models include "mixedbread-ai/mxbai-rerank-large-v1".
+
+Learn more: [Reranking API](https://mixedbread.ai/docs/reranking)
+
+```typescript
+const reranker = new MixedbreadAIReranker({ apiKey: 'your-api-key' });
+const documents = [{ pageContent: "To bake bread you need flour" }, { pageContent: "To bake bread you need yeast" }];
+const query = "What do you need to bake bread?";
+const result = await reranker.compressDocuments(documents, query);
+console.log(result);
+```
+
+## Development
+
+To develop the `@langchain/mixedbread-ai` package, follow these instructions:
+
+### Install dependencies
+
+```bash
+yarn install
+```
+
+### Build the package
+
+```bash
+yarn build
+```
+
+Or from the repo root:
+
+```bash
+yarn build --filter=@langchain/mixedbread-ai
+```
+
+### Run tests
+
+Test files should live within a `tests/` folder in the `src/` directory. Unit tests should end in `.test.ts` and integration tests should end in `.int.test.ts`:
+
+```bash
+yarn test
+yarn test:int
+```
+
+### Lint & Format
+
+Run the linter & formatter to ensure your code is up to standard:
+
+```bash
+yarn lint && yarn format
+```
+
+### Adding new entry points
+
+If you add a new file to be exported, either import & re-export from `src/index.ts`, or add it to the `entrypoints` field in the `config` variable located inside `langchain.config.js` and run `yarn build` to generate the new entry point.

libs/langchain-mixedbread-ai/jest.config.cjs

@@ -0,0 +1,21 @@
+/** @type {import('ts-jest').JestConfigWithTsJest} */
+module.exports = {
+  preset: "ts-jest/presets/default-esm",
+  testEnvironment: "./jest.env.cjs",
+  modulePathIgnorePatterns: ["dist/", "docs/"],
+  moduleNameMapper: {
+    "^(\\.{1,2}/.*)\\.js$": "$1",
+  },
+  transform: {
+    "^.+\\.tsx?$": ["@swc/jest"],
+  },
+  transformIgnorePatterns: [
+    "/node_modules/",
+    "\\.pnp\\.[^\\/]+$",
+    "./scripts/jest-setup-after-env.js",
+  ],
+  setupFiles: ["dotenv/config"],
+  testTimeout: 20_000,
+  passWithNoTests: true,
+  collectCoverageFrom: ["src/**/*.ts"],
+};