Skip to content

Commit

Permalink
Merge pull request #845 from MicrosoftDocs/main
Browse files Browse the repository at this point in the history
11/19 Publish
  • Loading branch information
huypub authored Nov 19, 2024
2 parents 52d2fcb + c7b2d22 commit b3e7fd6
Show file tree
Hide file tree
Showing 103 changed files with 4,405 additions and 723 deletions.
10 changes: 8 additions & 2 deletions articles/cosmos-db/TOC.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,12 @@
items:
- name: What is a vector database
href: vector-database.md
- name: Vector database in Azure Cosmos DB NoSQL
- name: Vector database in Azure Cosmos DB for NoSQL
href: nosql/vector-search.md
- name: Full text search in Azure Cosmos DB for NoSQL
href: gen-ai/full-text-search.md
- name: Hybrid search in Azure Cosmos DB for NoSQL
href: gen-ai/hybrid-search.md
- name: Vector database in Azure Cosmos DB for MongoDB
href: mongodb/vcore/vector-search.md
- name: AI Integrations
Expand Down Expand Up @@ -51,14 +55,16 @@
href: gen-ai/document-ingestion.md
- name: Cosmos AI Graph - Graph RAG
href: gen-ai/cosmos-ai-graph.md
- name: AI agent
- name: AI agents
href: ai-agents.md
- name: Real-time custom content generation
href: mongodb/vcore/ai-advertisement-generation.md
- name: Azure AI Advantage free trial
href: ai-advantage.md
expanded: true
expanded: true
- name: Visual Studio Code extension
href: visual-studio-code-extension.md
- name: NoSQL
href: nosql/toc.yml
- name: MongoDB
Expand Down
6 changes: 3 additions & 3 deletions articles/cosmos-db/ai-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,9 +169,9 @@ In place of all the standalone databases, Azure Cosmos DB can serve as a unified

### Speed

Azure Cosmos DB provides single-digit millisecond latency. This capability makes it suitable for processes that require rapid data access and management. These processes include caching (both traditional and [semantic caching](https://techcommunity.microsoft.com/t5/azure-architecture-blog/optimize-azure-openai-applications-with-semantic-caching/ba-p/4106867)), transactions, and operational workloads.
Azure Cosmos DB provides single-digit millisecond latency. This capability makes it suitable for processes that require rapid data access and management. These processes include caching (both traditional and semantic caching, transactions, and operational workloads.

Low latency is crucial for AI agents that need to perform complex reasoning, make real-time decisions, and provide immediate responses. In addition, the service's [use of the DiskANN algorithm](nosql/vector-search.md#enroll-in-the-vector-search-preview-feature) provides accurate and fast vector search with minimal memory consumption.
Low latency is crucial for AI agents that need to perform complex reasoning, make real-time decisions, and provide immediate responses. In addition, the service's [use of the DiskANN algorithm](nosql/vector-search.md#enable-the-vector-indexing-and-search-feature) provides accurate and fast vector search with minimal memory consumption.

### Scale

Expand Down Expand Up @@ -314,7 +314,7 @@ Here's the output of *main.py*:

### Build the AI travel agent by using Python FastAPI

The AI travel agent is hosted in a back end API through Python FastAPI, which facilitates integration with the front-end user interface. The API project processes agent requests by [grounding](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857) the LLM prompts against the data layer, specifically the vectors and documents in Azure Cosmos DB.
The AI travel agent is hosted in a back end API through Python FastAPI, which facilitates integration with the front-end user interface. The API project processes agent requests by grounding the LLM prompts against the data layer, specifically the vectors and documents in Azure Cosmos DB.

The agent makes use of various tools, particularly the Python functions provided at the API service layer. This article focuses on the code necessary for AI agents within the API code.

Expand Down
138 changes: 138 additions & 0 deletions articles/cosmos-db/emulator-linux.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
title: Linux-based emulator (preview)
titleSuffix: Azure Cosmos DB for NoSQL
description: Use the Azure Cosmos DB Linux-based emulator to test your applications against API for NoSQL endpoints.
author: TheovanKraay
ms.author: thvankra
ms.service: azure-cosmos-db
ms.topic: how-to
ms.date: 11/07/2024
# CustomerIntent: As a developer, I want to use the Linux-based Azure Cosmos DB emulator so that I can develop my application against a database during development.
---

# Linux-based emulator (preview)

The next generation of the Azure Cosmos DB emulator is entirely Linux-based and is available as a Docker container. It supports running on a wide variety of processors and operating systems.

> [!IMPORTANT]
> This version of the emulator only supports the API for NoSQL in [gateway mode](nosql/sdk-connection-modes.md#available-connectivity-modes), with a select subset of features. For more information, see [feature support](#feature-support).
## Prerequisites

- [Docker](https://www.docker.com/)

## Installation

Get the Docker container image using `docker pull`. The container image is published to the [Microsoft Artifact Registry](https://mcr.microsoft.com/) as `mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview`.

```bash
docker pull mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview
```

## Running

To run the container, use `docker run`. Afterwards, use `docker ps` to validate that the container is running.

```bash
docker run --detach --publish 8081:8081 --publish 1234:1234 mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview

docker ps
```

```output
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c1bb8cf53f8a mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview "/bin/bash -c /home/…" 5 seconds ago Up 5 seconds 0.0.0.0:1234->1234/tcp, :::1234->1234/tcp, 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp <container-name>
```

> [!NOTE]
> The emulator is comprised of two components:
>
> - **Data explorer** - interactively explore the data in the emulator. By default this runs on port `1234`
> - **Azure Cosmos DB emulator** - a local version of the Azure Cosmos DB database service. By default, this runs on port `8081`.
>
> The emulator gateway endpoint is typically available on port `8081` at the address <http://localhost:8081>. To navigate to the data explorer, use the address <http://localhost:1234> in your web browser. It may take a few seconds for data explorer to be available. The gateway endpoint is typically available immediately.
## Docker commands

The following table summarizes the available Docker commands for configuring the emulator. This table details the corresponding arguments, environment variables, allowed values, default settings, and descriptions of each command.

| Requirement | Arg | Env | Allowed values | Default | Description |
|---|---|---|---|---|---|
| Print the settings to stdout from the container | `--help`, `-h` | N/A | N/A | N/A | Display information on available configuration |
| Set the port of the Cosmos endpoint | `--port [INT]` | PORT | INT | 8081 | The port of the Cosmos endpoint on the container. You still need to publish this port (for example, `-p 8081:8081`). |
| Specify the protocol used by the Cosmos endpoint | `--protocol` | PROTOCOL | `https`, `http`, `https-insecure` | `http` | The protocol of the Cosmos endpoint on the container. |
| Enable the data explorer | `--enable-explorer` | ENABLE_EXPLORER | `true`, `false` | `true` | Enable running the Cosmos Data Explorer on the same container. |
| Set the port used by the data explorer | `--explorer-port` | EXPLORER_PORT | INT | 1234 | The port of the Cosmos Data Explorer on the container. You still need to publish this port (for example, `-p 1234:1234`). |
| User should be able to specify the protocol used by the explorer, otherwise default to what the Cosmos endpoint is using | `--explorer-protocol` | EXPLORER_PROTOCOL | `https`, `http`, `https-insecure` | `<the value of --protocol>` | The protocol of the Cosmos Data Explorer on the container. Defaults to the protocol setting on the Cosmos endpoint. |
| Specify the key via file | `--key-file [PATH]` | KEY_FILE | PATH | `<default secret>` | Override default key with the key specified in the file. You need to mount this file into the container (for example, if KEY_FILE=/mykey, you'd add an option like the following to your docker run: `--mount type=bind,source=./myKey,target=/myKey`) |
| Set the data path | `--data-path [PATH]` | DATA_PATH | PATH | `/data` | Specify a directory for data. Frequently used with `docker run --mount` option (for example, if DATA_PATH=/usr/cosmos/data, you'd add an option like the following to your docker run: `--mount type=bind,source=./.local/data,target=/usr/cosmos/data`) |
| Specify the cert path to be used for https | `--cert-path [PATH]` | CERT_PATH | PATH | `<default cert>` | Specify a path to a certificate for securing traffic. You need to mount this file into the container (for example, if CERT_PATH=/mycert.pfx, you'd add an option like the following to your docker run: `--mount type=bind,source=./mycert.pfx,target=/mycert.pfx`) |
| Specify the cert secret to be used for https | N/A | CERT_SECRET | string | `<default secret>` | The secret for the certificate specified on CERT_PATH. |
| Set the log level | `--log-level [LEVEL]` | LOG_LEVEL | `quiet`, `error`, `warn`, `info`, `debug`, `trace` | `info` | The verbosity of logs that emitted by the emulator and data explorer. |
| Enable diagnostic info being sent to Microsoft | `--enable-telemetry` | ENABLE_TELEMETRY | `true`, `false` | `true` | Enable sending logs to Microsoft to help us improve the emulator. |

## Feature support

This emulator is in active development and preview. As a result, not all Azure Cosmos DB features are supported. Some features will also not be supported in the future. This table includes the state of various features and their level of support.

| | Support |
|---|---|
| **Create database** | ✅ Supported |
| **Read database** | ✅ Supported |
| **Delete database** | ✅ Supported |
| **Read database feed** | ✅ Supported |
| **Create database twice conflict** | ✅ Supported |
| **Create collection** | ✅ Supported |
| **Read collection** | ✅ Supported |
| **Update collection** | ✅ Supported |
| **Delete collection** | ✅ Supported |
| **Read collection feed** | ✅ Supported |
| **Create collection twice conflict** | ✅ Supported |
| **Create collection with custom index policy** | ✅ Supported |
| **Create collection with ttl expiration** | ✅ Supported |
| **Create partitioned collection** | ✅ Supported |
| **Get and change collection performance** | ✅ Supported |
| **Create document** | ✅ Supported |
| **Read document** | ✅ Supported |
| **Update document** | ✅ Supported |
| **Patch document** | ✅ Supported |
| **Delete document** | ✅ Supported |
| **Read document feed** | ✅ Supported |
| **Insert large document** | ✅ Supported |
| **Create and read document with utf data** | ✅ Supported |
| **Query with sql query spec** | ✅ Supported |
| **Query with equality** | ✅ Supported |
| **Query with and filter and projection** | ⚠️ Not yet implemented |
| **Query with and filter** | ⚠️ Not yet implemented |
| **Query with equals on id** | ✅ Supported |
| **Query with inequality** | ⚠️ Not yet implemented |
| **Query with range operators on numbers** | ⚠️ Not yet implemented |
| **Query with range operators on strings** | ⚠️ Not yet implemented |
| **Query with range operators date times** | ⚠️ Not yet implemented |
| **Query with order by** | ✅ Supported |
| **Query with order by numbers** | ✅ Supported |
| **Query with order by strings** | ⚠️ Not yet implemented |
| **Query with aggregates** | ⚠️ Not yet implemented |
| **Query with subdocuments** | ⚠️ Not yet implemented |
| **Query with joins** | ⚠️ Not yet implemented |
| **Query with two joins** | ⚠️ Not yet implemented |
| **Query with two joins and filter** | ⚠️ Not yet implemented |
| **Query with single join** | ⚠️ Not yet implemented |
| **Query with string math and array operators** | ⚠️ Not yet implemented |
| **Query with paging** | ⚠️ Not yet implemented |
| **Query partitioned collection in parallel** | ⚠️ Not yet implemented |
| **Query with order by for partitioned collection** | ⚠️ Not yet implemented |
| **Stored procedure** | ❌ Not planned |
| **Triggers** | ❌ Not planned |
| **UDFs** | ❌ Not planned |

## Limitations

In addition to features not yet supported or not planned, the following list includes current limitations of the emulator.

- The .NET SDK for Azure Cosmos DB doesn't support bulk execution in the emulator.
- The .NET SDK doesn't support HTTP mode in the emulator.

## Reporting issues

If you encounter issues with using this version of the emulator, open an issue in the GitHub repository (<https://github.com/Azure/azure-cosmos-db-emulator-docker>) and tag it with the label `cosmosEmulatorVnextPreview`.
12 changes: 8 additions & 4 deletions articles/cosmos-db/emulator.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,28 @@ author: sajeetharan
ms.author: sasinnat
ms.service: azure-cosmos-db
ms.topic: concept-article
ms.date: 10/09/2024
ms.date: 11/07/2024
# CustomerIntent: As a developer, I want to use the Azure Cosmos DB emulator so that I can develop my application against a database during development.
---


# What is the Azure Cosmos DB emulator?

The Azure Cosmos DB emulator provides a local environment that emulates the Azure Cosmos DB service designed for development purposes. Using the emulator, you can develop and test your application locally, without creating an Azure subscription or incurring any service costs. When you're satisfied with how your application is working with the emulator, you can transition to using an Azure Cosmos DB account with minimal friction.

> [!IMPORTANT]
> We do not recommend the use of the emulator for production workloads.
> [!Tip]
> [!TIP]
> Visit our new **[Samples Gallery](https://aka.ms/AzureCosmosDB/Gallery)** for the latest samples for building new apps

## Differences between the emulator and cloud service

The emulator provides an environment on your developer workspace that isn't capable of emulating every aspect of the Azure Cosmos DB service. Here are a few key differences in functionality between the emulator and the equivalent cloud service.

> [!IMPORTANT]
> The Linux emulator currently doesn't support developer machines running on Apple silicon series or Microsoft ARM chips. A temporary workaround is to install a Windows virtual machine and run the emulator on that platform.
>
> Alternatively, you can used the new [Linux-based Azure Cosmos DB emulator (preview)](emulator-linux.md).
- The emulator's **Data Explorer** pane is only supported in the API for NoSQL and API for MongoDB.
- The emulator only supports **provisioned throughput**. The emulator doesn't support **serverless** throughput.
Expand Down Expand Up @@ -57,6 +57,10 @@ Every request made against the emulator must be authenticated using a key over T

In some cases, you may wish to manually import the TLS/SS certificate from the emulator's running container into your host machine. This step avoids bad practices like disabling TLS/SSL validation in the SDK. For more information, see [import certificate](how-to-develop-emulator.md#import-the-emulators-tlsssl-certificate).

## Related content

- [Linux-based Azure Cosmos DB emulator (preview)](emulator-linux.md)

## Next step

> [!div class="nextstepaction"]
Expand Down
6 changes: 5 additions & 1 deletion articles/cosmos-db/gen-ai/document-ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,14 @@ ms.collection:
- ce-skilling-ai-copilot
---

# Load document files into Azure Cosmos DB
# Load and process document files into Azure Cosmos DB for search

[!INCLUDE[NoSQL](../includes/appliesto-nosql.md)]


> [!NOTE]
> Document ingestion for Azure Cosmos DB is in private preview. If you're interested to join the preview, we encourage you to join the wait list by signing this form: https://aka.ms/Doc2CDBSignup
We introduce Doc2CDB for Azure Cosmos DB, a powerful accelerator designed to streamline the extraction, preprocessing, and management of large volumes of text data for vector similarity search. This solution uses the advanced vector indexing capabilities of Azure Cosmos DB and is powered by Azure AI Services to provide a robust and efficient pipeline that’s easily to set up and perfect for many use cases including:

- Vector Similarity Search over Text Data. Extract and vectorize text from document data to store in Azure Cosmos DB, makes it easy for you to perform semantic search to find documents that are contextually related to your queries. This allows them to discover relevant information that might not be found through traditional keyword searches, facilitating more comprehensive data retrieval.
Expand Down
Loading

0 comments on commit b3e7fd6

Please sign in to comment.