Skip to content

Latest commit

 

History

History
340 lines (296 loc) · 48.1 KB

aoai.md

File metadata and controls

340 lines (296 loc) · 48.1 KB
Raw

Azure OpenAI and Reference Architecture

Azure OpenAI vs OpenAI

  1. OpenAI offers the latest features and models, while Azure OpenAI provides a reliable, secure, and compliant environment with seamless integration into other Azure services.
  2. Azure OpenAI supports private networking, role-based authentication, and responsible AI content filtering.
  3. Azure OpenAI does not use user input as training data for other customers. Data, privacy, and security for Azure OpenAI. Azure OpenAI does not share user data, including prompts and responses, with OpenAI.

Microsoft Azure OpenAI LLM Framework

LLM Integration Frameworks

  1. Semantic Kernel (Feb 2023): An open-source SDK for integrating AI services like OpenAI, Azure OpenAI, and Hugging Face with conventional programming languages such as C# and Python. It's an LLM orchestrator, similar to LangChain. / git / x-ref GitHub Repo stars
  2. Kernel Memory (Jul 2023): An open-source service and plugin for efficient dataset indexing through custom continuous data hybrid pipelines. GitHub Repo stars
  3. Azure ML Prompt Flow (Jun 2023): A visual designer for prompt crafting using Jinja as a prompt template language. / ref / git GitHub Repo stars
  • A Memory in Semantic Kernel vs Kernel Memory (formerly Semantic Memory) is designed to handle large datasets and extended conversations, with the memory pipeline deployed as a separate service. ref

Prompt

  1. Prompt Engine (Jun 2022): A tool for crafting prompts for large language models in Python. / Python GitHub Repo stars GitHub Repo stars
  2. PromptBench (Jun 2023): A unified evaluation framework for large language models. GitHub Repo stars
  3. SAMMO (Apr 2024): A general-purpose framework for prompt optimization. / ref GitHub Repo stars
  4. Prompty (Apr 2024): A template language for integrating prompts with LLMs and frameworks, enhancing prompt management and evaluation. GitHub Repo stars
  5. guidance (Nov 2022): A domain-specific language (DSL) for controlling large language models, focusing on model interaction and implementing the "Chain of Thought" technique. GitHub Repo stars
  6. LMOps (Dec 2022): A toolkit for improving text prompts used in generative AI models, including tools like Promptist for text-to-image generation and Structured Prompting. GitHub Repo stars
  7. LLMLingua (Jul 2023): A tool for compressing prompts and KV-Cache, achieving up to 20x compression with minimal performance loss. LLMLingua-2 was released in Mar 2024. GitHub Repo stars
  8. TypeChat (Apr 2023): A tool that replaces prompt engineering with schema engineering, designed to build natural language interfaces using types. / git GitHub Repo stars
  9. GenAIScript (Aug 2023): Orchestrate LLMs, tools, and data in a single JavaScript. GitHub Repo stars

Agent

  1. JARVIS (Mar 2023): An interface for LLMs to connect numerous AI models for solving complex AI tasks. GitHub Repo stars
  2. Autogen (Mar 2023): A customizable and conversable agent framework. / ref / Autogen Studio (June 2024) GitHub Repo stars
  3. TaskWeaver (Sep 2023): A code-first agent framework for converting natural language requests into executable code with support for rich data structures and domain-adapted planning. GitHub Repo stars
  4. UFO (Mar 2024): A UI-focused agent for Windows OS interaction. GitHub Repo stars
  5. Semantic Workbench (Aug 2024): A development tool for creating intelligent agents. / ref GitHub Repo stars
  6. OmniParser (Sep 2024): A simple screen parsing tool towards pure vision based GUI agent. GitHub Repo stars
  7. TinyTroupe: LLM-powered multiagent persona simulation for imagination enhancement and business insights. [Mar 2024] GitHub Repo stars
  8. RD-Agent: open source R&D automation tool ref [Apr 2024] GitHub Repo stars
  9. Magentic-One: Built on AutoGen. A Generalist Multi-Agent System for Solving Complex Tasks [Nov 2024]
  10. AIOpsLab: Building AI agents for autonomous clouds. AI agents for incident root cause analysis (RCA) git [20 Dec 2024] GitHub Repo stars

Deep learning

  1. DeepSpeed (May 2020): A deep learning optimization library for easy, efficient, and effective distributed training and inference, featuring the Zero Redundancy Optimizer. GitHub Repo stars
  2. FLAML (Dec 2020): A lightweight Python library for efficient automation of machine learning and AI operations, offering interfaces for AutoGen, AutoML, and hyperparameter tuning. GitHub Repo stars

Risk Identification & LLMOPs

  1. PyRIT (Dec 2023): Python Risk Identification Tool for generative AI, focusing on LLM robustness against issues like hallucination, bias, and harassment. GitHub Repo stars
  2. AI Central (Oct 2023): An AI Control Center for monitoring, authenticating, and providing resilient access to multiple OpenAI services. GitHub Repo stars
  3. Azure AI Evaluation SDK (Oct 2024): Evaluate with the prompt flow SDK has been retired and replaced with Azure AI Evaluation SDK.

Data processing

  • Microsoft Fabric: Fabric integrates technologies like Azure Data Factory, Azure Synapse Analytics, and Power BI into a single unified product [May 2023]

Toolkit

Microsoft Copilot

Copilot Products

Agents

Copilot Development

  1. Microsoft AI and AI Studio
  2. Copilot Studio

Azure AI Search

  • doc:ref / blog:ref / git
  • Azure Cognitive Search rebranding Azure AI Search, it supports Vector search and semantic ranker. [16 Nov 2023]
  • In the vector databases category within Azure, several alternative solutions are available. However, the only option that provides a range of choices, including a conventional Lucene-based search engine and a hybrid search incorporating vector search capabilities.
  • Vector Search Sample Code: git [Apr 2023] GitHub Repo stars
  • Azure AI Search (FKA. Azure Cognitive Search) supports
    1. Text Search
    2. Pure Vector Search
    3. Hybrid Search (Text search + Vector search)
    4. Semantic Hybrid Search (Text search + Semantic search + Vector search)
  • A set of capabilities designed to improve relevance in these scenarios. We use a combination of hybrid retrieval (vector search + keyword search) + semantic ranking as the most effective approach for improved relevance out-of–the-box. TL;DR: Retrieval Performance; Hybrid search + Semantic rank > Hybrid search > Vector only search > Keyword only ref [18 Sep 2023]
    acs
  • Hybrid search using Reciprocal Rank Fusion (RRF): Reciprocal Rank Fusion (RRF) is an algorithm that evaluates the search scores from multiple, previously ranked results to produce a unified result set. In Azure Cognitive Search, RRF is used whenever there are two or more queries that execute in parallel. ref
  • Integrated vectorization: Automatically splits documents into chunks, creates embeddings with Azure OpenAI, maps them to an Azure AI Search index, and automates query vectorization. [24 Aug 2024]

Azure AI Enterprise Services

  • Azure OpenAI Service Offerings
    1. Offering: Standard (Default), Batch (Low-cost, Huge workload), Provisioned (High performance)
    2. Offering Region types: Global (World wide), Data_zones (Zone based), Regional (Region based)
  • Azure OpenAI Service On Your Data in Public Preview ref [19 Jun 2023]
  • Copilot (FKA. Bing Chat Enterprise) [18 Jul 2023] Privacy and Protection
    1. Doesn't have plugin support
    2. Only content provided in the chat by users is accessible to Bing Chat Enterprise.
  • Models as a Service (MaaS): A cloud-based AI approach that provides developers and businesses with access to pre-built, pre-trained machine learning models. [Jul 2023]
  • Azure OpenAI Finetuning: Babbage-002 is $34/hour, Davinci-002 is $68/hour, and Turbo is $102/hour. ref [16 Oct 2023]
  • Customer Copyright Commitment: protects customers from certain IP claims related to AI-generated content. ref [16 Nov 2023]
  • Assistants API: Code Interpreter, Function calling, Knowledge retrieval tool, and Threads (Truncated and optimized conversation history for the model's context length) in Azure [06 Feb 2024]
  • Azure AI Foundry: Azure AI Foundry portal (formerly Azure AI Studio), and the Azure AI Foundry SDK, our unified SDK with pre-built app templates. Rebranding for Enhancement: Azure Machine Learning Studio > Azure AI Studio > Azure AI Foundry. ref / ref: Documentation [19 Nov 2024]

Research

  1. Microsoft Office Copilot: Natural Language Commanding via Program Synthesis: [cnt]: Semantic Interpreter, a natural language-friendly AI system for productivity software such as Microsoft Office that leverages large language models (LLMs) to execute user intent across application features. [6 Jun 2023]
  2. NL2KQL: From Natural Language to Kusto Query [3 Apr 2024]
  3. GraphRAG (by Microsoft): RAG with a graph-based approach to efficiently answer both specific and broad questions over large text corpora1. ref / git / x-ref [24 Apr 2024] GitHub Repo stars
  4. SpreadsheetLLM: Introduces an efficient method to encode Excel sheets, outperforming previous approaches with 25 times fewer tokens.[12 Jul 2024]
  5. AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systems [9 Aug 2024]
  6. Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks [Nov 2024]
  7. PromptWizard: The future of prompt optimization through feedback-driven self-evolving prompts [17 Dec 2024]

Azure Reference Architectures

Azure OpenAI Application
Azure OpenAI Embeddings QnA [Apr 2023] Azure Cosmos DB + OpenAI ChatGPT C# blazor [Mar 2023]
embeddin_azure_csharp gpt-cosmos
C# Implementation ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search [Apr 2023] Simple ChatGPT UI application TypeScript, ReactJs and Flask [Apr 2023]
embeddin_azure_csharp gpt-cosmos
Azure Video Indexer demo Azure Video Indexer + OpenAI [Apr 2023] Miyagi Integration demonstrate for multiple langchain libraries [Feb 2023]
demo-videoindexer miyagi
ChatGPT + Enterprise data RAG (Retrieval-Augmented Generation)🏆 [Feb 2023] Chat with your data - Solution accelerator [Jun 2023]
demo-videoindexer
  • An open-source template gallery: 🏆AI template collection
  • ChatGPT + Enterprise data RAG (Retrieval-Augmented Generation) Demo git 🏆/ 📺: Can ChatGPT work with your enterprise data? [06 Apr 2023] [8 Feb 2023] GitHub Repo stars
  • GPT-Azure-Search-Engine: git Integration of Azure Bot Service with LangChain [Feb 2023] GitHub Repo stars
  • ChatGPT Plugin Quickstart using Python and FastAPI: git [May 2023] GitHub Repo stars
  • Azure AI CLI: Interactive command-line tool for ai [Jul 2023] GitHub Repo stars
  • Smart Components: Experimental, end-to-end AI features for .NET apps [Mar 2024] GitHub Repo stars
  • Prompt Buddy: 🏆Share and upvote favorite AI prompts. free Microsoft Teams Power App using Dataverse for Teams. [Mar 2024] GitHub Repo stars
  • eShopSupport:💡A reference .NET application using AI for a customer support ticketing system ref [Apr 2024] GitHub Repo stars
  • ARGUS: Hybrid approach with Azure Document Intelligence combined and GPT4-Vision to get better results without any pre-training. [Jun 2024] GitHub Repo stars
  • VoiceRAG:💡Voice Using Azure AI Search and the GPT-4o Realtime API for Audio ref [Sep 2024] GitHub Repo stars
  • AI Dev Gallery:💡Windows application featuring interactive samples powered by local AI models. [Nov 2024] GitHub Repo stars
  • Semantic Kernel sample application x-ref

Azure OpenAI Accelerator / samples

Referece: Use Case and Architecture