Skip to content

Latest commit

 

History

History
398 lines (252 loc) · 24 KB

build-bot-based-plugin.md

File metadata and controls

398 lines (252 loc) · 24 KB
Raw
title author description ms.localizationpriority ms.topic ms.author ms.date
Message Extension as Plugin for Copilot
v-ypalikila
Learn how to build a bot-based message extension plugin for Microsoft Copilot for Microsoft 365 using Teams Developer Portal and Teams Toolkit.
medium
overview
anclear
11/14/2023

Extend bot-based message extension as plugin for Copilot for Microsoft 365

Note

  • Ensure that Copilot for Microsoft 365 is available for your organization. You have two ways to get a developer environment for Copilot:
  • Bot-based search message extension plugin is available in public developer preview.
  • Only bot-based message extensions with search commands can be extended as plugins for Copilot for Microsoft 365.

Microsoft Copilot for Microsoft 365, powered by an advanced processing and orchestration engine, integrates Microsoft 365 apps, Microsoft Graph, and Large Language Models (LLMs) to transform your words into a potent productivity tool. Although Copilot for Microsoft 365 can utilize apps and data within the Microsoft 365 ecosystem, many users rely on various external tools and services for work management and collaboration. By extending your message extension as a plugin in Copilot for Microsoft 365, you can enable users to interact with third-party tools and services, therefore empowering them to achieve more with Copilot for Microsoft 365. You can achieve this extension by developing a plugin or connecting to an external data source.

:::image type="content" source="../assets/images/Copilot/ailib-copilot-diff.png" alt-text="Graphic shows the user interaction flow between the user, Microsoft Teams, and Copilot for Microsoft 365." lightbox="../assets/images/Copilot/ailib-copilot-diff.png":::

See the video to learn more about extending Copilot for Microsoft 365 using Teams message extensions:

[!VIDEO https://www.youtube.com/embed/vvNFCagkdcE?si=1jS5gsLuJFPANzoi]

What is a plugin?

A plugin allows Copilot for Microsoft 365 to interact directly with third-party data, apps, and services, enhance its capabilities, and broaden its range of capabilities. Plugins allow Copilot for Microsoft 365 to:

  • Fetch real-time information, such as the latest news coverage on a product launch.
  • Retrieve knowledge-based information, such as a team’s design files in Figma.

Descriptions enhance the usability and effectiveness of a message extension plugin. The following description offer a clear and concise summary of the app’s features:

  • App description: App description helps improve your app discoverability in the Teams Store.
  • Command description: Command description maps user intent and utterance to search command inside a plugin and must be built based on the analysis of the user intent and keywords.
  • Parameter description: Parameter description explains the requirements of the system in a natural language with output format.
  • Semantic description: Semantic description helps Copilot for Microsoft 365 generate content by providing a conceptual understanding of the plugin's capabilities and scenarios where it can help achieve user goals and match user’s intent with the plugin's capabilities.

For more information, see define descriptions.

All bot-based search message extensions are eligible for plugin support, subject to validation to ensure the plugin meets quality, security, privacy, and usefulness expectations. You can create a bot-based search message extension using Teams Toolkit for Visual Studio Code, Visual Studio, Teams Toolkit command line interface (CLI), or Developer Portal for Teams and extend the message extension to function as a plugin in Copilot for Microsoft 365.

Prerequisites

Before you get started, ensure that you're familiar with the following standards and guidelines for building message extension plugins for Copilot for Microsoft 365:

Create bot-based message extension

Important

Plugins for Microsoft Copilot for Microsoft 365 are in preview and only work in Copilot for Microsoft 365 in Microsoft Teams.

Let's create a bot-based search message extension that can search npm registries in Teams and Copilot and share results through the compose message area of the Microsoft Teams client.

Visual Studio Code

Before you get started, ensure that you install the following tools to build and deploy your message extension:

  • Install the latest Teams Toolkit prerelease version.
  • Ensure that the Develop Copilot Plugin feature flag is enabled. To enable the feature flag, follow these steps:
    1. Open Visual Studio Code.
    2. Go to Manage > Settings.
    3. Enter Teams Toolkit in the Search settings search box.
    4. Under Extensions, select the Fx-extension: Develop Copilot Plugin checkbox.

To create a bot-based search message extension plugin using Visual Studio Code, follow these steps:

  1. Open Visual Studio Code.

  2. From the left pane, select Teams Toolkit.

  3. Select Create a New App.

  4. Select Message Extension.

  5. Select Custom Search Results.

  6. Select Start with a Bot.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-start-with-bot.png" alt-text="Screenshot shows the Start with a Bot option to create a bot-based message extension in Visual Studio Code.":::

  7. Select a programming language.

  8. Select Default folder.

  9. Enter the name of your app and select Enter. Teams Toolkit scaffolds your app and creates a message extension.

To run your message extension in Teams, follow these steps:

  1. From the left pane, select Teams Toolkit.

  2. Under ACCOUNTS, perform the following steps:

    1. Select Sign in to Microsoft 365 and enter your Microsoft 365 credentials.

    2. Select Sign in to Azure and enter your Azure credentials.

      :::image type="content" source="../assets/images/Copilot/api-based-me-ttk-accounts.png" alt-text="Screenshot shows the Sign in to Microsoft 365 and Azure option under ACCOUNTS in Teams Toolkit for Visual Studio Code.":::

  3. From the left pane, select Run and Debug (Ctrl+Shift+D).

  4. Select Debug in Teams (Edge) or Debug in Teams (Chrome). Teams Toolkit launches your app in Teams using a web browser.

  5. Select Add. The app is installed on Teams.

  6. Go to a chat and select Actions and apps.

  7. From the message extension fly-out menu, enter the name of your message extension in the search box.

  8. Select your message extension and enter your search query.

  9. Select a product from the list. Teams unfurls the product as an Adaptive Card in the message compose area.

  10. Select Send.

Trigger message extension in Copilot for Microsoft 365

To trigger the message extension as plugin in Copilot for Microsoft 365 in Teams, follow these steps:

  1. Select Apps.

  2. Search for Copilot and open the Copilot for Microsoft 365 app.

  3. Select Plugins.

  4. From the list of plugins, turn on the toggle for your message extension.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-plugin-toggle.png" alt-text="Screenshot shows the Plugin option, list of plugins, and the toggle enabled for bot-based-ME-test-local plugin.":::

  5. From the message compose area, send a message to Copilot for Microsoft 365 to search for npm package information in Teams and Copilot. For example, find the npm package info on teamsfx-react in npm-searchlocal.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-trigger-plugin.png" alt-text="Screenshot shows the plugin prompt and the response from Copilot for Microsoft 365.":::

Note

This prompt might not always make Copilot for Microsoft 365 include a response from your message extension. If it happens, try some other prompts or leave feedback to us by downvoting the Copilot for Microsoft 365 response and leave a message.

Visual Studio

Before you get started, ensure that you install the following tools to build and deploy your message extension:

  • Install the Visual Studio Enterprise 2022 Preview version 17.9.0 Preview 1.0 or later and install the Microsoft Teams development tools under ASP.NET and web development workload.
  • Ensure that the Copilot feature flag is enabled. To enable the feature flag, follow these steps:
    1. Open Visual Studio.
    2. Go to Tools > Options.
    3. Enter Teams Toolkit in the Search Settings search box.
    4. Under Preview Features, select the Teams Toolkit: Develop Copilot Plugin checkbox.

To create a bot-based search message extension plugin using Visual Studio, follow these steps:

  1. Open Visual Studio.

  2. Go to File > New > Project... or Create a new Project.

  3. Search for Teams and select Microsoft Teams App.

  4. Select Custom Search Results.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-new-project.png" alt-text="Screenshot shows the custom search results option in Visual Studio.":::

  5. Select Create. The project is scaffolded.

  6. In the debug dropdown menu, select Dev Tunnels > Create a Tunnel.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-dev-tunnel.png" alt-text="Screenshot shows the create a tunnel option in Visual Studio.":::

    1. Select the account to create the tunnel. Azure, Microsoft Account (MSA), and GitHub are the account types that are supported.
    2. Name: Enter a name for the tunnel.
    3. Tunnel Type: Select Persistent or Temporary.
    4. Access: Select Public.
    5. Select OK. Visual Studio displays a confirmation message that a tunnel is created.

    The tunnel you created is listed under Dev Tunnels > (name of the tunnel).

  7. Go to Solution Explorer and select your project.

  8. Right-click the project menu and select Teams Toolkit > Prepare Teams App Dependencies.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-teams-app-dependencies.png" alt-text="Screenshot shows the Prepare Teams app dependencies option under Teams Toolkit in Visual Studio app project.":::

    If prompted, sign in with a Microsoft 365 account.

  9. In the debug dropdown menu, select Microsoft Teams (browser). Visual Studio launches Teams web client.

  10. Select Add. The message extension is added to Teams.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-add-app.png" alt-text="Screenshot shows an example of the Add option to add Contoso Pluginlocal app to Teams.":::

  11. Go to a chat and select Actions and apps.

  12. From the message extension fly-out menu, enter the name of your message extension in the search box.

  13. Select your message extension and enter your search query.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-invoke.png" alt-text="Screenshot shows an example of Contoso search query entered in the Contoso pluginlocal message extension and the message extension displays seven products results.":::

  14. Select a product from the list. Teams unfurls the product into an Adaptive Card in the message compose area.

  15. Select Send. The Adaptive Card is sent to the chat.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-send-card.png" alt-text="Screenshot shows Contoso. Base product sent as an Adaptive Card in Teams chat.":::

Trigger message extension in Copilot for Microsoft 365

To trigger the message extension as plugin in Copilot for Microsoft 365, follow these steps:

  1. Go to Visual Studio.

  2. In the debug dropdown, select Copilot (browser). Visual Studio launches Teams web client.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-debug-Copilot.png" alt-text="Screenshot shows the Copilot (Browser) debug option in Visual Studio.":::

  3. Go to Teams web client > Apps.

  4. Search for Copilot and open the Copilot app.

  5. Select Plugins.

  6. From the list of plugins, turn on the toggle for your app.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-me-plugin-toggle.png" alt-text="Screenshot shows the Plugin option, list of plugins, and the toggle enabled for bot-based-ME-test-local plugin.":::

  7. From the message compose area, send a message to Copilot for Microsoft 365 to search for npm package information in Teams and Copilot. For example, find the npm package info on teamsfx-react in npmsearchlocal.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-trigger-plugin.png" alt-text="Screenshot shows the plugin prompt and the Adaptive Card response with content and preview card from Copilot for Microsoft 365. The response contains a list of four products with Contoso product name. ":::

Note

This prompt might not always make Copilot for Microsoft 365 include a response from your message extension. If it happens, try some other prompts or leave feedback to us by downvoting the Copilot for Microsoft 365 response and leave a message.

Toolkit CLI

Before you get started, ensure that you set the environment variable DEVELOP_COPILOT_PLUGIN to true. To set the environment variables, follow these steps:

  1. On your computer, select the Windows + X key.

  2. Select System.

  3. Under About > Device specifications, select Advanced system settings.

  4. Under System Properties, select Environment Variables...

  5. Under User variables, select New... A New User Variable dialog opens.

  6. Update the following fields:

    1. Variable name: DEVELOP_COPILOT_PLUGIN
    2. Variable value: true

  7. Select OK.

    :::image type="content" source="../assets/images/Copilot/bot-based-plugin-CLI-add-env-variable.png" alt-text="Screenshot shows the Variable name and Variable value fields in the New User Variable dialog.":::

To create a bot-based search message extension plugin using Teams Toolkit CLI, follow these steps:

  1. Go to Command Prompt.

  2. Enter the following command:

    npm install -g @microsoft/teamsfx-cli@beta

  3. Enter teamsfx new in the terminal.

  4. Select Message Extension. Use the arrow keys to switch between options.

    :::image type="content" source="../assets/images/Copilot/api-based-me-CLI-new-project-me.png" alt-text="Screenshot shows Teams capabilities as options in the CLI interface.":::

  5. Select Custom Search Results.

  6. Select Start with a Bot.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-new-project.png" alt-text="Screenshot shows the message extension, custom search results, and start with a bot option selected in the CLI window.":::

  7. Enter the location for your project and select Enter.

  8. Enter the name of your application and select Enter.

    Teams Toolkit scaffolds and creates a project in the specified location.

  9. Go to the folder path where your project is created and enter the following command to provision your app in Azure:

     teamsfx provision --env dev

    Teams Toolkit CLI opens a browser window and requests you to sign in to your Microsoft Account.

  10. Sign in to your Microsoft account. Teams Toolkit CLI executes validation and provisions your app on Azure.

  11. From the list, select a subscription.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-resource-group.png" alt-text="Screenshot shows the sign-in to Azure and select a subscription option in the CLI window.":::

  12. From the list, select a resource group.

  13. If you receive a message, which reads Cost may incur according to the usage. Do you want to provision resources in dev environment using accounts listed above?, enter Y.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-provisoin-preview.png" alt-text="Screenshot shows the Cost may incur according to the usage. Do you want to provision resources in dev environment using accounts listed option in the CLI window.":::

    Teams Toolkit validates your app manifest and provisions your app on Azure.

  14. In the command prompt window, enter the following command to preview your app in Teams:

     teamsfx preview --env dev

    A new browser window with Teams web client opens. You can add your app to Teams.

  15. Select Add. The message extension is added to Teams.

  16. Go to a chat and select Actions and apps.

  17. From the message extension fly-out menu, enter the name of your message extension in the search box.

  18. Select your message extension and enter your search query.

Trigger message extension in Copilot for Microsoft 365

To trigger the message extension through Copilot for Microsoft 365, follow these steps:

  1. Select Apps.

  2. Search for Copilot and open the Copilot app.

  3. Select Plugins.

  4. From the list of plugins, turn on the toggle for your app.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-plugin-toggle.png" alt-text="Screenshot shows the Plugin option, list of plugins, and the toggle enabled for bot-based-ME-test-local plugin.":::

  5. From the message compose area, send a message to Copilot for Microsoft 365 to search for npm package information in Teams and Copilot. For example, find the npm package info on teamsfx-react in npm-searchlocal.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-trigger-plugin.png" alt-text="Screenshot shows the plugin prompt and the response from Copilot for Microsoft 365.":::

Note

This prompt might not always make Copilot for Microsoft 365 include a response from your message extension. If it happens, try some other prompts or leave feedback to us by downvoting the Copilot for Microsoft 365 response and leave a message tagged with [MessageExtension].

Developer Portal for Teams

To create a bot-based search message extension plugin using Developer portal for Teams, follow these steps:

  1. Go to Teams Developer Portal.

  2. Go to Apps.

  3. Select + New apps.

  4. Under Configure, select App features.

  5. Select Messaging extension.

    :::image type="content" source="../assets/images/Copilot/api-based-me-tdp-app-feature.png" alt-text="Screenshot shows the messaging extension option in Teams Developer Portal.":::

  6. Under Message extension type, select Bot.

    1. If you get a disclaimer that reads API Message extension is already in use by users. Would you like to change message extension type to bot?, select Yes, change.

    :::image type="content" source="../assets/images/Copilot/bot-based-me-tdp-type.png" alt-text="Screenshot shows API Message extension is already in use disclaimer when a user switches from API to bot message extension type.":::

  7. If you have an existing bot, select Existing bot or if you have a bot ID, select Enter Bot ID.

    1. If you don't have a Bot ID, you can select Create a new bot, to create a new bot and enter the bot ID of the new bot that you've created.

    :::image type="content" source="../assets/images/Copilot/bot-based-tdp-select-bot.png" alt-text="Screenshot shows the Bot, Existing bot, and Enter Bot ID options for messaging extension app feature in Developer Portal for Teams.":::

  8. Select Save.

  9. Under Command, select + Add a command.

    Add a command page appears.

  10. In the Add a command page, select the Search as the type of command and update the following fields:

    • Command ID
    • Command title
    • Command description
    • Context in which the command works
    • Parameter name
    • Parameter title
    • Parameter description

  11. Select Save. The command is added to the message extension.

  12. Select Save.

A bot-based search message extension is created.

:::image type="content" source="../assets/images/Copilot/bot-based-tdp-message-extension-created.png" alt-text="Screenshot shows the messaging extension created and listed in the App features page in Developer Portal for Teams.":::

To test your bot-based message extension created in the Developer Portal for Teams, you can use the following methods:

  • Preview in Teams: In Developer Portal, open your message extension and select Preview in Teams in the upper-right corner. You're redirected to Teams, where you can add the app to Teams to preview the app.

  • Download app package: On the message extension page, select App package from the left pane and then, in the upper-left corner of the window, select Download app package. The app package is downloaded to your local machine in a .zip file. You can upload the app package to teams and test the message extension.

For more on testing your plugin in Copilot for Microsoft 365, see Debugging plugin selection.

Enable message extension as a plugin for Copilot for meetings

Ensure that you've created a bot-based message extension and extended it as a plugin for Copilot for Microsoft 365. Before you get started, We recommend you to follow the guidelines listed in Copilot for Microsoft 365 extension guidelines to extend your plugin for Copilot for meetings.

To enable message extension as a plugin for Copilot for Teams meetings, follow these steps:

  1. Go to a Teams meeting.

  2. Select Join.

  3. In the meeting window, select Copilot.

  4. Select Start transcription.

    :::image type="content" source="../assets/images/Copilot/meeting-copilot-start-transcription.png" alt-text="Screenshot shows the Copilot option and start transcription button in a Teams meeting.":::

  5. Select the Spoken language and select Confirm. The Copilot for Teams pane appears.

    :::image type="content" source="../assets/images/Copilot/meeting-copilot-select-language.png" alt-text="Screenshot shows the dropdown to select a spoken language and a confirm button in Teams meeting.":::

  6. In the Copilot for Teams pane, at the bottom-right corner, select the Copilot Plugin Button.

    :::image type="content" source="../assets/images/Copilot/meeting-copilot-trigger-plugin.png" alt-text="Screenshot shows the plugin icon in the copilot pane in a Teams meeting.":::

  7. Search for your plugin and turn on the toggle for your plugin.

  8. From the compose area, select More prompts to send a static prompt or you can type your own prompt and select Send.

    :::image type="content" source="../assets/images/Copilot/meeting-copilot-static-prompts.png" alt-text="Screenshot shows the list of static prompts available in the copilot pane in a Teams meeting.":::

Meeting Copilot helps your meetings to be better and more productive. You can make meetings more interactive, which helps users by giving them a smoother and more interesting experience.

Step-by-step guide

Follow the step-by-step guide to build a bot-based search message extension plugin for Copilot for Microsoft 365.

See also