Skip to content

Commit

Permalink
Gradio components in gr.Chatbot() (#8131)
Browse files Browse the repository at this point in the history
* chatbot components

* demoi

* add changeset

* preprocess fix

* add changeset

* Make guide for tailwind more verbose (#8152)

* Lite wheel optimization (#7855)

* Add `pull_request.branches.main` as a trigger of the `publish` workflow

* [WIP] Comment out the publish steps

* Package and upload the NPM package for debug

* Skip the copy_frontend.py hook in the Lite build

* add changeset

* [WIP] Show gradio files

* [WIP] Show gradio files

* Comment out installing the gradio and gradio_client libraries

* Restore installing gradio_client because it's used by `python js/_website/generate_jsons/generate.py` that follows

* Restore installing gradio because it's used by `python js/_website/generate_jsons/generate.py` that follows

* Add code

* Revert "[WIP] Show gradio files"

This reverts commit e15fef2.

* Build the gradio wheel with the custom lite target

* add changeset

* Revert "[WIP] Show gradio files"

This reverts commit aef053f.

* Revert "Skip the copy_frontend.py hook in the Lite build"

This reverts commit ca296d0.

* Update .github/actions/install-frontend-deps/action.yml for hatch installation

* [WIP] Fix test-functional.yml and .github/actions/install-all-deps/action.yml to call the setup actions in this branch

* Revert "[WIP] Fix test-functional.yml and .github/actions/install-all-deps/action.yml to call the setup actions in this branch"

This reverts commit 571823b.

* Comment-in lines in publish.yml

* Move Lite build from publish.yml to deploy-spaces.yml

* Use the build_lite option of install-all-deps instead of running the build command

* [TMP] Change the branch of action files

* Fix the hatch Lite build setting

* Return pnpm pack to deploy-space

* Revert "[TMP] Change the branch of action files"

This reverts commit fe4e1c8.

* Remove dependencies for lite build

* [TMP] Change the branch of action files

* Revert "Remove dependencies for lite build"

This reverts commit 856a858.

* Install packaging>=23.2

* [TMP] Show packaging version

* Fix pip install

* Fix

* Uninstall packaging once

* Use `pip install -U` instead of uninstalling the exiting version

* Revert "[TMP] Show packaging version"

This reverts commit 910b6bb.

* Add `-U` flag

* Set packaging version as >=23.2

* Revert the changes on pip install

* Set packaging version as >=23.2 in requirements.txt

* Revert "Set packaging version as >=23.2"

This reverts commit 8aa77c8.

* Fix hook name

* Revert "Set packaging version as >=23.2 in requirements.txt"

This reverts commit fbd605c.

* Revert "Revert the changes on pip install"

This reverts commit 81ff38a.

* Add comments

* Revert "[TMP] Change the branch of action files"

This reverts commit 0d6aa48.

* Revert the trigger of .github/workflows/deploy-spaces.yml

* Remove unused `node_auth_token` and `npm_token` inputs from the `install-all-deps` action

* [TMP] Trigger CI based on this PR

* Remove packging installation

* Revert "Remove packging installation"

This reverts commit 4a4f18d.

* Revert "[TMP] Trigger CI based on this PR"

This reverts commit 6cea830.

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: freddyaboulton <alfonsoboulton@gmail.com>

* Add ETag to `/custom_component` route to control browser caching (#8170)

* Add code

* add changeset

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Implement JS Client tests (#8109)

* add msw setup and initialisation tests

* add changeset

* add walk_and_store_blobs improvements and add tests

* add changeset

* api_info tests

* add direct space URL link tests

* fix tests

* add view_api tests

* add post_message test

* tweak

* add spaces tests

* jwt and protocol tests

* add post_data tests

* test tweaks

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Remove hatch installation in js/app/package.json which is no longer needed (#8174)

* Remove hatch installation in js/app/package.json which is no longer needed

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Update test-functional.yml - Fix vulnerability code injection (#8145)

* Update test-functional.yml

* Update test-functional.yml

* tweaks

---------

Co-authored-by: pngwn <hello@pngwn.io>

* rework upload to be a class method + pass client into each component (#8179)

* rework upload to be a class method + pass client into each component

* add changeset

* Update client/js/src/utils/upload_files.ts

* fix storybook

* review comments

* Apply suggestions from code review

Co-authored-by: Hannah <hannahblair@users.noreply.github.com>

* format

* ts fix

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Hannah <hannahblair@users.noreply.github.com>

* chore(deps): update pnpm to v9 (#8123)

* chore(deps): update pnpm to v9

* update workflow

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pngwn <hello@pngwn.io>

* Use workspace version for code in _website (#8189)

* workspace

* add changeset

* remove circular import from preview

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Pass Error status in /dev/reload stream (#8106)

* get error message

* Support multiple clients

* add changeset

* add changeset

* add changeset

* Display in UI

* console.error the python traceback

* lint

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Convert sse calls in client from async to sync (#8182)

* convert sse calls in client from async to sync

* add changeset

* more sync

* lint

* more sync

* fix threadpool

* fix timeouts

* reuse executor

* lint

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* run python reload only if python file changed (#8194)

* run python reload only if python file changed

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Freddy Boulton <alfonsoboulton@gmail.com>

* fix: handling SIGINT correctly in reload.py, single entrance of block_thread in blocks.py (#8158)

* fix: handling SIGINT, single block_thread and fix popen

* Use pass

---------

Co-authored-by: freddyaboulton <alfonsoboulton@gmail.com>

* Add eventsource polyfill for Node.js and browser environments (#8118)

* add msw setup and initialisation tests

* add changeset

* add eventsource polyfill for node and browser envs

* add changeset

* add changeset

* config tweak

* types

* update eventsource usage

* add changeset

* add walk_and_store_blobs improvements and add tests

* add changeset

* api_info tests

* add direct space URL link tests

* fix tests

* add view_api tests

* add post_message test

* tweak

* add spaces tests

* jwt and protocol tests

* add post_data tests

* test tweaks

* dynamically import eventsource

* revet eventsource imports

* add node test

* lockfile

* add client test in root pkg file

* lcokfile

* remove eventsource from js/app

* add changeset

* remove ts ignore

* move eventsource polyfill to eventsource factory

* add changeset

* tweak

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Ensure connectivity to private HF spaces with SSE protocol (#8181)

* add msw setup and initialisation tests

* add changeset

* add eventsource polyfill for node and browser envs

* add changeset

* add changeset

* config tweak

* types

* update eventsource usage

* add changeset

* add walk_and_store_blobs improvements and add tests

* add changeset

* api_info tests

* add direct space URL link tests

* fix tests

* add view_api tests

* add post_message test

* tweak

* add spaces tests

* jwt and protocol tests

* add post_data tests

* test tweaks

* dynamically import eventsource

* revet eventsource imports

* add jwt param to sse requests

* add stream test

* add changeset

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Support custom components in gr.load (#8200)

* Add code

* add changeset

* Update fuzzy-mirrors-scream.md

* Update fuzzy-mirrors-scream.md

* Fix tests

* Update .changeset/fuzzy-mirrors-scream.md

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* Fix code

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* Refactor analytics to not use api.gradio.app (#8180)

* Analytics refactor

* add changeset

* add changeset

* Fix wasm?

* Fix python tests'

* Revert changes chrome

* use util function

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Specify the fastapi version on Lite to avoid ujson installation which is not available on Pyodide yet (#8204)

* Specify the fastapi version on Lite to avoid ujson installation which is not available on Pyodide yet

* add changeset

* Refactoring

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Set the show_api flag on Lite (#8205)

* Set the show_api flag on Lite

* add changeset

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Extend Interface.from_pipeline() to support Transformers.js.py pipelines on Lite (#8052)

* Extend Interface.from_pipeline() to support Transformers.js.py pipelines on Lite (wip: only object-detection in this commit)

* add changeset

* Add image-classification and image-segmentation

* Add zero-shot-image-classification and zero-shot-object-detection

* Add document-question-answering

* Add feature-extraction and fill-mask

* Add question-answering and summarization

* Fix an error message

* Add text2text-generation, text-classification, and text-generation

* Add translation andtranslation_xx_to_yy

* Add zero-shot-classification

* Add postprocess_takes_inputs to control the args passed to the postprocess function of each pipeline

* Add topk option to image-classification

* format_backend

* Add audio-classification, automatic-speech-recognition, and zero-shot-audio-classification

* Add image-to-text

* Add token-classification (with JSON component as an output. Is it correct?)

* Ignore import type failure of transformers_js_py

* Add image-feature-extraction

* Add image-to-image

* Add text-to-audio

* Add depth-estimation

* Remove `render=False`

* Reorder the if-blocks following the Transformers.js doc

* Update gradio/pipelines_utils.py

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* Update gradio/pipelines_utils.py

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* Fix feature-extraction demo

* Fix demo title

* Add guides/08_gradio-clients-and-lite/gradio-lite-and-transformers-js.md without contents

* Rename guides/08_gradio-clients-and-lite/*.md to fix the order

* Use pipeline.model.config._name_or_path for the demo title instead of pipeline.model.config.model_type

* Fix normal Interface.from_pipeline to use pipeline.model.config.name_or_path as the demo title

* Write an article about Gradio-Lite and Transformers.js

* Update the doc

* tweaks

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* merge

* allow the canvas size to be set on the `ImageEditor` (#8127)

* add canvas size kwarg to imageeditor

* add changeset

* fix tests

* fix cropsize

* changes

* notebooks

* update docstrings

* fix type

* fix undefined dimensions

* Update image_editor.py

Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* fix type

* format

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

* Rename `eventSource_Factory` and `fetch_implementation` (#8209)

* rename eventSource_factory -> stream_factory + rename event_source -> steam

* rename fetch_implementation -> fetch

* rename fetch to _fetch due to global.fetch conflict

* add changeset

* format

* format

* format

* format

* fix

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* remove redundant event source logic (#8211)

* remove redundant event source logic

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Only connect to heartbeat if needed (#8169)

* Add connect_heartbeat field

* fix types

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* chore: update versions (#8172)

* fixes

* type fixes

* type fixes

* notebook fix

* type ignore

* data object model

* remove component in tuple

* more fixes

* extend components

* remove test var

* extend to all components backend

* remove loading data models

* conflict fix

* test and type fixes

* playwright test

* PR fixes

* final changes

* Add pltly for 2e2 test

* pass loader to Gradio helper class

* fix things

* add changeset

* checks

* more fixy

* more fixy

* more fixy

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Simon Duerr <dev@simonduerr.eu>
Co-authored-by: Yuichiro Tachibana (Tsuchiya) <t.yic.yt@gmail.com>
Co-authored-by: freddyaboulton <alfonsoboulton@gmail.com>
Co-authored-by: Hannah <hannahblair@users.noreply.github.com>
Co-authored-by: Lê Ngọc Hoa <114990730+h2oa@users.noreply.github.com>
Co-authored-by: pngwn <hello@pngwn.io>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Ali Abdalla <ali.si3luwa@gmail.com>
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>
Co-authored-by: James Zhou <61927718+jameszhou02@users.noreply.github.com>
Co-authored-by: Tiger3018 <tiger3018of02@gmail.com>
  • Loading branch information
13 people authored Jun 14, 2024
1 parent 7fc0f51 commit bb504b4
Show file tree
Hide file tree
Showing 42 changed files with 1,835 additions and 626 deletions.
16 changes: 16 additions & 0 deletions .changeset/bitter-goats-chew.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
"@gradio/app": minor
"@gradio/audio": minor
"@gradio/chatbot": minor
"@gradio/gallery": minor
"@gradio/image": minor
"@gradio/multimodaltextbox": minor
"@gradio/plot": minor
"@gradio/simpleimage": minor
"@gradio/storybook": minor
"@gradio/utils": minor
"@gradio/video": minor
"gradio": minor
---

feat:Gradio components in `gr.Chatbot()`
5 changes: 4 additions & 1 deletion .config/basevite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,10 @@ export default defineConfig(({ mode }) => {
build: {
sourcemap: false,
target: "esnext",
minify: production
minify: production,
rollupOptions: {
external: ["virtual:component-loader"]
}
},
define: {
BUILD_MODE: production ? JSON.stringify("prod") : JSON.stringify("dev"),
Expand Down
6 changes: 3 additions & 3 deletions .config/playwright.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const base = defineConfig({
},
expect: { timeout: 15000 },
timeout: 30000,
testMatch: /.*.spec.ts/,
testMatch: /.*\.spec\.ts/,
testDir: "..",
workers: process.env.CI ? 1 : undefined,
retries: 3
Expand All @@ -37,13 +37,13 @@ const lite = defineConfig(base, {
},
testMatch: [
"**/file_component_events.spec.ts",
"**/chatbot_multimodal.spec.ts",
"**/kitchen_sink.spec.ts",
"**/gallery_component_events.spec.ts",
"**/image_remote_url.spec.ts" // To detect the bugs on Lite fixed in https://github.com/gradio-app/gradio/pull/8011 and https://github.com/gradio-app/gradio/pull/8026
],
workers: 1,
retries: 3
retries: 3,
timeout: 60000
});

lite.projects = undefined; // Explicitly unset this field due to https://github.com/microsoft/playwright/issues/28795
Expand Down
Binary file added demo/chatbot_core_components/files/audio.wav
Binary file not shown.
Binary file added demo/chatbot_core_components/files/avatar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions demo/chatbot_core_components/files/sample.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
hello friends
Binary file added demo/chatbot_core_components/files/world.mp4
Binary file not shown.
1 change: 1 addition & 0 deletions demo/chatbot_core_components/run.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_core_components"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/audio.wav https://github.com/gradio-app/gradio/raw/main/demo/chatbot_core_components/files/audio.wav\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatbot_core_components/files/avatar.png\n", "!wget -q -O files/sample.txt https://github.com/gradio-app/gradio/raw/main/demo/chatbot_core_components/files/sample.txt\n", "!wget -q -O files/world.mp4 https://github.com/gradio-app/gradio/raw/main/demo/chatbot_core_components/files/world.mp4"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import os\n", "import plotly.express as px\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "\n", "def random_plot():\n", " df = px.data.iris()\n", " fig = px.scatter(\n", " df,\n", " x=\"sepal_width\",\n", " y=\"sepal_length\",\n", " color=\"species\",\n", " size=\"petal_length\",\n", " hover_data=[\"petal_width\"],\n", " )\n", " return fig\n", "\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", " print(x.index, x.value, x.liked)\n", "\n", "\n", "def random_bokeh_plot():\n", " from bokeh.models import ColumnDataSource, Whisker\n", " from bokeh.plotting import figure\n", " from bokeh.sampledata.autompg2 import autompg2 as df\n", " from bokeh.transform import factor_cmap, jitter, factor_mark\n", "\n", " classes = list(sorted(df[\"class\"].unique()))\n", "\n", " p = figure(\n", " height=400,\n", " x_range=classes,\n", " background_fill_color=\"#efefef\",\n", " title=\"Car class vs HWY mpg with quintile ranges\",\n", " )\n", " p.xgrid.grid_line_color = None\n", "\n", " g = df.groupby(\"class\")\n", " upper = g.hwy.quantile(0.80)\n", " lower = g.hwy.quantile(0.20)\n", " source = ColumnDataSource(data=dict(base=classes, upper=upper, lower=lower))\n", "\n", " error = Whisker(\n", " base=\"base\",\n", " upper=\"upper\",\n", " lower=\"lower\",\n", " source=source,\n", " level=\"annotation\",\n", " line_width=2,\n", " )\n", " error.upper_head.size = 20\n", " error.lower_head.size = 20\n", " p.add_layout(error)\n", "\n", " p.circle(\n", " jitter(\"class\", 0.3, range=p.x_range),\n", " \"hwy\",\n", " source=df,\n", " alpha=0.5,\n", " size=13,\n", " line_color=\"white\",\n", " color=factor_cmap(\"class\", \"Light6\", classes),\n", " )\n", " return p\n", "\n", "\n", "def random_matplotlib_plot():\n", " import numpy as np\n", " import pandas as pd\n", " import matplotlib.pyplot as plt\n", "\n", " countries = [\"USA\", \"Canada\", \"Mexico\", \"UK\"]\n", " months = [\"January\", \"February\", \"March\", \"April\", \"May\"]\n", " m = months.index(\"January\")\n", " r = 3.2\n", " start_day = 30 * m\n", " final_day = 30 * (m + 1)\n", " x = np.arange(start_day, final_day + 1)\n", " pop_count = {\"USA\": 350, \"Canada\": 40, \"Mexico\": 300, \"UK\": 120}\n", " df = pd.DataFrame({\"day\": x})\n", " for country in countries:\n", " df[country] = x ** (r) * (pop_count[country] + 1)\n", "\n", " fig = plt.figure()\n", " plt.plot(df[\"day\"], df[countries].to_numpy())\n", " plt.title(\"Outbreak in \" + \"January\")\n", " plt.ylabel(\"Cases\")\n", " plt.xlabel(\"Days since Day 0\")\n", " plt.legend(countries)\n", " return fig\n", "\n", "\n", "def add_message(history, message):\n", " for x in message[\"files\"]:\n", " history.append(((x,), None))\n", " if message[\"text\"] is not None:\n", " history.append((message[\"text\"], None))\n", " return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "\n", "def bot(history, response_type):\n", " if response_type == \"plot\":\n", " history[-1][1] = gr.Plot(random_plot())\n", " elif response_type == \"bokeh_plot\":\n", " history[-1][1] = gr.Plot(random_bokeh_plot())\n", " elif response_type == \"matplotlib_plot\":\n", " history[-1][1] = gr.Plot(random_matplotlib_plot())\n", " elif response_type == \"gallery\":\n", " history[-1][1] = gr.Gallery(\n", " [os.path.join(\"files\", \"avatar.png\"), os.path.join(\"files\", \"avatar.png\")]\n", " )\n", " elif response_type == \"image\":\n", " history[-1][1] = gr.Image(os.path.join(\"files\", \"avatar.png\"))\n", " elif response_type == \"video\":\n", " history[-1][1] = gr.Video(os.path.join(\"files\", \"world.mp4\"))\n", " elif response_type == \"audio\":\n", " history[-1][1] = gr.Audio(os.path.join(\"files\", \"audio.wav\"))\n", " elif response_type == \"audio_file\":\n", " history[-1][1] = (os.path.join(\"files\", \"audio.wav\"), \"description\")\n", " elif response_type == \"image_file\":\n", " history[-1][1] = (os.path.join(\"files\", \"avatar.png\"), \"description\")\n", " elif response_type == \"video_file\":\n", " history[-1][1] = (os.path.join(\"files\", \"world.mp4\"), \"description\")\n", " elif response_type == \"txt_file\":\n", " history[-1][1] = (os.path.join(\"files\", \"sample.txt\"), \"description\")\n", " else:\n", " history[-1][1] = \"Cool!\"\n", " return history\n", "\n", "\n", "fig = random_plot()\n", "\n", "with gr.Blocks(fill_height=True) as demo:\n", " chatbot = gr.Chatbot(\n", " elem_id=\"chatbot\",\n", " bubble_full_width=False,\n", " scale=1,\n", " )\n", " response_type = gr.Radio(\n", " [\n", " \"audio_file\",\n", " \"image_file\",\n", " \"video_file\",\n", " \"txt_file\",\n", " \"plot\",\n", " \"matplotlib_plot\",\n", " \"bokeh_plot\",\n", " \"image\",\n", " \"text\",\n", " \"gallery\",\n", " \"video\",\n", " \"audio\",\n", " ],\n", " value=\"text\",\n", " label=\"Response Type\",\n", " )\n", "\n", " chat_input = gr.MultimodalTextbox(\n", " interactive=True,\n", " placeholder=\"Enter message or upload file...\",\n", " show_label=False,\n", " )\n", "\n", " chat_msg = chat_input.submit(\n", " add_message, [chatbot, chat_input], [chatbot, chat_input]\n", " )\n", " bot_msg = chat_msg.then(\n", " bot, [chatbot, response_type], chatbot, api_name=\"bot_response\"\n", " )\n", " bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", " chatbot.like(print_like_dislike, None, None)\n", "\n", "demo.queue()\n", "if __name__ == \"__main__\":\n", " demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
179 changes: 179 additions & 0 deletions demo/chatbot_core_components/run.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
import gradio as gr
import os
import plotly.express as px

# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.


def random_plot():
df = px.data.iris()
fig = px.scatter(
df,
x="sepal_width",
y="sepal_length",
color="species",
size="petal_length",
hover_data=["petal_width"],
)
return fig


def print_like_dislike(x: gr.LikeData):
print(x.index, x.value, x.liked)


def random_bokeh_plot():
from bokeh.models import ColumnDataSource, Whisker
from bokeh.plotting import figure
from bokeh.sampledata.autompg2 import autompg2 as df
from bokeh.transform import factor_cmap, jitter, factor_mark

classes = list(sorted(df["class"].unique()))

p = figure(
height=400,
x_range=classes,
background_fill_color="#efefef",
title="Car class vs HWY mpg with quintile ranges",
)
p.xgrid.grid_line_color = None

g = df.groupby("class")
upper = g.hwy.quantile(0.80)
lower = g.hwy.quantile(0.20)
source = ColumnDataSource(data=dict(base=classes, upper=upper, lower=lower))

error = Whisker(
base="base",
upper="upper",
lower="lower",
source=source,
level="annotation",
line_width=2,
)
error.upper_head.size = 20
error.lower_head.size = 20
p.add_layout(error)

p.circle(
jitter("class", 0.3, range=p.x_range),
"hwy",
source=df,
alpha=0.5,
size=13,
line_color="white",
color=factor_cmap("class", "Light6", classes),
)
return p


def random_matplotlib_plot():
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

countries = ["USA", "Canada", "Mexico", "UK"]
months = ["January", "February", "March", "April", "May"]
m = months.index("January")
r = 3.2
start_day = 30 * m
final_day = 30 * (m + 1)
x = np.arange(start_day, final_day + 1)
pop_count = {"USA": 350, "Canada": 40, "Mexico": 300, "UK": 120}
df = pd.DataFrame({"day": x})
for country in countries:
df[country] = x ** (r) * (pop_count[country] + 1)

fig = plt.figure()
plt.plot(df["day"], df[countries].to_numpy())
plt.title("Outbreak in " + "January")
plt.ylabel("Cases")
plt.xlabel("Days since Day 0")
plt.legend(countries)
return fig


def add_message(history, message):
for x in message["files"]:
history.append(((x,), None))
if message["text"] is not None:
history.append((message["text"], None))
return history, gr.MultimodalTextbox(value=None, interactive=False)


def bot(history, response_type):
if response_type == "plot":
history[-1][1] = gr.Plot(random_plot())
elif response_type == "bokeh_plot":
history[-1][1] = gr.Plot(random_bokeh_plot())
elif response_type == "matplotlib_plot":
history[-1][1] = gr.Plot(random_matplotlib_plot())
elif response_type == "gallery":
history[-1][1] = gr.Gallery(
[os.path.join("files", "avatar.png"), os.path.join("files", "avatar.png")]
)
elif response_type == "image":
history[-1][1] = gr.Image(os.path.join("files", "avatar.png"))
elif response_type == "video":
history[-1][1] = gr.Video(os.path.join("files", "world.mp4"))
elif response_type == "audio":
history[-1][1] = gr.Audio(os.path.join("files", "audio.wav"))
elif response_type == "audio_file":
history[-1][1] = (os.path.join("files", "audio.wav"), "description")
elif response_type == "image_file":
history[-1][1] = (os.path.join("files", "avatar.png"), "description")
elif response_type == "video_file":
history[-1][1] = (os.path.join("files", "world.mp4"), "description")
elif response_type == "txt_file":
history[-1][1] = (os.path.join("files", "sample.txt"), "description")
else:
history[-1][1] = "Cool!"
return history


fig = random_plot()

with gr.Blocks(fill_height=True) as demo:
chatbot = gr.Chatbot(
elem_id="chatbot",
bubble_full_width=False,
scale=1,
)
response_type = gr.Radio(
[
"audio_file",
"image_file",
"video_file",
"txt_file",
"plot",
"matplotlib_plot",
"bokeh_plot",
"image",
"text",
"gallery",
"video",
"audio",
],
value="text",
label="Response Type",
)

chat_input = gr.MultimodalTextbox(
interactive=True,
placeholder="Enter message or upload file...",
show_label=False,
)

chat_msg = chat_input.submit(
add_message, [chatbot, chat_input], [chatbot, chat_input]
)
bot_msg = chat_msg.then(
bot, [chatbot, response_type], chatbot, api_name="bot_response"
)
bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])

chatbot.like(print_like_dislike, None, None)

demo.queue()
if __name__ == "__main__":
demo.launch()
Binary file removed demo/chatbot_multimodal/files/lion.jpg
Binary file not shown.
1 change: 1 addition & 0 deletions demo/chatbot_multimodal/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
plotly
2 changes: 1 addition & 1 deletion demo/chatbot_multimodal/run.ipynb
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/files/avatar.png\n", "!wget -q -O files/lion.jpg https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/files/lion.jpg"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import os\n", "import time\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", " print(x.index, x.value, x.liked)\n", "\n", "def add_message(history, message):\n", " for x in message[\"files\"]:\n", " history.append(((x,), None))\n", " if message[\"text\"] is not None:\n", " history.append((message[\"text\"], None))\n", " return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "def bot(history):\n", " response = \"**That's cool!**\"\n", " history[-1][1] = \"\"\n", " for character in response:\n", " history[-1][1] += character\n", " time.sleep(0.05)\n", " yield history\n", "\n", "with gr.Blocks() as demo:\n", " chatbot = gr.Chatbot(\n", " [],\n", " elem_id=\"chatbot\",\n", " bubble_full_width=False\n", " )\n", "\n", " chat_input = gr.MultimodalTextbox(interactive=True, file_types=[\"image\"], placeholder=\"Enter message or upload file...\", show_label=False)\n", "\n", " chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])\n", " bot_msg = chat_msg.then(bot, chatbot, chatbot, api_name=\"bot_response\")\n", " bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", " chatbot.like(print_like_dislike, None, None)\n", "\n", "demo.queue()\n", "if __name__ == \"__main__\":\n", " demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio plotly"]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/files/avatar.png"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import os\n", "import plotly.express as px\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "def random_plot():\n", " df = px.data.iris()\n", " fig = px.scatter(df, x=\"sepal_width\", y=\"sepal_length\", color=\"species\",\n", " size='petal_length', hover_data=['petal_width'])\n", " return fig\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", " print(x.index, x.value, x.liked)\n", "\n", "def add_message(history, message):\n", " for x in message[\"files\"]:\n", " history.append(((x,), None))\n", " if message[\"text\"] is not None:\n", " history.append((message[\"text\"], None))\n", " return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "def bot(history):\n", " history[-1][1] = \"Cool!\"\n", " return history\n", "\n", "fig = random_plot()\n", "\n", "with gr.Blocks(fill_height=True) as demo:\n", " chatbot = gr.Chatbot(\n", " elem_id=\"chatbot\",\n", " bubble_full_width=False,\n", " scale=1,\n", " )\n", "\n", " chat_input = gr.MultimodalTextbox(interactive=True, placeholder=\"Enter message or upload file...\", show_label=False)\n", "\n", " chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])\n", " bot_msg = chat_msg.then(bot, chatbot, chatbot, api_name=\"bot_response\")\n", " bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", " chatbot.like(print_like_dislike, None, None)\n", "\n", "demo.queue()\n", "if __name__ == \"__main__\":\n", " demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
Loading

0 comments on commit bb504b4

Please sign in to comment.