Skip to content

Commit

Permalink
Initial docs (jupyterlab#22)
Browse files Browse the repository at this point in the history
* Initial docs commit

* Change authors

* Updates theming, adds image

* Removes stray whitespace

* Adds basic intro

* Removes indices

* Switches to Markdown; adds user docs file

* Adds user docs

* Adds configuration info

* Makes URLs into links

* Mentions jupyter_ai_chatgpt

* Early contributor docs

* Adds user docs with screen shots

* Adds AI module and task description

* Docs on creating a new AI module

* Notes that modality and insertion mode are extensible

* Adds docs/requirements.txt

* Removes colon_fence from requirements

* Adds definition of Jupyter AI at top level

* Adds image to top level docs

* Plugin -> AI modules

* "new tasks" are most important of the three
  • Loading branch information
JasonWeill authored and Marchlak committed Oct 28, 2024
1 parent c79f5fe commit 74e552f
Show file tree
Hide file tree
Showing 14 changed files with 368 additions and 0 deletions.
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
35 changes: 35 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.https://www.sphinx-doc.org/
exit /b 1
)

if "%1" == "" goto help

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%

:end
popd
2 changes: 2 additions & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
myst_parser
pydata_sphinx_theme
Binary file added docs/source/_static/codify-cell-menu.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/_static/gai-explain-code.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/_static/generate-output-dialog.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/_static/jupyter-ai-toolbar.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/_static/jupyter_logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
54 changes: 54 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information

project = 'Jupyter AI'
copyright = '2023, Project Jupyter'
author = 'Project Jupyter'
release = '0.3.0'

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = ["myst_parser"]
myst_enable_extensions = ["colon_fence"]

templates_path = ['_templates']
exclude_patterns = []

# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = "pydata_sphinx_theme"
html_static_path = ['_static']

# -- Jupyter theming -------------------------------------------------
html_logo = "_static/jupyter_logo.png"

html_theme_options = {
"logo": {
"text": "Jupyter AI",
},
"icon_links": [
{
# Label for this link
"name": "GitHub",
# URL where the link will redirect
"url": "https://github.com/jupyterlab/jupyter-ai", # required
# Icon class (if "type": "fontawesome"), or path to local image (if "type": "local")
"icon": "fab fa-github-square",
# The type of image to be used (see below for details)
"type": "fontawesome",
},
{
"name": "jupyter.org",
"url": "https://jupyter.org",
"icon": "_static/jupyter_logo.png",
"type": "local",
},
],
}
102 changes: 102 additions & 0 deletions docs/source/contributors/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Contributors

This page is intended for people interested in building new or modified functionality for Jupyter AI.

## AI module and task definitions

Jupyter AI is distributed as a monorepo, including the core `jupyter_ai` package and a set of AI modules.

An **AI module** provides an interface between a Jupyter notebook and a generative artificial intelligence (GAI) algorithm. It defines one or more **tasks**. A task has the following properties, as specified in `tasks.py`:

* **ID** (`id`) — A unique text identifier for the task.
* **Task name** (`name`) — A friendly name that appears in the Jupyter notebook user interface. It should be styled as a menu item would, with a verb and without definite articles.
* Good task names: "Generate image below", "Explain code"
* Bad task names: "Image generation" (no verb), "Explain the code" (definite article)
* **Prompt template** (`prompt_template`) — The prompt to be sent to the GAI algorithm, with the special string `{body}` to be replaced with the contents of the current cell or the highlighted portion of the cell.
* **Modality** (`modality`) — The type of information that the task takes as input and emits as output. Supported modalities include:
* `txt2txt` — Text input, text output (as code, markdown, or both)
* `txt2img` — Text input, image output
* **Insertion mode** (`insertion_mode`) — The way the task's output will be added to the notebook or document. Supported insertion modes include:
* `above` — AI output will be inserted above the selected text
* `replace` — AI output will replace the selected text
* `below` — AI output will be inserted below the selected text
* `above-in-cells` — AI output will be inserted above the current cell, in new notebook cells
* `below-in-cells` — AI output will be inserted below the current cell, in new notebook cells
* `below-in-image` — AI output will be inserted below the current cell, as an image

AI modules can register new tasks, modalities, and insertion modes.

## Prerequisites

You can develop Jupyter AI on any system that can run a supported Python version, including recent Windows, macOS, and Linux versions. If you have not already done so, [download Python](https://www.python.org/downloads/) and install it. The commands below presume that you can run `python` and `pip` from your preferred terminal.

## Development install

First, install the Hatch CLI, which installs the Hatchling build backend automatically.

```
pip install hatch
```

Then, enter the default hatch environment, which automatically installs all dependencies and executes development setup when entering for the first time. This command must be run from the root of the monorepo (`<jupyter-ai-top>`).

```
cd <jupyter-ai-top>
hatch shell
```

Set up your development environment and start the server:

```
jlpm setup:dev # only needs to be run once
jlpm dev
```

Finally, in a separate shell, enter the hatch environment and build the project after making any changes.

```
cd <jupyter-ai-top>
hatch shell
jlpm build
```

To exit the hatch environment, on a blank command prompt, run `exit` or press `Ctrl+D`.

If installation fails for any reason, you will have to first uninstall the hatch environment and then test your fix by reinstalling.

## Creating a new AI module

You can use the **Jupyter AI Module Cookiecutter** to create a new AI module easily. The AI module constructed from the template serves as a very simple example that can be extended however you wish.

First, install `cookiecutter`.

```
pip install cookiecutter
```

Then, from the root of your `jupyter-ai` repository, run these commands:

```
cd packages/
cookiecutter jupyter-ai-module-cookiecutter
```

Follow the prompts to create a new AI module under `packages/`. Your labextension name should use hyphens, whereas your Python name should use underscores.

To integrate the new AI module into the Jupyter AI monorepo, run this command from the AI module root:

```
rm -r .github/ binder/ CHANGELOG.md RELEASE.md
```

Rename the JS package to be scoped under `@jupyter-ai/`.

Finally, add the Python package to the `options.python_packages` field in `.jupyter-releaser.toml`.

## Development uninstall

To uninstall your Jupyter AI development environment, remove the Hatch environment:

```
hatch env remove default
```
18 changes: 18 additions & 0 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Jupyter AI

**Jupyter AI** is a package that lets Jupyter notebook users run tasks using generative AI (GAI) using APIs provided by GAI model vendors.

![Jupyter AI adds a cell toolbar button that can explain code in one click](_static/gai-explain-code.gif)

Jupyter AI provides a framework for AI modules that define GAI-powered tasks. For example, a "generate code" task uses a large language model (LLM) to generate source code from a text description, and an "explain code" task could use that same LLM to provide a plain English explanation for what some source code does. AI modules can read and write text, images, or other media formats, and can work on multiple file types — not just Jupyter notebooks. AI modules are Python packages that provide GAI interfaces and can provide JupyterLab extensions as well. They can register new models, new insertion modes, and new tasks.

## Contents

```{toctree}
---
maxdepth: 2
---
users/index
contributors/index
```
137 changes: 137 additions & 0 deletions docs/source/users/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Users

This page is intended for people interested in installing and using Jupyter AI. If you are interested in contributing a new or modified feature in JupyterLab, please see our {doc}`contributor's guide </contributors/index>`.

## Prerequisites

To use Jupyter AI, you will need to have JupyterLab ≥ 3.5 (*not* JupyterLab 4) installed.

To use Jupyter AI with OpenAI APIs, such as GPT-3 and ChatGPT, you will need
an OpenAI account and API key. You can generate an API key at
[https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys).

:::{attention}
:name: open-ai-cost
OpenAI may charge users for API usage. Jupyter AI users are responsible for all charges
they incur when they make API requests. See [OpenAI's pricing page](https://openai.com/pricing)
for details.
:::

## Installing

To install the extension, execute:

```bash
pip install jupyter_ai
```

The command above should install two extensions: a server extension and a frontend extension.

You can also install a particular AI module by running `pip install`. For example, to install the ChatGPT AI module, run:

```bash
pip install jupyter_ai_chatgpt
```

You can check that the Jupyter AI server extension is enabled by running:

```bash
jupyter server extension list
```

To verify that the frontend extension is installed, run:

```bash
jupyter labextension list
```

## Configuring with OpenAI

The Jupyter AI package includes AI modules that work with OpenAI's ChatGPT and GPT-3 models.
To use these models, you will need to create an OpenAI account and create API keys.
You can generate an API key at [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys).
Please note that [OpenAI may charge for API usage](#open-ai-cost).

### Configuring GPT-3

To use the `GPT3ModelEngine` in `jupyter_ai`, you will need an OpenAI API key.
Copy the API key and then create a Jupyter config file locally at `config.py` to
store the API key.

```python
c.GPT3ModelEngine.api_key = "<your-api-key>"
```

Finally, start a new JupyterLab instance pointing to this configuration file.

```bash
jupyter lab --config=config.py
```

If you are doing this in a Git repository, you can ensure you never commit this
file by accident by adding it to `.git/info/exclude`.

Alternatively, you can also specify your API key while launching JupyterLab.

```bash
jupyter lab --GPT3ModelEngine.api_key=<api-key>
```

### Configuring ChatGPT

To use the `ChatGptModelEngine` in `jupyter_ai`, add your ChatGPT API key to `config.py`.

```python
c.ChatGptModelEngine.api_key = "<your-api-key>"
```

Then, start a new JupyterLab instance pointing to this configuration file.

```bash
jupyter lab --config=config.py
```

If you would prefer not to use a configuration file, you can specify your API key
while launching JupyterLab.

```bash
jupyter lab --ChatGptModelEngine.api_key=<api-key>
```

## Using the extension in JupyterLab

### Basic usage

In JupyterLab, once the Jupyter AI extension is installed and configured, you should see a Jupyter AI button in the cell toolbar:

![The Jupyter AI button appears at the left of the cell toolbar](../_static/jupyter-ai-toolbar.png)

If you click the button on a markdown cell, it will run the "generate code" task, and it will create new cells with markdown and code.

If you click the button on a code cell, it will run the "explain code" tasks, and it will create a new cell with a markdown explanation of what this code does.

You can also run the "generate code" or "explain code" task by right-clicking on a cell and clicking "Codify cell with AI".

![The JupyterLab context menu includes an option to "Codify cell with AI"](../_static/codify-cell-menu.png)

### Advanced usage

You can highlight all or some of the text in a cell, and select "Generate output from selection with AI…", to bring up a dialog with more AI options.

![If you select text in a cell, the JupyterLab context menu includes an option to "Generate output from selection with AI…"](../_static/generate-output-from-selection.png)

This dialog will show all available tasks and it shows a preview of what the prompt will look like. The special string `{body}` in the prompt template will be replaced with the selected text.

![The dialog shows all tasks from all model engines and includes a prompt template for what will be sent to the specified model](../_static/generate-output-dialog.png)

If the selected task has a description, the dialog will show that as well.

![The dialog will show a task description if a task has one.](../_static/generate-code-dialog-description.png)

## Uninstalling

To remove the extension, execute:

```bash
pip uninstall jupyter_ai
```

0 comments on commit 74e552f

Please sign in to comment.