Skip to content

Commit

Permalink
docs: reflect new configuration setup (#119)
Browse files Browse the repository at this point in the history
Signed-off-by: Panos Vagenas <35837085+vagenas@users.noreply.github.com>
  • Loading branch information
vagenas authored Jul 27, 2023
1 parent e616cd8 commit a58f770
Show file tree
Hide file tree
Showing 9 changed files with 94 additions and 102 deletions.
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
*Interact with the Deep Search platform for new knowledge explorations and discoveries*


The Deep Search Toolkit is a Python SDK allowing a user to interact with the Deep Search platform. The toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.
The Deep Search Toolkit is a Python SDK and CLI allowing users to interact with the Deep Search platform.
The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.


[Learn about IBM Deep Search](https://ds4sd.github.io/)
Expand Down Expand Up @@ -53,14 +54,14 @@ Python 3.8+

## Start using the toolkit

### Login to Deep Search
For details, check the [authentication docs](https://ds4sd.github.io/deepsearch-toolkit/getting_started/#authentication).
### Set up a profile
For details, check [Profiles](https://ds4sd.github.io/deepsearch-toolkit/guide/configuration#profiles).
```console
deepsearch login
deepsearch profile config
```

### Convert a document
For details, check the [document conversion docs](https://ds4sd.github.io/deepsearch-toolkit/guide/convert_doc).
For details, check [Document conversion](https://ds4sd.github.io/deepsearch-toolkit/guide/convert_doc).
```console
deepsearch documents convert -p 1234567890abcdefghijklmnopqrstvwyz123456 -u https://arxiv.org/pdf/2206.00785.pdf
```
Expand Down
8 changes: 3 additions & 5 deletions deepsearch/model/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,12 @@ app.run(host="127.0.0.1", port=8000)

### Settings
App configuration is done in [`Settings`](server/config.py) based on
[Pydantic settings management](https://docs.pydantic.dev/latest/usage/settings/).
[Pydantic Settings with dotenv support](https://docs.pydantic.dev/dev-v1/usage/settings/).

E.g. the required API key can be set via env var `DS_MODEL_API_KEY` (for precedence rules,
check the
[Pydantic docs](https://docs.pydantic.dev/latest/usage/settings/#field-value-priority)).
E.g. the required API key can be injected via env var `DS_MODEL_API_KEY`.

### OpenAPI
The OpenAPI UI is served under `/docs`, e.g. http://127.0.0.1:8000/docs.
The OpenAPI UI is served under `/docs`, i.e. by default at http://127.0.0.1:8000/docs.

## Developing a new model
To develop a new model class for an existing [kind](kinds/), inherit from the base model
Expand Down
20 changes: 9 additions & 11 deletions docs/getting_started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,26 @@ After registering with [Deep Search](https://ds4sd.github.io/), you can obtain y


---
### Configure a profile
### Set up a profile

=== "CLI"
Configure a profile using [`deepsearch profile config`](../cli-reference.md#profile).
Set up a [profile][profiles] using `deepsearch profile config`.
In its basic form it looks like this:
<div class="termy">

```console
$ deepsearch profile config

Host: # (1)
Username: # (2)
Api key: # (3)
Host: https://ds.example.com # the Deep Search instance you are using
Username: name@example.com # your username
Api key: (hidden) # your API key
```

</div>

1. Input the Deep Search deployment host to use.
2. Enter your `User name`.
3. Enter your `API key`.

By providing a profile name (via option `--profile-name`) you can configure multiple
different profiles, which you can then easily switch between and manage.

For details, check [Profiles](../guide/configuration.md#profiles).
For details, check [Profiles][profiles].

### Validate the setup

Expand Down Expand Up @@ -89,3 +84,6 @@ Here, we show a simple way to convert documents using [Deep Search](https://ds4s
```

---


[profiles]: ../guide/configuration.md#profiles
7 changes: 2 additions & 5 deletions docs/guide/cli_plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,9 @@ Options:
--help Show this message and exit.
Commands:
config Manage CLI config files
cps Interact with DeepSearch CPS component
[ ... ] [ ... ]
example
login Login to DeepSearch platform
query Interact with DeepSearch Query component
version Print the client and server version information
[ ... ] [ ... ]
```

And running our newly added command gives us...
Expand Down
125 changes: 61 additions & 64 deletions docs/guide/configuration.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
# Configuration

The Toolkit can be configured both via the CLI and via
environment variables.
The Toolkit can be configured via the CLI and via environment variables.

Besides *global* settings, the Toolkit also allows the configuration of multiple
*profiles* for enabling users to easily work with different Deep Search deployments.

## Profiles

The Toolkit provides the capability of easily interacting with different Deep Search
instances through the use of *profiles*. A user may define multiple profiles, identified
by profile name, and can easily switch between them.

### Overview

For an overview of the profile management commands check `deepsearch profile --help`:
Expand All @@ -32,8 +35,8 @@ $ deepsearch profile --help

### Profile setup

To setup a profile use [`deepsearch profile config`](../cli-reference.md#profile),
providing options as needed (for a full reference run with `--help`).
To set up a profile use `deepsearch profile config`, providing options as needed (for a
full reference check `deepsearch profile config --help`).

Here is a basic invocation example:

Expand All @@ -44,76 +47,70 @@ Username: name@example.com
Api key:
```

> NOTE: If you had used the meanwhile deprecated `deepsearch login` command to set up a
config file in the default location, that will automatically be migrated to a profile.
!!! note
If you had used the meanwhile deprecated `deepsearch login` command to set up a
config file in the default location, that will automatically be migrated to a profile.

### Usage

Here some usage examples:

=== "CLI"

```console
$ # configure a profile
$ deepsearch profile config --profile-name foo --host ds-1.example.com ...
$ # -> "foo" now available and selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
$
$ # configure another profile
$ deepsearch profile config --profile-name bar --host ds-2.example.com ...
$ # -> "bar" now available and selected as the active profile
$
$ deepsearch profile list
$ # -> displays all profiles, with "bar" marked as the active one
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "bar"
$
$ # switch to previous profile
$ deepsearch profile use foo
$ # -> "foo" now selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
```

=== "Python"

```python
from deepsearch.cps.client.api import CpsApi

# using the active profile; let's assume that is "foo"
api = CpsApi.from_env()
print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "foo"

# using a profile by name
api = CpsApi.from_env(profile_name="bar")
print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "bar"
```

---
#### Usage in CLI

```console
$ # configure a profile
$ deepsearch profile config --profile-name foo --host ds-1.example.com ...
$ # -> "foo" now available and selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
$
$ # configure another profile
$ deepsearch profile config --profile-name bar --host ds-2.example.com ...
$ # -> "bar" now available and selected as the active profile
$
$ deepsearch profile list
$ # -> displays all profiles, with "bar" marked as the active one
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "bar"
$
$ # switch to previous profile
$ deepsearch profile use foo
$ # -> "foo" now selected as the active profile
$
$ deepsearch cps projects list
$ # -> outputs projects corresponding to "foo"
```

## Environment variables
#### Usage in Python

Under the hood, configuration management in the Toolkit is implemented based on [Pydantic
Settings with dotenv support](https://docs.pydantic.dev/1.10/usage/settings).
To use the active profile (recommended usage pattern):
```python
from deepsearch.cps.client.api import CpsApi

Configuration is persisted in Toolkit-managed dotenv files, but can be overriden via
environment variables.
api = CpsApi.from_env()

To see which environment variables are supported, check the relevant [Pydantic Settings
classes][settings_file], also taking into account any defined prefixes.
print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to active profile
```

For example you can set the profile for the scope of a single operation (e.g. a project
listing) as follows:
To use a specific profile:
```python
from deepsearch.cps.client.api import CpsApi

```console
DEEPSEARCH_PROFILE=bar deepsearch cps projects list
api = CpsApi.from_env(profile_name="foo")

print([p.name for p in api.projects.list()])
# -> outputs projects corresponding to "foo"
```

## Environment variables

Under the hood, the Toolkit leverages [Pydantic Settings with dotenv
support][pydantic_settings], so configuration settings can be easily overriden via
environment variables. This can be useful e.g. in a containerization scenario.

To see which environment variables are supported, check the relevant [Pydantic Settings
classes][settings_file], also taking into account any defined prefixes.

[pydantic_settings]: https://docs.pydantic.dev/dev-v1/usage/settings
[settings_file]: https://github.com/DS4SD/deepsearch-toolkit/blob/main/deepsearch/core/client/settings.py
8 changes: 4 additions & 4 deletions docs/guide/data_indices.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Suppose you want to create an index called `NAME`. Optionally, a description,`DE
</div>
=== "Python"

After you have generated the api object (from [login configuration](../installation/login.md)),
After you have generated the api object (from a [profile](../configuration#usage-in-python)):

```python
api.data_indices.create(proj_key=PROJ_KEY, name=NAME, desc=DESC)
Expand Down Expand Up @@ -159,9 +159,9 @@ Attachments can be added to an index item in a project. Briefly, attachments hav
attachment_key = "usr_my_attachment" # optional. if set need start with 'usr_' and be snake_case

index.add_item_attachment(
api=api,
index_item_id=index_item_id,
attachment_path=attachment_path,
api=api,
index_item_id=index_item_id,
attachment_path=attachment_path,
attachment_key=attachment_key, # optional
)
```
6 changes: 3 additions & 3 deletions docs/guide/kgs.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,20 +64,20 @@ More example queries are available in the [examples gallery](../gallery/index.md
$ deepsearch cps kgs download -p 'abc123' -k 'zxc987'

url
--------------------------------
--------------------------------
https://host.domain/download-url
```

</div>

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication))
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)):

```python
proj_key = "abc123" # get your proj_key from the UI details
kg_key = "xyz987" # get kg_key (also called bag_key) from the UI details
kg = api.knowledge_graphs.get(proj_key, kg_key)

download_url = kg.download()
```
```
7 changes: 3 additions & 4 deletions docs/guide/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo
## Project management

!!! info
Make sure to [configure your login](../getting_started/index.md#authentication) before
using the project management features listed below.
Make sure to [complete your profile setup](../getting_started/index.md#set-up-a-profile) before using the project management features listed below.

### Creating a project

Expand All @@ -39,7 +38,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication)), creating a project is very easy.
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)), creating a project is very easy.

```python
proj = api.projects.create(name="my-project")
Expand Down Expand Up @@ -70,7 +69,7 @@ A collaborator may be added to a project as `owner`, `editor`, or `viewer`. Belo

=== "Python"

After you have generated the `api` object (from [login configuration](../getting_started/#authentication)), listing projects is very easy.
After you have generated the `api` object (from a [profile](../configuration#usage-in-python)), listing projects is very easy.

```python
projects = api.projects.list() # returns list of projects
Expand Down
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@

*Interact with the Deep Search platform for new knowledge explorations and discoveries*

The Deep Search Toolkit provides a Python SDK and a CLI allowing a user to interact with the Deep Search platform. The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.
The Deep Search Toolkit is a Python SDK and CLI allowing users to interact with the Deep Search platform.
The Toolkit provides easy-to-use functionalities for several common processes such as document conversion, graph creation and querying.


[Learn about IBM Deep Search :octicons-link-external-16:](https://ds4sd.github.io/){ .md-button }

Expand Down

0 comments on commit a58f770

Please sign in to comment.