From 758ad085d623ccff4b7aaa4ed6561cb2d6966c59 Mon Sep 17 00:00:00 2001 From: Chris Riccomini Date: Tue, 10 Oct 2023 14:41:41 -0700 Subject: [PATCH] Update cLI and concpets for URL scheme --- docs/cli.md | 90 +++++++++++------------------------------------- docs/concepts.md | 37 +++----------------- 2 files changed, 25 insertions(+), 102 deletions(-) diff --git a/docs/cli.md b/docs/cli.md index 3c036a6..077b65c 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -15,114 +15,64 @@ Interacting with Recap from the command line. ## Introduction -Recap ships with a command line interface (CLI). You can use it to manage configuration, browse systems, read schemas, and start Recap's [gateway server](/docs/gateway). +Recap ships with a command line interface (CLI). You can use it to manage configuration, browse systems, read schemas, and start Recap's [gateway](/docs/gateway) and [registry](/docs/registry). ## Commands -### `add` - -Add a system to the config. - -#### Usage -{: .no_toc } - -``` -recap add [OPTIONS] SYSTEM URL -``` - -#### Parameters -{: .no_toc } - -| Option | Type | Description | Default | Required | -| :-- | :-- | :-- | :-: | :-: | -| SYSTEM | TEXT | The name of the system. | | YES | -| URL | TEXT | URL for the system. | | YES | - -#### Example -{: .no_toc } - -``` -recap add my_pg postgresql://user:pass@host:port/dbname -``` - ### `ls` -List the children of a path. - -#### Usage -{: .no_toc } - -``` -recap ls [OPTIONS] [PATH] -``` - -#### Parameters -{: .no_toc } - -| Option | Type | Description | Default | Required | -| :-- | :-- | :-- | :-: | :-: | -| PATH | TEXT | Path to list children of. | / | YES | - -#### Example -{: .no_toc } - -``` -recap ls my_pg/testdb -``` - -### `remove` - -Remove a system from the config. +List the children of a URL. #### Usage {: .no_toc } ``` -recap remove [OPTIONS] SYSTEM URL +recap ls [OPTIONS] [URL] ``` #### Parameters {: .no_toc } -| Option | Type | Description | Default | Required | -| :-- | :-- | :-- | :-: | :-: | -| SYSTEM | TEXT | The name of the system. | | YES | +| Option | Type | Description | Default | +| :-- | :-- | :-- | :-: | +| URL | TEXT | URL to parent. | - | #### Example {: .no_toc } ``` -recap remove my_pg +recap ls postgres://user:pass@localhost:5432/testdb ``` ### `schema` -Get the schema of a path. +Get the schema of a URL. #### Usage {: .no_toc } ``` -recap schema [OPTIONS] [PATH] +recap schema [OPTIONS] URL ``` #### Parameters {: .no_toc } -| Option | Type | Description | Default | Required | -| :-- | :-- | :-- | :-: | :-: | -| PATH | TEXT | Path to get schema of. | / | YES | +| Option | Type | Description | Default | +| :-- | :-- | :-- | :-: | +| URL | TEXT | URL to schema. | - | +| \-\-output-format -of | TEXT | Schema output format. `[avro|json|protobuf|recap]` | recap | #### Example {: .no_toc } ``` -recap schema my_pg/testdb/public/my_table +recap schema postgres://user:pass@localhost:5432/testdb/public/test_types ``` ### `serve` -Start Recap's HTTP/JSON gateway server. Uses [uvicorn](https://www.uvicorn.org/) under the hood. +Start Recap's HTTP/JSON server. #### Usage {: .no_toc } @@ -134,11 +84,11 @@ recap serve [OPTIONS] #### Parameters {: .no_toc } -| Option | Type | Description | Default | Required | -| :-- | :-- | :-- | :-: | :-: | -| `--host` | TEXT | Hostname to bind the server to. | / | YES | -| `--port` | TEXT | Port to bind the server to. | 8000 | YES | -| `--log-level` | TEXT | Log level for the server. | info | YES | +| Option | Type | Description | Default | +| :-- | :-- | :-- | :-: | +| `--host` | TEXT | Hostname to bind the server to. | 127.0.0.1 | +| `--port` | TEXT | Port to bind the server to. | 8000 | +| `--log-level` | TEXT | Log level for the server. | info | #### Example {: .no_toc } diff --git a/docs/concepts.md b/docs/concepts.md index 4675b48..2ccefcc 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -17,37 +17,6 @@ Understanding Recap's design. You should understand the following concepts before using Recap. -## Systems - -Systems are Recap's term for remote schema sources. They're typically databases, schema registries, data catalogs, and so on. - -Each system has a name and a URL. Systems are normally defined as environment variables. - -```bash -export MY_PG=postgresql://user:pass@host:port/dbname -``` - -See Recap's [configuration](/docs/configuration) documentation for more information. - -## Paths - -Recap takes a path as an argument when listing or reading schemas. Each system has its own path structure. For example, PostgreSQL has paths like: - -``` -[system]/[database]/[schema]/[table] -``` - -You can browse paths using the `ls` command: - -```bash -recap ls my_pg -recap ls my_pg/testdb -recap ls my_pg/testdb/public -``` - -{: .note } -This example uses Recap's [CLI](/docs/cli). You can also use Recap's [Python API](/docs/python) or [gateway](/docs/gateway). - ## Clients Each system has a [client](/docs/python#clients) implementation. Clients connect to remote systems, read schemas, and convert them to Recap schemas. Clients use converters to convert to Recap schemas. @@ -62,10 +31,14 @@ Recap's [CLI](/docs/cli) is a command line interface for connecting to systems a ## Gateway -Recap's [gateway](/docs/gateway) is a stateless HTTP/JSON server that you can use to query systems and schemas. The server doesn't require any database or storage. It's simply connects to systems and returns Recap schemas. +Recap's [gateway](/docs/gateway) is a stateless HTTP/JSON API that you can use to query systems and schemas. The server doesn't require any database or storage. It's simply connects to systems and returns Recap schemas. The gateway is useful if you're not using Python, but want to use Recap programmatically. +## Registry + +Recap's [registry](/docs/registry) is an HTTP/JSON API that stores schemas in a filesystem or object store like S3 or GCS. The registry is useful for caching schemas or acting as a repository when using Recap schemas as a source of truth. + ## Python API Recap is written in Python and is designed to be used as a Python library. See the [Python](/docs/python) documentation for more information. \ No newline at end of file