Skip to content

Commit

Permalink
Convert more connector docs to markdown
Browse files Browse the repository at this point in the history
- All JBDC based connectors
- Kafka and related connectors with some layout improvements
- Blackhole connector
- All used fragments
  • Loading branch information
mosabua committed Aug 3, 2023
1 parent f5eddd8 commit fa23a09
Show file tree
Hide file tree
Showing 63 changed files with 8,739 additions and 9,005 deletions.
3 changes: 2 additions & 1 deletion docs/src/main/sphinx/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ def setup(app):

myst_enable_extensions = [
"colon_fence",
"deflist"
"deflist",
"substitution"
]

# -- Options for HTML output ---------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
ALTER SCHEMA
^^^^^^^^^^^^
### ALTER SCHEMA

The connector supports renaming a schema with the ``ALTER SCHEMA RENAME``
statement. ``ALTER SCHEMA SET AUTHORIZATION`` is not supported.
The connector supports renaming a schema with the `ALTER SCHEMA RENAME`
statement. `ALTER SCHEMA SET AUTHORIZATION` is not supported.
15 changes: 7 additions & 8 deletions docs/src/main/sphinx/connector/alter-table-limitation.fragment
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
ALTER TABLE RENAME TO
^^^^^^^^^^^^^^^^^^^^^
### ALTER TABLE RENAME TO

The connector does not support renaming tables across multiple schemas. For
example, the following statement is supported:

.. code-block:: sql

ALTER TABLE example.schema_one.table_one RENAME TO example.schema_one.table_two
```sql
ALTER TABLE example.schema_one.table_one RENAME TO example.schema_one.table_two
```

The following statement attempts to rename a table across schemas, and therefore
is not supported:

.. code-block:: sql

ALTER TABLE example.schema_one.table_one RENAME TO example.schema_two.table_two
```sql
ALTER TABLE example.schema_one.table_one RENAME TO example.schema_two.table_two
```
39 changes: 18 additions & 21 deletions docs/src/main/sphinx/connector/avro-decoder.fragment
Original file line number Diff line number Diff line change
@@ -1,28 +1,28 @@
Avro decoder
""""""""""""
#### Avro decoder

The Avro decoder converts the bytes representing a message or key in Avro format
based on a schema. The message must have the Avro schema embedded. Trino does
not support schemaless Avro decoding.

The ``dataSchema`` must be defined for any key or message using ``Avro``
decoder. ``Avro`` decoder should point to the location of a valid Avro
The `dataSchema` must be defined for any key or message using `Avro`
decoder. `Avro` decoder should point to the location of a valid Avro
schema file of the message which must be decoded. This location can be a remote
web server (e.g.: ``dataSchema: 'http://example.org/schema/avro_data.avsc'``) or
local file system(e.g.: ``dataSchema: '/usr/local/schema/avro_data.avsc'``). The
web server (e.g.: `dataSchema: 'http://example.org/schema/avro_data.avsc'`) or
local file system(e.g.: `dataSchema: '/usr/local/schema/avro_data.avsc'`). The
decoder fails if this location is not accessible from the Trino cluster.

The following attributes are supported:

* ``name`` - Name of the column in the Trino table.
* ``type`` - Trino data type of column.
* ``mapping`` - A slash-separated list of field names to select a field from the
Avro schema. If the field specified in ``mapping`` does not exist in the
original Avro schema, a read operation returns ``NULL``.
- `name` - Name of the column in the Trino table.
- `type` - Trino data type of column.
- `mapping` - A slash-separated list of field names to select a field from the
Avro schema. If the field specified in `mapping` does not exist in the
original Avro schema, a read operation returns `NULL`.

The following table lists the supported Trino types that can be used in ``type``
The following table lists the supported Trino types that can be used in `type`
for the equivalent Avro field types:

```{eval-rst}
.. list-table::
:widths: 40, 60
:header-rows: 1
Expand All @@ -43,11 +43,11 @@ for the equivalent Avro field types:
- ``ARRAY``
* - ``MAP``
- ``MAP``
```

No other types are supported.

Avro schema evolution
+++++++++++++++++++++
##### Avro schema evolution

The Avro decoder supports schema evolution with backward compatibility. With
backward compatibility, a newer schema can be used to read Avro data created
Expand All @@ -57,16 +57,13 @@ default value in the Avro schema file.

The schema evolution behavior is as follows:

* Column added in new schema: Data created with an older schema produces a
- Column added in new schema: Data created with an older schema produces a
*default* value when the table is using the new schema.

* Column removed in new schema: Data created with an older schema no longer
- Column removed in new schema: Data created with an older schema no longer
outputs the data from the column that was removed.

* Column is renamed in the new schema: This is equivalent to removing the column
- Column is renamed in the new schema: This is equivalent to removing the column
and adding a new one, and data created with an older schema produces a
*default* value when the table is using the new schema.

* Changing type of column in the new schema: If the type coercion is supported
- Changing type of column in the new schema: If the type coercion is supported
by Avro, then the conversion happens. An error is thrown for incompatible
types.
Loading

0 comments on commit fa23a09

Please sign in to comment.