diff --git a/docs/understanding-airbyte/airbyte-specification.md b/docs/understanding-airbyte/airbyte-specification.md index 99bdf5a1ae72..ba1957c498d6 100644 --- a/docs/understanding-airbyte/airbyte-specification.md +++ b/docs/understanding-airbyte/airbyte-specification.md @@ -67,7 +67,7 @@ read(Config, AirbyteCatalog, State) -> Stream * Input: 1. none. * Output: - 1. `spec` - a [ConnectorSpecification](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L180-L199) wrapped in an `AirbyteMessage` of type `spec`. + 1. `spec` - a [ConnectorSpecification](https://github.com/airbytehq/airbyte/blob/922bfd08a9182443599b78dbb273d70cb9f63d30/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L256-L306) wrapped in an `AirbyteMessage` of type `spec`. * The objective of the spec command is to pull information about how to use a source. The `ConnectorSpecification` contains this information. * The `connectionSpecification` of the `ConnectorSpecification` must be valid JsonSchema. It describes what inputs are needed in order for the source to interact with the underlying data source. * e.g. If using a Postgres source, the `ConnectorSpecification` would specify that a `hostname`, `port`, and `password` are required in order for the connector to function. @@ -79,7 +79,7 @@ read(Config, AirbyteCatalog, State) -> Stream * Input: 1. `config` - A configuration JSON object that has been validated using the `ConnectorSpecification`. * Output: - 1. `connectionStatus` - an [AirbyteConnectionStatus](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L94-L107) wrapped in an `AirbyteMessage` of type `connection_status`. + 1. `connectionStatus` - an [AirbyteConnectionStatus](https://github.com/airbytehq/airbyte/blob/922bfd08a9182443599b78dbb273d70cb9f63d30/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L99-L112) wrapped in an `AirbyteMessage` of type `connection_status`. * The `check` command attempts to connect to the underlying data source in order to verify that the provided credentials are usable. * e.g. If the given the credentials, it can connect to the Postgres database, it will return a success response. If it fails \(perhaps the password is incorrect\), it will return a failed response and \(when possible\) a helpful error message. @@ -88,7 +88,7 @@ read(Config, AirbyteCatalog, State) -> Stream * Input: 1. `config` - A configuration JSON object that has been validated using the `ConnectorSpecification`. * Output: - 1. `catalog` - an [AirbyteCatalog](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L108-L118) wrapped in an `AirbyteMessage` of type `catalog`. + 1. `catalog` - an [AirbyteCatalog](https://github.com/airbytehq/airbyte/blob/922bfd08a9182443599b78dbb273d70cb9f63d30/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L113-L123) wrapped in an `AirbyteMessage` of type `catalog`. * This command detects the _structure_ of the data in the data source. * An `AirbyteCatalog` describes the structure of data in a data source. It has a single field called `streams` that contains a list of `AirbyteStream`s. Each of these contain a `name` and `json_schema` field. The `json_schema` field accepts any valid JsonSchema and describes the structure of a stream. This data model is intentionally flexible. That can make it a little hard at first to mentally map onto your own data, so we provide some examples below: * If we are using a data source that is a traditional relational database, each table in that database would map to an `AirbyteStream`. Each column in the table would be a key in the `properties` field of the `json_schema` field. @@ -215,7 +215,7 @@ For the sake of brevity, we will not re-describe `spec` and `check`. They are ex ## The Airbyte Protocol -* All messages passed to and from connectors must be wrapped in an `AirbyteMesage` envelope and serialized as JSON. The JsonSchema specification for these messages can be found [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml). +* All messages passed to and from connectors must be wrapped in an `AirbyteMessage` envelope and serialized as JSON. The JsonSchema specification for these messages can be found [here](https://github.com/airbytehq/airbyte/blob/922bfd08a9182443599b78dbb273d70cb9f63d30/airbyte-protocol/models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml#L13-L45). * Even if a record is wrapped in an `AirbyteMessage` it will only be processed if it appropriate for the given command. e.g. If a source `read` action includes AirbyteMessages in its stream of type Catalog for instance, these messages will be ignored as the `read` interface only expects `AirbyteRecordMessage`s and `AirbyteStateMessage`s. The appropriate `AirbyteMessage` types have been described in each command above. * **ALL** actions are allowed to return `AirbyteLogMessage`s on stdout. For brevity, we have not mentioned these log messages in the description of each action, but they are always allowed. An `AirbyteLogMessage` wraps any useful logging that the connector wants to provide. These logs will be written to Airbyte's log files and output to the console. * I/O: