Skip to content

Commit

Permalink
Fixes relative links above docs to be URIs (#17850)
Browse files Browse the repository at this point in the history
also adds a missing exit 1 to deploy docs
when the deploy SHOULD fail
  • Loading branch information
supertopher authored Oct 11, 2022
1 parent 286b51b commit 9c46e5a
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 8 deletions.
16 changes: 8 additions & 8 deletions docs/understanding-airbyte/database-data-catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
* The `release_stage` describes the certification level of the connector (e.g. Alpha, Beta, Generally Available).
* The `docker_repository` field is the name of the docker image associated with the connector definition. `docker_image_tag` is the tag of the docker image and the version of the connector definition.
* The `source_type` field is only used for Sources, and represents the category of the connector definition (e.g. API, Database).
* The `resource_requirements` field sets a default resource requirement for any connector of this type. This overrides the default we set for all connector definitions, and it can be overridden by a connection-specific resource requirement. The column is a JSON blob with the schema defined in [ActorDefinitionResourceRequirements.yaml](airbyte-config/config-models/src/main/resources/types/ActorDefinitionResourceRequirements.yaml)
* The `resource_requirements` field sets a default resource requirement for any connector of this type. This overrides the default we set for all connector definitions, and it can be overridden by a connection-specific resource requirement. The column is a JSON blob with the schema defined in [ActorDefinitionResourceRequirements.yaml](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/ActorDefinitionResourceRequirements.yaml)
* The `public` boolean column, describes if a connector is available to all workspaces or not. For non, `public` connector definitions, they can be provisioned to a workspace using the `actor_definition_workspace_grant` table. `custom` means that the connector is written by a user of the platform (and not packaged into the Airbyte product).
* Each record contains additional metadata and display data about a connector (e.g. `name` and `icon`), and we should add additional metadata here over time.
* `actor_definition_workspace_grant`
Expand All @@ -31,9 +31,9 @@
* todo (cgardens) - should we remove the `modified_at` column? These records should be immutable.
* `connection`
* Each record in this table configures a connection (`source_id`, `destination_id`, and relevant configuration).
* The `resource_requirements` field sets a default resource requirement for the connection. This overrides the default we set for all connector definitions and the default set for the connector definitions. The column is a JSON blob with the schema defined in [ResourceRequirements.yaml](airbyte-config/config-models/src/main/resources/types/ResourceRequirements.yaml).
* The `resource_requirements` field sets a default resource requirement for the connection. This overrides the default we set for all connector definitions and the default set for the connector definitions. The column is a JSON blob with the schema defined in [ResourceRequirements.yaml](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/ResourceRequirements.yaml).
* The `source_catalog_id` column is a foreign key to the `sourc_catalog` table and represents the catalog that was used to configure the connection. This should not be confused with the `catalog` column which contains the [ConfiguredCatalog](airbyte-protocol.md#catalog) for the connection.
* The `schedule_type` column defines what type of schedule is being used. If the `type` is manual, then `schedule_data` will be null. Otherwise, `schedule_data` column is a JSON blob with the schema of [StandardSync#scheduleData](airbyte-config/config-models/src/main/resources/types/StandardSync.yaml#79) that defines the actual schedule. The columns `manual` and `schedule` are deprecated and should be ignored (they will be dropped soon).
* The `schedule_type` column defines what type of schedule is being used. If the `type` is manual, then `schedule_data` will be null. Otherwise, `schedule_data` column is a JSON blob with the schema of [StandardSync#scheduleData](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/StandardSync.yaml#L74) that defines the actual schedule. The columns `manual` and `schedule` are deprecated and should be ignored (they will be dropped soon).
* The `namespace_type` column configures whether the namespace for the connection should use that defined by the source, the destination, or a user-defined format (`custom`). If `custom` the `namespace_format` column defines the string that will be used as the namespace.
* The `status` column describes the activity level of the connector: `active` - current schedule is respected, `inactive` - current schedule is ignored (the connection does not run) but it could be switched back to active, and `deprecated` - the connection is permanently off (cannot be moved to active or inactive).
* `state`
Expand All @@ -49,8 +49,8 @@
* Each record in this table represents a stream in a connection that is enqueued to be reset or is currently being reset. It can be thought of as a queue. Once the stream is reset, the record is removed from the table.
* `operation`
* The `operation` table transformations for a connection beyond the raw output produced by the destination. The two options are: `normalization`, which outputs Airbyte's basic normalization. The second is `dbt`, which allows a user to configure their own custom dbt transformation. A connection can have multiple operations (e.g. it can do `normalization` and `dbt`).
* If the `operation` is `dbt`, then the `operator_dbt` column will be populated with a JSON blob with the schema from [OperatorDbt](airbyte-config/config-models/src/main/resources/types/OperatorDbt.yaml).
* If the `operation` is `normalization`, then the `operator_dbt` column will be populated with a JSON blob with the scehma from [OperatorNormalization](airbyte-config/config-models/src/main/resources/types/OperatorNormalization.yaml).
* If the `operation` is `dbt`, then the `operator_dbt` column will be populated with a JSON blob with the schema from [OperatorDbt](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/OperatorDbt.yaml).
* If the `operation` is `normalization`, then the `operator_dbt` column will be populated with a JSON blob with the scehma from [OperatorNormalization](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/OperatorNormalization.yaml).
* Operations are scoped by workspace, using the `workspace_id` column.
* `connection_operation`
* This table joins the `operation` table to the `connection` for which it is configured.
Expand All @@ -72,7 +72,7 @@
* Each record in this table represents a job.
* The `config_type` column captures the type of job. We only make jobs for `sync` and `reset` (we do not use them for `spec`, `check`, `discover`).
* A job represents an attempt to use a connector (or a pair of connectors). The goal of this model is to capture the input of that run. A job can have multiple attempts (see the `attempts` table). The guarantee across all attempts is that the input into each attempt will be the same.
* That input is captured in the `config` column. This column is a JSON Blob with the schema of a [JobConfig](airbyte-config/config-models/src/main/resources/types/JobConfig.yaml). Only `sync` and `resetConnection` are ever used in that model.
* That input is captured in the `config` column. This column is a JSON Blob with the schema of a [JobConfig](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/JobConfig.yaml). Only `sync` and `resetConnection` are ever used in that model.
* The other top-level fields are vestigial from when `spec`, `check`, `discover` were used in this model (we will eventually remove them).
* The `scope` column contains the `connection_id` for the relevant connection of the job.
* Context: It is called `scope` and not `connection_id`, because, this table was originally used for `spec`, `check`, and `discover`, and in those cases the `scope` referred to the relevant actor or actor definition. At this point the scope is always a `connection_id`.
Expand All @@ -81,10 +81,10 @@
* Each record in this table represents an attempt.
* Each attempt belongs to a job--this is captured by the `job_id` column. All attempts for a job will run on the same input.
* The `id` column is a unique id across all attempts while the `attempt_number` is an ascending number of the attempts for a job.
* The output of each attempt, however, can be different. The `output` column is a JSON blob with the schema of a [JobOutput](airbyte-config/config-models/src/main/resources/types/StandardSyncOutput.yaml). Only `sync` is used in that model. Reset jobs will also use the `sync` field, because under the hood `reset` jobs end up just doing a `sync` with special inputs. This object contains all the output info for a sync including stats on how much data was moved.
* The output of each attempt, however, can be different. The `output` column is a JSON blob with the schema of a [JobOutput](ahttps://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/StandardSyncOutput.yaml). Only `sync` is used in that model. Reset jobs will also use the `sync` field, because under the hood `reset` jobs end up just doing a `sync` with special inputs. This object contains all the output info for a sync including stats on how much data was moved.
* The other top-level fields are vestigial from when `spec`, `check`, `discover` were used in this model (we will eventually remove them).
* The `status` column contains the attempt status. The lifecycle of a job / attempt is explained in detail in the [Jobs & Workers documentation](jobs.md#job-state-machine).
* If the attempt fails, the `failure_summary` column will be populated. The column is a JSON blob with the schema of (AttemptFailureReason)[airbyte-config/config-models/src/main/resources/types/AttemptFailureSummary.yaml].
* If the attempt fails, the `failure_summary` column will be populated. The column is a JSON blob with the schema of [AttemptFailureReason](https://github.com/airbytehq/airbyte/blob/master/airbyte-config/config-models/src/main/resources/types/AttemptFailureSummary.yaml).
* The `log_path` column captures where logs for the attempt will be written.
* `created_at`, `started_at`, and `ended_at` track the run time.
* The `temporal_workflow_id` column keeps track of what temporal execution is associated with the attempt.
Expand Down
1 change: 1 addition & 0 deletions tools/bin/deploy_docusaurus
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ yarn install
if ! yarn build; then
echo -e "$red_text""yarn build has failed. Documentation probably has broken links""$default_text"
echo -e "$red_text""please fix the links, commit, and try again""$default_text"
exit 1
fi

# Check tty for local/remote deploys (we expect cloud to be non-interactive)
Expand Down

0 comments on commit 9c46e5a

Please sign in to comment.