Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CRD/APIService sections to CSV topic #15089

Merged
merged 1 commit into from
Sep 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions applications/operator_sdk/osdk-generating-csvs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ include::modules/osdk-how-csv-gen-works.adoc[leveloffset=+1]
include::modules/osdk-csv-composition-configuration.adoc[leveloffset=+1]
include::modules/osdk-manually-defined-csv-fields.adoc[leveloffset=+1]
include::modules/osdk-generating-a-csv.adoc[leveloffset=+1]
include::modules/osdk-crds.adoc[leveloffset=+1]
include::modules/osdk-apiservices.adoc[leveloffset=+1]

////
TODO: discuss whether multiple CSV files can be present, each with a unique file name (ex. `app-operator.csv.0.1.1.yaml`), or a single `app-operator.csv.yaml` file that relies on VCS (git) to version the file.
Expand Down
141 changes: 141 additions & 0 deletions modules/osdk-apiservices.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
// Module included in the following assemblies:
//
// * applications/operator_sdk/osdk-generating-csvs.adoc

[id="osdk-apiservices_{context}"]
= Understanding your API services

As with CRDs, there are two types of APIServices that your Operator may use:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ecordell Do we have the two sections for APIServices in the csv object? I'm sorry, but I haven't seen any csv which contains the owned/required for APIService in community-operators. I can only find the csv which contain owned/required for CRD.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @njhale since I believe you authored the upstream version via operator-framework/operator-lifecycle-manager#492.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there are both sections for apiservices as well. I beleive the template service broker and ansible service broker make use of them?

But yes, the vast majority of operators are CRD-based (and this is a good thing)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, got it, thanks!

_owned_ and _required_.

[id="osdk-apiservices-owned_{context}"]
== Owned APIServices

When a CSV owns an APIService, it is responsible for describing the deployment
of the extension `api-server` that backs it and the `group-version-kinds` it
provides.

An APIService is uniquely identified by the `group-version` it provides and can
be listed multiple times to denote the different kinds it is expected to
provide.

.Owned APIService fields
[cols="2a,5a,2",options="header"]
|===
|Field |Description |Required/Optional

|`Group`
|Group that the APIService provides, for example `database.example.com`.
|Required

|`Version`
|Version of the APIService, for example `v1alpha1`.
|Required

|`Kind`
|A kind that the APIService is expected to provide.
|Required

|`Name`
|The plural name for the APIService provided
|Required

|`DeploymentName`
|Name of the deployment defined by your CSV that corresponds to your APIService
(required for owned APIServices). During the CSV pending phase, the OLM Operator
searches your CSV's InstallStrategy for a deployment spec with a matching name,
and if not found, does not transition the CSV to the install ready phase.
|Required

|`DisplayName`
|A human readable version of your APIService name, for example `MongoDB Standalone`.
|Required

|`Description`
|A short description of how this APIService is used by the Operator or a
description of the functionality provided by the APIService.
|Required

|`Resources`
a|Your APIServices own one or more types of Kubernetes objects. These are listed
in the resources section to inform your users of the objects they might need to
troubleshoot or how to connect to the application, such as the Service or
Ingress rule that exposes a database.

It is recommended to only list out the objects that are important to a human,
not an exhaustive list of everything you orchestrate. For example, ConfigMaps
that store internal state that should not be modified by a user should not
appear here.
|Optional

|`SpecDescriptors`, `StatusDescriptors`, and `ActionDescriptors`
|Essentially the same as for owned CRDs.
|Optional
|===

[id="osdk-apiservices-resource-creation_{context}"]
=== APIService Resource Creation

The Operator Lifecycle Manager (OLM) is responsible for creating or replacing
the Service and APIService resources for each unique owned APIService:

* Service Pod selectors are copied from the CSV deployment matching the
APIServiceDescription's `DeploymentName`.

* A new CA key/cert pair is generated for for each installation and the
base64-encoded CA bundle is embedded in the respective APIService resource.

[id="osdk-apiservices-service-certs_{context}"]
=== APIService Serving Certs

The OLM handles generating a serving key/cert pair whenever an owned APIService
is being installed. The serving certificate has a CN containing the host name of
the generated Service resource and is signed by the private key of the CA bundle
embedded in the corresponding APIService resource.

The cert is stored as a type `kubernetes.io/tls` Secret in the deployment
namespace, and a Volume named `apiservice-cert` is automatically appended to the
Volumes section of the deployment in the CSV matching the
APIServiceDescription's `DeploymentName` field.

If one does not already exist, a VolumeMount with a matching name is also
appended to all containers of that deployment. This allows users to define a
VolumeMount with the expected name to accommodate any custom path requirements.
The generated VolumeMount's path defaults to
`/apiserver.local.config/certificates` and any existing VolumeMounts with the
same path are replaced.

[id="osdk-apiservice-required_{context}"]
== Required APIServices

The OLM ensures all required CSVs have an APIService that is available and all
expected `group-version-kinds` are discoverable before attempting installation.
This allows a CSV to rely on specific kinds provided by APIServices it does not
own.

.Required APIService fields
[cols="2a,5a,2",options="header"]
|===
|Field |Description |Required/Optional

|`Group`
|Group that the APIService provides, for example `database.example.com`.
|Required

|`Version`
|Version of the APIService, for example `v1alpha1`.
|Required

|`Kind`
|A kind that the APIService is expected to provide.
|Required

|`DisplayName`
|A human readable version of your APIService name, for example `MongoDB Standalone`.
|Required

|`Description`
|A short description of how this APIService is used by the Operator or a
description of the functionality provided by the APIService.
|Required
|===
218 changes: 218 additions & 0 deletions modules/osdk-crds.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
// Module included in the following assemblies:
//
// * applications/operator_sdk/osdk-generating-csvs.adoc

[id="osdk-crds_{context}"]
= Understanding your Custom Resource Definitions (CRDs)

There are two types of Custom Resource Definitions (CRDs) that your Operator may
use: ones that are _owned_ by it and ones that it depends on, which are
_required_.

[id="osdk-crds-owned_{context}"]
== Owned CRDs

The CRDs owned by your Operator are the most important part of your CSV. This
establishes the link between your Operator and the required RBAC rules,
dependency management, and other Kubernetes concepts.

It is common for your Operator to use multiple CRDs to link together concepts,
such as top-level database configuration in one object and a representation of
ReplicaSets in another. Each one should be listed out in the CSV file.

.Owned CRD fields

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ecordell Do we need to point out the must or optional fields? For example, etcd csv, there are no Group and ActionDescriptors fields.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ecordell what do you think?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense to me; though the fields are defined required/optional in the schema for the api.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated Tables 3, 4, 5, and 6 with a Required/Optional column, based on what was listed as required for customresourcedefinitions and apiservicedefinitions in https://github.com/operator-framework/operator-lifecycle-manager/blob/master/manifests/0000_50_olm_03-clusterserviceversion.crd.yaml.

[cols="2a,5a,2",options="header"]
|===
|Field |Description |Required/Optional

|`Name`
|The full name of your CRD.
|Required

|`Version`
|The version of that object API.
|Required

|`Kind`
|The machine readable name of your CRD.
|Required

|`DisplayName`
|A human readable version of your CRD name, for example `MongoDB Standalone`.
|Required

|`Description`
|A short description of how this CRD is used by the Operator or a description of
the functionality provided by the CRD.
|Required

|`Group`
|The API group that this CRD belongs to, for example `database.example.com`.
|Optional

|`Resources`
a|Your CRDs own one or more types of Kubernetes objects. These are listed in the resources section to inform your users of the objects they might need to troubleshoot or how to connect to the application, such as the Service or Ingress rule that exposes a database.

It is recommended to only list out the objects that are important to a human,
not an exhaustive list of everything you orchestrate. For example, ConfigMaps
that store internal state that should not be modified by a user should not
appear here.
|Optional

|`SpecDescriptors`, `StatusDescriptors`, and `ActionDescriptors`
a|These Descriptors are a way to hint UIs with certain inputs or outputs of your
Operator that are most important to an end user. If your CRD contains the name
of a Secret or ConfigMap that the user must provide, you can specify that here.
These items are linked and highlighted in compatible UIs.

There are three types of descriptors:

* `SpecDescriptors`: A reference to fields in the `spec` block of an object.
* `StatusDescriptors`: A reference to fields in the `status` block of an object.
* `ActionDescriptors`: A reference to actions that can be performed on an object.

All Descriptors accept the following fields:

* `DisplayName`: A human readable name for the Spec, Status, or Action.
* `Description`: A short description of the Spec, Status, or Action and how it is
used by the Operator.
* `Path`: A dot-delimited path of the field on the object that this descriptor describes.
* `X-Descriptors`: Used to determine which "capabilities" this descriptor has and
which UI component to use. See the *openshift/console* project for a canonical
link:https://github.com/openshift/console/blob/master/frontend/public/components/operator-lifecycle-manager/descriptors/types.ts#L5-L26[list of React UI X-Descriptors] for {product-title}.

Also see the *openshift/console* project for more information on
link:https://github.com/openshift/console/tree/master/frontend/public/components/operator-lifecycle-manager/descriptors[Descriptors]
in general.
|Optional

|===

The following example depicts a `MongoDB Standalone` CRD that requires some user
input in the form of a Secret and ConfigMap, and orchestrates Services,
StatefulSets, Pods and ConfigMaps:

[id="osdk-crds-owned-example_{context}"]
.Example owned CRD
[source,yaml]
----
- displayName: MongoDB Standalone
group: mongodb.com
kind: MongoDbStandalone
name: mongodbstandalones.mongodb.com
resources:
- kind: Service
name: ''
version: v1
- kind: StatefulSet
name: ''
version: v1beta2
- kind: Pod
name: ''
version: v1
- kind: ConfigMap
name: ''
version: v1
specDescriptors:
- description: Credentials for Ops Manager or Cloud Manager.
displayName: Credentials
path: credentials
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:selector:core:v1:Secret'
- description: Project this deployment belongs to.
displayName: Project
path: project
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:selector:core:v1:ConfigMap'
- description: MongoDB version to be installed.
displayName: Version
path: version
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:label'
statusDescriptors:
- description: The status of each of the Pods for the MongoDB cluster.
displayName: Pod Status
path: pods
x-descriptors:
- 'urn:alm:descriptor:com.tectonic.ui:podStatuses'
version: v1
description: >-
MongoDB Deployment consisting of only one host. No replication of
data.
----

[id="osdk-crds-required_{context}"]
== Required CRDs

Relying on other required CRDs is completely optional and only exists to reduce
the scope of individual Operators and provide a way to compose multiple
Operators together to solve an end-to-end use case.

An example of this is an Operator that might set up an application and install
an etcd cluster (from an etcd Operator) to use for distributed locking and a
Postgres database (from a Postgres Operator) for data storage.

The Operator Lifecycle Manager (OLM) checks against the available CRDs and

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ecordell OLM will resolve and install the required CRD, right? Do we need to point out it here?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is describing the "single-operator" workflow. If you have no catalogs and no subscriptions, OLM will still check that your required CRDs exist (if, for example, you have installed it outside of OLM)

Operators in the cluster to fulfill these requirements. If suitable versions are
found, the Operators are started within the desired namespace and a Service
Account created for each Operator to create, watch, and modify the Kubernetes
resources required.

.Required CRD fields
[cols="2a,5a,2",options="header"]
|===
|Field |Description |Required/Optional

|`Name`
|The full name of the CRD you require.
|Required

|`Version`
|The version of that object API.
|Required

|`Kind`
|The Kubernetes object kind.
|Required

|`DisplayName`
|A human readable version of the CRD.
|Required

|`Description`
|A summary of how the component fits in your larger architecture.
|Required
|===

.Example required CRD
[source,yaml]
----
required:
- name: etcdclusters.etcd.database.coreos.com
version: v1beta2
kind: EtcdCluster
displayName: etcd Cluster
description: Represents a cluster of etcd nodes.
----

[id="osdk-crds-templates_{context}"]
== CRD templates

Users of your Operator will need to be aware of which options are required
versus optional. You can provide templates for each of your CRDs with a minimum
set of configuration as an annotation named `alm-examples`. Compatible UIs will
pre-fill this template for users to further customize.

The annotation consists of a list of the `kind`, for example, the CRD name and
the corresponding `metadata` and `spec` of the Kubernetes object.

The following full example provides templates for `EtcdCluster`, `EtcdBackup`
and `EtcdRestore`:

[source,yaml]
----
metadata:
annotations:
alm-examples: >-
[{"apiVersion":"etcd.database.coreos.com/v1beta2","kind":"EtcdCluster","metadata":{"name":"example","namespace":"default"},"spec":{"size":3,"version":"3.2.13"}},{"apiVersion":"etcd.database.coreos.com/v1beta2","kind":"EtcdRestore","metadata":{"name":"example-etcd-cluster"},"spec":{"etcdCluster":{"name":"example-etcd-cluster"},"backupStorageType":"S3","s3":{"path":"<full-s3-path>","awsSecret":"<aws-secret>"}}},{"apiVersion":"etcd.database.coreos.com/v1beta2","kind":"EtcdBackup","metadata":{"name":"example-etcd-cluster-backup"},"spec":{"etcdEndpoints":["<etcd-cluster-endpoints>"],"storageType":"S3","s3":{"path":"<full-s3-path>","awsSecret":"<aws-secret>"}}}]
----
3 changes: 1 addition & 2 deletions modules/osdk-manually-defined-csv-fields.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,7 @@ ensure uniqueness, for example `app-operator.v0.1.1`.
|`spec.customresourcedefinitions`
|Any CRDs the Operator uses. This field is populated automatically by the
Operator SDK if any CRD YAML files are present in `deploy/`. However, several
fields not in the CRD manifest spec that require user input (more details in the
link:https://github.com/operator-framework/operator-lifecycle-manager/blob/master/Documentation/design/building-your-csv.md#owned-crds[CSV CRD spec section]):
fields not in the CRD manifest spec require user input:

- `description`: description of the CRD.
- `resources`: any Kubernetes resources leveraged by the CRD, for example Pods and StatefulSets.
Expand Down