Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[KOGITO-9177] Introduce workflowproj module #143

Merged
merged 4 commits into from
Jun 5, 2023
Merged

[KOGITO-9177] Introduce workflowproj module #143

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ef174a7
[KOGITO-9177] Introduce workflowproj module
ricardozanini Jun 2, 2023
8b5e4cd
Fix unit test with subresource
ricardozanini Jun 2, 2023
aa1eb13
Incorporating reviews, adding nil check on save files
ricardozanini Jun 5, 2023
283f696
Adding filename formatting options
ricardozanini Jun 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ jobs:
with:
python-version: '3.10'

# TODO: cache this installation
- name: Build operator image
run: |
pip install cekit==4.5.0
Expand All @@ -104,6 +105,7 @@ jobs:
kubectl version
kubectl get pods -A

# TODO: install the operator-sdk first, then cache the installation
- name: Run tests
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://issues.redhat.com/browse/KOGITO-9264

run: |
make test-e2e
3 changes: 2 additions & 1 deletion Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ COPY go.mod go.mod
COPY go.sum go.sum

# Copy internal dependency
COPY api/ api/
COPY workflowproj/ workflowproj/
COPY container-builder/ container-builder/

# cache deps before building and copying source so that we don't need to re-download as much
Expand All @@ -29,7 +31,6 @@ RUN go mod download

# Copy the go source
COPY main.go main.go
COPY api/ api/
COPY controllers/ controllers/
COPY utils/ utils/
COPY version/ version/
Expand Down
14 changes: 11 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,11 +93,11 @@ help: ## Display this help.

.PHONY: manifests
manifests: generate ## Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects.
$(CONTROLLER_GEN) rbac:roleName=manager-role crd:allowDangerousTypes=true webhook paths="./..." output:crd:artifacts:config=config/crd/bases
$(CONTROLLER_GEN) rbac:roleName=manager-role crd:allowDangerousTypes=true webhook paths="./api/..." paths="./controllers/..." output:crd:artifacts:config=config/crd/bases

.PHONY: generate
generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..."
$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./api/..." paths="./container-builder/api/..."

.PHONY: fmt
fmt: ## Run go fmt against code.
Expand All @@ -110,9 +110,17 @@ vet: ## Run go vet against code.
go vet ./...

.PHONY: test
test: manifests generate envtest vet fmt ## Run tests.
test: manifests generate envtest vet fmt test-workflowproj ## Run tests.
KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" go test $(shell go list ./... | grep -v /test/) -coverprofile cover.out

.PHONY: test-container-builder
test-container-builder:
cd container-builder && make test

.PHONY: test-workflowproj
test-workflowproj:
cd workflowproj && make test

##@ Build

.PHONY: build
Expand Down
231 changes: 32 additions & 199 deletions README.md
View file
Open in desktop
ricardozanini marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,222 +1,55 @@
# Kogito Serverless Operator

- [Kogito Serverless Operator](#kogito-serverless-operator)
- [Getting Started](#getting-started)
- [Prepare a Minikube instance](#prepare-a-minikube-instance)
- [Run the operator](#run-the-operator)
- [Test the Greeting workflow](#test-the-greeting-workflow)
- [Prerequisites](#prerequisites)
- [Prepare for the build](#prepare-for-the-build)
- [Install your workflow](#install-your-workflow)
- [Cleanup your cluster](#cleanup-your-cluster)
- [Use Workflow Development Mode](#use-workflow-development-mode)
- [Development mode: External files](#development-mode-external-files)
- [Use local scripts](#use-local-scripts)
- [Development and Contributions](#development-and-contributions)
The Kogito Serverless Operator defines a set
of [Kubernetes Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
to help users to deploy Kogito Serverless
Workflow projects on Kubernetes and OpenShift.

The Kogito Serverless Operator is built in order to help the Kogito Serverless users to build and deploy easily on
Kubernetes/Knative/OpenShift a service based on Kogito that will be able to execute a workflow.
Please [visit our official documentation](https://kiegroup.github.io/kogito-docs/serverlessworkflow/latest/cloud/operator/install-serverless-operator.html)
to know more.

The CustomResources defined and managed by this operator are the following:
- Workflow
- Platform
- Build
## Available modules for integrations

## Getting Started
If you're a developer, and you are interested in integrating your project or application with the Kogito Serverless Operator
ecosystem, this repository provides a few Go Modules described below.

You’ll need a Kubernetes cluster to run against. You can use:
### Kogito Serverless Operator Types (api)

- [KIND](https://sigs.k8s.io/kind)
- [MINIKUBE](https://minikube.sigs.k8s.io/)
- [Openshift Local](https://console.redhat.com/openshift/create/local)
Every custom resource managed by the operator is exported in the module [api](api). You can use it to programmatically
create any custom type managed by the operator.
To use it, simply run:

to get a local cluster for testing, or run against a remote cluster.

**Note:** Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster `kubectl cluster-info` shows).

### Prepare a Minikube instance

```sh
minikube start --cpus 4 --memory 4096 --addons registry --addons metrics-server --insecure-registry "10.0.0.0/24" --insecure-registry "localhost:5000"
```

**Note:** To speed up, you can increase cpus and memory options. For example, use `--cpus 12 --memory 16384`.

**Tip:** If it does work with the default driver, aka `docker`, you can try to start with the `podman` driver:
```sh
minikube start [...] --driver podman
```

**Important:** There are some issues with the `crio` container runtime and Kaniko that the operator is using. Reference: https://github.com/GoogleContainerTools/kaniko/issues/2201

### Run the operator

1. Install the CRDs

```sh
make install
```

2. Build and push your image to the location specified by `IMG`:

```sh
make docker-build docker-push IMG=<some-registry>/<context_namespace>/kogito-serverless-operator:tag
```

3. Deploy the controller to the cluster with the image specified by `IMG`:

```sh
make deploy IMG=<some-registry>/kogito-serverless-operator:tag
```

This will deploy the operator into the `kogito-serverless-operator-system` namespace.

## Test the Greeting workflow

A good starting point to check that everything is working well, it is the [Greeting workflow](https://github.com/kiegroup/kogito-examples/blob/stable/README.md#serverless-workflow-getting-started).

### Prerequisites

* Operator is deployed on the cluster
See [Getting started](#getting-started)


### Prepare for the build

Follow these steps to create a container that you can than deploy as a Service on Kubernetes or KNative.

1. Create a namespace for the building phase

```sh
kubectl create namespace kogito-workflows
```shell
go get github.com/kiegroup/kogito-serverless-workflow/api
```

2. Create a secret for the container registry authentication
Then you can create any type programmatically, for example:

```sh
kubectl create secret docker-registry regcred --docker-server=<registry_url> --docker-username=<registry_username> --docker-password=<registry_password> --docker-email=<registry_email> -n kogito-workflows
```go
workflow := &v1alpha08.KogitoServerlessWorkflow{
ObjectMeta: metav1.ObjectMeta{Name: w.name, Namespace: w.namespace},
Spec: v1alpha08.KogitoServerlessWorkflowSpec{Flow: *myWorkflowDef>}
}
```

or you directly import your local docker config into your kubernetes cluster:
You can use the [Kubernetes client-go library](https://github.com/kubernetes/client-go) to manipulate these objects in
the cluster.

```sh
kubectl create secret generic regcred --from-file=.dockerconfigjson=${HOME}/.docker/config.json --type=kubernetes.io/dockerconfigjson -n kogito-workflows
```

3. Create a Platform containing the configuration (i.e. registry address, secret) for building your workflows:

You can find a basic Platform CR example in the [config](config/samples/sw.kogito_v1alpha08_kogitoserverlessplatform.yaml) folder.

```sh
kubectl apply -f config/samples/sw.kogito_v1alpha08_kogitoserverlessplatform.yaml -n kogito-workflows
```

**Note:** In this Custom Resource, `spec.platform.registry.secret` is the name of the secret you created just before.
You might need to register our schemes:

**Tip:** You can also update "on-the-fly" the platform CR registry field with this command (change `<YOUR_REGISTRY>`):
```sh
cat config/samples/sw.kogito_v1alpha08_kogitoserverlessplatform.yaml | sed "s|address: .*|address: <YOUR_REGISTRY>|g" | kubectl apply -n kogito-workflows -f -
```go
s := scheme.Scheme
utilruntime.Must(v1alpha08.AddToScheme(s))
```

### Install your workflow

1. Install the Serverless Workflow custom resource:

```sh
kubectl apply -f config/samples/sw.kogito_v1alpha08_kogitoserverlessworkflow.yaml -n kogito-workflows
```

2. You can check the logs of the build of your workflow via:

```sh
kubectl logs kogito-greeting-builder -n kogito-workflows
```

The final pushed image should be printed into the logs at the end of the build.

## Cleanup your cluster

You will need to remove the different resources you created.

1. Remove created workflow resources

```sh
kubectl delete -f config/samples/sw.kogito_v1alpha08_kogitoserverlessworkflow.yaml -n kogito-workflows
```

2. Remove the `kogito-workflows` namespace

```sh
kubectl delete namespace kogito-workflows
```

3. Remove the operator

```sh
make undeploy
```

## Use Workflow Development Mode
In the development mode, a user can edit and reload the files using the quarkus dev mode.
To enable the dev mode add this annotation into the KogitoServerlessWorkflow CR:
```
sw.kogito.kie.org/profile: dev
```

### Development mode External files

In development mode, different external files can be edited.
Each file type should be saved into a different configmap.
In order to be able to edit a file via a configmap, you must set up an annotation in your KogitoServerlessWorkflow CR which reference the specific configmap which is already existing into the namespace.
Then, each annotation entry will be turned into a file inside the path `src/main/resources/<File type specific folder>` and using the key as the filename.
See the table below for the supported types and the corresponding annotations and folder paths.

File type | Annotation | Folder path
-----------|-------------------------------------|---
CamelRoute | sw.kogito.kie.org/resource-camel | src/main/resources/routes
AsyncApi | sw.kogito.kie.org/resource-openapi | src/main/resources/templates
OpenApi | sw.kogito.kie.org/resource-asyncapi | src/main/resources/specs
Generic | sw.kogito.kie.org/resource-generic | src/main/resources/generic

You could see an example of configmap and a workflow with one of this annotation in the file
```
https://github.com/kiegroup/kogito-serverless-operator/tree/main/config/samples/sw.kogito_v1alpha08_kogitoserverlessworkflow_devmodeWithConfigMapAndExternalResource.yaml:
```

## Use Workflow Development Mode

#### Override Builder image and version

By default the builder image use a stable version aligned to the Kogito version.

In the dev mode you can override the default builder image using the devBaseImage in the KogitoServerlessPlatform:

```sh
apiVersion: sw.kogito.kie.org/v1alpha08
kind: KogitoServerlessPlatform
metadata:
name: kogito-workflow-platform
spec:
cluster: kubernetes
devBaseImage: quay.io/<your_org>/<your-swf-builder>
platform:
registry:
address: quay.io/kiegroup
secret: regcred
```

## Use local scripts

You can find some scripts in the [hack](./hack/local/) folder.
## Container Builder (container-builder)

- `greeting_example_deploy.sh` will install operator and deploy all resources in your current cluster
- `greeting_example_remove.sh` will remove the created workflow resource from `greeting_example_deploy.sh` script.
If you give the `-A` or `--all` option, it will also remove the operator from the cluster.
Please see the module's [README file](container-builder/README.md).

## Tekton Pipeline to build and deploy the Operator
## Workflow Project Handler (workflowproj)

Setup a [pipeline](docs/PIPELINE.md) on a Openshift cluster.
Please see the module's [README file](workflowproj/README.md).

## Development and Contributions
## Development and Contributions

Contributing is easy, just take a look at our [contributors](docs/CONTRIBUTING.md)'guide.
51 changes: 51 additions & 0 deletions api/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
module github.com/kiegroup/kogito-serverless-operator/api

go 1.19

require (
github.com/serverlessworkflow/sdk-go/v2 v2.2.3
k8s.io/api v0.27.2
k8s.io/apimachinery v0.27.2
knative.dev/pkg v0.0.0-20230525143525-9bda38b21643
sigs.k8s.io/controller-runtime v0.15.0
)

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-playground/locales v0.14.0 // indirect
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.11.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/senseyeio/duration v0.0.0-20180430131211-7c2a214ada46 // indirect
github.com/stretchr/testify v1.8.2 // indirect
go.uber.org/atomic v1.10.0 // indirect
golang.org/x/crypto v0.1.0 // indirect
golang.org/x/mod v0.10.0 // indirect
golang.org/x/net v0.10.0 // indirect
golang.org/x/oauth2 v0.7.0 // indirect
golang.org/x/sys v0.8.0 // indirect
golang.org/x/term v0.8.0 // indirect
golang.org/x/text v0.9.0 // indirect
golang.org/x/time v0.3.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.3.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/client-go v0.27.2 // indirect
k8s.io/klog/v2 v2.90.1 // indirect
k8s.io/utils v0.0.0-20230209194617-a36077c30491 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
Loading
Loading