Add katib to Helm Chart and update example/helm

[kromanow94]

Description

I discussed with @doncorsean that I wanted to do some changes and testing for his original PR in #6. We aligned that I'll take his branch and put my changes on it top of it to honor his contribution and move forward.

I did some changes and testing and from the general perspective it seems to be working but we need a closer look. I asked @MaxKavun from my team to have a look at what's happening here, compare what we have with what we expect and possibly make some changes for formatting, parameterization and readability.

Other contributions in form of testing, suggestions, or even PRs to this PR are also welcomed!

Things already included

• katib manifests to the Kubeflow Helm Chart.
• CRDs for Experiments, Suggestions, Trials to the Kubeflow Helm Chart.
• Example katib experiment in example/helm/katib-experiment.random.yaml.
• values.yaml for MySQL contains initdbScripts that adds katib databse.
• Configured katib dbmanager to use the same MySQL Instance as for Kubeflow in example/helm/values.kubeflow.yaml but with different database.

Todo

• Have a look at the conventions, maybe there is something that might be improved.
• Not all manifests are properly parameterized (charts/kubeflow/templates/katib/user-roles/*, maybe more).
• Have a look at the values.yaml::katib.config - cleanup needed and possibly there are some differences between upstream 1.8.0 and this.
• Have a general and criticizing look at everything because it's a lot and it's easy to miss something.

[MaxKavun]

Does it make sense to create default db configuration since we use the same secrets/host for

1. pipeline component
2. katib

And add ability to override for each component ? I see this is quite common approach to avoid repeating the same configs

[kromanow94]

This is a good idea but let's handle that as a separate issue.

#9

[MaxKavun]

I also not sure if this is right approach to tight such scripts with database deployment itself

Bitnami mentions it's only for fresh instance and first boot, should we consider a separate Job for database initialisation instead ?

[kromanow94]

This is for example configuration. Having this in mind, it should be fine. Our main product is the Helm Chart and the example configuration is just for reference. But, we should document this.

I created an issue for documentation here: #10

[kromanow94]

Please also generate the kubeflow/manifests for Kubeflow 1.8 with Kustomize and have a look if there any differences between the generated Helm Chart and the generated manifests Kustomize.

You can use this example kustomization.yaml to generate the manifest directly from upstream:

$ cat kustomization.yaml 
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- github.com/kubeflow/manifests/example?ref=v1.8.1

$ kustomize build . > kubeflow-manifests-aio-1.8.1.yaml

[kromanow94]

@MaxKavun can you just fix the CICD for Lint and Test Charts / list-tests? After that we can squash and merge!