Init for chart with local dependencies is failing

[AssafKatz3]

Bug Report

What did you do?

1. Clone the our git repository
2. Set the current directory to root of HELM chart
3. Running:

operator-sdk init --plugins helm --domain=run.ai --helm-chart=.

The Chart.yaml contains:

  - name: prometheus-pushgateway
    repository: "file://./subcharts/prometheus-pushgateway"
    version: 1.0.1
    condition: prometheus-pushgateway.enabled 

Note: subcharts/prometheus-pushgateway is valid HELM chart

What did you expect to see?

I expect the command to pass without errors.

What did you see instead? Under which circumstances?

I see the help message of init with helm plugin and the following error:
FATA[0000] failed to initialize project with "": failed to fetch chart dependencies: directory /tmp/osdk-helm-chart946952279/runai-operator/subcharts/prometheus-pushgateway not found

Environment

Operator type:
language helm
version.BuildInfo{Version:"v3.5.3", GitCommit:"041ce5a2c17a58be0fcd5f5e16fb3e7e95fea622", GitTreeState:"dirty", GoVersion:"go1.15.8"}

Kubernetes cluster type:

OpenShift

$ operator-sdk version

operator-sdk version: "v1.5.0", commit: "98f30d59ade2d911a7a8c76f0169a7de0dec37a0", kubernetes version: "1.19.4", go version: "go1.15.5", GOOS: "linux", GOARCH: "amd64"

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1-5-g76a04fc", GitCommit:"c66c03f3012a10f16eb86fdce6330433adf6c9ee", GitTreeState:"clean", BuildDate:"2021-03-01T03:35:08Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0+5fbfd19", GitCommit:"5fbfd197c16d3c5facbaa1d7b9f3ea58cf6b36e9", GitTreeState:"clean", BuildDate:"2021-02-17T15:21:33Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

Additional context

I am trying to create operator bundle.

[camilamacedo86]

The problem here shows related to the chart itself and not to operator-sdk. Are you enable to deploy and use the chart without changing its default config? Are you not facing the same issue by doing that without creating the operator?

[AssafKatz3]

The problem here shows related to the chart itself and not to operator-sdk. Are you enable to deploy and use the chart without changing its default config? Are you not facing the same issue by doing that without creating the operator?

Yes, I can create it, the question is why it is trying to find the chart in different location (/tmp/osdk-helm-chart946952279)?

[camilamacedo86]

Could you pass the Helm chart used for we are able to try to reproduce it?

[AssafKatz3]

Hi,
I built the master and then I tried to run:

./go/bin/operator-sdk init --plugins helm --domain=run.ai --helm-chart=./runai-operator-2.0.3.tgz

And got:

Error: failed to initialize project: unable to scaffold with "base.helm.sdk.operatorframework.io/v1": stat config/manager/kustomization.yaml: not a directory

Then I extract it to test directory and run:

../go/bin/operator-sdk init --plugins helm --domain=run.ai --helm-chart=runai-operator

And got:

Error: failed to create API: unable to scaffold with "base.helm.sdk.operatorframework.io/v1": failed to fetch chart dependencies: directory /home/assaf/test/helm-charts/runai-operator/subcharts/prometheus-pushgateway not found

I checked and this directory doesn't exist, actually:

ls helm-charts/runai-operator/subchartsls

Returns:

cannot access 'helm-charts/runai-operator/subcharts': No such file or directory

While:

ls helm-charts/runai-operator/charts/

Print:

    admission-controller-2.0.0.tgz      agent-0.1.0.tgz                  prometheus-pushgateway-1.0.1.tgz    postgresql-9.7.2.tgz                   ...

Note that this chart is for HELM 3.x and the subcharts was feature of HELM 2.x.

Thanks

[camilamacedo86]

Removed from the Backlog for we are able to re-check that in the bug triage meeting again.
We have the data and seems that the user is facing problems with the chart and master branch.
So, it seems like an issue that requires further analysis to be properly addressed.

c/c @estroz @jmrodri

[camilamacedo86]

Hi @AssafKatz3,

The error:

Error: failed to create API: unable to scaffold with "base.helm.sdk.operatorframework.io/v1": failed to fetch chart dependencies: directory /home/assaf/test/helm-charts/runai-operator/subcharts/prometheus-pushgateway not found

It is because the chart was not found.

Note that you privately shared with us a tar.gz which has an operator project (./runai-operator-2.0.3.tgz) and not only helm-charts. Also, is expected that you inform 1 chart per time. You cannot create 1 API for Many charts. You need one API per chart and that might be your issue. You should not inform an tar.gz of a helm-based operator.

However, I also might found an issue on master, see:

$ operator-sdk create api --helm-chart=/Users/camilamacedo/Desktop/runai-operator.tar.gz
Error: failed to create API: unable to inject the resource to "base.helm.sdk.operatorframework.io/v1": repo not found

The Helm should not be looking for the field repo.

c/c @Adirio @estroz

[AssafKatz3]

Hi,
The agree that /home/assaf/test/helm-charts/runai-operator/subcharts/prometheus-pushgateway isn't existed, but the problem is why operator SDK search for it in first place. The relevant section in Chart.yaml is:

• condition: prometheus-pushgateway.enabled
  name: prometheus-pushgateway
  repository: file://./subcharts/prometheus-pushgateway
  version: 1.0.1
  Therefore the missing chart is in subcharts folder (per HELM 3) while operator SDK is searching it in charts folder (per HELM 2).

I have the 1 chart that I try to create API for it - runai-operator. This chart contains (conditional) references to other charts and they happen to be saved inside its folder. Its seems that operator SDK misinterprets the situation, assumes that I tries to create APIs to multi project and therefore searches for subcharts folder.

I don't understand the statement:
The Helm should not be looking for the field repo
But operator SDK should ignore the all folders which aren't part of HELM spec, including charts folder. Its seems that it does look for field repository in Chart.yaml and do it wrongly.

Thanks

[camilamacedo86]

Hi @AssafKatz3,

The scenario/mock sent is not valid to be used with the create api subcommand as described in #4689 (comment). The tool expects only 1 chart which means that each API will be created based on 1 specific chart.

Seems that you are trying to use the create api to create one API with many charts ( data sent in private ) or based on your latest comment that you are trying to use a scenario where 1 chart has sub-charts which seems not supported either. You can check the logic used here.

[AssafKatz3]

Hi @camilamacedo86 ,
I checked the logic, and even I don't know golang well, it seems that the logic is correct - the download success and the code try to find the HELM chart in wrong location. I think that the only issue is the missing expanding of sub charts in ScaffoldChart function at internal/plugins/helm/v1/chartutil/chart.
Thanks

[camilamacedo86]

HI @AssafKatz3,

By looking in the old open issues here which are related to your scenario I found some interesting things. See: #3522 (comment) (so subhcart are supported) and then see that your scenario seems indeed duplicated of #2942. Could you please check that?

Also, note that the file send private was not a chart and was an invalid scenario (#4689 (comment)). And then, the best approach to get help from the community would be to create a scenario where we can reproduce the issue and then, submit that on GitHub. Otherwise, collaborators from the community have no chance to try to help on that.

[AssafKatz3]

Hi @camilamacedo86
#3522 contains workaround that I will try to use variant of it then I will check if it is really duplicated as #2942.

I cannot send the real chart to public but I will try to craft variant of example file in #3522 with our scainrio (It will be much simpler anyway then real complex chart).

Thanks

[AssafKatz3]

Hi,
I created public example for this issue, based on #2942 example alpine.zip. The problem is about .helmignore file which contains (as required) subcharts/*.
Thanks

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[AssafKatz3]

/remove-lifecycle stale

[estroz]

@varshaprasad96 @fabianvf have you seen this issue, and will the new helm-operator fix this issue?

[varshaprasad96]

I haven't tested out this particular scenario, but this issue needs to be investigated. If the bug is related to SDK looking at the wrong location for dependent helm charts while creating api, then this is something which would occur with the new hybrid-helm operator too.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[AssafKatz3]

/remove-lifecycle stale

[asmacdo]

@tlwu2013 will verify this is still a problem.

[tlwu2013]

Confirm still seeing this issue with SDK version v1.15.0:
FATA[0000] failed to create API: unable to scaffold with "base.helm.sdk.operatorframework.io/v1": failed to fetch chart dependencies: directory /Users/tony_wu/Documents/My_Tasks/go_path/src/github.com/operator-framework/operator-sdk/runai-dec-2021/helm-charts/runai-operator/subcharts/prometheus-pushgateway not found

[tlwu2013]

Update the workarounds for runai-operator-2.0.3.tgz (https://storage.googleapis.com/runai-charts/runai-operator-2.0.3.tgz):

--
Step 0). download and unpack runai-operator-2.0.3.tgz

Step 1). open runai-operator folder, and change the sub-folder name from charts to subcharts
--> (to match the local repository path for dependencies in Chart.yaml)

Step 2). operator-sdk init --plugins helm --helm-chart ./runai-operator
--> Successfully generated the projects with subcharts.

$ cd config/
$ tree
.
├── crd
│   ├── bases
│   │   └── charts.my.domain_runaioperators.yaml
│   └── kustomization.yaml
├── default
│   ├── kustomization.yaml
│   ├── manager_auth_proxy_patch.yaml
│   └── manager_config_patch.yaml
├── manager
│   ├── controller_manager_config.yaml
│   ├── kustomization.yaml
│   └── manager.yaml
├── manifests
│   └── kustomization.yaml
├── prometheus
│   ├── kustomization.yaml
│   └── monitor.yaml
├── rbac
│   ├── auth_proxy_client_clusterrole.yaml
│   ├── auth_proxy_role.yaml
│   ├── auth_proxy_role_binding.yaml
│   ├── auth_proxy_service.yaml
│   ├── kustomization.yaml
│   ├── leader_election_role.yaml
│   ├── leader_election_role_binding.yaml
│   ├── role.yaml
│   ├── role_binding.yaml
│   ├── runaioperator_editor_role.yaml
│   ├── runaioperator_viewer_role.yaml
│   └── service_account.yaml
├── samples
│   ├── charts_v1alpha1_runaioperator.yaml
│   └── kustomization.yaml
└── scorecard
    ├── bases
    │   └── config.yaml
    ├── kustomization.yaml
    └── patches
        ├── basic.config.yaml
        └── olm.config.yaml

11 directories, 29 files

[theishshah]

Closed with documented workaround, please feel free to re-open if this does not solve the issue

[mateusoliveira43]

Hi everyone 🖖

I believe this is still a problem 😭

For example, in the project I work, the helm charts has local dependy

dependencies:
  - name: exporters
    version: 2.0.10-rc.5
    repository: file://./charts/exporters

And if I run this commands in the project root

mkdir bug
cd bug
operator-sdk init --plugins=helm --domain=test-bug --project-name=test-bug
operator-sdk create api --helm-chart=../charts/pelorus/

I get

FATA[0000] failed to create API: unable to scaffold with "base.helm.sdk.operatorframework.io/v1": failed to fetch chart dependencies: directory PATH/pelorus/bug/helm-charts/pelorus/charts/exporters not found

And I belive the problem is that operator-sdk does copy the dependency (bug/helm-charts/pelorus/charts/exporters-2.0.10-rc.5.tgz), but as tgz file instead of copying the files recursively.

my work around for this is copying the files before running create-api command

These steps work

mkdir bug
cd bug
operator-sdk init --plugins=helm --domain=test-bug --project-name=test-bug
mkdir -p helm-charts/pelorus/charts/exporters
cp -r ../charts/pelorus/charts/exporters/* helm-charts/pelorus/charts/exporters
operator-sdk create api --helm-chart=../charts/pelorus/

The responsibility of copying the dependency as files instead of tgz should be operator-sdk's?