How to change playbook inventory for each custom resource

[yhidai]

Type of question

Best practices
How to implement a specific feature

Question

What did you do?

I am trying to create an operator with Ansible Operator.
I'm trying to define Redis information in a custom resource, create a VM with an IaaS provider based on that data, and set up Redis.

What did you expect to see?

I would like to change the host that runs Ansible playbooks for each custom resource processed by the reconcile loop.
Is there such functionality in Ansible Operator?

What did you see instead? Under which circumstances?

Environment

Operator type:

/language ansible

Kubernetes cluster type:

vanilla

$ operator-sdk version

operator-sdk version: "v1.23.0", commit: "1eaeb5adb56be05fe8cc6dd70517e441696846a4", kubernetes version: "1.24.2", go version: "go1.18.5", GOOS: "linux", GOARCH: "amd64"

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:16:20Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.15", GitCommit:"b84cb8ab29366daa1bba65bc67f54de2f6c34848", GitTreeState:"clean", BuildDate:"2022-12-08T10:42:57Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}

Additional context

[varshaprasad96]

This issue was brought up in the community meeting. We would need inputs from the ansible operator folks to get more idea on this. Tagging @everettraven for help in following this up.

[everettraven]

@yhidai Could you elaborate a bit more on what you are attempting to do? It sounds like on reconciliation of a CR you are wanting to:

• Create VM with an IaaS provider
• Run a playbook against (on?) that VM to configure Redis on the VM

Does that sound right?

[yhidai]

@varshaprasad96
Thank you for your reply. I ask him.

@everettraven
Thank you for answering my question.

Could you elaborate a bit more on what you are attempting to do? It sounds like on reconciliation of a CR you are wanting to:

• Create VM with an IaaS provider
• Run a playbook against (on?) that VM to configure Redis on the VM
  Does that sound right?

Yes, that's right.
I would like to know if the following use cases are possible with Ansible Operator.

• The Operator reconcile the following custom resources.

apiVersion: redis.example.com
kind: Redis
metadata:
  name: redis-standalone
spec:
  host:
    name: redis-standalone
  redis:
    version: 7.0.10
    memory: 16GiB

• Operator creates a VM in CloudStack with hostname of spec.host.name in custom resource.
• Once the VM is created, the Operator will record the IPAddress in the CR.

apiVersion: redis.example.com
kind: Redis
metadata:
  name: redis-standalone
spec:
  host:
    name: redis-standalone
    ipAddress: 10.41.0.38
  redis:
    version: 7.0.10
    memory: 16GiB

• Operator executes a playbook to build Redis specified in spec.redis.version for VM of spec.host.ipAddress in CR

[everettraven]

@yhidai Thanks for the elaboration! I'll have to try and get in contact with some folks that have more ansible experience and how this might be possible.

[everettraven]

@yhidai I was able to reach out to some folks offline and this was their recommendation:

here’s what I would try:

• Create an inventory file in that input directory that contains localhost (which is the controller manager container) and the external host you want to manage. See here for more info on the expected structure.
• Figure out how to get the SSH key into the operator’s environment (maybe mounting a secret to ~/.ssh/id_rsa, I dunno)
• Play around with a playbook that sets the top-level hosts to something other than localhost.

Note that you will need to be careful about where the k8s tasks end up running. Ansible will run tasks on all hosts defined in the inventory, and the hosts pattern is how to control where things run. You may end up needing delegate_to and/or run_once.

You can also try having 2 “plays” in the same playbook, one that runs against localhost and the other than runs against a remote host. I have no idea what the sdk will do in this case.

Does this give you enough information to try some of the recommendations? Unfortunately I'm not sure when I would have some time to test these recommendations myself

The input directory they are referring to is found here:

operator-sdk/internal/ansible/runner/runner.go

Line 207 in 78c5643

Path: filepath.Join("/tmp/ansible-operator/runner/", r.GVK.Group, r.GVK.Version, r.GVK.Kind,

[yhidai]

@everettraven Thank you for the detailed information.
I will check the information you advised.

From the information you gave me, I understood that using Ansible Operator to manage the IaaS resources I was trying to test was an unexpected usage for Ansible Operator.

I understand that Ansible Operator is for managing resources on Kubernetes using Ansible playbooks, is that correct?

[everettraven]

From the information you gave me, I understood that using Ansible Operator to manage the IaaS resources I was trying to test was an unexpected usage for Ansible Operator.

Yeah, this is a use case I've personally never seen but that doesn't mean it can't be done - I just don't know how it should be done :)

I understand that Ansible Operator is for managing resources on Kubernetes using Ansible playbooks, is that correct?

Yeah, but there shouldn't be anything that is stopping you from being able to do what you are attempting to do. An additional thing you could try that I didn't think of until now is using a second custom resource for when the VM is created to trigger the redis configuration playbook. So you would have a CR that triggers a playbook to create the VM on your IaaS provider and create a CR that triggers a playbook to configure Redis using the information on that CR

[yhidai]

@everettraven Thank you very much for your kind reply.

After learning more about Ansible Operator and Operator SDK functions, I will try it based on the information you gave me.

[everettraven]

Just to follow up on this because I got another recommendation from one of the original authors of the ansible plugin:

if you pass the ANSIBLE_INVENTORY environment variable to the operator container it will take precedence over the generated inventory. So you can build an inventory file like normal, put it in your container, make sure the envvar is set and reference the proper hosts in your plays and it should all behave as normal

[Goend]

@yhidai As you mentioned earlier,, Do you have the best practices for running custome ansible playbooks with Ansible Operator? I want to use it for installing k8s and other components.
Look forward to your soonest reply.

[yhidai]

@Goend Sorry, I haven't tried the advice given in this issue yet.