How can I run once in a reconcile loop?

[imaimaibah]

I am creating an operator by kubebuilder, which accesses external cloud services and create resource there.
If I create a lot of objects, the operator sends api calls per object. I would like to run once and get all data in a loop and compare the data with the current state, and take an action if needed.

Is it possible to do that? If yes, how can I do that?

I would appreciate any help.
Thanks,

[camilamacedo86]

The controller’s reconciliation loop needs to be idempotent when developing operators. By following the Operator pattern, you will create Controllers which provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.

That means the idea is to observe the resources on the cluster (watches feature) and then ensure that when anything that matters changes, the controllers will reconcile the Kinds and ensure the desired state. Maybe look at can be helpful: https://sdk.operatorframework.io/docs/best-practices/common-recommendation/

[imaimaibah]

So answer is I can't do and I shouldn't do? Do you have any suggestion to reduce api calls to external service? If I create 100 objects, it will call 100 times at periodic reconcile loop.

[camilamacedo86]

If your go is to reconcile resources of some kind then every time that you create a resource the reconciliation will be called.
If the reconciliation or re-trigged because of changes in a resource watched you should check if the resource is or was not created before created. The goal is you built an idempotent solution to ensure a desire on the cluster.

So, I'd like to recommend you check out the scaffold done with the new plugin deploy.image/v1-alpha, see: https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v3-with-deploy-image/controllers/busybox_controller.go

Also, the following links to getting started and better understand its purpose:

https://dev4devs.com/2020/08/16/how-to-getting-started-develop-go-operators-from-scratch-with-sdk-1-0/
https://sdk.operatorframework.io/docs/best-practices/common-recommendation/.

I am marking this one as sorted out. If you still need help please feel free open new issues/discussions after that letting us know more exactly what you are doing, what you expected and why, and what you are facing so that we can better assist you. Thank you.

[mogsie]

The accepted answer is correct in that this is how you need to think about these operators. However, this doesn't stop you from accessing the API in a smart way. Kubebuilder (and its related projects) already do this for the k8s API itself:

• it watches for resources
• it lists all instances
• it caches them
• whenever something is modified, it updates the cache.

All this happens transparently, so when you try to say k8sClient.Get() it actually only looks in the cache.

There's nothing stopping you from doing something similar, or something different against the cloud API.

The naïve approach is to access the cloud API as you describe, once per reconcile. There is nothing wrong with your reconcile loop implementing something like this:

• List all objects (1 API call, 100 objects)
• Cache them
• When you reconcile object X, inspect the cache and reconcile based on that
• Periodically refresh the cache (e.g. by the reconcile loop forcing a refresh when you get an event, and the cache is older than X seconds).

If your cloud API supports watching events (like the k8s watch API), then you can reduce the amount of stale reconciles you will have.

[camilamacedo86]

Closing since it is answered.