Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watcher does not support Unstructured objects #422

Closed
crimsonfaith91 opened this issue Oct 9, 2018 · 9 comments
Closed

Watcher does not support Unstructured objects #422

crimsonfaith91 opened this issue Oct 9, 2018 · 9 comments

Comments

@crimsonfaith91
Copy link
Contributor

crimsonfaith91 commented Oct 9, 2018

I spent a full day working on getting the watcher works for Unstructured objects, but failed to do any progress, so I deduce the watcher does not work for Unstructured objects.

For example,

err = controller.Watch(&source.Kind{Type: someUnstructuredResource.DeepCopyObject()}, &handler.EnqueueRequestForOwner{
	IsController: true,
	OwnerType:    &operatorv1alpha1.SomeOwnerResource{},
})

.DeepCopyObject() is a generated function returning a runtime.Object. When I modify an instance of someUnstructuredResource, the sync loop is not triggered.

To verify watcher does not work, I change someUnstructuredResource.DeepCopyObject() to &corev1.Pod{} and modify the pod. The sync loop is triggered.

@pwittrock
@droot

@droot
Copy link
Contributor

droot commented Oct 9, 2018

@shawn-hurley Can you pl. here since your are the most familiar with unstructured codebase.

@shawn-hurley
Copy link

@droot will take a look either today or tomorrow.

I am personally using watches with unstructured objects right now. I wonder if you are setting the GVK for the unstructured object?

@crimsonfaith91
Copy link
Contributor Author

crimsonfaith91 commented Oct 9, 2018

@shawn-hurley Yes. I am able to obtain the GVK through .GetKind() and GetAPIVersion().

Btw, are you setting the watcher by specifying .Watch() call within function below?

func add(mgr manager.Manager, r reconcile.Reconciler) error

I currently register the watcher within the sync loop (Reconcile function). I found out that the mutex of the watcher blocks indefinitely too when attempting to register the watcher within the sync loop. I couldn't register the watcher in add() function above because I can only know type of the Unstructured objects within the sync loop when reconciling.

@shawn-hurley
Copy link

currently register the watcher within the sync loop (Reconcile function)

So it sounds like you are attempting to call controller.Watch during an already started controller.

I am seeing the same blocking behavior if I use unstructured or &v1.Pod{} in a watch triggered inside a loop. This is because the start method of the controller takes the lock and does not let go until it is stopped.

This will not work for any scenario, unstructured or otherwise AFAIK. @droot @DirectXMan12 can you verify that is the case?

@droot
Copy link
Contributor

droot commented Oct 9, 2018

This is because the start method of the controller takes the lock and does not let go until it is stopped.

Correct. we just verified that from the code. Taking a deep look at it to see how we can support this use-case.

@crimsonfaith91
Copy link
Contributor Author

crimsonfaith91 commented Oct 10, 2018

@droot @shawn-hurley Sorry for not making it clearer in my previous comment.

I am aware of Start function of controller has a mutex lock too, so the mutex block should be a deadlock. However, this is not my main concern, which is: I am only able to know about GVK of the unstructured object after controller starts. This means that, IIUC, I am restricted to only call controller.Watch dynamically in the sync loop (i.e., after the controller has started).

What I want to achieve is the ability to watch an object with dynamic type. I debugged by temporarily un-comment the mutex codes of controller.Watch function so that I can call the function within the sync loop. I then tried to watch a pod and the unstructured object - the pod's change triggered resync, but not the unstructured object. I think there may be another bug that the watcher does not handle unstructured objects correctly (I may be wrong), but everything looks fine after examining fields of the unstructured object.

@crimsonfaith91 crimsonfaith91 changed the title Watcher does not support Unstructured objects [high priority] Watcher does not support Unstructured objects Oct 10, 2018
@crimsonfaith91
Copy link
Contributor Author

crimsonfaith91 commented Oct 10, 2018

I mocked a dummy Unstructured object to check whether it works with watcher. I encountered an error: `no kind "SomeResourceList" is registered for version "some.resource.group/version".

I speculate that various components of Kubebuilder (not just the watcher) requires SchemeBuilder.Register(&SomeResource{}, &SomeResourceList{}) to work probably, but my use case requires "dynamic" type, so registering a set of types upfront is not an option. I also added SchemeBuilder.Register(&Unstructured{}, &UnstructuredList{}), but the error persists.

For now, I reduce resync period to have the change of Unstructured objects being picked up.

I now think this may be more of a feature request than a bug.

@DirectXMan12
Copy link
Contributor

Pretty much any type you want to work with has to be registered with discovery. If a given resource is not available in discovery, we can't work with it. It doesn't necessarily need to be registered in the scheme, but if you can't find it at the /apis endpoint, then you will see that error

@crimsonfaith91
Copy link
Contributor Author

@DirectXMan12 Oops, this problem has resolved. I will close it now.

To fix the problem, it requires adding the scheme dynamically apart from this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants