Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event Notification and Polling, Inbound Event Sources #720

Merged
merged 24 commits into from
Dec 15, 2021

Conversation

csviri
Copy link
Collaborator

@csviri csviri commented Nov 29, 2021

No description provided.

@csviri
Copy link
Collaborator Author

csviri commented Dec 9, 2021

Refactored the MySQL example to use the PerResourcePollingEventSource.


protected void handleDelete(ResourceID relatedResourceID) {
if (!isRunning()) {
log.debug("Received event but event for {} source is not running", relatedResourceID);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logging doesn't make sense

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The though was to make is easier on debug level to find out the life cycle of an event. If you prefer I can remove this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant the message that's output… I'm not sure what it means :)


protected void handleEvent(T value, ResourceID relatedResourceID) {
if (!isRunning()) {
log.debug("Received event but event for {} source is not running", relatedResourceID);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logging doesn't make sense

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same :)

cache.remove(relatedResourceID);
// we only propagate event if the resource was previously in cache
if (cachedValue != null
&& (eventFilter == null || eventFilter.acceptDelete(cachedValue, relatedResourceID))) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would you not want to propagate a delete event?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Wanted to make a generic filter, but this might be over engineered. We can remove the filters for now and see if someone is asking for it.


import io.javaoperatorsdk.operator.processing.event.ResourceID;

public interface EventFilter<T> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concept of event filter actually doesn't really make much sense to me… the events come from event sources which you write, so presumably, the event source decide of which events trigger the event handler so I'm not sure why we need to filter them again…

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, the idea was to separate that two thing, maybe to reuse some filters. But agree, I will remove filtering

*
* @param resourceID of the target related resource
* @return the cached value of the resource, if not present it gets the resource from the
* supplier. If the supplier provides a value it is cached, so there will be no new event
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last sentence is not very clear…

values.forEach((k, v) -> super.handleEvent(v, k));
var keysToRemove = StreamSupport.stream(cache.spliterator(), false)
.filter(e -> !values.containsKey(e.getKey())).map(Cache.Entry::getKey)
.collect(Collectors.toList());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not remove the keys directly instead of collecting them in a list first?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed.


public MySQLSchemaReconciler(KubernetesClient kubernetesClient, MySQLDbConfig mysqlDbConfig) {
this.kubernetesClient = kubernetesClient;
this.mysqlDbConfig = mysqlDbConfig;
}

@Override
public void prepareEventSources(EventSourceRegistry<MySQLSchema> eventSourceRegistry) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that this new implementation actually makes things more complicated than before: you have to write more code to achieve the same result (or at least, that's how it looks to me). Granted, I'm not familiar with this example / use case at all but it doesn't appear simpler now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for now it was not receiving events if the schema was deleted or changed.

Copy link
Collaborator

@metacosm metacosm Dec 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK so that's probably what I'm missing 😄
Like I said, I'm not really familiar with the example so just looking at the code seemed like things were now more complex.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also need to think about how to integrate this new feature with the dependent resources work.

@metacosm metacosm merged commit 2e9bd22 into main Dec 15, 2021
@metacosm metacosm deleted the polling-event-source branch December 15, 2021 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Umbrella Issue for improved and Utility EventSources provided for V2
2 participants