Umbrella Issue for improved and Utility EventSources provided for V2

[csviri]

Event Sources provides functionality to get notifications, thus trigger reconciliation if a dependent resource changes. Withing this issue the topic is revisited, and will try to improve and provide implementation of event sources that would help to cover most of the use cases to a certain level of abstraction.

The goal is to have "batteries included", so users could use and/or easily implement their own custom event sources utilizing the ones we provide. This is intended to be the first design / iteration, plan is to evolve these based on user feedback.

Event Source Categories

We can divide event sources into two big categories:

1. Event Sources for Kuberentes Resource or InformerEventSource
   This event source covers the watching and caching the Kubernetes resources. Already present in the system. Provides a
   cache of the observed resources too.
2. Non Kubernetes (external) Resources. Think of a object that is not managed by Kubernetes it self, from Database users, to git repositories, etc. But still here we communicate with an API. Ideally still through well defined REST API, thus managing resources in a declarative way. Based how these objects and related events are handled we can divide this to two sub-categories.
   1. Pull based - when we have to basically poll the resource to get the actual state, in regular intervals.
   2. Push based - when a notification in some form is received about the changes of the resource. Like a webhook or sqs queue, etc.

Polling Event Source for Pull Based Updates

For this purpose two implementations are provided:

1. PerResourcePollingEventSource - Periodically calls a provided function for each custom resource.
2. PollingEventSource - Is for use cases when resources can be queried from an api, but it is done for a generic way. Like listing some object with realted tags. Then these are mapped to ResourceID-s.

Both of these provide caches of the resources. The events are propagated only if resources changed (checked by equals method on pojos).

Inbound Event Sources for Push Based Updates

Inbound event sources are for the case when we receive some notifications about the resources. For now we don't support specific systems, but provide basic implementation that could be used or extended to easily support any notification system. Like webhooks, or messaging systems.

1. SimpleInboundEventSource - tyically for use cases when the notification is not resource based. This event source make it just easy to propagate any event into the system. Does not provide caching.

2. CachingInboundEventSource - Typically for use cases when the notification is resource based, in other works we receive the whole resource within the event. This implementation provides caching.

Note that these event sources also provide addtional functionalities, like filtering of events.

Abstract Event Sources to Support Custom Ones

CachingFilteringEventSource - is an abstract event source (serves as a base class for some above), that supports caching and event filtering.

Custom Resource Event Notifications for Event Source

There are cases when an Event Source needs to be aware of the custom resources managed by the system, for example:

• TimerEventSource - wants to cancel all the timed tasks if a custom resource is deleted
• PerResourcePollingEventSource - also needs to stop polling for the custom resource if it's not present in the system. But also want't start polling in case a custom resource appeared (or reached a certain state - see predicate in the implementation).

For this purpose the ResourceEventAware was introduced. If an event sources implements this interface will receive events about the custom resources in the system. Allowing to implement logic based on that.

See also:

#651
#665
#666
#673

[csviri]

@metacosm @lburgazzoli @adam-sandor @heesuk-ahn if you have time pls take a look, and let me know what do you think.

[heesuk-ahn]

This seems like a good first step. 👍

It is good that the user can directly create a Custom EventSource through the abstraction layer, or that Cache or Filter can be directly injected from the user through the constructor.

This point is considered to be an advantage in that it is possible to access Cache or Filter outside of the framework during development.

It is not yet clear how each EventSource will be used, but I'll take a closer look at this part. :)

[csviri]

Next step will include these to examples.