Metrics improvements

[morhidi]

Hi team,

we've successfully implemented and using the existing Metrics interface from JOSDK in the Flink Operator. Great job!

We'd be happy to see a few more metrics and improvements around the following areas:

• Exposure of the complete CR in the Metrics interface
• CR delay (time between the CR created and first detected by an operator)
• Kubernetes API access (basically HTTP request/response metrics, thus this might be a better fit for fabric8 client, having an http client independent interceptor would be awesome)
• A few other metrics about JOSDK internals (queue size, thread count, informer count, etc.)

Let me know what you think.

Thanks,
Matyas

[metacosm]

All of this sounds reasonable. 😄

[metacosm]

There's an HTTP client independent interceptor interface in Fabric8 6: https://github.com/fabric8io/kubernetes-client/blob/master/kubernetes-client-api/src/main/java/io/fabric8/kubernetes/client/http/Interceptor.java

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[csviri]

Just some thoughts:
CR delay (time between the CR created and first detected by an operator) : This is a little problematic since, we don't know if a resource was reconciled before or not if an operator starts. Also when an operator is not running and CR is created might lead to very high values in the metrics.

It would be possible to do a metric to measure resource event received and first processed time delay. But event this might be an issue if there is already a reconciliation running - that might also complicate things.

So queue since and thread count might be a simpler indicator to measure the pressure on the operator I guess.

[csviri]

will close this issue, added then mentioned metrics, expect the one I commented above, and informer count (since it might not trivial to add in the current architecture, also it's fairly simple, in general if the namespaces are not changing dynamically are static values). In case @morhidi you think it still makes sense pls create a separate issue for that.

[morhidi]

Thanks @csviri