[FR] Enable metrics endpoint to handles metrics for kubernetes purpose.

[wdesplas]

Is your feature request related to a problem? Please describe.

I have move the cloud-service-broker into kubernetes as a stateless componenent and to use HPA (horizontal pod autoscaling) using custom metrics, we to have the /metrics endpoint.
This will be helpfull once moved the cloud-service-broker to kubernetes to be HA.

Describe the solution you'd like
I have found a package https://github.com/VictoriaMetrics/metrics that help go packages to have metrics.

Describe alternatives you've considered
The alternative is to use a prometheus but is will be quite difficult on some cluster for RBAC purpose.

Additional Context

Priority

Priority Context
Low
It's only a nice to have in this case as the cloud-service-broker works fine without this implementation.

Platform
N/A

Applicable Services
N/A

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/182373654

The labels on this github issue will be updated when the story is started.

[FelisiaM]

Hi @wdesplas

Thank you for your request.

Would you mind elaborating more on what type of metrics would be needed to emit from this new endpoint?

Currently, we are at capacity and given this is low priority we don't think we will get around to implementing it for a while. However, we would be happy to review a PR.

Thanks
Felisia & @pivotal-marcela-campo

[wdesplas]

Hi @FelisiaM ,

In my point of view, the following metrics will be helpfull

• number of terraform process <= most important
• number of retrieving catalog request
• number of provision request
• number of unprovisioning request
• number of binding request
• number of unbinding request
• more ? ...

It can be usefull for HPA or grafana dashboard.

If it suite your point of view, I will start the implementation before performing a PR.

Regards

[pivotal-marcela-campo]

Hi @wdesplas,
Just a not that we don't currently test the broker with more than one instance running at the time, so if you are intending to use this for HPA you might want to test that first.
It makes sense otherwise, and definitely nice to have. We will be happy to review the PR.

Thanks!

[wdesplas]

Hi @pivotal-marcela-campo ,
Juste for information, the CSB is stateless by design.
Therefore, we have curently test the CSB with at least 3 instances in // on a docker swarm cluster and it works fine without any issues.
Now I have make it working on kubernetes with HPA based on metrics as CPU and I think it will be helpfull to have more metrics. I will make a PR as soon I have something relevent.

Regards,

[pivotal-marcela-campo]

Sounds great!
Thanks