No containers instructions ?

[wally007]

Is your feature request related to a problem? Please describe.

Can you please described if it is possible to run this project with existing Prometheus/Grafana setup but without containers ?

We are not using docker (or any containers) in Production but we still would like to monitor our vSAN performance cluster.
We already have Prometheus and Grafana running - all we would like to is to scrape the metrics into existing Prometheus instance(s) ( for example from a single VM running red hat/cent os )

I tried to make some sense from the instruction and just simply extract needed binaries but documentation is confusing (I'm not sure which binaries exactly are needed even after re-reading "simple getting started" multiple times I'm totally lost)

Thank you

Describe the solution you'd like

I'd like instruction how to monitor VSAN using existing Prometheus/Grafana infrastructure but without the need to deploy containers.

Describe alternatives you've considered

We are using Telegraf at the moment but we've moved away from INFLUX and into native Prometheus/VictoriaMetrics database and VMware monitoring is the last solution where we cant find official vmware/vsan exporter binary.
(community supported vmware-exporter does not scrape vsan metrics)

Additional context

No response

[JDDavila]

We are in the same situation. Documentation around scraping the native /vsanmetrics endpoint would be ideal.

[ndd314]

Thanks for PR @wally007 and @JDDavila. We'll looking into document a way to integrate with existing Prometheus/Grafana.

[JDDavila]

I was able to get this working without containers.

1. Download the repo
2. Use vsan-promethetus-setup to get the bearer token. NOTE: you will need to install pyvmomi

python vsanSetupToken.py -s <vcenter> -u <user> -c <cluster>

3. Copy the bearer token that is generated.
4. Now, use vsan-prometheus-servicediscovery
5. Export the environment variables needed

export CONFIG_DIR=<Prometheus targets file>
export BEARER_TOKEN=<generated in previous step>
export VCENTER=<vcenter IP address>
export SCHEME="http"

6. Run the vsanServicediscovery.py
   python vsanServicediscovery.py
7. Use the generated targets file in your prometheus config. It will go in the file_sd_configs section.

  - job_name: "vsan"
    bearer_token: <BEARER_TOKEN>
    scheme: https
    tls_config:
      insecure_skip_verify: true
    file_sd_configs:
    - files:
       - "/etc/prometheus/vsan_targets.json"

[wally007]

I tried with CentOS7 and RHEL7.9 and AlmaLinux8/RHEL8.6 and both are getting the same error

[root@monitor vsan-prometheus-setup]# python3 vsanSetupToken.py -s
Traceback (most recent call last):
  File "vsanSetupToken.py", line 19, in <module>
    import vsanmgmtObjects
  File "/root/vsan-integration-for-prometheus/vsan-prometheus-setup/vsanmgmtObjects.py", line 7, in <module>
    from pyVmomi.VmomiSupport import CreateDataType, CreateManagedType, CreateEnumType, AddVersion, AddVersionParent, F_LINK, F_LINKABLE, F_OPTIONAL, F_SECRET, newestVersions, stableVersions, publicVersions
ImportError: cannot import name 'stableVersions'
[root@monitor vsan-prometheus-setup]#

then I thought, i might as well try to get the docker version working but that's also failing

[root@docker-1 vsan-integration-for-prometheus]# TAG=vsan_mon ./scripts/docker-build.sh
Sending build context to Docker daemon  46.08kB
Step 1/5 : FROM photon:3.0
3.0: Pulling from library/photon
ea3cba57e745: Pull complete
Digest: sha256:2beb96bb0c3cb3df7d84c24f6d7e63febf16a00066141405f360768b529e597c
Status: Downloaded newer image for photon:3.0
 ---> a596ddcf019f
Step 2/5 : RUN tdnf install -y python3 python3-pip && pip3 install setuptools && pip3 install requests && tdnf clean all
 ---> Running in 37a1ac7c0ded
Error(1229) : Timeout was reached
Error: Failed to synchronize cache for repo 'VMware Photon Extras 3.0 (x86_64)' from 'https://packages.vmware.com/photon/3.0/photon_extras_3.0_x86_64'
^C[root@docker-1 vsan-integration-for-prometheus]#
[root@docker-1 vsan-integration-for-prometheus]#
[root@docker-1 vsan-integration-for-prometheus]#
[root@docker-1 vsan-integration-for-prometheus]# curl https://packages.vmware.com/photon/3.0/photon_extras_3.0_x86_64
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<HTML>
 <HEAD>
  <TITLE>Index of /45848/photon/3.0/photon_extras_3.0_x86_64</TITLE>
 </HEAD>
 <BODY>
<H1>Index of /45848/photon/3.0/photon_extras_3.0_x86_64</H1>
<PRE>   Name                              Last modified        Size
<HR>
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="photon_extras_3.0_x86_64/../">Parent Directory</A>                  01-Jan-1970 00:00      -
<IMG SRC="/icons/dir.gif" ALT="[DIR]"> <A HREF="photon_extras_3.0_x86_64/repodata/">repodata/</A>                         22-Apr-2020 17:11      -
</PRE><HR>
</BODY></HTML>
[root@docker-1 vsan-integration-for-prometheus]#

This is on brand new AlmaLinux/RHEL8.6 and docker ce installation. Obviously connectivity is there to packages.vmware site (see curl command) so i'm not sure what the next step I can make to get this running :-(

[JDDavila]

The first error looks like you're missing pyvmomi.

Also, you are missing the required parameters, "-s vcenter -u user -c cluster"

[wally007]

Thanks for the response.
Unfortunately pyvmomi is installed

[root@docker-1 vsan-prometheus-setup]# pip3 install pyvmomi
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Requirement already satisfied: pyvmomi in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: requests>=2.3.0 in /usr/local/lib/python3.6/site-packages (from pyvmomi)
Requirement already satisfied: six>=1.7.3 in /usr/local/lib/python3.6/site-packages (from pyvmomi)
Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /usr/local/lib/python3.6/site-packages (from requests>=2.3.0->pyvmomi)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests>=2.3.0->pyvmomi)
Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /usr/local/lib/python3.6/site-packages (from requests>=2.3.0->pyvmomi)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests>=2.3.0->pyvmomi)

Originally i left out the IP/username because the error is the same whether i provide credentials or not.

[root@docker-1 vsan-prometheus-setup]# python3 vsanSetupToken.py -s 192.168.123.252 -u root -c Room43
Traceback (most recent call last):
  File "vsanSetupToken.py", line 19, in <module>
    import vsanmgmtObjects
  File "/root/vsan-integration-for-prometheus/vsan-prometheus-setup/vsanmgmtObjects.py", line 7, in <module>
    from pyVmomi.VmomiSupport import CreateDataType, CreateManagedType, CreateEnumType, AddVersion, AddVersionParent, F_LINK, F_LINKABLE, F_OPTIONAL, F_SECRET, newestVersions, stableVersions, publicVersions
ImportError: cannot import name 'stableVersions'
[root@docker-1 vsan-prometheus-setup]#

[silverzhaojr]

Please try to copy the file vsan-prometheus-exporter/vsanmgmtObjects.py to overwrite the file vsan-prometheus-setup/vsanmgmtObjects.py to see if it works.

[wally007]

Please try to copy the file vsan-prometheus-exporter/vsanmgmtObjects.py to overwrite the file vsan-prometheus-setup/vsanmgmtObjects.py to see if it works.

Thank you - that has worked. I was able to get it working after that without containers or even exporters.

1, I wish there was a way to export the bearer token from within the vSphere GUI - or is there ? Then we could simply adjust Prometheus configs and scrape metrics away - for static environments.

2, Is it documented anywhere in VMware or perhaps vSAN documents what DOM OWNER LEAF or LASOM etc. acronyms mean ? We do use vSAN but we arent specialist by any stretch in vSAN.

3, I know this is a stretch, but is there anything like this but for ESXi host metrics ?
( I know we could use telegraf but if there are already native metrics provided like for vSAN - one less tool to worry about in getting the performance metrics into Prometheus/Grafana )

Thank you for your help

[JDDavila]

I found this link helpful in explaining the various vSAN components.

https://vinception.fr/vmware-vsan-architecture-components/

I found the CompMgr role to match the "backend" statistics in the vCenter GUI.

[Kybeer]

3, I know this is a stretch, but is there anything like this but for ESXi host metrics ?

We use this exporter to scrape vCenter (non-vSAN) metrics, including ESXi hosts.

https://github.com/pryorda/vmware_exporter

[wally007]

3, I know this is a stretch, but is there anything like this but for ESXi host metrics ?

We use this exporter to scrape vCenter (non-vSAN) metrics, including ESXi hosts.

https://github.com/pryorda/vmware_exporter

I know about that exporter but its a bit buggy (if you clone vm for example) plus incomplete metrics (for example you cant get basic metrics such as network bandwidth utilization per interface - only network usage per host which is useless for hosts with several nics - ive got esxi server with 12 nic's for example)

It'd be nice to be able to query esxi hosts directly for metrics without exporter in between like we can do here with vsan.

[Kybeer]

It'd be nice to be able to query esxi hosts directly for metrics without exporter in between like we can do here with vsan.

Out of curiosity, are you using the method above by @JDDavila to scrape vSAN metrics? I'm currently looking for something similar.

[wally007]

It'd be nice to be able to query esxi hosts directly for metrics without exporter in between like we can do here with vsan.

Out of curiosity, are you using the method above by @JDDavila to scrape vSAN metrics? I'm currently looking for something similar.

yes, im able to get vSAN metrics.