puppetdb SD - how to use metadata from facts (facter)

[WojciechKuk]

I have a large puppet database, and each host in database has some facts, that I need to use as labels (metadata) in discovery.

Example facts (why I need them):

• dcnum (we have few datacenters - I need prometheus, to scrape targets only from their datacenter - how to achieve this in pdb query?)
• fqdn (often similar to certname, but I can't use node/certname for 100% hosts)
• private_ipaddress (many hosts have few IP addresses, and only one is "private" and accessible for prometheus)
• hostgroup (hosts are divided into groups - i.e. some_service/mysql_master some_service/haproxy_img)
• service_uid (hosts are linked to the Service Catalog)

I need to use those metadata from facter in discovery (I know, that some labels I can add on exporter side, but if I have them in puppet, why can't use them?) - especially private_ipaddress is crucial for me.

There are no __meta_puppetdb_fact_xxxxxx discovered labels.

How can I use metadata from facts?

[roidelapluie]

We do not support facts I think because they are not fetchables in the same API call.

[WojciechKuk]

I think, if it's possible to use tags in some class - i.e.

tag => [ "priv_ip=${::private_ipaddress}", "group=${::hostgroup}" ]

and then, recover metadata from __meta_puppetdb_tags using regexp?

[roidelapluie]

We try to avoid business logic in our service discoveries.

[WojciechKuk]

there are many business related metadata in i.e. kubernetes_sd ;)

[roidelapluie]

Can you show me where? I don't think we expect any specific format from kubernetes.

[lchopfpt]

I don't think "we try to avoid business logic" makes sense here in this context. They want metadata of their nodes. Gathering that metadata is not "business logic".... They then want to use that metadata in their rules and alerts, which is where the business logic occurs.

Querying by a fact is possible today. And I would suggest that often people do need queries here... Maybe to separate lab from prod for instance. There can be any number of reasons.

The query is able to handle querying BY a fact:
resources {certname in facts[certname]{ name = "<fact name>" and value!~"<fact value>"}}
So first it get all the nodes' certnames that match the inner query of fact name and value. Then it will get all the resources where the certname key matches those that were found in the query.

As for "outputting" facts and being able to use as labels to then query on with prometheus rules/alertmanager. This is a reasonable request.
It's pretty typical to want to include some metadata as labels and if the service discovery can do that, even better. For instance, my "Node Down" alert is based on the scraping of an exporter. If that exproter is not reachable, then the only labels are get are that of the instance/target and the prometheus job name. Which means I cannot do any other logic or group bys.

I would suggest that the puppetdb_sd provider could allow for more options than just "resources" being queried. If one could return a facts array instead of resources array, it'd personally be more valuable. And then those facts are the meta parameters that folks can choose from.

Seems like the workaround of updating a module with tags, or parameters that include your facts could be helpful.