Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Elasticsearch plugin. #950

Merged
merged 3 commits into from Nov 9, 2022
Merged

Fix Elasticsearch plugin. #950

merged 3 commits into from Nov 9, 2022

Conversation

fujiwara
Copy link
Contributor

@fujiwara fujiwara commented Nov 8, 2022

Hi.

I resolved some problems in mackerel-plugin-elasticsearch.

Support to self-signed cert and basic authorization.

Added -insecure,-user, and -password flags.

Elastcsearch(ES) v8.5 official docker image runs on https with self-signed certificates and requires Basic authentication.

docker run -p 9200:9200 -e "ELASTIC_PASSWORD=pass" -e "discovery.type=single-node" elasticsearch:8.5.0

This command runs the HTTPS server on port 9200, and basic auth 'elastic:pass' is required.

Add -suppress-missing-error flag.

Recent ES versions no longer serve some stats values that are expected by this plugin.

For example, for ES v8.5.0, many errors occurred, as below.

<metrics.plugin.elasticsearch> Failed to find 'filter_cache_size': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'evictions_filter_cache': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_index': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'total_suggest': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_suggest': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_listener': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_snapshot_data': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'total_percolate': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_bench': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_bulk': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_optimize': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_percolate': Cannot handle as a hash
<metrics.plugin.elasticsearch> Failed to find 'threads_merge': Cannot handle as a hash

These stat values were removed in several versions step by step. (I wrote MISSINGvX comments in codes.)

For backward compatibility, I do not remove these metrics, but I hope to suppress errors.

Added script metrics.

ES has a limitation of script compilation rate.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-using.html

If you compile too many unique scripts within a short time,
Elasticsearch rejects the new dynamic scripts with a circuit_breaking_exception error.

Script metrics help monitor this limitation.

Support to
- Self-signed cert
- Basic authentication
- Suppress errors for missing value
ES has a limitation of script compilation rate.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting-using.html
> If you compile too many unique scripts within a short time,
> Elasticsearch rejects the new dynamic scripts with a circuit_breaking_exception error.

Script metrics are helpful in monitoring this limitation.
Some metrics have been removed in recent ES versions.
Copy link
Contributor

@yseto yseto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems good.
Thanks for your contribution.

@yseto yseto merged commit 66cf1f8 into mackerelio:master Nov 9, 2022
@yseto
Copy link
Contributor

yseto commented Nov 9, 2022

@fujiwara
Copy link
Contributor Author

fujiwara commented Nov 9, 2022

@yseto Thank you!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants