Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jaeger DEV: Integrate with ES #1481

Closed
Jose-Matsuda opened this issue Dec 14, 2022 · 3 comments
Closed

Jaeger DEV: Integrate with ES #1481

Jose-Matsuda opened this issue Dec 14, 2022 · 3 comments
Assignees
Labels
component/elasticsearch ElasticSearch and Kibana

Comments

@Jose-Matsuda
Copy link
Contributor

Jose-Matsuda commented Dec 14, 2022

From #1462 (comment)

When trying to start up a Jaeger instance some of the created pods crash out complaining about Elastic.

Example in simple-prod-collector (same error in simple-prod-query)

"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: 
Head \"https://elastic-monitoring-es-http.monitoring-system.svc:9200\": read tcp ... 
read: connection reset by peer no ElasticSearch node available

Of note is that Souheil was able to remove the ES configuration and have it run in memory to just check things out and it worked fine.

@Souheil-Yazji Souheil-Yazji self-assigned this Dec 14, 2022
@Souheil-Yazji Souheil-Yazji added the component/elasticsearch ElasticSearch and Kibana label Dec 14, 2022
@Souheil-Yazji
Copy link
Contributor

ES service cannot be found by Jaeger-Operator spawned pods. Istio sidecarts injected into them successfully. Support ticket to CNS created:
https://jirab.statcan.ca/browse/CLOUD-17384

@Souheil-Yazji
Copy link
Contributor

Souheil-Yazji commented Jan 11, 2023

After correctly configuring the ES service thanks to Will, we ran into an issue where the Jaeger Collector, a component created by the operator, was running into the following error:
"msg":"Failed to create span writer","error":"elastic: Error 400 (Bad Request): unknown key [template] in the template

After doing some research, it seems that the latest release of Jaeger (1.41) does not yet support ES v8.x+. There is a long-run issue open in the Jaeger repository for this.
Also supporting this discovery https://www.jaegertracing.io/docs/1.41/deployment/#elasticsearch, does not include 8.

For reference, our ElasticSearch instance is currently at v8.5.0

According to ES, it should be possible to integrate https://www.elastic.co/guide/en/apm/guide/current/jaeger-integration.html

@Souheil-Yazji
Copy link
Contributor

Closing this issue as Jaeger successfully connects to the ES instance and that is the normal scope. Continuing work on the last comment in #1505

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/elasticsearch ElasticSearch and Kibana
Projects
None yet
Development

No branches or pull requests

2 participants