Kafka Output fails to publish events due to Bug in Sarama

[simioa]

• Version: Metricbeat 7.7.1
• Operating System: RHEL 7.8

It looks like the recent Sarama Update introduced a bug which causes the Kafka Output, if configured to use multiple TLS broker, to fail because the Server Name does not match to the actual connected node.

More about this issue can be found here: Connection with multiple TLS brokers fails to negotiate

• Steps to Reproduce:
  • Update Metricbeat to 7.7.1
  • Use Kafka Output with TLS enabled and multiple brokers

Example Output Configuration:

output.kafka:
  enabled: true
  hosts: ["kafka-01:9093","kafka-02:9093","kafka-03:9093","kafka-04:9093"]

  topic: metricbeat
  max_message_bytes: 1900000

  client_id: client

  ssl.certificate_authorities: ["ca.pem"]
  ssl.certificate: "cert.pem"
  ssl.key: "key.pem"

Error in Log:

2020-06-16T15:15:55.974+0200	INFO	[publisher_pipeline_output]	pipeline/output.go:111	Connection to kafka(kafka-01:9093,kafka-02:9093,kafka-03:9093,kafka-04:9093) established
2020-06-16T15:15:56.515+0200	DEBUG	[kafka]	kafka/client.go:276	finished kafka batch
2020-06-16T15:15:56.515+0200	DEBUG	[kafka]	kafka/client.go:290	Kafka publish failed with: x509: certificate is valid for kafka-04, not kafka-01

This does not happen when using Metricbeat 7.7.0

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[jsoriano]

Probably related to #18711.

[sim0nx]

Facing the same issue in filebeats after an upgrade to the mentioned version.

[jsoriano]

This kubernetes manifest can be used to reproduce a failing scenario: https://gist.github.com/jsoriano/32ea46daa1e24ae4c86c5f91f729264d

[dedemorton]

When you fix this bug, can you please update the docs to say that the issue is resolved (or let me know, and I'll do it)? Here's an issue with details about where to make the updates: #19308

[lucabelluccini]

Hello @jsoriano
It seems the Sarama bug was fixed in https://github.com/Shopify/sarama/tree/v1.26.4 ?

[jsoriano]

@lucabelluccini yes, this issue seems to be fixed after #19527, closing this.