TLS / Certificate Hostname Verification against IP fails with "address family must be specified"

[JSchy65]

Describe the bug

Using Forward Output Plugin with TLS. Hostname is an IP address. Server certificate contains it's correct IP in it's SubjectAlternativeNames extention. tls_verify_hostname is switched to on.

The TLS connection to the server is not established: address family must be specified
On server side: close socket due to unexpected ssl error: SSL_read: unexpected eof while reading

Works if the hostname is a name (not an IP).
Works if the hostname is an IP and tls_verify_hostname is off.

The problem is in the Ruby implementation. A bug is existing, see
https://bugs.ruby-lang.org/issues/19770

To Reproduce

Have fluentd running on the server side with:
root certificate
server certificate with AlternativeSubjectName extention containing the IP address of the server
server key

Start fluentd on client side with:
root certificate
client certificate
client key
TLS config with tls_insecure_mode false and tls_verify_hostname true

Now, with every send of the heartbeat, you will see the error below.

The attached file contains all certificates etc. to reproduce the behaviour together with the server- and client configurations below. Your server must have the IP 192.168.55.11.
tls.zip

Expected behavior

The TLS connection should be established successfully and heartbeat and other data should be send.

Your Environment

- Fluentd version: 1.16.2 (log says 1.16.1 ??)
- TD Agent version: td-agent 4.5.0 fluentd 1.16.1 (0a6d706a9cee5882d751b2cc6169696709df0134)
- Operating system: 22.04.2 LTS (Jammy Jellyfish)
- Kernel version: 5.19.0-46-generic

Your Configuration

On server side:

<source>
  @type forward
  bind 0.0.0.0
  port 24224
  <transport tls>
    ca_path /fluentd/etc/fluent-root.crt
    cert_path /fluentd/etc/fluent-backoffice.crt
    private_key_path /fluentd/etc/fluent-backoffice.key
    private_key_passphrase abcd
  </transport>
</source>

<match *>
  @type copy
  <store>
    @type file
    path /fluentd/log/${container_name}/${container_name}
    append true
    <buffer time,container_name>
      timekey 5
      timekey_wait 1
    </buffer>
    <format>
      @type single_value
      message_key log
    </format>
  </store>
  <store>
    @type stdout
    <buffer>
      flush_interval 1
    </buffer>
  </store>
</match>

On client side:

<source>
  @type forward
  bind 0.0.0.0
  port 24224
</source>

<match *>
  @type copy
  <store>
    @type forward
    transport tls
    tls_cert_path /fluentd/etc/fluent-root.crt
    tls_client_cert_path /fluentd/etc/fluent-all-clients.crt
    tls_client_private_key_path /fluentd/etc/fluent-all-clients.key
    tls_client_private_key_passphrase efgh
    tls_verify_hostname true
    tls_allow_self_signed_cert true
    tls_insecure_mode false
    <server>
      host 192.168.55.11
      port 24224
    </server>
    <buffer>
      flush_interval 2
    </buffer>
  </store>
  <store>
    @type file
    path /fluentd/log/${container_name}/${container_name}
    append true
    <buffer time,container_name>
      timekey 5
      timekey_wait 1
    </buffer>
    <format>
      @type single_value
      message_key log
    </format>
   </store>
</match>

Your Error Log

On client side:

2023-07-18 06:19:27 +0000 [trace]: #0 fluent/log.rb:317:trace: sending heartbeat host="192.168.55.11" port=24224 heartbeat_type=:transport
2023-07-18 06:19:27 +0000 [debug]: #0 fluent/log.rb:339:debug: connect new socket
2023-07-18 06:19:27 +0000 [trace]: #0 fluent/log.rb:317:trace: loading system default certificate store
2023-07-18 06:19:27 +0000 [trace]: #0 fluent/log.rb:317:trace: adding CA cert path="/fluentd/etc/fluent-root.crt"
2023-07-18 06:19:27 +0000 [trace]: #0 fluent/log.rb:317:trace: setting TLS context mode="peer" ciphers="ALL:!aNULL:!eNULL:!SSLv2"
2023-07-18 06:19:28 +0000 [trace]: #0 fluent/log.rb:317:trace: entering TLS handshake
2023-07-18 06:19:28 +0000 [trace]: #0 fluent/log.rb:317:trace: enqueueing all chunks in buffer instance=2180
2023-07-18 06:19:28 +0000 [trace]: #0 fluent/log.rb:317:trace: enqueueing all chunks in buffer instance=2220
2023-07-18 06:19:28 +0000 [trace]: #0 fluent/log.rb:317:trace: checking peer's certificate subject=#<OpenSSL::X509::Name CN=--backoffice-->
2023-07-18 06:19:28 +0000 [debug]: #0 fluent/log.rb:339:debug: unexpected error happen during heartbeat host="192.168.55.11" port=24224 heartbeat_type=:transport error_class=IPAddr::AddressFamilyError error="address family must be specified"

On server side:

2023-07-18 06:19:28 +0000 [warn]: #0 fluent/log.rb:381:warn: close socket due to unexpected ssl error: SSL_read: unexpected eof while reading
2023-07-18 06:19:28 +0000 [warn]: #0 fluent/log.rb:381:warn: close socket due to unexpected ssl error: SSL_read: unexpected eof while reading

Additional context

The root cause of the problem is in Ruby. The method parsing the IP address raises the exception. It raises always the exception if the hostname is a string containing an IP.

This problem is already communicated to Ruby, see:
https://bugs.ruby-lang.org/issues/19770

To make your product more reliable, please support to fix that problem :-)

[kenhys]

Thank you for feedback.

Works if the hostname is an IP and tls_verify_hostname is off.

It seems that in this case, above solution is reasonable, I'm not sure warn in such a case or support fallback when invalid fqdn is assigned.