:tls_alert, 'handshake failure', https://www.bbc.co.uk

[docapotamus]

Hello,

I originally posted this in the HTTPoison repository but I was informed it is as issue with hackney/ssl (edgurgel/httpoison#164). Please feel free to point me in the right direction if I am wrong again.

I am trying to check https://www.bbc.co.uk from HTTPoison and receiving:

[error] SSL: :hello: ssl_alert.erl:97:Fatal error: handshake failure

With the following tuple returned:

{:error, %HTTPoison.Error{id: nil, reason: {:tls_alert, 'handshake failure'}}}}

I can't see anything wrong with the certificate when using openssl s_client -connect www.bbc.co.uk:443:

CONNECTED(00000003)
depth=2 C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
verify return:1
depth=1 C = BE, O = GlobalSign nv-sa, CN = GlobalSign Organization Validation CA - SHA256 - G2
verify return:1
depth=0 C = GB, ST = London, L = London, O = British Broadcasting Corporation, CN = *.bbc.co.uk
verify return:1

---
Certificate chain
 0 s:/C=GB/ST=London/L=London/O=British Broadcasting Corporation/CN=*.bbc.co.uk
   i:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
 1 s:/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2
   i:/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA

---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF0zCCBLugAwIBAgISESGmCn6XQw4M0Yr34Hxqn/0FMA0GCSqGSIb3DQEBCwUA
MGYxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTwwOgYD
VQQDEzNHbG9iYWxTaWduIE9yZ2FuaXphdGlvbiBWYWxpZGF0aW9uIENBIC0gU0hB
MjU2IC0gRzIwHhcNMTYwMzE0MTcwMTA2WhcNMTcwMzE1MTcwMTA2WjBwMQswCQYD
VQQGEwJHQjEPMA0GA1UECBMGTG9uZG9uMQ8wDQYDVQQHEwZMb25kb24xKTAnBgNV
BAoTIEJyaXRpc2ggQnJvYWRjYXN0aW5nIENvcnBvcmF0aW9uMRQwEgYDVQQDDAsq
LmJiYy5jby51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJhk1p3B
nY+PBKhGHtyVKEJWONitLVBPAqK9E0265Nv9LY4A5mhui9dgnQK31cUdCYM8mzSc
iLtGzbcOZ9Wp95nfGQalcjG2PakE5vlbeVxR5K2khJD6Uvys6X3frxJ7CWjaY+ms
VURe5YeSZp8pMdB2QPFoSG1Yu7SM21yHtz0WbEITXtNLGBVUMzlT1wdyIWa0pNYH
IxKPZZN7oMtQuIzTFq+lLiuJAUGYBmw90yZcoC9pgq9W0qCV5phy19QtJj++t5QJ
2ZOqHqyP3VL1sMHfI09EBftLwzsWY0RXbZ84ePjf5ldfeLiEjBIstWzd+rNH2yww
2BsKBmLH5Ns/CU8CAwEAAaOCAm8wggJrMA4GA1UdDwEB/wQEAwIFoDBJBgNVHSAE
QjBAMD4GBmeBDAECAjA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9iYWxz
aWduLmNvbS9yZXBvc2l0b3J5LzCBtQYDVR0RBIGtMIGqggsqLmJiYy5jby51a4IK
YmJjaS5jby51a4IHYmJjLmNvbYIObGl2ZS5iYmMuY28udWuCD2xpdmUuYmJjaS5j
by51a4IMbGl2ZS5iYmMuY29tggwqLmJiY2kuY28udWuCCSouYmJjLmNvbYIQKi5s
aXZlLmJiYy5jby51a4IRKi5saXZlLmJiY2kuY28udWuCDioubGl2ZS5iYmMuY29t
ggliYmMuY28udWswCQYDVR0TBAIwADAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYB
BQUHAwIwSQYDVR0fBEIwQDA+oDygOoY4aHR0cDovL2NybC5nbG9iYWxzaWduLmNv
bS9ncy9nc29yZ2FuaXphdGlvbnZhbHNoYTJnMi5jcmwwgaAGCCsGAQUFBwEBBIGT
MIGQME0GCCsGAQUFBzAChkFodHRwOi8vc2VjdXJlLmdsb2JhbHNpZ24uY29tL2Nh
Y2VydC9nc29yZ2FuaXphdGlvbnZhbHNoYTJnMnIxLmNydDA/BggrBgEFBQcwAYYz
aHR0cDovL29jc3AyLmdsb2JhbHNpZ24uY29tL2dzb3JnYW5pemF0aW9udmFsc2hh
MmcyMB0GA1UdDgQWBBR08gvI1HXfa/ExprxIjz0ovqm9vTAfBgNVHSMEGDAWgBSW
3mHxvRwWKVMcwMx9O4MAQOYafDANBgkqhkiG9w0BAQsFAAOCAQEAXymgwHvukz+R
HEpVFWwLKQJoXL6EoQPiQb0v+bX3lOlh2B+plG+Ev0GVU7XR+okpsZXuhu6LrLvm
TBnny1RTxozy4hRqQ44pylDo7KyN90ynDYSgm8W+9V/c51ZWlhhg2KkklJubnuH5
DUL5t+G3EVawUit+XuEbNfC3yBYvbQpY4rjelr1uNukcnvRyQ6+8PjGH50ZzyAni
o+ydZvbVLpTosye/KhCmnmVv7QsKrF73BzMVcsfob/nQ8zvM5vXtDPoI6FzD+YSW
J8LOf3GJf7EuwxB9N2uoTR5ademUF2eN+SjaMB/porVkHIMrmY9QR5cd9kY1cTvn
vhIf+cM6UA==
-----END CERTIFICATE-----
subject=/C=GB/ST=London/L=London/O=British Broadcasting Corporation/CN=*.bbc.co.uk
issuer=/C=BE/O=GlobalSign nv-sa/CN=GlobalSign Organization Validation CA - SHA256 - G2

---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits

---
SSL handshake has read 3136 bytes and written 434 bytes

---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 77AFF54A148AEC8BD6359129C69A2B33E22B86205BDBA720CA19DAD3934A5C09
    Session-ID-ctx: 
    Master-Key: 67730137790D1F41E4E99F24511B75C64D4A4C6004485E3C89CD4F6E5FF56B6D8B3CAD3A5D87C2C45FE2EB15C3450744
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1470851488
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)

---

I can provide any more details that are needed. Its seems to be working in all browsers and via curl. I tested this against a fresh home brew installation of OpenSSL but I am also getting the issue on up to date Debian Jessie Machine.

I am using the Erlangsolutions packages on Debian.

Not sure how the Erlang TLS/SSL module works but I'm wondering if it is an issue there.

Thanks in advance

[ericmj]

What OTP version are you using?

[docapotamus]

Erlang/OTP 19 [erts-8.0] [source-6dc93c1] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]

Is this whats needed? I'm not too sure how to get more information that the Shell startup.

[ericmj]

Can you try checking if OTP 18 works? I have seen a similar regression on OTP 19.

[docapotamus]

@ericmj I get the same issue with 18.3.4 from Erlang Solutions

Erlang/OTP 18 [erts-7.3.1] [source-25741fd] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]

[benoitc]

@docapotamus can you retry with the latest master? I don't reproduce it right now:

Eshell V7.2.1  (abort with ^G)
1> application:ensure_all_started(hackney).
{ok,[idna,mimerl,certifi,ssl_verify_fun,metrics,hackney]}
2> hackney:get("https://www.bbc.co.uk").
{ok,200,
    [{<<"Server">>,<<"nginx">>},
     {<<"Content-Type">>,<<"text/html; charset=utf-8">>},
     {<<"ETag">>,<<"W/\"2ea59-t07wZlKNKmIwdVEaVGr5Iw\"">>},
     {<<"X-Frame-Options">>,<<"SAMEORIGIN">>},
     {<<"x-origin-route">>,<<"xrt-ext">>},
     {<<"Content-Length">>,<<"191065">>},
     {<<"Date">>,<<"Thu, 25 Aug 2016 14:28:48 GMT">>},
     {<<"Connection">>,<<"keep-alive">>},
     {<<"Set-Cookie">>,
      <<"BBC-UID=35678baf90804ab0792a65e611b99a63ed19b791271444463a80"...>>},
     {<<"X-Cache-Action">>,<<"HIT">>},
     {<<"X-Cache-Hits">>,<<"2">>},
     {<<"X-Cache-Age">>,<<"17">>},
     {<<"Cache-Control">>,
      <<"private, max-age=0, must-revalidate">>},
     {<<"Vary">>,
      <<"Accept-Encoding, X-CDN, X-BBC-Edge-Schem"...>>}],
    #Ref<0.0.2.2025>}
3>

[docapotamus]

I just used the Rebar example from the README

$ rebar3 compile
===> Verifying dependencies...
===> Fetching hackney ({git,"git://github.com/benoitc/hackney.git",
                                   {branch,"master"}})
===> Package certifi-0.5.0 not found. Fetching registry updates and trying again...
===> Updating package registry...
===> Writing registry to /home/joe/.cache/rebar3/hex/default/registry
===> Generating package index...
===> [cloudi_service_oauth1:1.5.1] Only existing version of cloudi_service_db_riak is 1.3.3 which does not match constraint ~> 1.5.1. Using anyway, but it is not guaranteed to work.
===> Writing index to /home/joe/.cache/rebar3/hex/default/packages.idx
===> Fetching certifi ({pkg,<<"certifi">>,<<"0.5.0">>})
===> Downloaded package, caching at /home/joe/.cache/rebar3/hex/default/packages/certifi-0.5.0.tar
===> Fetching idna ({pkg,<<"idna">>,<<"1.2.0">>})
===> Downloaded package, caching at /home/joe/.cache/rebar3/hex/default/packages/idna-1.2.0.tar
===> Fetching metrics ({pkg,<<"metrics">>,<<"1.0.1">>})
===> Downloaded package, caching at /home/joe/.cache/rebar3/hex/default/packages/metrics-1.0.1.tar
===> Fetching mimerl ({pkg,<<"mimerl">>,<<"1.0.2">>})
===> Downloaded package, caching at /home/joe/.cache/rebar3/hex/default/packages/mimerl-1.0.2.tar
===> Fetching ssl_verify_fun ({pkg,<<"ssl_verify_fun">>,<<"1.1.0">>})
===> Downloaded package, caching at /home/joe/.cache/rebar3/hex/default/packages/ssl_verify_fun-1.1.0.tar
===> Compiling mimerl
===> Compiling certifi
===> Compiling ssl_verify_fun
===> Compiling metrics
===> Compiling idna
===> Compiling hackney


Erlang/OTP 19 [erts-8.0.2] [source-9503fff] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V8.0.2  (abort with ^G)
1> application:ensure_all_started(hackney).
{ok,[idna,mimerl,certifi,ssl_verify_fun,metrics,hackney]}
2> hackney:get("https://www.bbc.co.uk").

=ERROR REPORT==== 25-Aug-2016::16:09:38 ===
SSL: hello: ssl_alert.erl:97:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}
3>
[0] 

Still have the same issue.

I am still trying on OTP19. I am using a fresh install of Debian with all updates each time on Digital Ocean just so I know I'm not breaking anything.

[benoitc]

@docapotamus i reproduced it with erlang 19 (was using erlang 18). I am checking which change could cause this error...

[docapotamus]

Thanks @benoitc. I should have tested it with Erlang 18 really.

[benoitc]

@docapotamus it should now work with erlang 19:

Erlang/OTP 19 [erts-8.0.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:0] [kernel-poll:false]

Eshell V8.0.3  (abort with ^G)
1> application:ensure_all_started(hackney).
{ok,[idna,mimerl,certifi,ssl_verify_fun,metrics,hackney]}
2> hackney:get("https://www.bbc.co.uk").
{ok,200,
    [{<<"Server">>,<<"nginx">>},
     {<<"Content-Type">>,<<"text/html; charset=utf-8">>},
     {<<"ETag">>,<<"W/\"2c837-nLA+fDke2xr7Rl5EAJxYRw\"">>},
     {<<"X-Frame-Options">>,<<"SAMEORIGIN">>},
     {<<"x-origin-route">>,<<"xrt-ext">>},
     {<<"Content-Length">>,<<"182327">>},
     {<<"Date">>,<<"Fri, 26 Aug 2016 09:34:55 GMT">>},
     {<<"Connection">>,<<"keep-alive">>},
     {<<"Set-Cookie">>,
      <<"BBC-UID=35574c10a0cd238fd8fecc42b1a94721c21aac85871474e67aa0"...>>},
     {<<"X-Cache-Action">>,<<"HIT">>},
     {<<"X-Cache-Hits">>,<<"5">>},
     {<<"X-Cache-Age">>,<<"104">>},
     {<<"Cache-Control">>,
      <<"private, max-age=0, must-revalidate">>},
     {<<"Vary">>,
      <<"Accept-Encoding, X-CDN, X-BBC-Edge-Schem"...>>}],
    #Ref<0.0.1.2055>}

[docapotamus]

@benoitc Thank you very much.

[nkezhaya]

Any way to bump hackney to 1.6.2? This was a pretty critical fix for us Erlang 19 users.

[jastkand]

@benoitc Hi, I have the same issue using Erlang 19.1 on macOS Sierra. Should I create the separate issue for that? Here is my deps definition:

{hackney, ".*", {git, "git://github.com/benoitc/hackney.git", {branch, "master"}}}

Erlang/OTP 19 [erts-8.1] [source-77fb4f8] [64-bit] [smp:4:4] [async-threads:0] [kernel-poll:false]

Eshell V8.1  (abort with ^G)
1> application:ensure_all_started(hackney).
{ok,[idna,mimerl,certifi,ssl_verify_fun,metrics,hackney]}
2> hackney:get("https://www.bbc.co.uk").

=ERROR REPORT==== 28-Oct-2016::15:56:54 ===
SSL: hello: ssl_alert.erl:97:Fatal error: handshake failure
{error,{tls_alert,"handshake failure"}}

[jakesgordon]

I'm still seeing this issue with hackney 1.6.3 on Erlang/OTP 19 (via Elixir 1.3.3 on Ubuntu 16.04).

I also see the same problem just doing :ssl.connect

iex> :ssl.connect 'www.bbc.co.uk', 443, []
{:error, {:tls_alert, 'handshake failure'}}

(while browsers and cURL can connect to "https://www.bbc.co.uk" just fine)

So maybe this is a problem in underlying erlang ssl library?