Error on cy.visit to site with invalid content-type is very confusing

[bahmutov]

Cypress 3.1.4

When visiting a site with invalid content-type, like this one https://dlive.tv/ it returns content type with extra ,text/html in charset.

$ curl -I https://dlive.tv/
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8,text/html

The following Cypress tests just visits this site

it('loads', () => {
  cy.visit('https://dlive.tv/')
})

and here is the Cypress UI

Here are the relevant logs from running with DEBUG=cypress:* npx cypress open in this case

cypress:server:socket backend:request { eventName: 'resolve:url', args: [ 'https://dlive.tv/', { failOnStatusCode: true, auth: null } ] } +0ms
  cypress:server:server resolving visit { url: 'https://dlive.tv/', headers: { host: 'localhost:51800', connection: 'Upgrade', pragma: 'no-cache', 'cache-control': 'no-cache', upgrade: 'websocket', origin: 'http://localhost:51800', 'sec-websocket-version': '13', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Cypress/3.1.4 Chrome/59.0.3071.115 Electron/1.8.2 Safari/537.36', 'accept-encoding': 'gzip, deflate', 'accept-language': 'en-US', 'sec-websocket-key': '8M/8qzHSxXOmyCptUZt3Ig==', 'sec-websocket-extensions': 'permessage-deflate; client_max_window_bits' }, options: { failOnStatusCode: true, auth: null } } +0ms
  cypress:server:server Getting remote state: { auth: undefined, props: null, origin: 'http://localhost:51800', strategy: 'file', visiting: undefined, domainName: 'localhost', fileServer: 'http://localhost:51801' } +1ms
  cypress:server:cookies getting:cookies { url: 'https://dlive.tv/' } +1ms
  cypress:server:cookies received get:cookies [] +0ms
  cypress:server:request sending request as stream { auth: null, gzip: false, url: 'https://dlive.tv/', headers: { accept: 'text/html,*/*', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Cypress/3.1.4 Chrome/59.0.3071.115 Electron/1.8.2 Safari/537.36' }, followRedirect: [Function], strictSSL: false } +0ms
  cypress:server:timers queuing timer id 23 after 30000 ms +50ms
  cypress:server:timers child received timer id 23 +52ms
  cypress:server:timers clearing timer id 23 from queue { '6': { args: [], ms: 2500, cb: [Function: cb] }, '22': { args: [], ms: 85000, cb: [Function] }, '23': { args: [], ms: 30000, cb: [Function] } } +6ms
  cypress:server:server got resolve:url response { headers: { 'content-type': 'text/html; charset=utf-8,text/html', 'transfer-encoding': 'chunked', connection: 'close', 'cache-control': 'public, max-age=0, s-maxage=20', date: 'Wed, 09 Jan 2019 15:16:43 GMT', etag: 'W/"1c030-8Kk9IAAIZ0pJy3ynZuU0D6+O+wM"', expires: 'Thu, 10 Jan 2019 15:16:43 GMT', server: 'envoy', 'set-cookie': [ '_130d6=http://10.27.87.149:80; Path=/' ], 'x-envoy-decorator-operation': 'linoapp-web.linoapp.svc.cluster.local:80/*', 'x-envoy-upstream-service-time': '412', 'x-frame-options': 'DENY, DENY', 'x-powered-by': 'Express', vary: 'Accept-Encoding,Accept-Encoding', 'x-cache': 'Miss from cloudfront', via: '1.1 2f7eec78b53c7625bd656dcd08ed1823.cloudfront.net (CloudFront)', 'x-amz-cf-id': 'HLAW4lrjbGrz-wkhffs5XBz4njVwAWCLkAYmFrQHDGbuFdQNri5QCA==' }, statusCode: 200 } +871ms

I think we can display a much better and helpful error in this case

[jennifer-shehane]

I'm not seeing where you are going in terms of suggestions. We display the content-type as null and say that it failed because the responses must have content-type: 'text/html', what more are you suggesting?

[bahmutov]

but it is not null, is it - the server sends Content-Type: text/html; charset=utf-8,text/html so we need to show that string and tell user - this is incorrect content type

[jennifer-shehane]

It seems that this is throwing here: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/util/headers.js#L6 which the modules docs say

Throws a TypeError if the Content-Type header is missing or invalid.

Initially called from here: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/server.coffee#L394

I don't know why it's doing this :/

[jennifer-shehane]

Related issue: #1727

[cypress-bot]

Released in 3.1.5.

[jennifer-shehane]

We've changed our implementation some regarding this issue. If a site's content-type is null, undefined or cannot be determined during cy.visit(), we now look at the content of the site and evaluate whether it looks like HTML. If it does look like HTML we no longer error and continue as if it the site's content-type is text/html.

If it does not look like HTML, then it will still error as shown above in this issue.

The code for this is done in #4321, but this has yet to be released. We'll update this issue and reference the changelog when it's released.

[cypress-bot]

Released in 3.3.2.