Assertion error truncates byte strings

[nickwilliams-eventbrite]

• a detailed description of the bug or suggestion
  Even when using arguments -vvvv --full-trace, an explicit assertion error is truncated. This is different from comparisons/diffs. Those are not truncated. But when an explicit assertion error message is provided, its contents are truncated. Example:

  def test_get_events_for_user(self):
      response = self.get(f'/v3/organizations/{self.test_organization_id}/events/')
      assert response.status_code == 200, response.content
  

  Where self.get returns a Python Requests Response object. The point of this assertion line is that, if the status code is not correct, the entire response content is dumped so that the error code(s) and message(s) is the JSON response can be viewed. However, this is the result, even with full verbosity:

  >       assert response.status_code == 200, response.content
  E       AssertionError: b'{"error": "SERVER_ERROR", "error_description": "Error(code=u\'SERVER_ERROR\', message=u\\"Internal server error: \'E...54, in get_response\
  E             raise self.JobError(response.errors)\
  E         JobError: Error executing job:", "variables": null}]}'
  E       assert 500 == 200
  E         -500
  E         +200
  

  The response content, in this case, was ~5000 bytes, but only ~250 are displayed, with an ellipse in the middle. With anything -vv or greater, the entire assertion message should be displayed.

• output of pip list from the virtual environment you are using

  pytest                 4.4.1             
  pytest-cov             2.6.1             
  pytest-django          3.4.8             
  pytest-runner          4.4
  ...
  requests               2.21.0 
  

• pytest and operating system versions
  Ubuntu 16.04

  $ python --version
  Python 3.7.1
  $ pytest --version
  This is pytest version 4.4.1, imported from /usr/local/lib/python3.7/site-packages/pytest.py
  setuptools registered plugins:
    pytest-cov-2.6.1 at /usr/local/lib/python3.7/site-packages/pytest_cov/plugin.py
    pytest-django-3.4.8 at /usr/local/lib/python3.7/site-packages/pytest_django/plugin.py
  

• minimal example if possible
  See above.

[nicoddemus]

Thanks @nickwilliams-eventbrite for the report!

Closing as a duplicate of #5192

[nickwilliams-eventbrite]

@nicoddemus, after further experimentation, this is definitely not a duplicate of #5192. It's actually caused by the fact that response.content is a bytestring.

On a simpler test case, I get the following when ipsum is a 5000-byte bytestring:

>       assert foo == 'bar', ipsum
E       AssertionError: b'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris tempor tincidunt urna, at euismod lacus tempor eu. C...elementum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nulla orci aliquam.'
E       assert 'foo' == 'bar'
E         - foo
E         + bar

However, if ipsum is a 5000-byte unicode string, I get the full output without any truncation, whether I use no arguments, -v, -vv, -vvv, or -vvvv.

>       assert foo == 'bar', ipsum
E       AssertionError: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris tempor tincidunt urna, at euismod lacus tempor eu. Curabitur ac nibh et metus maximus egestas. Donec sit amet tempus massa. Pellentesque ac gravida enim, id consequat massa. Quisque interdum faucibus porta. Nulla bibendum tincidunt vestibulum. Duis finibus a dolor non maximus. Nulla dignissim scelerisque mi non sodales. Donec quis ligula velit. Praesent et turpis ex. Donec erat lectus, pellentesque a semper in, ullamcorper sit amet quam. Sed mauris ante, suscipit eget ligula ac, viverra tincidunt nulla.
E         
E         Aenean sem sem, vulputate in metus in, hendrerit finibus sapien. Proin in tempor ex. Maecenas turpis lectus, fermentum sed ante sit amet, imperdiet semper mi. Integer vitae maximus velit, id molestie odio. Cras iaculis tincidunt lacus vitae interdum. Nullam at semper dui. Curabitur sapien velit, consequat non neque ut, condimentum rutrum urna. Sed eu elit leo. Nullam rutrum ullamcorper tellus, non suscipit quam sodales a. Aenean malesuada quam in quam ultricies bibendum in ut mauris.
E         
E         Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Suspendisse sit amet elit luctus, tempor erat in, lobortis leo. Aliquam quis nibh lacus. Sed posuere erat consequat, condimentum ipsum eget, vehicula metus. Fusce egestas ullamcorper orci sit amet euismod. Aliquam ut turpis tincidunt, posuere orci eget, rhoncus erat. Maecenas vitae orci et dolor rhoncus fringilla ut ut mi.
E         
E         Aliquam at ex mi. Quisque sit amet justo odio. Nullam dapibus egestas viverra. In pellentesque leo a vulputate commodo. Aenean maximus bibendum orci, sit amet vehicula lorem sagittis ut. Morbi ex elit, congue pretium est eu, ultrices dictum nunc. Nullam eros risus, consequat in posuere quis, iaculis sit amet lectus. Etiam facilisis aliquet diam nec tempor. Aliquam id dolor in eros feugiat luctus. Mauris sit amet tincidunt nibh, sed dictum nisl. Nam et venenatis massa. Donec pellentesque luctus justo a elementum. Donec molestie laoreet rhoncus. Ut sapien lorem, scelerisque ac feugiat et, consequat sit amet urna. Aenean in tincidunt enim. Vivamus auctor velit vel nibh imperdiet ultricies.
E         
E         Curabitur bibendum velit leo, sit amet aliquam mauris placerat et. Proin venenatis justo et urna placerat, vel pulvinar tellus iaculis. Etiam ullamcorper dui in gravida mollis. Integer tincidunt lectus in urna rutrum dictum. Vivamus malesuada lobortis erat, a ullamcorper augue. Pellentesque et lacus sed libero malesuada varius. Suspendisse ultricies scelerisque iaculis. Praesent rutrum quis neque dignissim ultricies. Cras lectus tortor, venenatis quis egestas eget, sagittis sit amet mauris. Phasellus mattis purus et felis semper, eget fringilla lacus vulputate. Vivamus consectetur purus ipsum, id consectetur dolor tempus sed. Quisque vestibulum vehicula nunc, consectetur scelerisque libero lobortis nec. Vestibulum ut ornare neque. In tincidunt posuere turpis, nec semper sapien gravida nec. Nunc eget mollis mi, sed imperdiet diam. Duis ac cursus ipsum, nec fermentum felis.
E         
E         Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nunc finibus ex id ex consequat maximus. Morbi laoreet elit ac volutpat mollis. Integer cursus efficitur lacus, eget commodo ipsum ultrices vel. Curabitur pretium tristique est id gravida. Mauris molestie viverra sapien quis posuere. Fusce at tempus massa. Duis efficitur pulvinar justo, eu laoreet lacus cursus quis. Suspendisse ut rutrum velit. Pellentesque tincidunt, diam sed tempor pellentesque, nisi mauris fermentum turpis, blandit facilisis metus erat eget odio. Etiam nec arcu ut risus pulvinar facilisis ut ac erat. Nunc fermentum diam vel ligula dignissim, a venenatis mauris molestie. Curabitur fermentum blandit quam sagittis faucibus. Mauris venenatis arcu vitae turpis imperdiet laoreet. Nulla in orci mollis, egestas ante sodales, vehicula mauris.
E         
E         Vestibulum a finibus odio, vel vulputate est. Maecenas commodo ipsum sit amet magna tempor varius. Ut malesuada urna non odio iaculis vestibulum. Nulla congue eget eros eu luctus. Phasellus ultricies sem quis faucibus posuere. Phasellus dignissim varius gravida. Quisque ante libero, bibendum vitae ex vitae, lobortis pellentesque arcu. Maecenas rhoncus ligula non posuere faucibus. Donec mollis magna purus, nec ultrices arcu ullamcorper et.
E         
E         Mauris vulputate ante eget turpis aliquet, ut efficitur orci venenatis. Fusce a condimentum tortor. Phasellus aliquet euismod ornare. Maecenas gravida, lectus ut dictum sagittis, erat justo hendrerit velit, at aliquet arcu ex nec ante. Maecenas maximus, nulla ac mattis malesuada, velit nulla egestas quam, nec pretium sem elit aliquet nulla. In lobortis odio ac dolor euismod mattis. Duis odio diam, molestie vitae sem elementum, pharetra euismod odio. Vestibulum placerat dolor sit amet feugiat porta. Nam finibus erat et nulla volutpat elementum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nulla orci aliquam.
E       assert 'foo' == 'bar'
E         - foo
E         + bar

As you can see, it's clearly not related to #5192, because the verbosity does not affect the issue. So what remains to be determined is:

1. Should PyTest apply the same truncation rules to bytestrings that it applies to unicode strings? I think it should, but it isn't here. Or,
2. Should PyTest decode the assertion error message when it's a bytestring instead of a unicode string? That could present a new set of challenges, but you could just use the bytestring as-is if it fails to decode into a unicode string.

[nicoddemus]

@nickwilliams-eventbrite thanks for the follow up, you are correct.

I've edited the title of the issue to reflect the underlying cause.

I'm not sure what to do here; your byte strings are actually text in this example, but we cannot assume this in general; for example the byte strings might be an image, in which case trying to display any part of it in the assertion message will only result in garbage. I think it is reasonable to expect that users do the decoding themselves, as they know best what are the underlying contents of the byte string.

[Zac-HD]

Closing this under the same reasoning as #9482; at least for core Pytest we think it's important to 'refuse the temptation to guess'. This would be a good feature for a plugin though!