UnicodeEncodeError on valid RSS feed

[samuelclay]

Nothing special going on, here's the repro:

>>> url = "https://news.google.com/news/rss/local/section/geo/Valbonne,%20France/Valbonne,%20Provence-Alpes-Côte%20d%27
Azur,%20France?ned=fr&hl=fr&gl=FR"
>>> import feedparser
>>> feedparser.parse(url)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/api.py", line 214, in parse
    data = _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/api.py", line 114, in _open_resource
    return http.get(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
  File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/http.py", line 158, in get
    f = opener.open(request)
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1385, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1342, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1266, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1104, in putrequest
    self._output(self._encode_request(request))
  File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1184, in _encode_request
    return request.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\xf4' in position 79: ordinal not in range(128)
>>> feedparser.__version__
'6.0.2'

And here's the valid RSS: https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fnews.google.com%2Frss%2Ftopics%2FCAAqIggKIhxDQkFTRHdvSkwyMHZNRGhvTlRGb0VnSmxiaWdBUAE%3Fhl%3Dfr%26gl%3DFR%26ceid%3DFR%3Afr

[kurtmckee]

@samuelclay, I am going to rip out all HTTP client code in feedparser and use the requests module for very basic functionality in the future.

However, feedparser shouldn't crash so I'll investigate and release a hotfix soon.

[kurtmckee]

This is fixed in feedparser 6.0.4. Thanks for reporting this, Samuel!