Getting exception for certain URLs that work with curl and other tools

[erezsh]

I'm not really sure how to describe this in idna terms. I never used idna directly.

Here's the issue I opened for requests: psf/requests#4569

They told me to hand it over to you (they're too lazy to do it themselves).

Anyway, hope this is helpful. Sorry if not.

P.S. here's the exception, just for convenience:

idna.core.IDNAError: The label b'xn--mn8ha4uc' is not a valid A-label

[kjd]

What is the expected behavior? The domain name is not a legal IDN. That said, is it being passed as a A-label or as a U-label? Is the expectation to pass-through the invalid label?

See #18 (comment) for related discussion.

[erezsh]

Honestly, I don't know. But I do know that if I put that url in curl, firefox/chrome, or even a different configuration of requests, then it works.

So it seems to me that it's either a bug in idna, or you're enforcing a standard that no-one else cares about..

[jribbens]

I think it's a bug in pyopenssl; the name contains emoji and hence idna is right to refuse it, but it's not actually the domain name you're fetching, it's just a redundant subjectAltName on an SSL cert, so pyopenssl should just be ignoring the invalid altName.

[kjd]

Per @jribbens, the root issue here seems to be pyopenssl trying to do an unnecessary IDNA conversion that is not relevant to the operation (which is connecting to https://thegroovecartel.com/feed/ per the parent issue in the requests library). Closing.