exceptiongroup monkeypatching traceback.TracebackException.__init__ breaks PyPy's Name/AttributeError suggestions

[cfbolz]

PyPy implements CPython 3.10's "Did you mean" suggestions for NameError and AttributeError. We backported this feature to 3.9, and it is implemented in normal Python code in the traceback module. In CPython, the suggestions are implemented in PyErr_Display in C, but traceback will also get an implementation in 3.12: python/cpython#97008

At some point between versions 1.0.0rc9 and 1.0.2, the monkeypatching of exceptiongroup started breaking this feature:

$ pypy3.9
Python 3.9.12 (51afa45d7c16841bf2b7c58bc5a52b13d78d983f, Jul 14 2022, 15:06:05)
[PyPy 7.3.10-alpha0 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> dira
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'dira' is not defined. Did you mean: dir?
>>>> import exceptiongroup
>>>> dira
Traceback (most recent call last):
  File "/home/cfbolz/bin/pypy-c-jit-105829-51afa45d7c16-linux64/lib/pypy3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<stdin>", line 1, in <module>
NameError: name 'dira' is not defined

Git bisect points to a0ec8cc. The reason for the breackage is that PyPy's TracebackException.__init__ has another check for NameError/AttributeError, just after the special case for SyntaxError:

...
        if exc_type and issubclass(exc_type, SyntaxError):
            # Handle SyntaxError's specially
            self.filename = exc_value.filename
            lno = exc_value.lineno
            self.lineno = str(lno) if lno is not None else None
            self.text = exc_value.text
            self.offset = exc_value.offset
            self.msg = exc_value.msg
        elif exc_type and issubclass(exc_type, (NameError, AttributeError)) and \
                getattr(exc_value, "name", None) is not None:
            suggestion = _compute_suggestion_error(exc_value, exc_traceback)
            if suggestion:
                self._str += ". Did you mean: %s?" % (suggestion, )
        if lookup_lines:
            self._load_lines()

I know that the monkeypatching can be turned off, but I wasn't even using exceptiongroup myself, only hypothesis which does.

I'm open to working on a fix, but would need some pointers on how you would want to do this. Ultimately the traceback module is to blame here, in my opinion, because it does not really have any useful ways to extend its functionality without copying large parts of it outright.

/cc @mattip

[cfbolz]

ah, I just realized that using exceptiongroup will also disable suggestions on CPython 3.10, so maybe this is a know property of the library?

[agronholm]

This problem is unique to this project. It gave me no joy to duplicate the entire TracebackException.__init__ method but there was no other way to make it work right.

ah, I just realized that using exceptiongroup will also disable suggestions on CPython 3.10, so maybe this is a know property of the library?

No, I didn't know that. I thought I had duplicated all of the functionality of the initializer. Does PyPY need a special code path? Either way I'm open to a PR that fixes this.

[cfbolz]

No, the problem is that in 3.10 there is no way to get those suggestions using the traceback module. The C logic for exception formatting has more features than traceback.py in 3.10, specifically the suggestions. So unless you want to backport this commit (which I would find reasonable, I think?) there is no way to get suggestions on CPython if you base everything on traceback.py (which you have to).

[cfbolz]

I could give this approach a try and we see how annoying it is. It would fix the problem both on cpy 3.10 and on pypy.