Xonsh crashes on start, importing prompt_toolkit -> wcwidth fails

[jonaslb]

Xonsh crashes on startup with a "NotImplementedError" coming from pkg_resources during import of prompt_toolkit -> wcwidth.
The error does not occur with xonsh --no-rc, however using source .config/xonsh/rc.xsh works fine.
Using print statements I have also checked that the rc actually is fully loaded before the crash.

The error started happening after fedora updated wcwidth from 0.1.9 to 0.2.3.
As a temporary workaround, manually downgrading to 0.1.9 seems to restore working function to xonsh.
Another workaround, keeping the newer version, is to place from wcwidth import wcwidth in xonshrc (thus skipping xonsh's custom module loading code, I assume).

xonfig

This is the xonfig when using xonsh --no-rc followed by source .config/xonsh/rc.xsh.

$ xonfig                                                                       
+------------------+---------------------+
| xonsh            | 0.9.17.dev2         |
| Git SHA          | 97a921c7            |
| Commit Date      | Apr 8 14:48:23 2020 |
| Python           | 3.8.3               |
| PLY              | 3.11                |
| have readline    | True                |
| prompt toolkit   | 2.0.10              |
| shell type       | prompt_toolkit2     |
| pygments         | 2.4.2               |
| on posix         | True                |
| on linux         | True                |
| distro           | fedora              |
| on darwin        | False               |
| on windows       | False               |
| on cygwin        | False               |
| on msys2         | False               |
| is superuser     | False               |
| default encoding | utf-8               |
| xonsh encoding   | utf-8               |
| encoding errors  | surrogateescape     |
+------------------+---------------------+

Traceback

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/xonsh/main.py", line 404, in main
    args = premain(argv)
  File "/usr/lib/python3.8/site-packages/xonsh/main.py", line 357, in premain
    env = start_services(shell_kwargs, args)
  File "/usr/lib/python3.8/site-packages/xonsh/main.py", line 306, in start_services
    builtins.__xonsh__.shell = Shell(execer=execer, **shell_kwargs)
  File "/usr/lib/python3.8/site-packages/xonsh/shell.py", line 181, in __init__
    elif not ptk_above_min_supported():
  File "/usr/lib/python3.8/site-packages/xonsh/platform.py", line 175, in ptk_above_min_supported
    return ptk_version_info()[:2] >= minimum_required_ptk_version
  File "/usr/lib/python3.8/site-packages/xonsh/platform.py", line 167, in ptk_version_info
    return tuple(int(x) for x in ptk_version().strip("<>+-=.").split("."))
  File "/usr/lib/python3.8/site-packages/xonsh/platform.py", line 156, in ptk_version
    import prompt_toolkit
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/__init__.py", line 18, in <module>
    from .application import Application
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/__init__.py", line 3, in <module>
    from .application import Application
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/application.py", line 13, in <module>
    from prompt_toolkit.buffer import Buffer
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/buffer.py", line 17, in <module>
    from .application.current import get_app
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/application/current.py", line 5, in <module>
    from prompt_toolkit.eventloop.context import TaskLocal, TaskLocalNotSetError
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/eventloop/__init__.py", line 3, in <module>
    from .async_generator import (
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/eventloop/async_generator.py", line 24, in <module>
    from .coroutine import From, Return
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/eventloop/coroutine.py", line 5, in <module>
    from prompt_toolkit.eventloop.defaults import get_event_loop
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/eventloop/defaults.py", line 5, in <module>
    from prompt_toolkit.utils import is_windows
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/prompt_toolkit/utils.py", line 14, in <module>
    from wcwidth import wcwidth
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/usr/lib/python3.8/site-packages/xonsh/imphooks.py", line 294, in exec_module
    rtn = self.loader.exec_module(module)
  File "/usr/lib/python3.8/site-packages/wcwidth/__init__.py", line 25, in <module>
    __version__ = _get_package_version()
  File "/usr/lib/python3.8/site-packages/wcwidth/wcwidth.py", line 408, in _get_package_version
    pkg_resources.resource_string(
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1156, in resource_string
    return get_provider(package_or_requirement).get_resource_string(
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1401, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1570, in _get
    raise NotImplementedError(
NotImplementedError: Can't perform this operation for loaders without 'get_data()'
Xonsh encountered an issue during launch
Failback to /bin/sh

[swedneck]

I was just bitten by this as well, it resulted in an infinite login loop since i had my (and root's) default shell set to xonsh.

[jquast]

I maintain the wcwidth library, and this line of code that is crashing has always been a part of wcwidth since the first release in 2015.

I'm sorry it is causing trouble. The code in pkg_resources throwing the error is:

      def _get(self, path):
          if hasattr(self.loader, 'get_data'):
              return self.loader.get_data(path)
          raise NotImplementedError(
              "Can't perform this operation for loaders without 'get_data()'"
          )

Anyway, I will produce a new wcwidth release to pypi, 0.2.4, where the package version is hardcoded, I hope that helps.

[jonaslb]

Thanks a bunch jquast, I can only agree with the comment in your new code about different issues popping up quite often in xonsh. However I still like it too much to leave over it :)

I think this issue should remain open at least until 0.2.4 is in the fedora repos. Then I guess it won't affect anyone anymore.

[Achimh3011]

Updating to wcwidth 0.2.4 did not help me: now it's other methods calling pkg_resources failing. I guess the problem is not with wcwidth at all.

[jquast]

Sorry I couldn't help more, best of luck

[Achimh3011]

Problem seems to be that xonsh supplies its own wrapper loader (in order to support importing '.xsh' files) and this wrapper loader class does not support get_data() (and probably other semi-standard loader functions as well). Adding a redirection for get_data to the underlying loader solves the problem for me.

However, I was not able to find out what causes the problem to occur recently, as the xonsh code in question is some years old. Furthermore, the problem for me only occurs with xonsh and prompt_toolkit on my Linux machine while the same xonsh version on my Mac (also using prompt_toolkit) works fine.

[laloch]

There seems to be something wrong with Python in Fedora recently. @swedneck, if I remember correctly, has also some issues with Python paths and loading of modules. Xonsh should, however, not crash like this. A PR would be very welcome.