Pyodide stack overflows

[hoodmane]

In Chrome with Jedi completions there are reasonably frequent RangeError s which result from Python overflowing the C / wasm stack. In Firefox I guess the wasm stack is much larger and so this is rarer (note that other UI stuff doesn't work in firefox though).
This seems to trigger in the Jedi "cache to file system" when Jedi is pickling parse trees -- the parse trees are pretty deep and pickle presumably recurses for each level of the tree. I wonder if there is a way to disable this feature: of course it's not very useful in pyodide because the cache gets thrown away when we refresh (unless we decided to persist the EmscriptenFS to IndexedDB or something dumb like that).

Pyodide is supposed to use sys.setrecursionlimit to prevent these C stack overflows from happening, but when I use sys.getrecursionlimit I see 3000 which is way too big. On my chrome, it seems like 154 is the max value that avoids triggering a low level stack overflow.

Weirdly enough, the Python runtime seems to get screwed up FASTER when I set the lower recursion limit. The low level stack overflow has as it's most obvious symptom that sys.exc_info() sometimes has an exception permanently left on it even when no exception has occurred. Currently I notice this in Execution and leave those stack frames off of error messages. Sometimes eventually the CPython VM segfaults or throws an assertion error.

See this issue:
pyodide/pyodide#373

[dalcde]

Is it okay to close this issue now?

[hoodmane]

Yes this is resolved upstream in Pyodide v0.19.

[hoodmane]

I also figured out how to build Rust packages for Pyodide:
pyodide/pyodide#2378

So if we get the sseq Python package working again we could very likely build it to run in Pyodide. I haven't worked out what exactly is required to do an out of tree build for Pyodide though -- I think in principle only the Python includes should be necessary, but we need pyconfig.h which is only generated by the ./configure step. This is included in the Pyodide prebuilt docker image but I think all that is really needed for out of tree build is:

1. `./emsdk install v2.0.27
2. cat pyodide-emscripten-patches/* | patch -p1
3. get Pyodide's pyconfig.h (and the rest of the Python includes from CPython's Include directory)
4. some strange Rust invocation

In particular, we don't need to compile/link anything but our own stuff.