Python stubs loaded from extensions folder even when using an explicit interpreter

[Always-Self-Hosted]

Hopefully the title isn't too confusing, But let me explain the steps i went through and the problem im seeing.

1. Set an interpreter that isn't default (I'm using one in a venv)
2. Install custom stubs to that venv (I did pip install micropython-esp8266-stubs)
3. Open a python file using packages installed in that venv that you have stubs for (I am using gc and machine)
4. Observe that the machine package import lints correctly and if you go to definition it will open the .pyi file in that venv location, but if you do the same for the gc package it opens the .pyi file in the default extension location (in my example, it opens the path to the jedi/third_party/typeshed/stdlib/* file)

I'm happy to be told im wrong, but this seems like an oversight to me. In my program i am getting lint warnings, and not getting function previews/auto complete for my correct code, because the implementation of gc from micropython is different to the python standard library gc, but i am developing in a venv for this reason, and i can't seem to find any extension settings that allow me to explicitly tell vs code (or in my case, vscodium) to only use stubs in my venv location (or atleast change the priority order that they are looked for)

Extension version: 2024.0.0
VSCodium version: VSCodium 1.82.2 (a5004432af6600c99fa4f918a3f2f8df44df71e6, 2023-09-14T18:27:11.315Z)
OS version: Linux x64 6.1.0-17-amd64

[davidhalter]

In the venv is the gc.py file next to the gc.pyi file? If that' the case, Jedi probably should load that file first. Otherwise it prefers the typeshed file. This is wrong according to some PEP, typeshed should be a fallback. But for performance reasons this is probably going to stay like that. I'm sorry.

But I'm currently working on a Rust version of Jedi/Mypy that should improve this.

[Always-Self-Hosted]

There is no gc.py next to the gc.pyi file, but that is the same for the machine.pyi also. There only seems to be .pyi files in the ~/.venv/lib/python3.11/site-packages/ directory. Which is what gets opened by vscodium when I "Go to definition".

Does this point to a different issue if I am missing the .py files?

[davidhalter]

I'm unsure if this is an issue. Can you give me a small reproduction zip that I could try?

On the other hand I'm also personally interested to know why you prefer Jedi to Pyright/Pylance in VSCode.

[Always-Self-Hosted]

If you just pip install micropython-esp8266-stubs into a fresh venv this is the exact setup I have, and then make sure to use that venv interpreter.

Also I am on linux using vscodium. I actually thought i was using pylance and reported this issue over there first: microsoft/pylance-release#5444

and sure enough after that post, i looked in the output tab and can see :

Language server is set to "Default". Jedi is used as current language server.
IntelliCode Python suggests to use Pylance as language server. Details about Pylance: 'https://aka.ms/vscode-pylance'.

and trying to then change the settings.json to use pylance doesn't work, it wont start up for some reason. I have not gone any deeper than this sorry, As when i couldn't get the correct auto complete to work for micropython i put the coding on pause.

[Always-Self-Hosted]

Also a bit more context for this, i tried adding an empty gc.py file next to the gc.pyi file in my venv, but even after restarting vscodium it still doesn't load the correct stubs.

[davidhalter]

The problem is that with whatever sys.path you import gc with, it will never overwrite the builtin module in normal Python. So this is actually strictly related to stub loading. Therefore a gc.py module will never be loaded.

For now I do not think this is important enough and it will also worsen performance. I will keep it open though so if someone comes along and implements it, we can evaluate the impacts and maybe merge.

[Always-Self-Hosted]

Is there a way to force jedi to use gc.pyi in a typings folder in the cwd? Or is there simply no workaround possible with jedi for this problem?

Also to be clear, i appreciate you explaining the rationale for the way this works, and as the creator of this project i can respect your opinion on saying this isn't important enough to degrade performance trade offs made in this project just to fix the problem. But I am keen to try and find a workaround IF one exists. If there isn't, I guess i will have to live with it, or give up on using vscodium and accept that i need to use vscode + pylance. But would very very very much like to avoid that :D

[davidhalter]

Yes, I understand that very much. I unfortunately don't think there's a solution here for now. If correctly done the performance tradeoffs might not even be that bad, but it's just too much work for me now. So if you want to you try and play with the import system and if it doesn't make tests slower by a lot, I might merge a pull request. But I just don't have the time for that now (nor is it very important).