leiden alg with igraph flavor causes out of bounds freezing #2969
Comments
|
@RubenVanEsch Can you provide a more minimally reproducible example? For example, paring down a bit the above to just: import scanpy as sc
em_adata = sc.datasets.pbmc3k()
sc.pp.pca(em_adata, n_comps=50)
sc.pp.neighbors(em_adata)
sc.tl.umap(em_adata)
sc.tl.leiden(em_adata,flavor='igraph',n_iterations=2,random_state=1653,directed=False)does not yield any error. |
|
@ilan-gold your minimal example causes the exact same error: Exception ignored in: <class 'ValueError'> if you are curious, it spits the error out 14.210 times (71050 lines of error message) EDIT: the random state does not seem to matter btw, also happens with different random states |
|
Ok @RubenVanEsch we have to assume that this is a windows problem then. I think we will try to set up a test job and hopefully this catches the problem, although will likely catch others. What happens without a |
|
@ilan-gold same thing without random state |
|
@RubenVanEsch Yes, and the issue there is that we're not the ones calling |
|
If the problem is windows, it's possible it will be solved by numpy 2.0. Not sure how easy the upgrade path to numpy 2.0 will be, however. |
|
I got the test runner to do windows and while there were other errors, this one was seemingly not present: https://dev.azure.com/scverse/scanpy/_build/results?buildId=6287&view=logs&j=4eb20215-89fc-58e4-6218-2c2fa88ddf72&t=482e4b16-75d9-5f8c-9594-aadcd098d2cb&l=3977 We have a test that is strikingly similar to the more minimal example from above: https://github.com/scverse/scanpy/blob/main/scanpy/tests/notebooks/test_pbmc3k.py minus the umap. Could you try this test (which doesn't call In general there will be some back and forth here until we find someone near us with a windows machine since using CI to fix this problem isn't really feasible, but at least we can narrow the scope. |
|
@RubenVanEsch, are you able to run this in WSL? Also, does the number you pass for random seed matter? |
From @RubenVanEsch :
|
|
@ivirshup @ilan-gold just got back to this, thought i could not install wsl as I am on a somewhat company restricted laptop, but turns out i can. installing it now (and probably using that from here on out). will run the tester in a bit and let you know |
@melonora, would you mind running this on your windows machine with the latest scanpy release to see if you can reproduce it? |
|
Yes I will and report back. Most likely in the evening. |
|
I can reproduce, this is the error that I get: From a first glance it seems like the default for randint is used which is |
|
I will see if I can reproduce on main and pinpoint where the problem arises. |
|
Do you guys still want me to try and run the test from @ilan-gold ? Or is it fine now that it is reproduced on your side as well? |
|
It is reproduced. It is due to the The part of the code that fails due to this is when using the context manager to perform the leiden clustering with igraph flavor. |
|
In particular here is the piece of code: scanpy/scanpy/tools/_leiden.py Lines 184 to 185 in a33111f Though the |
|
Where would you put the |
|
It wouldn't be on our side. As far as I know the numpy random number generator is called from within c code within igraph itself. |
Please make sure these conditions are met
What happened?
was running the standard pipeline on some data and when i run
sc.tl.leiden(em_adata,flavor='igraph',n_iterations=2,random_state=1653,directed=False)it spits out infinite lines of ignored exceptions. it does not actually crash the kernel, but does bog it down and causes everything to to take much more time than necesarry.
I am working in a conda env on a Win 10 , 64bit, x64 system
the problem also occurs using the pbmc3k dataset
Minimal code sample
Error output
Versions
The text was updated successfully, but these errors were encountered: