Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: spectralModularityMaximizationClustering fails for large graphs or large num_eigen_vects #4392

Closed
2 tasks done
CyanideCentral opened this issue May 7, 2024 · 5 comments
Closed
2 tasks done

[BUG]: spectralModularityMaximizationClustering fails for large graphs or large num_eigen_vects #4392

CyanideCentral opened this issue May 7, 2024 · 5 comments
Assignees
@ChuckHastings @seunghwak
Labels
bug Something isn't working

Comments

@CyanideCentral
Copy link

Version

24.04

Which installation method(s) does this occur on?

Pip

Describe the bug.

I am using spectralModularityMaximizationClustering on a graph with 2.4M nodes and 110M nonzero entries in its adjacency matrix. In the following code, the intended number of clusters is k=47.

M = graph_adj.tocoo()
edges = cudf.DataFrame()
edges['source'] = cudf.Series(M.row)
edges['target'] = cudf.Series(M.col)
G = cugraph.Graph()
G.from_cudf_edgelist(edges, source='source', destination='target')
df = cugraph.spectralModularityMaximizationClustering(G, k, num_eigen_vects=nev)

I started with num_eigen_vects=47=k, but encountered the following error. (Is this due to insufficient buffer size configured for the eigenvectors?)

File "spectral_modularity_maximization.pyx", line 143, in pylibcugraph.spectral_modularity_maximization.spectral_modularity_maximization
  File "utils.pyx", line 53, in pylibcugraph.utils.assert_success
RuntimeError: non-success value returned from cugraph_spectral_modularity_maximization: CUGRAPH_UNKNOWN_ERROR CUDA error encountered at: file=/__w/cugraph/cugraph/python/cugraph/build/cp39-cp39-manylinux_2_17_x86_64/_deps/raft-src/cpp/include/raft/spectral/detail/modularity_maximization.hpp line=105: call='cudaPeekAtLastError()', Reason=cudaErrorInvalidConfiguration:invalid configuration argument
Obtained 23 stack frames
#1 in /home/xxx/anaconda3/lib/python3.9/site-packages/cugraph/structure/../libcugraph.so: raft::cuda_error::cuda_error(std::string const&) +0x14b [0x7f2998bc0fcb]
#2 in /home/xxx/anaconda3/lib/python3.9/site-packages/cugraph/structure/../libcugraph.so: std::tuple<int, float, int> raft::spectral::detail::modularity_maximization<int, float, raft::spectral::lanczos_solver_t<int, float, int>, raft::spectral::kmeans_solver_t<int, float, int> >(raft::resources const&, raft::spectral::matrix::detail::sparse_matrix_t<int, float> const&, raft::spectral::lanczos_solver_t<int, float, int> const&, raft::spectral::kmeans_solver_t<int, float, int> const&, int*, float*, float*) +0xb40 [0x7f299a295790]
#3 in /home/xxx/anaconda3/lib/python3.9/site-packages/cugraph/structure/../libcugraph.so: void cugraph::ext_raft::detail::spectralModularityMaximization_impl<int, int, float>(cugraph::legacy::GraphCSRView<int, int, float> const&, int, int, float, int, float, int, int*, float*, float*) +0x290 [0x7f299a296240]
#4 in /home/xxx/anaconda3/lib/python3.9/site-packages/cugraph/structure/../libcugraph.so: void cugraph::ext_raft::spectralModularityMaximization<int, int, float>(cugraph::legacy::GraphCSRView<int, int, float> const&, int, int, float, int, float, int, int*) +0x3ad [0x7f299a2975ed]
#5 in /home/xxx/anaconda3/lib/python3.9/site-packages/pylibcugraph/libcugraph_c.so: cugraph_spectral_modularity_maximization +0xef7 [0x7f2bcb2ed487]
#6 in /home/xxx/anaconda3/lib/python3.9/site-packages/pylibcugraph/spectral_modularity_maximization.cpython-39-x86_64-linux-gnu.so(+0x7793) [0x7f2bce067793]
#7 in python(+0xfe58e) [0x5639a85f758e]

Referring to Issue #710, I set smaller values for num_eigen_vects, and the algorithm only finishes without crashing when nev<=4. Still, with nev=4, the result contains 36 clusters instead of k=47.

In addition, I experimented with a subgraph consisting of the first 100k nodes, and the algorithm can finish with nev=47. For the subgraph with the first 1M nodes, the algorithm can only finish with nev<=16.

p.s. I first tried to instantiate the Graph object with adjlist from the CSR sparse matrix in the following code. However, directly calling spectralModularityMaximizationClustering afterward failed with the error 'NoneType' object has no attribute 'edgelist_df'.

M = graph.tocsr()
offsets = cudf.Series(M.indptr)
indices = cudf.Series(M.indices)
G = cugraph.Graph()
G.from_cudf_adjlist(offsets, indices, None)
df = cugraph.spectralModularityMaximizationClustering(G, k, num_eigen_vects=nev)

Minimum reproducible example

No response

Relevant log output

No response

Environment details

<details><summary>Click here to see environment details</summary><pre>
     
     **git***
     Not inside a git repository
     
     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=20.04
     DISTRIB_CODENAME=focal
     DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
     NAME="Ubuntu"
     VERSION="20.04.6 LTS (Focal Fossa)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 20.04.6 LTS"
     VERSION_ID="20.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=focal
     UBUNTU_CODENAME=focal
     Linux sig2 5.4.0-173-generic #191-Ubuntu SMP Fri Feb 2 13:55:07 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
     
     ***GPU Information***
     Tue May  7 07:48:29 2024
     +---------------------------------------------------------------------------------------+
     | NVIDIA-SMI 545.29.02              Driver Version: 545.29.02    CUDA Version: 12.3     |
     |-----------------------------------------+----------------------+----------------------+
     | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
     |                                         |                      |               MIG M. |
     |=========================================+======================+======================|
     |   0  NVIDIA GeForce RTX 3090        Off | 00000000:18:00.0 Off |                  N/A |
     | 35%   37C    P8              27W / 350W |    555MiB / 24576MiB |      0%      Default |
     |                                         |                      |                  N/A |
     +-----------------------------------------+----------------------+----------------------+
     |   1  NVIDIA GeForce RTX 3090        Off | 00000000:3B:00.0 Off |                  N/A |
     | 34%   37C    P8              22W / 350W |      8MiB / 24576MiB |      0%      Default |
     |                                         |                      |                  N/A |
     +-----------------------------------------+----------------------+----------------------+
     |   2  NVIDIA GeForce RTX 3090        Off | 00000000:86:00.0 Off |                  N/A |
     | 30%   30C    P8              29W / 350W |      8MiB / 24576MiB |      0%      Default |
     |                                         |                      |                  N/A |
     +-----------------------------------------+----------------------+----------------------+
     
     +---------------------------------------------------------------------------------------+
     | Processes:                                                                            |
     |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
     |        ID   ID                                                             Usage      |
     |=======================================================================================|
     |    0   N/A  N/A   2389731      C   /home/xxx/anaconda3/bin/python          254MiB |
     |    0   N/A  N/A   2519479      C   /home/xxx/anaconda3/bin/python              282MiB |
     +---------------------------------------------------------------------------------------+
     
     ***CPU***
     Architecture:                       x86_64
     CPU op-mode(s):                     32-bit, 64-bit
     Byte Order:                         Little Endian
     Address sizes:                      46 bits physical, 48 bits virtual
     CPU(s):                             64
     On-line CPU(s) list:                0-63
     Thread(s) per core:                 2
     Core(s) per socket:                 16
     Socket(s):                          2
     NUMA node(s):                       2
     Vendor ID:                          GenuineIntel
     CPU family:                         6
     Model:                              85
     Model name:                         Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz
     Stepping:                           7
     CPU MHz:                            1200.068
     CPU max MHz:                        3900.0000
     CPU min MHz:                        1200.0000
     BogoMIPS:                           5800.00
     Virtualization:                     VT-x
     L1d cache:                          1 MiB
     L1i cache:                          1 MiB
     L2 cache:                           32 MiB
     L3 cache:                           44 MiB
     NUMA node0 CPU(s):                  0-15,32-47
     NUMA node1 CPU(s):                  16-31,48-63
     Vulnerability Gather data sampling: Mitigation; Microcode
     Vulnerability Itlb multihit:        KVM: Mitigation: Split huge pages
     Vulnerability L1tf:                 Not affected
     Vulnerability Mds:                  Not affected
     Vulnerability Meltdown:             Not affected
     Vulnerability Mmio stale data:      Mitigation; Clear CPU buffers; SMT vulnerable
     Vulnerability Retbleed:             Mitigation; Enhanced IBRS
     Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
     Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
     Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
     Vulnerability Srbds:                Not affected
     Vulnerability Tsx async abort:      Mitigation; TSX disabled
     Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
     
     ***CMake***
     /usr/bin/cmake
     cmake version 3.16.3
     
     CMake suite maintained and supported by Kitware (kitware.com/cmake).
     
     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
     Copyright (C) 2019 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
     
     
     ***nvcc***
     /usr/local/cuda-11.6/bin/nvcc
     nvcc: NVIDIA (R) Cuda compiler driver
     Copyright (c) 2005-2021 NVIDIA Corporation
     Built on Fri_Dec_17_18:16:03_PST_2021
     Cuda compilation tools, release 11.6, V11.6.55
     Build cuda_11.6.r11.6/compiler.30794723_0
     
     ***Python***
     /home/xxx/anaconda3/bin/python
     Python 3.9.12
     
     ***Environment Variables***
     PATH                            : /usr/local/cuda-11.6/bin:/usr/local/cuda-11.6/bin:/home/xxx/anaconda3/bin:/home/xxx/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
     LD_LIBRARY_PATH                 : /usr/local/cuda-11.6/lib64:/usr/local/cuda-11.6/lib64:
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/xxx/anaconda3
     PYTHON_PATH                     :
     
     ***conda packages***
     /home/xxx/anaconda3/bin/conda
     # packages in environment at /home/xxx/anaconda3:
     #
     # Name                    Version                   Build  Channel
     _ipyw_jlab_nb_ext_conf    0.1.0            py39h06a4308_1
     _libgcc_mutex             0.1                        main
     _openmp_mutex             4.5                       1_gnu
     absl-py                   1.2.0                    pypi_0    pypi
     aiohttp                   3.8.1            py39h7f8727e_1
     aiosignal                 1.2.0              pyhd3eb1b0_0
     alabaster                 0.7.12             pyhd3eb1b0_0
     anaconda                  2022.05                  py39_0
     anaconda-client           1.9.0            py39h06a4308_0
     anaconda-navigator        2.2.0            py39h06a4308_0
     anaconda-project          0.10.2             pyhd3eb1b0_0
     anndata                   0.8.0                    pypi_0    pypi
     anyio                     3.5.0            py39h06a4308_0
     appdirs                   1.4.4              pyhd3eb1b0_0
     argon2-cffi               21.3.0             pyhd3eb1b0_0
     argon2-cffi-bindings      21.2.0           py39h7f8727e_0
     arrow                     1.2.2              pyhd3eb1b0_0
     astroid                   2.6.6            py39h06a4308_0
     astropy                   5.0.4            py39hce1f21e_0
     asttokens                 2.0.5              pyhd3eb1b0_0
     astunparse                1.6.3                    pypi_0    pypi
     async-timeout             4.0.1              pyhd3eb1b0_0
     atomicwrites              1.4.0                      py_0
     attrs                     21.4.0             pyhd3eb1b0_0
     automat                   20.2.0                     py_0
     autopep8                  1.6.0              pyhd3eb1b0_0
     babel                     2.9.1              pyhd3eb1b0_0
     backcall                  0.2.0              pyhd3eb1b0_0
     backports                 1.1                pyhd3eb1b0_0
     backports.functools_lru_cache 1.6.4              pyhd3eb1b0_0
     backports.tempfile        1.0                pyhd3eb1b0_1
     backports.weakref         1.0.post1                  py_1
     bcrypt                    3.2.0            py39he8ac12f_0
     beautifulsoup4            4.11.1           py39h06a4308_0
     binaryornot               0.4.4              pyhd3eb1b0_1
     biopython                 1.80                     pypi_0    pypi
     bitarray                  2.4.1            py39h7f8727e_0
     bkcharts                  0.2              py39h06a4308_0
     black                     19.10b0                    py_0
     blas                      1.0                         mkl
     bleach                    4.1.0              pyhd3eb1b0_0
     blosc                     1.21.0               h8c45485_0
     bokeh                     2.4.2            py39h06a4308_0
     boto3                     1.21.32            pyhd3eb1b0_0
     botocore                  1.24.32            pyhd3eb1b0_0
     bottleneck                1.3.4            py39hce1f21e_0
     brotli                    1.0.9                he6710b0_2
     brotlipy                  0.7.0           py39h27cfd23_1003
     brunsli                   0.1                  h2531618_0
     bzip2                     1.0.8                h7b6447c_0
     c-ares                    1.18.1               h7f8727e_0
     ca-certificates           2022.3.29            h06a4308_1
     cachetools                4.2.2              pyhd3eb1b0_0
     certifi                   2021.10.8        py39h06a4308_2
     cffi                      1.15.0           py39hd667e15_1
     cfitsio                   3.470                hf0d0db6_6
     chardet                   4.0.0           py39h06a4308_1003
     charls                    2.2.0                h2531618_0
     charset-normalizer        2.0.4              pyhd3eb1b0_0
     click                     8.1.7                    pypi_0    pypi
     cloudpickle               2.0.0              pyhd3eb1b0_0
     clyent                    1.2.2            py39h06a4308_1
     colorama                  0.4.4              pyhd3eb1b0_0
     colorcet                  2.0.6              pyhd3eb1b0_0
     conda                     4.13.0           py39hf3d152e_1    conda-forge
     conda-build               3.21.9           py39h06a4308_0
     conda-content-trust       0.1.1              pyhd3eb1b0_0
     conda-env                 2.6.0                         1
     conda-pack                0.6.0              pyhd3eb1b0_0
     conda-package-handling    1.8.1            py39h7f8727e_0
     conda-repo-cli            1.0.5            py39h06a4308_0
     conda-token               0.3.0              pyhd3eb1b0_0
     conda-verify              3.4.2                      py_1
     constantly                15.1.0             pyh2b92418_0
     cookiecutter              1.7.3              pyhd3eb1b0_0
     cryptography              3.4.8            py39hd23ed53_0
     cssselect                 1.1.0              pyhd3eb1b0_0
     cubinlinker-cu11          0.3.0.post2              pypi_0    pypi
     cuda-python               11.8.1                   pypi_0    pypi
     cudatoolkit               11.3.1               h2bc3f7f_2
     cudf-cu11                 24.4.1                   pypi_0    pypi
     cugraph-cu11              24.4.0                   pypi_0    pypi
     cupy-cuda11x              12.0.0                   pypi_0    pypi
     curl                      7.82.0               h7f8727e_0
     cycler                    0.11.0             pyhd3eb1b0_0
     cython                    0.29.28          py39h295c915_0
     cytoolz                   0.11.0           py39h27cfd23_0
     daal4py                   2021.5.0         py39h78b71dc_0
     dal                       2021.5.1           h06a4308_803
     dask                      2024.1.1                 pypi_0    pypi
     dask-cuda                 24.4.0                   pypi_0    pypi
     dask-cudf-cu11            24.4.1                   pypi_0    pypi
     dask-expr                 0.4.0                    pypi_0    pypi
     dataclasses               0.8                pyh6d0b6a4_7
     datashader                0.13.0             pyhd3eb1b0_1
     datashape                 0.5.4            py39h06a4308_1
     dbus                      1.13.18              hb2f20db_0
     debugpy                   1.5.1            py39h295c915_0
     decorator                 5.1.1              pyhd3eb1b0_0
     defusedxml                0.7.1              pyhd3eb1b0_0
     diff-match-patch          20200713           pyhd3eb1b0_0
     distributed               2024.1.1                 pypi_0    pypi
     docutils                  0.17.1           py39h06a4308_1
     entrypoints               0.4              py39h06a4308_0
     et_xmlfile                1.1.0            py39h06a4308_0
     executing                 0.8.3              pyhd3eb1b0_0
     expat                     2.4.4                h295c915_0
     faiss-gpu                 1.7.3           py3.9_h28a55e0_0_cuda11.3    pytorch
     fastrlock                 0.8.1                    pypi_0    pypi
     filelock                  3.6.0              pyhd3eb1b0_0
     flake8                    3.9.2              pyhd3eb1b0_0
     flask                     1.1.2              pyhd3eb1b0_0
     flatbuffers               1.12                     pypi_0    pypi
     fontconfig                2.13.1               h6c09931_0
     fonttools                 4.25.0             pyhd3eb1b0_0
     freetype                  2.11.0               h70c0345_0
     frozenlist                1.2.0            py39h7f8727e_0
     fsspec                    2022.2.0           pyhd3eb1b0_0
     funcy                     1.17                     pypi_0    pypi
     future                    0.18.2           py39h06a4308_1
     gast                      0.4.0                    pypi_0    pypi
     gensim                    4.1.2            py39h295c915_0
     giflib                    5.2.1                h7b6447c_0
     glib                      2.69.1               h4ff587b_1
     glob2                     0.7                pyhd3eb1b0_0
     gmp                       6.2.1                h2531618_2
     gmpy2                     2.1.2            py39heeb90bb_0
     google-api-core           1.25.1             pyhd3eb1b0_0
     google-auth               1.33.0             pyhd3eb1b0_0
     google-auth-oauthlib      0.4.6                    pypi_0    pypi
     google-cloud-core         1.7.1              pyhd3eb1b0_0
     google-cloud-storage      1.31.0                     py_0
     google-crc32c             1.1.2            py39h27cfd23_0
     google-pasta              0.2.0                    pypi_0    pypi
     google-resumable-media    1.3.1              pyhd3eb1b0_1
     googleapis-common-protos  1.53.0           py39h06a4308_0
     greenlet                  1.1.1            py39h295c915_0
     grpcio                    1.42.0           py39hce63b2e_0
     gst-plugins-base          1.14.0               h8213a91_2
     gstreamer                 1.14.0               h28cd5cc_2
     h5py                      3.6.0            py39ha0f2276_0
     hdf5                      1.10.6               hb1b8bf9_0
     hdf5storage               0.1.19                   pypi_0    pypi
     heapdict                  1.0.1              pyhd3eb1b0_0
     holoviews                 1.14.8             pyhd3eb1b0_0
     hvplot                    0.7.3              pyhd3eb1b0_1
     hyperlink                 21.0.0             pyhd3eb1b0_0
     icu                       58.2                 he6710b0_3
     idna                      3.3                pyhd3eb1b0_0
     imagecodecs               2021.8.26        py39h4cda21f_0
     imageio                   2.9.0              pyhd3eb1b0_0
     imagesize                 1.3.0              pyhd3eb1b0_0
     importlib-metadata        7.1.0                    pypi_0    pypi
     importlib_metadata        4.11.3               hd3eb1b0_0
     incremental               21.3.0             pyhd3eb1b0_0
     inflection                0.5.1            py39h06a4308_0
     infomap                   2.7.1                    pypi_0    pypi
     iniconfig                 1.1.1              pyhd3eb1b0_0
     intake                    0.6.5              pyhd3eb1b0_0
     intel-openmp              2021.4.0          h06a4308_3561
     intervaltree              3.1.0              pyhd3eb1b0_0
     ipykernel                 6.9.1            py39h06a4308_0
     ipython                   8.2.0            py39h06a4308_0
     ipython_genutils          0.2.0              pyhd3eb1b0_1
     ipywidgets                7.6.5              pyhd3eb1b0_1
     isort                     5.9.3              pyhd3eb1b0_0
     itemadapter               0.3.0              pyhd3eb1b0_0
     itemloaders               1.0.4              pyhd3eb1b0_1
     itsdangerous              2.0.1              pyhd3eb1b0_0
     jax                       0.4.24                   pypi_0    pypi
     jaxlib                    0.4.24                   pypi_0    pypi
     jaxopt                    0.8.3                    pypi_0    pypi
     jdcal                     1.4.1              pyhd3eb1b0_0
     jedi                      0.18.1           py39h06a4308_1
     jeepney                   0.7.1              pyhd3eb1b0_0
     jinja2                    2.11.3             pyhd3eb1b0_0
     jinja2-time               0.2.0              pyhd3eb1b0_3
     jmespath                  0.10.0             pyhd3eb1b0_0
     joblib                    1.3.2                    pypi_0    pypi
     jpeg                      9e                   h7f8727e_0
     jq                        1.6               h27cfd23_1000
     json5                     0.9.6              pyhd3eb1b0_0
     jsonschema                4.4.0            py39h06a4308_0
     jupyter                   1.0.0            py39h06a4308_7
     jupyter_client            6.1.12             pyhd3eb1b0_0
     jupyter_console           6.4.0              pyhd3eb1b0_0
     jupyter_core              4.9.2            py39h06a4308_0
     jupyter_server            1.13.5             pyhd3eb1b0_0
     jupyterlab                3.3.2              pyhd3eb1b0_0
     jupyterlab_pygments       0.1.2                      py_0
     jupyterlab_server         2.10.3             pyhd3eb1b0_1
     jupyterlab_widgets        1.0.0              pyhd3eb1b0_1
     jxrlib                    1.1                  h7b6447c_2
     keras                     2.9.0                    pypi_0    pypi
     keras-preprocessing       1.1.2                    pypi_0    pypi
     keyring                   23.4.0           py39h06a4308_0
     kiwisolver                1.3.2            py39h295c915_0
     krb5                      1.19.2               hac12032_0
     lazy-object-proxy         1.6.0            py39h27cfd23_0
     lcms2                     2.12                 h3be6417_0
     ld_impl_linux-64          2.35.1               h7274673_9
     lerc                      3.0                  h295c915_0
     libaec                    1.0.4                he6710b0_1
     libarchive                3.4.2                h62408e4_0
     libclang                  14.0.1                   pypi_0    pypi
     libcrc32c                 1.1.1                he6710b0_2
     libcurl                   7.82.0               h0b77cf5_0
     libdeflate                1.8                  h7f8727e_5
     libedit                   3.1.20210910         h7f8727e_0
     libev                     4.33                 h7f8727e_1
     libfaiss                  1.7.3           hfc2d529_0_cuda11.3    pytorch
     libffi                    3.3                  he6710b0_2
     libgcc-ng                 9.3.0               h5101ec6_17
     libgfortran-ng            7.5.0               ha8ba4b0_17
     libgfortran4              7.5.0               ha8ba4b0_17
     libgomp                   9.3.0               h5101ec6_17
     libidn2                   2.3.2                h7f8727e_0
     liblief                   0.11.5               h295c915_1
     libllvm11                 11.1.0               h3826bc1_1
     libnghttp2                1.46.0               hce63b2e_0
     libpng                    1.6.37               hbc83047_0
     libprotobuf               3.19.1               h4ff587b_0
     libsodium                 1.0.18               h7b6447c_0
     libspatialindex           1.9.3                h2531618_0
     libssh2                   1.10.0               h8f2d780_0
     libstdcxx-ng              9.3.0               hd4cf53a_17
     libtiff                   4.2.0                h85742a9_0
     libunistring              0.9.10               h27cfd23_0
     libuuid                   1.0.3                h7f8727e_2
     libwebp                   1.2.2                h55f646e_0
     libwebp-base              1.2.2                h7f8727e_0
     libxcb                    1.14                 h7b6447c_0
     libxml2                   2.9.12               h03d6c58_0
     libxslt                   1.1.34               hc22bd24_0
     libzopfli                 1.0.3                he6710b0_0
     linetimer                 0.1.5                    pypi_0    pypi
     llvmlite                  0.42.0                   pypi_0    pypi
     locket                    1.0.0                    pypi_0    pypi
     lxml                      4.8.0            py39h1f438cf_0
     lz4-c                     1.9.3                h295c915_1
     lzo                       2.10                 h7b6447c_2
     markdown                  3.3.4            py39h06a4308_0
     markdown-it-py            3.0.0                    pypi_0    pypi
     markupsafe                2.0.1            py39h27cfd23_0
     mat73                     0.59                     pypi_0    pypi
     matplotlib                3.5.1            py39h06a4308_1
     matplotlib-base           3.5.1            py39ha18d171_1
     matplotlib-inline         0.1.2              pyhd3eb1b0_2
     mccabe                    0.6.1            py39h06a4308_1
     mdurl                     0.1.2                    pypi_0    pypi
     metis                     0.2a5                    pypi_0    pypi
     mistune                   0.8.4           py39h27cfd23_1000
     mkl                       2021.4.0           h06a4308_640
     mkl-service               2.4.0            py39h7f8727e_0
     mkl_fft                   1.3.1            py39hd3c417c_0
     mkl_random                1.2.2            py39h51133e4_0
     ml-dtypes                 0.3.2                    pypi_0    pypi
     mock                      4.0.3              pyhd3eb1b0_0
     mpc                       1.1.0                h10f8cd9_1
     mpfr                      4.0.2                hb69a4c5_1
     mpi                       1.0                       mpich
     mpich                     3.3.2                hc856adb_0
     mpmath                    1.2.1            py39h06a4308_0
     msgpack-python            1.0.2            py39hff7bd54_1
     multidict                 5.2.0            py39h7f8727e_2
     multipledispatch          0.6.0            py39h06a4308_0
     munkres                   1.1.4                      py_0
     mypy_extensions           0.4.3            py39h06a4308_1
     natsort                   8.2.0                    pypi_0    pypi
     navigator-updater         0.2.1                    py39_1
     nbclassic                 0.3.5              pyhd3eb1b0_0
     nbclient                  0.5.13           py39h06a4308_0
     nbconvert                 6.4.4            py39h06a4308_0
     nbformat                  5.3.0            py39h06a4308_0
     ncurses                   6.3                  h7f8727e_2
     nest-asyncio              1.5.5            py39h06a4308_0
     networkx                  2.7.1              pyhd3eb1b0_0
     nltk                      3.7                pyhd3eb1b0_0
     nose                      1.3.7           pyhd3eb1b0_1008
     notebook                  6.4.8            py39h06a4308_0
     numba                     0.59.0                   pypi_0    pypi
     numexpr                   2.8.1            py39h6abb31d_0
     numpy                     1.24.4                   pypi_0    pypi
     numpydoc                  1.2                pyhd3eb1b0_0
     nvtx                      0.2.10                   pypi_0    pypi
     oauthlib                  3.2.0                    pypi_0    pypi
     olefile                   0.46               pyhd3eb1b0_0
     oniguruma                 6.9.7.1              h27cfd23_0
     openjpeg                  2.4.0                h3ad879b_0
     openpyxl                  3.0.9              pyhd3eb1b0_0
     openssl                   1.1.1n               h7f8727e_0
     opt-einsum                3.3.0                    pypi_0    pypi
     packaging                 21.3               pyhd3eb1b0_0
     pandas                    2.2.1                    pypi_0    pypi
     pandocfilters             1.5.0              pyhd3eb1b0_0
     panel                     0.13.0           py39h06a4308_0
     param                     1.12.0             pyhd3eb1b0_0
     parsel                    1.6.0            py39h06a4308_0
     parso                     0.8.3              pyhd3eb1b0_0
     partd                     1.2.0              pyhd3eb1b0_1
     patchelf                  0.13                 h295c915_0
     pathlib                   1.0.1              pyhd3eb1b0_1
     pathspec                  0.7.0                      py_0
     patsy                     0.5.2            py39h06a4308_1
     pcre                      8.45                 h295c915_0
     pep8                      1.7.1            py39h06a4308_0
     pexpect                   4.8.0              pyhd3eb1b0_3
     pickleshare               0.7.5           pyhd3eb1b0_1003
     pillow                    9.0.1            py39h22f2fdc_0
     pip                       21.2.4           py39h06a4308_0
     pkginfo                   1.8.2              pyhd3eb1b0_0
     plotly                    5.6.0              pyhd3eb1b0_0
     pluggy                    1.0.0            py39h06a4308_1
     poyo                      0.5.0              pyhd3eb1b0_0
     prometheus_client         0.13.1             pyhd3eb1b0_0
     prompt-toolkit            3.0.20             pyhd3eb1b0_0
     prompt_toolkit            3.0.20               hd3eb1b0_0
     protego                   0.1.16                     py_0
     protobuf                  4.25.3                   pypi_0    pypi
     psutil                    5.8.0            py39h27cfd23_1
     ptxcompiler-cu11          0.8.1.post1              pypi_0    pypi
     ptyprocess                0.7.0              pyhd3eb1b0_2
     pure_eval                 0.2.2              pyhd3eb1b0_0
     py                        1.11.0             pyhd3eb1b0_0
     py-lief                   0.11.5           py39h295c915_1
     pyaml                     23.9.5                   pypi_0    pypi
     pyarrow                   14.0.2                   pypi_0    pypi
     pyasn1                    0.4.8              pyhd3eb1b0_0
     pyasn1-modules            0.2.8                      py_0
     pycodestyle               2.7.0              pyhd3eb1b0_0
     pycosat                   0.6.3            py39h27cfd23_0
     pycparser                 2.21               pyhd3eb1b0_0
     pyct                      0.4.6            py39h06a4308_0
     pycurl                    7.44.1           py39h8f2d780_1
     pydispatcher              2.0.5            py39h06a4308_2
     pydocstyle                6.1.1              pyhd3eb1b0_0
     pyerfa                    2.0.0            py39h27cfd23_0
     pyflakes                  2.3.1              pyhd3eb1b0_0
     pygments                  2.17.2                   pypi_0    pypi
     pyhamcrest                2.0.2              pyhd3eb1b0_2
     pyjwt                     2.4.0            py39h06a4308_0
     pylibcugraph-cu11         24.4.0                   pypi_0    pypi
     pylibraft-cu11            24.4.0                   pypi_0    pypi
     pylint                    2.9.6            py39h06a4308_1
     pyls-spyder               0.4.0              pyhd3eb1b0_0
     pymetis                   2022.1                   pypi_0    pypi
     pynndescent               0.5.7                    pypi_0    pypi
     pynvml                    11.4.1                   pypi_0    pypi
     pyodbc                    4.0.32           py39h295c915_1
     pyopenssl                 21.0.0             pyhd3eb1b0_1
     pyparsing                 3.0.4              pyhd3eb1b0_0
     pyqt                      5.9.2            py39h2531618_6
     pyrsistent                0.18.0           py39heee7806_0
     pysocks                   1.7.1            py39h06a4308_0
     pytables                  3.6.1            py39h77479fe_1
     pytest                    7.1.1            py39h06a4308_0
     python                    3.9.12               h12debd9_0
     python-dateutil           2.8.2              pyhd3eb1b0_0
     python-fastjsonschema     2.15.1             pyhd3eb1b0_0
     python-libarchive-c       2.9                pyhd3eb1b0_1
     python-lsp-black          1.0.0              pyhd3eb1b0_0
     python-lsp-jsonrpc        1.0.0              pyhd3eb1b0_0
     python-lsp-server         1.2.4              pyhd3eb1b0_0
     python-slugify            5.0.2              pyhd3eb1b0_0
     python-snappy             0.6.0            py39h2531618_3
     python_abi                3.9                      2_cp39    conda-forge
     pytz                      2021.3             pyhd3eb1b0_0
     pyviz_comms               2.0.2              pyhd3eb1b0_0
     pywavelets                1.3.0            py39h7f8727e_0
     pyxdg                     0.27               pyhd3eb1b0_0
     pyyaml                    6.0              py39h7f8727e_1
     pyzmq                     22.3.0           py39h295c915_2
     qdarkstyle                3.0.2              pyhd3eb1b0_0
     qstylizer                 0.1.10             pyhd3eb1b0_0
     qt                        5.9.7                h5867ecd_1
     qtawesome                 1.0.3              pyhd3eb1b0_0
     qtconsole                 5.3.0              pyhd3eb1b0_0
     qtpy                      2.0.1              pyhd3eb1b0_0
     queuelib                  1.5.0            py39h06a4308_0
     raft-dask-cu11            24.4.0                   pypi_0    pypi
     rama                      0.0.7                    pypi_0    pypi
     rapids-dask-dependency    24.4.1                   pypi_0    pypi
     readline                  8.1.2                h7f8727e_1
     regex                     2022.3.15        py39h7f8727e_0
     requests                  2.27.1             pyhd3eb1b0_0
     requests-file             1.5.1              pyhd3eb1b0_0
     requests-oauthlib         1.3.1                    pypi_0    pypi
     rich                      13.7.1                   pypi_0    pypi
     ripgrep                   12.1.1                        0
     rmm-cu11                  24.4.0                   pypi_0    pypi
     rope                      0.22.0             pyhd3eb1b0_0
     rsa                       4.7.2              pyhd3eb1b0_1
     rtree                     0.9.7            py39h06a4308_1
     ruamel_yaml               0.15.100         py39h27cfd23_0
     s3transfer                0.5.0              pyhd3eb1b0_0
     scann                     1.2.7                    pypi_0    pypi
     scanpy                    1.9.1                    pypi_0    pypi
     scikit-image              0.19.2           py39h51133e4_0
     scikit-learn              1.2.2                    pypi_0    pypi
     scikit-learn-intelex      2021.5.0         py39h06a4308_0
     scikit-optimize           0.9.0                    pypi_0    pypi
     scipy                     1.9.1                    pypi_0    pypi
     scrapy                    2.6.1            py39h06a4308_0
     seaborn                   0.11.2             pyhd3eb1b0_0
     secretstorage             3.3.1            py39h06a4308_0
     send2trash                1.8.0              pyhd3eb1b0_1
     service_identity          18.1.0             pyhd3eb1b0_1
     session-info              1.0.0                    pypi_0    pypi
     setuptools                61.2.0           py39h06a4308_0
     sip                       4.19.13          py39h295c915_0
     six                       1.16.0             pyhd3eb1b0_1
     smart_open                5.1.0              pyhd3eb1b0_0
     snappy                    1.1.9                h295c915_0
     sniffio                   1.2.0            py39h06a4308_1
     snowballstemmer           2.2.0              pyhd3eb1b0_0
     sortedcollections         2.1.0              pyhd3eb1b0_0
     sortedcontainers          2.4.0              pyhd3eb1b0_0
     soupsieve                 2.3.1              pyhd3eb1b0_0
     sparse-dot-mkl            0.9.0                    pypi_0    pypi
     sphinx                    4.4.0              pyhd3eb1b0_0
     sphinxcontrib-applehelp   1.0.2              pyhd3eb1b0_0
     sphinxcontrib-devhelp     1.0.2              pyhd3eb1b0_0
     sphinxcontrib-htmlhelp    2.0.0              pyhd3eb1b0_0
     sphinxcontrib-jsmath      1.0.1              pyhd3eb1b0_0
     sphinxcontrib-qthelp      1.0.3              pyhd3eb1b0_0
     sphinxcontrib-serializinghtml 1.1.5              pyhd3eb1b0_0
     spyder                    5.1.5            py39h06a4308_1
     spyder-kernels            2.1.3            py39h06a4308_0
     sqlalchemy                1.4.32           py39h7f8727e_0
     sqlite                    3.38.2               hc218d9a_0
     stack_data                0.2.0              pyhd3eb1b0_0
     statsmodels               0.13.2           py39h7f8727e_0
     stdlib-list               0.8.0                    pypi_0    pypi
     sympy                     1.10.1           py39h06a4308_0
     tabulate                  0.8.9            py39h06a4308_0
     tbb                       2021.5.0             hd09550d_0
     tbb4py                    2021.5.0         py39hd09550d_0
     tblib                     1.7.0              pyhd3eb1b0_0
     tenacity                  8.0.1            py39h06a4308_0
     tensorboard               2.9.1                    pypi_0    pypi
     tensorboard-data-server   0.6.1                    pypi_0    pypi
     tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
     tensorflow                2.9.1                    pypi_0    pypi
     tensorflow-estimator      2.9.0                    pypi_0    pypi
     tensorflow-io-gcs-filesystem 0.26.0                   pypi_0    pypi
     termcolor                 1.1.0                    pypi_0    pypi
     terminado                 0.13.1           py39h06a4308_0
     testpath                  0.5.0              pyhd3eb1b0_0
     text-unidecode            1.3                pyhd3eb1b0_0
     textdistance              4.2.1              pyhd3eb1b0_0
     threadpoolctl             3.1.0                    pypi_0    pypi
     three-merge               0.1.1              pyhd3eb1b0_0
     tifffile                  2021.7.2           pyhd3eb1b0_2
     tikzplotlib               0.10.1                   pypi_0    pypi
     tinycss                   0.4             pyhd3eb1b0_1002
     tk                        8.6.11               h1ccaba5_0
     tldextract                3.2.0              pyhd3eb1b0_0
     toml                      0.10.2             pyhd3eb1b0_0
     tomli                     1.2.2              pyhd3eb1b0_0
     toolz                     0.11.2             pyhd3eb1b0_0
     torch                     1.13.0+cu116             pypi_0    pypi
     torch-sparse              0.6.15                   pypi_0    pypi
     torchaudio                0.13.0+cu116             pypi_0    pypi
     torchvision               0.14.0+cu116             pypi_0    pypi
     tornado                   6.1              py39h27cfd23_0
     tqdm                      4.64.0           py39h06a4308_0
     traitlets                 5.1.1              pyhd3eb1b0_0
     twisted                   22.2.0           py39h7f8727e_0
     typed-ast                 1.4.3            py39h7f8727e_1
     typing-extensions         4.1.1                hd3eb1b0_0
     typing_extensions         4.1.1              pyh06a4308_0
     tzdata                    2024.1                   pypi_0    pypi
     ucx-py-cu11               0.37.0                   pypi_0    pypi
     ujson                     5.1.0            py39h295c915_0
     umap-learn                0.5.3                    pypi_0    pypi
     unidecode                 1.2.0              pyhd3eb1b0_0
     unixodbc                  2.3.9                h7b6447c_0
     urllib3                   1.26.9           py39h06a4308_0
     w3lib                     1.21.0             pyhd3eb1b0_0
     watchdog                  2.1.6            py39h06a4308_0
     wcwidth                   0.2.5              pyhd3eb1b0_0
     webcolors                 1.12                     pypi_0    pypi
     webencodings              0.5.1            py39h06a4308_1
     websocket-client          0.58.0           py39h06a4308_4
     werkzeug                  2.0.3              pyhd3eb1b0_0
     wget                      1.21.3               h0b77cf5_0
     wheel                     0.37.1             pyhd3eb1b0_0
     widgetsnbextension        3.5.2            py39h06a4308_0
     wrapt                     1.12.1           py39he8ac12f_1
     wurlitzer                 3.0.2            py39h06a4308_0
     xarray                    0.20.1             pyhd3eb1b0_1
     xlrd                      2.0.1              pyhd3eb1b0_0
     xlsxwriter                3.0.3              pyhd3eb1b0_0
     xz                        5.2.5                h7b6447c_0
     yaml                      0.2.5                h7b6447c_0
     yapf                      0.31.0             pyhd3eb1b0_0
     yarl                      1.6.3            py39h27cfd23_0
     zeromq                    4.3.4                h2531618_0
     zfp                       0.5.5                h295c915_6
     zict                      3.0.0                    pypi_0    pypi
     zipp                      3.7.0              pyhd3eb1b0_0
     zlib                      1.2.12               h7f8727e_2
     zope                      1.0              py39h06a4308_1
     zope.interface            5.4.0            py39h7f8727e_0
     zstd                      1.4.9                haebb681_0
     
</pre></details>

Other/Misc.

No response

Code of Conduct

  • I agree to follow cuGraph's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
@CyanideCentral CyanideCentral added ? - Needs Triage Need team to review and classify bug Something isn't working labels May 7, 2024
@BradReesWork BradReesWork removed the ? - Needs Triage Need team to review and classify label May 9, 2024
@ChuckHastings
Copy link
Collaborator

I have been able to create a graph that reproduces this error. I will investigate.

@ChuckHastings
Copy link
Collaborator

I have isolated the failing call. Working on a bug fix.

@ChuckHastings
Copy link
Collaborator

rapidsai/raft#2319 should address this. Once it gets deployed we will retest internally. Goal is to address this in 24.06 release.

@ChuckHastings
Copy link
Collaborator

I have verified in the 24.06 branch that this bug fix eliminates the crash.

So this should be corrected in 24.06. If you are testing with conda packages, you can pull the latest nightly builds. If you are testing from source you should be able to do a clean build from source to pick up this change. If you are testing from released versions, version 24.06 will be released next month.

@CyanideCentral
Copy link
Author

I have verified in the 24.06 branch that this bug fix eliminates the crash.

So this should be corrected in 24.06. If you are testing with conda packages, you can pull the latest nightly builds. If you are testing from source you should be able to do a clean build from source to pick up this change. If you are testing from released versions, version 24.06 will be released next month.

Thanks for your effort! We have tested 24.06 and can confirm that this issue is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ChuckHastings ChuckHastings

@seunghwak seunghwak

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@CyanideCentral @BradReesWork @ChuckHastings @seunghwak