Ubuntu 20.04 CUDA 11.5 error

[FredDgrv]

Building on ubuntu up-to-date 20.04 with just installed nvcc

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_Sep_13_19:13:29_PDT_2021
Cuda compilation tools, release 11.5, V11.5.50
Build cuda_11.5.r11.5/compiler.30411180_0

g++ (Ubuntu 10.3.0-1ubuntu1) 10.3.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Following the section "Building From Source" in guideline

mkdir build
cd build
cmake .. -DUSE_CUDA=ON
make -j4

I get the error "CUB requires a definition of CUB_NS_QUALIFIER when CUB_NS_PREFIX/POSTFIX are defined." (more details below).
Please could you help?

error

[ 65%] Building CUDA object src/CMakeFiles/objxgboost.dir/c_api/c_api.cu.o
nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
In file included from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/cub/util_namespace.cuh:41,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/cub/util_math.cuh:37,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/adjacent_difference.h:47,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/execution_policy.h:38,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/scan_by_key.h:36,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/adl/scan_by_key.h:42,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/scan.inl:29,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/scan.h:1560,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/reduce_by_key.inl:37,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/reduce_by_key.h:87,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/reduce.inl:28,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/reduce.h:781,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/find.inl:19,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/find.h:61,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/find.inl:25,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/find.h:381,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/mismatch.inl:21,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/mismatch.h:56,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/mismatch.inl:27,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/mismatch.h:257,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/equal.inl:21,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/equal.h:46,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/equal.inl:26,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/equal.h:235,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.inl:26,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.h:586,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_vector.h:26,
from /repos/github/xgboost/src/c_api/../data/../common/device_helpers.cuh:6,
from /repos/github/xgboost/src/c_api/../data/device_adapter.cuh:10,
from /repos/github/xgboost/src/c_api/c_api.cu:7:
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/cub/version.cuh:46: warning: "CUB_VERSION" redefined
46 | #define CUB_VERSION 101301
|
In file included from /repos/github/xgboost/cub/cub/util_namespace.cuh:36,
from /repos/github/xgboost/cub/cub/util_arch.cuh:37,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/util.h:32,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/malloc_and_free.h:29,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/adl/malloc_and_free.h:42,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/memory.inl:20,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/memory.h:69,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/reference.h:23,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/memory.h:25,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_ptr.h:25,
from /repos/github/xgboost/src/c_api/../data/../common/device_helpers.cuh:5,
from /repos/github/xgboost/src/c_api/../data/device_adapter.cuh:10,
from /repos/github/xgboost/src/c_api/c_api.cu:7:
/repos/github/xgboost/cub/cub/version.cuh:46: note: this is the location of the previous definition
46 | #define CUB_VERSION 101100
|
In file included from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/cub/util_math.cuh:37,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/adjacent_difference.h:47,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/execution_policy.h:38,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/cuda/detail/scan_by_key.h:36,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/adl/scan_by_key.h:42,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/scan.inl:29,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/scan.h:1560,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/reduce_by_key.inl:37,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/reduce_by_key.h:87,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/reduce.inl:28,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/reduce.h:781,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/find.inl:19,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/find.h:61,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/find.inl:25,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/find.h:381,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/mismatch.inl:21,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/mismatch.h:56,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/mismatch.inl:27,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/mismatch.h:257,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/equal.inl:21,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/system/detail/generic/equal.h:46,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/equal.inl:26,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/equal.h:235,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.inl:26,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.h:586,
from /usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_vector.h:26,
from /repos/github/xgboost/src/c_api/../data/../common/device_helpers.cuh:6,
from /repos/github/xgboost/src/c_api/../data/device_adapter.cuh:10,
from /repos/github/xgboost/src/c_api/c_api.cu:7:
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/cub/util_namespace.cuh:46:2: error: #error CUB requires a definition of CUB_NS_QUALIFIER when CUB_NS_PREFIX/POSTFIX are defined.
46 | #error CUB requires a definition of CUB_NS_QUALIFIER when CUB_NS_PREFIX/POSTFIX are defined.
| ^~~~~
make[2]: *** [src/CMakeFiles/objxgboost.dir/build.make:862: src/CMakeFiles/objxgboost.dir/c_api/c_api.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:313: src/CMakeFiles/objxgboost.dir/all] Error 2
make: *** [Makefile:171: all] Error 2

cmake

-- The CXX compiler identification is GNU 10.3.0
-- The C compiler identification is GNU 10.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- CMake version 3.18.4
-- Performing Test XGBOOST_MM_PREFETCH_PRESENT
-- Performing Test XGBOOST_MM_PREFETCH_PRESENT - Success
-- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT
-- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT - Success
-- xgboost VERSION: 1.6.0
-- Setting build type to 'Release' as none was specified.
-- Configured CUDA host compiler: /usr/bin/c++
-- The CUDA compiler identification is NVIDIA 11.5.50
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-11.5/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMAKE_CUDA_ARCHITECTURES: 86-real;86-virtual;80-real;80-virtual;75-real;75-virtual;70-real;70-virtual;61-real;61-virtual;60-real;60-virtual;52-real;52-virtual;50-real;50-virtual;
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /usr/include
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- /repos/github/xgboost/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /repos/github/xgboost/build

[trivialfis]

Please check out #7379 .

[FredDgrv]

Thanks for the help

I did rm build and cmake .. -DBUILD_WITH_CUDA_CUB=ON -DUSE_CUDA=ON instead

Now I have this error
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/pointer.h(79): error: class "thrust::detail::eval_if<false, thrust::detail::identity_, thrust::detail::remove_cv<>>" has no member "type"
detected during:
instantiation of class "thrust::detail::pointer_base<Element, Tag, Reference, Derived> [with Element=, Tag=thrust::device_system_tag, Reference=thrust::device_reference<>, Derived=thrust::device_ptr<>]"
(125): here
instantiation of class "thrust::pointer<Element, Tag, Reference, Derived> [with Element=, Tag=thrust::device_system_tag, Reference=thrust::device_reference<>, Derived=thrust::device_ptr<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_ptr.h(61): here
instantiation of class "thrust::device_ptr [with T=]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_malloc_allocator.h(78): here
instantiation of class "thrust::device_malloc_allocator [with T=]"
/home/frederic/repos/github/xgboost/src/predictor/../common/device_helpers.cuh(435): here
instantiation of class "dh::detail::XGBDefaultDeviceAllocatorImpl [with T=]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/allocator/allocator_traits.h(244): here
instantiation of class "thrust::detail::allocator_traits [with Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/contiguous_storage.h(41): here
instantiation of class "thrust::detail::contiguous_storage<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.h(46): here
instantiation of class "thrust::detail::vector_base<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_vector.h(54): here
instantiation of class "thrust::device_vector<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/home/frederic/repos/github/xgboost/src/predictor/gpu_predictor.cu(528): here

and also

/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/type_traits/pointer_traits.h(203): error: class "thrust::detail::pointer_difference<>" has no member "type"
detected during:
instantiation of class "thrust::detail::pointer_traits [with Ptr=]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/allocator/allocator_traits.h(350): here
instantiation of class "thrust::detail::allocator_traits [with Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/contiguous_storage.h(41): here
instantiation of class "thrust::detail::contiguous_storage<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/detail/vector_base.h(46): here
instantiation of class "thrust::detail::vector_base<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/usr/local/cuda-11.5/bin/../targets/x86_64-linux/include/thrust/device_vector.h(54): here
instantiation of class "thrust::device_vector<T, Alloc> [with T=, Alloc=dh::detail::XGBDefaultDeviceAllocatorImpl<>]"
/home/frederic/repos/github/xgboost/src/predictor/gpu_predictor.cu(528): here

Please do you have an idea?

[trivialfis]

Hi, thanks for testing. Are you building from the PR I linked?

[FredDgrv]

Yes I pulled the 2 commits on your PR

[trivialfis]

Can you verify that your git submodules are up-to-date?

git submodule update

[FredDgrv]

I get that output

Submodule path 'dmlc-core': checked out 'f00e3ec7abc9f293a1b7061157b0a4e22a735cf5'
Submodule path 'gputreeshap': checked out 'c78fe621e429117cbca45e7b23eb5c3b6280fa3a'

[trivialfis]

Great! Now try to run cmake and compile it again.

[FredDgrv]

Many thanks that was the issue and the project is built now.

I believe this issue should be closed with the PR. Therefore I will let that issue opened. I hope that's ok

[trivialfis]

Yup, let me take care of the rest.