Use rapids-build-backend

[KyleFromNVIDIA]

Contributes to rapidsai/build-planning#31.

[bdice]

@KyleFromNVIDIA I'm going to merge the upstream into this PR to get a fresh CI run. This is the only open PR targeting 24.06 and it's a good candidate to look at. I want to investigate the latest CI failures, which @rjzamora indicated may be related to dask-expr.

I'll compare its failures to those on #5815.

[jameslamb]

This looks good to me, and I think it's ready to merge. Those failing test jobs are optional, and the failures are unrelated to this PR's changes.

[jameslamb]

Those 3 failing cudf-pandas-integration test jobs are optional and the failures are not a result of this PR's changes. I'm confident in that because I see the exact same failures on other unrelated PRs, like #5905.

  File "/opt/conda/envs/test/lib/python3.9/site-packages/cuml/preprocessing/LabelEncoder.py", line 121, in LabelEncoder
  Failed example:
      print(str_label)
  Exception raised:
      Traceback (most recent call last):
        File "/opt/conda/envs/test/lib/python3.9/doctest.py", line 1334, in __run
          exec(compile(example.source, filename, "single",
        File "<doctest LabelEncoder[17]>", line 1, in <module>
          print(str_label)
      NameError: name 'str_label' is not defined

(build link)

This PR looks like it has all the changes we want for the rapids-build-backend rollout, so I'm merging it.

[jameslamb]

/merge

[jameslamb]

Oh interesting, I see this now

Waiting on code owner review from rapidsai/cuml-cmake-codeowners and/or rapidsai/cuml-python-codeowners.

Even though I don't see those groups in the list of reviewers.

I guess the groups were dropped because @bdice and @vyasr are a part of them and left earlier reviews "on behalf of" those groups.

[bdice]

Just a few minor alignment questions -- we'd like the build scripts to look as identical as possible across RAPIDS.

[bdice]

We changed up how this works for some other packages, to something like rapids-generate-version > ./VERSION. Does this need a similar change?

e.g. https://github.com/rapidsai/cudf/blob/branch-24.08/ci/build_python.sh

[jameslamb]

Yes I think so, you're right. I'll change that here.

[jameslamb]

I just pushed 29911b6 doing that.

Made a similar change in build_cpp.sh as well (which doesn't need the version to be written to the VERSION file).

[jameslamb]

The devcontainer pip job is failing like this:

ERROR: Cannot install cuml-cu11 because these package versions have conflicting dependencies.

The conflict is caused by:
    raft-dask-cu11 24.8.0a20 depends on distributed-ucxx-cu11==0.39.*
    raft-dask-cu11 24.8.0a19 depends on distributed-ucxx-cu11==0.39.*

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

(build link)

I believe the root cause is bugs from rapidsai/raft#2331, not caught in CI because of rapidsai/raft#2348, which should be fixed by some combination of rapidsai/raft#2349 and rapidsai/raft#2347.