GPUTreeShap

[RAMitchell]

Initial GPU acceleration for shap values.

The following shows around 14x speed up with a 1080Ti vs AMD Ryzen 7, 8 core.

import xgboost as xgb
import numpy as np
import time

rng = np.random.RandomState(5)
n = 1000
m = 1000
X = rng.rand(m, n)
y = rng.rand(m)
m_tests = [100000]
predictors = ["cpu_predictor", "gpu_predictor"]
model = xgb.XGBRegressor(tree_method='gpu_hist', max_depth=10, n_estimators=500,
                         predictor="gpu_predictor",
                         gpu_id=0,
                         min_child_weight=0.01)
model.fit(X, y)
results = {p: [] for p in predictors}
for m_test in m_tests:
    X_test = rng.rand(m_test, n)
    dtest = xgb.DMatrix(X_test)
    for p in predictors:
        model.get_booster().set_param({"predictor": p})
        start = time.perf_counter()
        xgb_shap = model.get_booster().predict(dtest, pred_contribs=True)
        results[p].append(time.perf_counter() - start)

print(results)

{'cpu_predictor': [26.5260732], 'gpu_predictor': [1.7506337000000016]}

[codecov-commenter]

Codecov Report

Merging #6038 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6038   +/-   ##
=======================================
  Coverage   79.10%   79.10%           
=======================================
  Files          12       12           
  Lines        3044     3044           
=======================================
  Hits         2408     2408           
  Misses        636      636           

Impacted Files	Coverage Δ
python-package/xgboost/sklearn.py	91.44% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1fd29ed...39c9445. Read the comment docs.

[trivialfis]

Any chance you can write a demo in XGBoost? I know we can look into treeshap repository, but it would be nice to have one here that we can run in CI.

[slundberg]

I didn't trace each line out, but this looks good to me. My one comment is that with deep trees you can sometimes get numerical stability issues (like >50). It looks like the GPU is using the same precision as the CPU, but if it ever uses lower precision for the weights in the alg. that is something to watch out for.

[hcho3]

LGTM. See minor comments.