[Roadmap] 1.4.0 Roadmap

[trivialfis]

@dmlc/xgboost-committer Please add your items here by editing this post. Let's ensure that

• Each item has to be associated with a ticket
• Major design/refactoring are associated with a RFC before committing the code
• Blocking issue must be marked as blocking
• Breaking change must be marked as breaking

For other contributors who have no permission to edit the post, please comment here about what you think should be in 1.4.0.

Main

• Initial support for Python type hints. (Better support for mypy. #6496, [dask] Add type hints. #6519)
• Improve ranking implementation. ([RFC] Revising ranking objectives. #6450)
• Add base margin to evaluation dataset. (Add base_margin for evaluation dataset. #6591)
• Further development for inplace prediction. (Feature completeness of inplace prediction. #6304 Enhance inplace prediction. #6653)
• Further development on categorical data support. (Categorical data support. #6503)
• Add score tree interval to early stopping. (Add score tree interval #5238, [Feature Request] Add score_tree_interval during early stopping #5090)
• Remove saving binary format for memory snapshot. (Drop binary format for memory snapshot. #6513)
• Staged predict ([Feature proposal] Reuse negative ntreelimit argument #5006)
• Global configuration. (Add global configuration #6414)
• Training continuation with early stopping. (Support early stopping with training continuation, correct num boosted rounds. #6506)
• Add use_rmm flag to global configuration (Add use_rmm flag to global configuration #6656)
• Deprecate ntree_limit in Python.
• Use inplace_predict for sklearn. (Use inplace predict for sklearn. #6718)
• Make all prediction functions for gbtree thread safe. (Make prediction thread safe. #6648)
• Reimplement AUC metric (Re-implement ROC-AUC. #6747).
• Fix gamma-deviance metric. (Bugs in Metrics #6731, Fix gamma deviance #6761)

Dask

• Make dask scikit-learn interface feature complete. ([dask] Complete features in regressor and classifier. #6471, [dask] Random forest estimators #6602)
  • Add ranking for dask. ([dask] Add DaskXGBRanker #6576)
  • Add other random forest interfaces. ([dask] Random forest estimators #6602)
  • Use inplace prediction for dask. ([breaking] Add prediction fucntion for DMatrix and use inplace predict for dask. #6668)
• Support training multiple models in parallel using dask. ([dask] Use distributed.MultiLock #6743)
• Optimize prediction performance. ([dask] Specify shape in prediction contrib and interaction. #6614)
• Support shap value computation. ([dask] Add a dummy sample to infer output shape. #6645)

For brief notes, at 1.4, dask interface should be feature complete, categorical data support for GPU is ready for public testing and inplace prediction will be more mature.

[Roffild]

#6507

[SmirnovEgorRu]

I want to propose to replace 'approx' to 'hist when tree_method is set to 'auto' on CPU. I observed #5178 was about this, but it's closed.

hist is faster than approx and my observations that even accuracy is better/on par. GPUs also have only 'hist' method, not 'approx'.

Do we have any concerns not to do this?

CC: @trivialfis, @hcho3, @ShvetsKS

[trivialfis]

I will put up some documents on theoretical aspect of various tree methods, then we can decide together.

[trivialfis]

in #6564

[SmirnovEgorRu]

@trivialfis, do we need to run experiments to decide, probably?

[trivialfis]

@SmirnovEgorRu I don't have objection of changing the default in this or next release. I mentioned there's a huge refactor for CPU implementations to @ShvetsKS . I would like to see some parts of it merged before making the change so we can make some fair comparison. Will come back to it after sorting out issues in dask interface. (which should be quite fast as most of the features are now supported).

[trivialfis]

I closed the PR you referenced because I couldn't get all tests passing, I think even if we decided to change the default now, we still have some blockers to track down. So refactoring first might help making the change clearer and easier.

[trivialfis]

Hi @ShvetsKS @SmirnovEgorRu I have been trying to refactor the CPU code for categorical data support based on the efficient CPU Hist code. I found that on URL dataset the cpu hist is slower than approx. It's not a conventional dataset as it's unusually wide and sparse. Just curious if you have plan on optimizing it.

[trivialfis]

The approx implementation is parallelizing on features with dynamic scheduling, so it has an advantage on these kind of datasets.

[SmirnovEgorRu]

@trivialfis, yep, we are thinking how to tune wide data sets as well. I suppose we can outperform approx with hist on URL.

[Denisevi4]

"Support training multiple models in parallel using dask". Does this include cross-validation with early stopping?

[trivialfis]

@Denisevi4 No, it's for running multiple training sessions on a single cluster simultaneously. But it's a basic requirement for cv.

[Roffild]

#6731

[trivialfis]

@hcho3 I would like to get the 1.4 out once we got AUC re-implemented. I can try fixing the gamma metric if the AUC re-implementation goes well.

[trivialfis]

I will branch out next week.

[Roffild]

Will you fix other metrics (gamma-nloglik, logloss)?

[trivialfis]

Yeah, I will take a deeper look into them this weekend.

[trivialfis]

@Roffild Will reply on the original thread: #6731

[trivialfis]

1.4 is out, submit status will be on #6793 .