Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support external memory in CPU histogram building. #7372

Merged
merged 3 commits into from Nov 22, 2021

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Oct 27, 2021

Extracted from #7214

The perf change for the complete PR is in #7214 (comment) . Might have some small overhead (within variance), but the benefit is hist/approx will be able to support both categorical data and external memory in the future.

  • Support external memory in histogram building.
  • Remove unused SubtractionTrick function. The builder class can be completely removed, but I will keep the cleanup out of the scope.

@trivialfis trivialfis mentioned this pull request Oct 27, 2021
8 tasks
@trivialfis
Copy link
Member Author

cc @ShvetsKS

Copy link
Collaborator

@hcho3 hcho3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me. Since I'm not the expert in this area of code, I suggest you get additional review from @ShvetsKS

@trivialfis
Copy link
Member Author

@ShvetsKS Could you please take a look into this PR? We can set hist to default once we have external memory support (it will be feature complete against all the other tree methods).

Port the rest.

Comment.

Start test.

Debug.

Remove.

test.

Format.

Fix test..
@trivialfis
Copy link
Member Author

@ShvetsKS friendly ping.

@trivialfis trivialfis added this to 1.6 In Progress in 2.0 Roadmap Nov 12, 2021
@trivialfis
Copy link
Member Author

I will merge this PR next week if there's no objection.

@trivialfis trivialfis merged commit 176110a into dmlc:master Nov 22, 2021
2.0 Roadmap automation moved this from 1.6 In Progress to 1.6 Done Nov 22, 2021
@trivialfis trivialfis deleted the external-mem-hist branch November 22, 2021 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants