Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement secure boost scheme phase 2 - enable secure xgboost via processor interface #10124

Open
wants to merge 56 commits into
base: vertical-federated-learning
Choose a base branch
from

Conversation

ZiyueXu77
Copy link

Add extensions to the vertical pipeline implemented in phase 1 to call Processor interface at proper locations to enable secure features:

  • Before broadcasting g/h pairs, call interface to prepare buffer
  • Communicate GHistIndexMatrix and RowSetCollection to interface for local gRPC handler in response to AllGather call

src/learner.cc Outdated Show resolved Hide resolved
ZiyueXu77

This comment was marked as resolved.

@nvidianz nvidianz mentioned this pull request Apr 19, 2024
@ZiyueXu77
Copy link
Author

Hi @trivialfis, this PR is ready for review, we have some remaining failed checks due to two issues:
3 windows checks: Cannot open include file: 'dlfcn.h' -> @nvidianz is looking into this
2 mac_os: Error: The version '3.8' with architecture 'arm64' was not found for macOS 14.4.1. -> this seems to be a recent issue impacting everyone.


/*! \brief An processor interface to handle tasks that require external library through plugins */
class Processor {
public:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is my understanding about these interfaces. Please confirm if it's correct:
ProcessGHPairs: active party encrypts g&h pairs
HandleGHPairs: passive parties save encrypted g&h pairs in local
ProcessAggregation: passive parties each cacluate it's histogram based on encrypted g&h pairs, the output is ciphertext
HandleAggregation: active party decrypts the allgathered historgram ciphertexts
ProcessHistograms: active party encrypted allgathered & decrypted histograms?
HandleHistograms: active parth decrypts the allgathered historgram ciphertexts?

I have two more questions:

  1. Where are ProcessHistograms and HandleHistograms needed in secure xgboost?
  2. Passive parties need a public key to perform HE addition calculations. The public key is generated by the active party. So I think there should be two more interfaces like ProcessPublicKey and HandlePublicKey.

Copy link
Author

@ZiyueXu77 ZiyueXu77 Apr 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @shaojian-ant , thanks for the question!
"ProcessHistograms" and "HandleHistograms" are for the upcoming secure horizontal scheme, I will remove these two from this PR to avoid confusion.
Under secure federated mode (see RFC #9987), the HE is supposed to be performed outside xgboost via the interface, hence the key management, etc. will not be part of xgboost lib (and hence in xgboost we only have a "mock" interface inplementation to illustrate the functionality)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants