Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PySpark XGBoost integration #8020

Merged
merged 74 commits into from Jul 13, 2022
Merged

PySpark XGBoost integration #8020

merged 74 commits into from Jul 13, 2022

Commits on Jun 21, 2022

  1. init 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 21, 2022
    Copy the full SHA
    0a15487 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2022

  1. fix 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    a04a1d0 View commit details
    Browse the repository at this point in the history

  2. clean 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    d2dbb8d View commit details
    Browse the repository at this point in the history

  3. remove external mode 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    34827ef View commit details
    Browse the repository at this point in the history

  4. update doc style 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    50ebb1f View commit details
    Browse the repository at this point in the history

  5. black 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    f386ee1 View commit details
    Browse the repository at this point in the history

  6. update 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jun 26, 2022
    Copy the full SHA
    5d4122c View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2022

  1. refactor 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    0424c19 View commit details
    Browse the repository at this point in the history

  2. update params code 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    f8f33bd View commit details
    Browse the repository at this point in the history

  3. pyspark param alias 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    ba4787f View commit details
    Browse the repository at this point in the history

  4. fix 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    9447486 View commit details
    Browse the repository at this point in the history

  5. add gpu param check test 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    016b1b7 View commit details
    Browse the repository at this point in the history

  6. update _repartition_needed

    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    72af029 View commit details
    Browse the repository at this point in the history

  7. merge fit/fit_distributed

    @WeichenXu123
    WeichenXu123 committed Jul 1, 2022
    Copy the full SHA
    55fa052 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2022

  1. fix base margin support 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 2, 2022
    Copy the full SHA
    7d9c37d View commit details
    Browse the repository at this point in the history

  2. remove dump file code 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 2, 2022
    Copy the full SHA
    14392a4 View commit details
    Browse the repository at this point in the history

  3. fix verbose param

    @WeichenXu123
    WeichenXu123 committed Jul 2, 2022
    Copy the full SHA
    5bee831 View commit details
    Browse the repository at this point in the history

  4. update _unsupported_xgb_params 

    Signed-off-by: Weichen Xu <weichen.xu@databricks.com>
    @WeichenXu123
    WeichenXu123 committed Jul 2, 2022
    Copy the full SHA
    a75ee88 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2022

  1. set nthread to be spark.task.cpus

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    d71e7e0 View commit details
    Browse the repository at this point in the history

  2. support feature_types and feature_names

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    75cfe91 View commit details
    Browse the repository at this point in the history

  3. update _repartition_needed

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    7f68346 View commit details
    Browse the repository at this point in the history

  4. support use array as features column

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    dfffe8e View commit details
    Browse the repository at this point in the history

  5. gpu mode support oss spark

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    9a87909 View commit details
    Browse the repository at this point in the history

  6. update comment

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    2aeaee8 View commit details
    Browse the repository at this point in the history

  7. avoid call pd.Series.to_list

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    10bf6b2 View commit details
    Browse the repository at this point in the history

  8. avoid data concatenation in predict_udf

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    53b1a5b View commit details
    Browse the repository at this point in the history

  9. update comment

    @WeichenXu123
    WeichenXu123 committed Jul 3, 2022
    Copy the full SHA
    c907164 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2022

  1. fix

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    3a92fac View commit details
    Browse the repository at this point in the history

  2. fix tests

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    9e6ad55 View commit details
    Browse the repository at this point in the history

  3. forbid camel case param in setParams

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    d24512d View commit details
    Browse the repository at this point in the history

  4. rename 2 camel case params

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    b3fa185 View commit details
    Browse the repository at this point in the history

  5. address comments

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    8cee5cb View commit details
    Browse the repository at this point in the history

  6. update doc

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    18956d1 View commit details
    Browse the repository at this point in the history

  7. remove feature_types pyspark param

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    d4f048d View commit details
    Browse the repository at this point in the history

  8. fix-test

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    9e66d2f View commit details
    Browse the repository at this point in the history

  9. update-doc

    @WeichenXu123
    WeichenXu123 committed Jul 5, 2022
    Copy the full SHA
    c60ccad View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2022

  1. fix tests

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    4f247ab View commit details
    Browse the repository at this point in the history

  2. fix lint, refactor

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    40afa4f View commit details
    Browse the repository at this point in the history

  3. fix test

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    508a36b View commit details
    Browse the repository at this point in the history

  4. fix test

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    39e2b45 View commit details
    Browse the repository at this point in the history

  5. support feature weights

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    e657a21 View commit details
    Browse the repository at this point in the history

  6. fix-ci

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    60e2561 View commit details
    Browse the repository at this point in the history

  7. update-ci-conda-env

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    70b2da2 View commit details
    Browse the repository at this point in the history

  8. add gpu test

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    b40bc14 View commit details
    Browse the repository at this point in the history

  9. clean test

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    6274aa0 View commit details
    Browse the repository at this point in the history

  10. ignore mypy

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    3207256 View commit details
    Browse the repository at this point in the history

  11. ignore mypy

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    76c1f8d View commit details
    Browse the repository at this point in the history

  12. update doc

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    035ac68 View commit details
    Browse the repository at this point in the history

  13. handle missing param

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    a2ead7e View commit details
    Browse the repository at this point in the history

  14. update doc

    @WeichenXu123
    WeichenXu123 committed Jul 6, 2022
    Copy the full SHA
    1472432 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. [CI] Install PySpark in Python env

    @hcho3
    hcho3 committed Jul 7, 2022
    Copy the full SHA
    7b5afa1 View commit details
    Browse the repository at this point in the history

  2. fix spark-xgb-model params

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    f8aea44 View commit details
    Browse the repository at this point in the history

  3. add spark config for printing full error stack and avoid task retries

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    c838437 View commit details
    Browse the repository at this point in the history

  4. clean test temp dir

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    fac1c2a View commit details
    Browse the repository at this point in the history

  5. fix import

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    62bf8bb View commit details
    Browse the repository at this point in the history

  6. add pyspark env config in CI

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    1b77f23 View commit details
    Browse the repository at this point in the history

  7. update discoveryScript config path

    @WeichenXu123
    WeichenXu123 committed Jul 7, 2022
    Copy the full SHA
    0487223 View commit details
    Browse the repository at this point in the history

  8. Skip tests.

    @trivialfis
    trivialfis committed Jul 7, 2022
    Copy the full SHA
    37938d8 View commit details
    Browse the repository at this point in the history

  9. Missing dependency.

    @trivialfis
    trivialfis committed Jul 7, 2022
    Copy the full SHA
    024fc9e View commit details
    Browse the repository at this point in the history

  10. black.

    @trivialfis
    trivialfis committed Jul 7, 2022
    Copy the full SHA
    4bb7404 View commit details
    Browse the repository at this point in the history

  11. missing dependency.

    @trivialfis
    trivialfis committed Jul 7, 2022
    Copy the full SHA
    b92959e View commit details
    Browse the repository at this point in the history

  12. Rest of them.

    @trivialfis
    trivialfis committed Jul 7, 2022
    Copy the full SHA
    b7bb690 View commit details
    Browse the repository at this point in the history

  13. Merge remote-tracking branch 'upstream/master' into xgb-spark-py

    @hcho3
    hcho3 committed Jul 7, 2022
    Copy the full SHA
    b7dbc41 View commit details
    Browse the repository at this point in the history

  14. Fix message

    @hcho3
    hcho3 committed Jul 7, 2022
    Copy the full SHA
    8885247 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. fix CI config

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    22f2103 View commit details
    Browse the repository at this point in the history

  2. setup python-test action jdk

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    277984d View commit details
    Browse the repository at this point in the history

  3. update macos ci config

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    0f9ffa0 View commit details
    Browse the repository at this point in the history

  4. fix callback test

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    3534c75 View commit details
    Browse the repository at this point in the history

  5. disable spark test on macos

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    197c267 View commit details
    Browse the repository at this point in the history

  6. remove disable lint

    @WeichenXu123
    WeichenXu123 committed Jul 8, 2022
    Copy the full SHA
    ee78b56 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2022

  1. address pylint errors

    @WeichenXu123
    WeichenXu123 committed Jul 10, 2022
    Copy the full SHA
    42d0ce5 View commit details
    Browse the repository at this point in the history

  2. address pylint errors

    @WeichenXu123
    WeichenXu123 committed Jul 10, 2022
    Copy the full SHA
    dc02023 View commit details
    Browse the repository at this point in the history

  3. address pylint issues

    @WeichenXu123
    WeichenXu123 committed Jul 10, 2022
    Copy the full SHA
    520d754 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. fix lint ci action

    @WeichenXu123
    WeichenXu123 committed Jul 11, 2022
    Copy the full SHA
    5237d81 View commit details
    Browse the repository at this point in the history