Unpin XGBoost Version upon XGBoost package update

[bchen1116]

Now that the XGBoost versioning issue has been merged, they will be updating the pip package to version 1.3.0.post0. This issue tracks removing our XGBoost version limitations once the new package is released.

[hcho3]

The 1.3.0.post0 version of XGBoost is now available on PyPI: https://pypi.org/project/xgboost/1.3.0.post0/#files. XGBoost should no longer crash upon import.

[bchen1116]

@hcho3 thanks for the update!

[bchen1116]

Observations:

• Docs build with XGBoost=1.3.0.post0 on CircleCI is failing due to it timing out (> 10.0 mins with no output) during the build

  • Timing out on Cost_benefit_matrix.ipynb, which succeeds locally for this version

• Running the Cost_benefit_matrix.ipynb file in CircleCI (ssh) also fails (times out)

• Changing OMP_THREADS in circleci to 16 and 32 workers resulted in the same timeout

• Running a simple XGBoost pipeline (Imputer + XGB Classifier) on CircleCI raises the following warning messages:
  

  • Changing loss from 'logloss' to 'error' results in runtime of 161.34s versus 244.852s, which is still much longer than with XGBoost=1.2.0
  • Changing this in our XGBoost implementation COULD solve the timeout issue for now, but I think the root causes is different since this runtime is still much larger than expected.

• Docs build with XGBoost=1.2.0 works as expected on CircleCI

• Docs build with both XGBoost=1.3.0.post0 and XGBoost=1.2.0 succeed on my local, using the same pip dependencies for the other packages.

  • 1.3.0.post0 version takes marginally longer (14m51s vs 14m40s) to build docs locally compared to 1.2.0

• Both versions of XGBoost fit to data in about the same amount of time locally (.1029s vs 0.0722s for 1.3.0.post0 and 1.2.0, respectively)

  • Tested using load_breast_cancer data, pipeline=['Imputer', 'Xgboost Classifier']

Next steps: Figure out why XGBoost=1.3.0.post0 is taking much longer to run on CircleCI. Fix dependency as needed. Closing PR 1538 for now.

[chukarsten]

@bchen1116 1.3.1 is released, should we uncap for this?

[bchen1116]

Writeup here, but this issue is still open

[angela97lin]

@chukarsten @dsherry

I found this: https://discuss.xgboost.ai/t/n-jobs-1-no-longer-uses-all-cores/1955/2 while looking into XGBoost that indicates it might be a good idea to try to unpin XGBoost (since the version we have, 1.2.1, falls within the package versions that seem to be suffering from this) if we can.