[CRAN] Major issues from CRAN check for most platforms.

[hetong007]

Please refer to the result page for more details. We have time before Jan 12th to patch in order to retain on CRAN.

Take the result from debian-gcc environment as an example, I see one error and one warning on the compatibility of models from earlier versions.

The error:

  Error (test_model_compatibility.R:69:3): Models from previous versions of XGBoost can be loaded
  Error: [02:01:34] amalgamation/../src/objective/./regression_loss.h:89: Check failed: base_score > 0.0f && base_score < 1.0f: base_score must be in (0,1) for logistic loss, got: -0
  Stack trace:
    [bt] (0) /home/hornik/tmp/R.check/r-devel-gcc/Work/build/Packages/xgboost/libs/xgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x5f) [0x7f0487cf7a9f]
    [bt] (1) /home/hornik/tmp/R.check/r-devel-gcc/Work/build/Packages/xgboost/libs/xgboost.so(xgboost::obj::LogisticRegression::ProbToMargin(float)+0xbf) [0x7f0487d83acf]
    [bt] (2) /home/hornik/tmp/R.check/r-devel-gcc/Work/build/Packages/xgboost/libs/xgboost.so(xgboost::LearnerIO::LoadModel(dmlc::Stream*)+0x9c0) [0x7f0487e10100]
    [bt] (3) /home/hornik/tmp/R.check/r-devel-gcc/Work/build/Packages/xgboost/libs/xgboost.so(XGBoosterLoadModel+0x233) [0x7f0487d141c3]
    [bt] (4) /home/hornik/tmp/R.check/r-devel-gcc/Work/build/Packages/xgboost/libs/xgboost.so(XGBoosterLoadModel_R+0x38) [0x7f0487cf3e88]
    [bt] (5) /home/hornik/tmp/R.check/r-devel-gcc/Work/b
  Backtrace:
   1. base::lapply(...) test_model_compatibility.R:69:2
   2. xgboost:::FUN(X[[i]], ...)
   3. xgboost::xgb.load(model_file) test_model_compatibility.R:79:6
   4. xgboost:::xgb.Booster.handle(modelfile = modelfile)

The warning:

  Warning (test_model_compatibility.R:69:3): Models from previous versions of XGBoost can be loaded
  The model had been generated by XGBoost version 1.0.0 or earlier and was loaded from a RDS file. We strongly ADVISE AGAINST using saveRDS() function, to ensure that your model can be read in current and upcoming XGBoost releases. Please use xgb.save() instead to preserve models for the long term. For more details and explanation, see https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html
  Backtrace:
    1. base::lapply(...) test_model_compatibility.R:69:2
    2. xgboost:::FUN(X[[i]], ...)
    4. xgboost:::predict.xgb.Booster(booster, newdata = pred_data)
    5. xgboost::xgb.Booster.complete(object, saveraw = FALSE)
    6. xgboost:::xgb.Booster.handle(modelfile = object$raw)
    7. xgboost::xgb.unserialize(modelfile)
    8. base::tryCatch(...)
    9. base:::tryCatchList(expr, classes, parentenv, handlers)
   10. base:::tryCatchOne(expr, names, parentenv, handlers[[1L]])
   11. value[[3L]](cond)

[trivialfis]

We recently added a test for loading old logitraw model, which may break tests on older version. This will be fixed in #6518 , which is about to be released at 21.

[hetong007]

Thanks for the message, but I'm still curious about how does this recent change affect the CRAN check status? The CRAN version was uploaded on September.

[trivialfis]

Those old models used for tests are hosted on our CI, so when we upload a new file there, anyone running the tests will receive it. You can check out the tests in backtrace you have provided.

[hcho3]

@hetong007 The 1.3.1 patch release should fix the issue. (Use the current release_1.3.0 branch.) If the new patch release passes the checks, can we keep XGBoost on CRAN? Or do we have to patch older versions too?

[hetong007]

@hcho3 As long as we have the patch submitted to CRAN before Jan 12th, I think we'll be fine.

[hcho3]

@hetong007 Can we submit the 1.3.1 version to CRAN some time this week or next?

Ps. Happy new year.

[hetong007]

@hcho3 Happy new year!

Just submitted. Will update the status in this thread.

[hetong007]

1.3.1.1 is submitted to CRAN. So far I observe an error on solaris: https://www.r-project.org/nosvn/R.check/r-patched-solaris-x86/xgboost-00install.html

[hcho3]

@trivialfis Can we bring back the rabit_empty stub? Otherwise, it would be quite difficult to get XGBoost working on Solaris.

[hcho3]

Fortunately for us, there is now a free service to try building package on the Solaris platform. https://builder.r-hub.io/.

@hetong007 Do we not get to keep XGBoost on CRAN if the error on Solaris isn't fixed by Jan 12? How soon should it be fixed?

[hetong007]

That service is a good news! I'll mark that for future submissions.

I think that Jan 12 is set for the previous large-scale error. As long as solaris is the only platform having trouble, I guess it is more likely to keep it on CRAN for a longer time.

[hetong007]

OK I've just heard from Prof. Ripley:

Please correct before 2021-01-20 to safely retain your package on CRAN.

Does it sound reasonable to fix Solaris before Jan 20th?

[mayer79]

Hello. Just to confirm that this might be a problem also for R packages on CRAN depending on XGBoost.

This is for one of my packages ("flashlight"). It uses XGBoost in "suggest"ed packages in its description vignettes.

Result: NOTE
Package suggested but not available for checking: ‘xgboost’
Flavor: r-patched-solaris-x86

Mail from Cran: "Please correct before 2021-01-20 to safely retain your package on CRAN."

[hcho3]

@hetong007 @mayer79 Got it. I'll try to fix it before Jan 20.

[mayer79]

@hcho3: CRAN issues are the best way to start a day ;). Thanks a lot for looking into this.

[jaredlander]

Hello. Just to confirm that this might be a problem also for R packages on CRAN depending on XGBoost.

This is for one of my packages ("flashlight"). It uses XGBoost in "suggest"ed packages in its description vignettes.

Result: NOTE
Package suggested but not available for checking: ‘xgboost’
Flavor: r-patched-solaris-x86
Mail from Cran: "Please correct before 2021-01-20 to safely retain your package on CRAN."

Yes, {coefplot} got dinged with that.

[trivialfis]

Sorry for being late to the discussion. I will also look into the issue in parallel with @hcho3

[trivialfis]

Hi, I created a PR on #6578 for fixing the system command symbol issue. But there seems to be some toolchain issues on the test site mentioned by @hcho3. Could anyone here help taking a look and provide suggestions?

[FranciscoPalomares]

Hi, I have a package that depends on xgboost, when do you think it will be fixed? Thank you.

[trivialfis]

We plan to make a patch release on 11 this month. I expect the CRAN package to be released soon.

[hcho3]

1.3.2.1 is now on CRAN. Early results suggest that XGBoost builds and runs successfully on Solaris now:

[hcho3]

Will close this issue once tests pass on all platforms.

[hetong007]

Excellent! Thanks a lot for @trivialfis and @hcho3's quick respond and kind help!

[hcho3]

Closing now, since tests are now passing for all platforms: https://cran.r-project.org/web/checks/check_results_xgboost.html