[PySpark Training] Eval results are not saved during training

[JoHacksCode]

Hello,

I am using the SparkXGBRegressor to train my models, v2.0.0. I've realized that even though they are logged during the training, the evals results are not saved, and cannot be accessed anywhere but in the logs.
This has a big impact, because I wish to use eval_metrics to compare models but cannot do so without the results.

Looking at the code, we can see that the evals_results are saved into a variable that is not used nor saved in the model.

xgboost/python-package/xgboost/spark/core.py

Line 964 in 4d7a187

evals_result: Dict[str, Any] = {}

Would it be possible to have a quick fix for this please

[hcho3]

According to #8352, evals_result was not part of the initial PySpark implementation. I am marking this as a feature request.

@wbo4958 @WeichenXu123 How difficult do you think adding evals_result would be for the PySpark interface?

[wbo4958]

Yeah, We can support that.