D2_pinball_score

[Manjubn777]

Hello team, I’m currently utilizing RandomizedSearchCV . Specifically, I’m working on probabilistic forecasting. However, when I use D2_pinball_score as the scoring metric, I encounter an error despite using the supported version. Any insights or guidance would be greatly appreciated.

[ogrisel]

"d2_pinball_score" is not a valid scorer name. You should instead pass a custom scorer object that wraps the d2_pinball_score callable with the value of alpha you are interested in:

from sklearn.metrics import make_scorer
from sklearn.metrics import d2_pinball_score

pinball_95_scorer = make_scorer(d2_pinball_score, alpha=0.95) 

Then you should be able to pass scoring=pinball_95_scorer to the RandomizedSearchCV constructor.

More details:

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html

I agree that both the documentation for the scoring parameter, the make_scorer utility, the d2_pinball_score metric and the error message could be improved because this is not easy to discover the correct usage by yourself.

[siddu1324]

I'm working on scikit-learn/sklearn/model_selection/_search.py