Accuracy metric number of classes warning question

[Vichoko]

For context, I'm running a heavy multi-label classification model with small batches (~10 elements) and a large number of classes (~50 classes). While training I'm getting this warning on accuracy metric calculation:

You have set 32 number of classes which is different from predicted (6) and target (28) number of classes

Should I be worried? I'm not sure why this should be a warning.
I keep wondering why is it bad having fewer classes in predicted output compared with the expected number of classes.

[SkafteNicki]

Which version of lightning are you using?
We did a revamp of metrics lately and the class based Accuracy metric does not throw this warning anymore (functional still does).
It was originally put in place to warn users that we estimated a different number of labels (from pred and target) than what the user provided, and it could mean that your output was biased and not estimating values within the hole label space. That said, it does not necessarily means that there is anything wrong with your model.

[Vichoko]

Thank you for the clarification!

I'm using version 0.9.0. Actually the warning message is:

UserWarning: You have set 32 number of classes if different from predicted (6) and target (28) number of classes warnings.warn(*args, **kwargs)

I updated the message to match the current version.