CodeCov PR comments have stopped working

[TimvdLippe]

For some reason, we no longer receive CodeCov PR comments. The reports get uploaded just fine and CodeCov does have data, but it doesn't actually post the comment here on GitHub. It's unclear why that has changed. Maybe it has something to do with our release task being separate and we don't actually run it? We can manually set after_n_builds to mitigate the impact maybe.

[TimvdLippe]

Interestingly enough, we did get a coverage report on #2458 (comment)

[temp-droid]

Hi @TimvdLippe, it seems you got another one here: #2461 (comment)

Could it be linked to #2447? Which also got its code coverage.

If it started to work again, I wouldn't think of an authentication issue, even if the documentation points to that: https://docs.codecov.com/docs/error-reference#no-statuses-or-pull-requests-are-posting

[TimvdLippe]

Yeah it now publishes comments sporadically. We have seen it before, where we also didn't get any comments. This might actually be a CodeCov + GitHub issue, where they might get throttled. Based on my understanding, I don't think we have a configuration problem on our side, as it works fine at times.

[temp-droid]

@codecov-commenter profile recommends to use the Codecov Github App. It is also recommended on the Team Bot documentation page: https://docs.codecov.com/docs/team-bot

The best way to integrate with Codecov.io is to Install Codecov's GitHub App.
Once installed, you are done! You do not need to set a Team Bot because Codecov will use the integration to post statuses and comments.

I am not fully aware how your setup is working, but maybe we should switch to their Github App? (if the project isn't using it already).

[TimvdLippe]

I am not exactly sure how that GitHub App works tbh. I would have assumed that it would only take care of us uploading, which we already do. Since the PR comments have returned, let's close this for now but monitor the situation.