Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add statistic calculation for Provider's testing. #21564

Merged
merged 1 commit into from Feb 14, 2022

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Feb 14, 2022

This script allows to calculate some basic stats of testing for
provider releases.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

This script allows to calculate some basic stats of testing for
provider releases.
@potiuk
Copy link
Member Author

potiuk commented Feb 14, 2022

I calculated some stats based on the community testing for Providers:

Issue Num Providers Num Issues Tested Issues Tested (%) Involved Commenting Involved who commented Extra people User response (%)
16456 67 72 28 38 48 21 19 2 39
17037 14 28 4 14 21 9 6 3 28
17268 31 65 29 44 36 13 12 1 33
17922 67 44 26 59 30 17 16 1 53
18028 3 5 2 40 3 2 1 1 33
18638 17 55 26 47 35 15 14 1 40
19328 56 53 27 50 48 18 16 2 33
19414 2 10 6 60 8 3 1 2 12
19895 14 43 20 46 31 12 11 1 35
20097 1 8 3 37 4 4 1 3 25
20220 1 10 9 90 5 2 1 1 20
20615 23 90 36 40 48 23 22 1 45
20766 2 3 0 0 3 1 1 0 33
21348 25 53 5 9 24 9 6 3 25
21443 27 73 43 58 33 17 14 3 42
Total 350 612 264 43 196 92 88 4 44

Some summary.

The stats cover period June 2021 - Feb 2022.

In total we had 15 releases, of 350 provider packages. All those provider packages solved 612 issues. Out of those 612 issues total 264 have been tested (which means our community tested 43% of all issues - features, bugfixes). The ratio of "per-provider" tests was much higher (hard to say without some more "complex" logic).

There were 196 people involved, out of which 88 actively took part and commented on their issues (either with "yes works" or "no it does not work"). Which is 44% "response rate".

There were 4 people who commented who were not involved (this basically means that in this 15 releases 4 people commented on the issue when they were not directly mentioned in the issue).

@github-actions github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Feb 14, 2022
@github-actions
Copy link

The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

Copy link
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested Issues is in fact tested PRs right? As we are really testing the PRs and PRs may not have issue related.

@potiuk potiuk merged commit f08c2d5 into apache:main Feb 14, 2022
@potiuk potiuk deleted the add-stats-for-providers-testing branch February 14, 2022 21:04
@potiuk potiuk restored the add-stats-for-providers-testing branch April 26, 2022 20:46
@potiuk potiuk deleted the add-stats-for-providers-testing branch July 29, 2022 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools okay to merge It's ok to merge this PR as it does not require more tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants