Allow to only comment on changes, failures, ... (#248)

Adds new `comment_mode`s and deprecates `create new` and `update last`.

README.md

@@ -174,7 +174,9 @@ The list of most notable options:
 |:-----|:-----:|:----------|
 |`files`|`*.xml`|File patterns to select the test result XML files, e.g. `"test-results/**/*.xml"`. Use multiline string for multiple patterns. Supports `*`, `**`, `?`, `[]`. Excludes files when starting with `!`. |
 |`check_name`|`"Unit Test Results"`|An alternative name for the check result.|
-|`comment_title`|same as `check_name`|An alternative name for the pull request comment.|
+|`comment_title`|same as `check_name`|An alternative title for the pull request comment.|
+|`comment_mode`|`always`|The action posts comments to pull requests that are associated with the commit. Set to:<br/>`always` - always comment<br/>`changes` - comment only when changes to the target branch exist<br/>`failures` - when failures or errors exist<br/>`errors` - when (only) errors exist<br/>`off` - to not create pull request comments<br/>With `changes in failures` and `changes in errors`, it only comments when changes in the number of failures and errors exist, respectively.|
+|`ignore_runs`|`false`|Does not process test run information by ignoring `<testcase>` elements in the XML files, which is useful for very large XML files. This disables any check run annotations.|
 
 <details>
 <summary>Options related to Git and GitHub</summary>
@@ -198,15 +200,13 @@ The list of most notable options:
 |:-----|:-----:|:----------|
 |`time_unit`|`seconds`|Time values in the XML files have this unit. Supports `seconds` and `milliseconds`.|
 |`job_summary`|`true`| Set to `true`, the results are published as part of the [job summary page](https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/) of the workflow run.|
-|`comment_mode`|`update last`|The action posts comments to a pull request that is associated with the commit. Set to `create new` to create a new comment on each commit, `update last` to create only one comment and update later on, `off` to not create pull request comments.|
 |`hide_comments`|`"all but latest"`|Configures which earlier comments in a pull request are hidden by the action:<br/>`"orphaned commits"` - comments for removed commits<br/>`"all but latest"` - all comments but the latest<br/>`"off"` - no hiding|
 |`compare_to_earlier_commit`|`true`|Test results are compared to results of earlier commits to show changes:<br/>`false` - disable comparison, `true` - compare across commits.'|
-|`test_changes_limit`|`10`|Limits the number of removed or skipped tests reported on PR comments. This report can be disabled with a value of `0`.|
+|`test_changes_limit`|`10`|Limits the number of removed or skipped tests reported on pull request comments. This report can be disabled with a value of `0`.|
 |`report_individual_runs`|`false`|Individual runs of the same test may see different failures. Reports all individual failures when set `true`, and the first failure only otherwise.|
 |`deduplicate_classes_by_file_name`|`false`|De-duplicates classes with same name by their file name when set `true`, combines test results for those classes otherwise.|
 |`check_run_annotations`|`all tests, skipped tests`|Adds additional information to the check run. This is a comma-separated list of any of the following values:<br>`all tests` - list all found tests,<br>`skipped tests` - list all skipped tests<br> Set to `none` to add no extra annotations at all.|
-|`check_run_annotations_branch`|`event.repository.default_branch` or `"main, master"`|Adds check run annotations only on given branches. If not given, this defaults to the default branch of your repository, e.g. `main` or `master`. Comma separated list of branch names allowed, asterisk `"*"` matches all branches. Example: `main, master, branch_one`|
-|`ignore_runs`|`false`|Does not process test run information by ignoring `<testcase>` elements in the XML files, which is useful for very large XML files. This disables any check run annotations.|
+|`check_run_annotations_branch`|`event.repository.default_branch` or `"main, master"`|Adds check run annotations only on given branches. If not given, this defaults to the default branch of your repository, e.g. `main` or `master`. Comma separated list of branch names allowed, asterisk `"*"` matches all branches. Example: `main, master, branch_one`.|
 |`json_file`|no file|Results are written to this JSON file.|
 |`json_thousands_separator`|`" "`|Formatted numbers in JSON use this character to separate groups of thousands. Common values are "," or ".". Defaults to punctuation space (\u2008).|
 |`fail_on`|`"test failures"`|Configures the state of the created test result check run. With `"test failures"` it fails if any test fails or test errors occur. It never fails when set to `"nothing"`, and fails only on errors when set to `"errors"`.|

action.yml

@@ -19,7 +19,11 @@ inputs:
     default: 'Unit Test Results'
     required: false
   comment_title:
-    description: 'Title of PR comments, defaults to value of check_name input'
+    description: 'An alternative title for the pull request comment. Defaults to value of check_name input'
+    required: false
+  comment_mode:
+    description: 'The action posts comments to pull requests that are associated with the commit. Set to: "always" - always comment, "changes" - comment only when changes to the target branch exist, "failures" - when failures or errors exist, "errors" - when (only) errors exist, "off" - to not create pull request comments. With "changes in failures" and "changes in errors", it only comments when changes in the number of failures and errors exist, respectively'
+    default: 'always'
     required: false
   fail_on:
     description: 'The created test result check run has failure state if any test fails or test errors occur. Never fails when set to "nothing", fails only on errors when set to "errors". Default is "test failures".'
@@ -49,10 +53,6 @@ inputs:
   comment_on_pr:
     description: 'Deprecated, please use comment_mode instead!'
     required: false
-  comment_mode:
-    description: 'Control PR comments: off - disable PR comments, create new - create comments on PRs, each time a new one, update last - create comment on PR, reuse an existing one'
-    default: 'update last'
-    required: false
   job_summary:
     description: 'Set to `true`, the results are published as part of the job summary page of the workflow run'
     default: 'true'
@@ -72,14 +72,14 @@ inputs:
     description: 'An alternative event name to use. Useful to replace a "workflow_run" event name with the actual source event name: github.event.workflow_run.event.'
     required: false
   test_changes_limit:
-    description: 'Limits the number of removed or skipped tests reported on PR comments. This report can be disabled with a value of 0. The default is 10.'
+    description: 'Limits the number of removed or skipped tests reported on pull request comments. This report can be disabled with a value of 0. The default is 10.'
     required: false
   check_run_annotations:
     description: 'Adds additional information to the check run. This is a comma-separated list of any of the following values: "all tests" - list all found tests, "skipped tests" - list all skipped tests. Set to "none" to add no extra annotations at all'
     default: 'all tests, skipped tests'
     required: false
   check_run_annotations_branch:
-    description: 'Adds check run annotations only on given branches. Comma separated list of branch names allowed, asterisk "*" matches all branches. Defaults to event.repository.default_branch or "main, master".'
+    description: 'Adds check run annotations only on given branches. Comma-separated list of branch names allowed, asterisk "*" matches all branches. Defaults to event.repository.default_branch or "main, master".'
     required: false
   seconds_between_github_reads:
     description: 'Sets the number of seconds the action waits between concurrent read requests to the GitHub API. This throttles the API usage to avoid abuse rate limits: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#abuse-rate-limits'

composite/action.yml

@@ -19,7 +19,11 @@ inputs:
     default: 'Unit Test Results'
     required: false
   comment_title:
-    description: 'Title of PR comments, defaults to value of check_name input'
+    description: 'An alternative title for the pull request comment. Defaults to value of check_name input'
+    required: false
+  comment_mode:
+    description: 'The action posts comments to pull requests that are associated with the commit. Set to: "always" - always comment, "changes" - comment only when changes to the target branch exist, "failures" - when failures or errors exist, "errors" - when (only) errors exist, "off" - to not create pull request comments. With "changes in failures" and "changes in errors", it only comments when changes in the number of failures and errors exist, respectively'
+    default: 'always'
     required: false
   fail_on:
     description: 'The created test result check run has failure state if any test fails or test errors occur. Never fails when set to "nothing", fails only on errors when set to "errors". Default is "test failures".'
@@ -49,10 +53,6 @@ inputs:
   comment_on_pr:
     description: 'Deprecated, please use comment_mode instead!'
     required: false
-  comment_mode:
-    description: 'Control PR comments: off - disable PR comments, create new - create comments on PRs, each time a new one, update last - create comment on PR, reuse an existing one'
-    default: 'update last'
-    required: false
   job_summary:
     description: 'Set to `true`, the results are published as part of the job summary page of the workflow run'
     default: 'true'
@@ -72,14 +72,14 @@ inputs:
     description: 'An alternative event name to use. Useful to replace a "workflow_run" event name with the actual source event name: github.event.workflow_run.event.'
     required: false
   test_changes_limit:
-    description: 'Limits the number of removed or skipped tests reported on PR comments. This report can be disabled with a value of 0. The default is 10.'
+    description: 'Limits the number of removed or skipped tests reported on pull request comments. This report can be disabled with a value of 0. The default is 10.'
     required: false
   check_run_annotations:
     description: 'Adds additional information to the check run. This is a comma-separated list of any of the following values: "all tests" - list all found tests, "skipped tests" - list all skipped tests. Set to "none" to add no extra annotations at all'
     default: 'all tests, skipped tests'
     required: false
   check_run_annotations_branch:
-    description: 'Adds check run annotations only on given branches. Comma separated list of branch names allowed, asterisk "*" matches all branches. Defaults to event.repository.default_branch or "main, master".'
+    description: 'Adds check run annotations only on given branches. Comma-separated list of branch names allowed, asterisk "*" matches all branches. Defaults to event.repository.default_branch or "main, master".'
     required: false
   seconds_between_github_reads:
     description: 'Sets the number of seconds the action waits between concurrent read requests to the GitHub API. This throttles the API usage to avoid abuse rate limits: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#abuse-rate-limits'
@@ -149,6 +149,7 @@ runs:
         COMMIT: ${{ inputs.commit }}
         CHECK_NAME: ${{ inputs.check_name }}
         COMMENT_TITLE: ${{ inputs.comment_title }}
+        COMMENT_MODE: ${{ inputs.comment_mode }}
         FAIL_ON: ${{ inputs.fail_on }}
         FILES: ${{ inputs.files }}
         TIME_UNIT: ${{ inputs.time_unit }}
@@ -157,7 +158,6 @@ runs:
         IGNORE_RUNS: ${{ inputs.ignore_runs }}
         HIDE_COMMENTS: ${{ inputs.hide_comments }}
         COMMENT_ON_PR: ${{ inputs.comment_on_pr }}
-        COMMENT_MODE: ${{ inputs.comment_mode }}
         COMPARE_TO_EARLIER_COMMIT: ${{ inputs.compare_to_earlier_commit }}
         PULL_REQUEST_BUILD: ${{ inputs.pull_request_build }}
         EVENT_FILE: ${{ inputs.event_file }}

python/publish/__init__.py

@@ -19,13 +19,27 @@
 punctuation_space = ' '
 
 comment_mode_off = 'off'
-comment_mode_create = 'create new'
-comment_mode_update = 'update last'
+comment_mode_create = 'create new'   # deprecated
+comment_mode_update = 'update last'  # deprecated
+comment_mode_always = 'always'
+comment_mode_changes = 'changes'
+comment_mode_changes_failures = 'changes in failures'  # includes comment_mode_changes_errors
+comment_mode_changes_errors = 'changes in errors'
+comment_mode_failures = 'failures'  # includes comment_mode_errors
+comment_mode_errors = 'errors'
 comment_modes = [
     comment_mode_off,
-    comment_mode_create,
-    comment_mode_update
+    comment_mode_always,
+    comment_mode_changes,
+    comment_mode_changes_failures,
+    comment_mode_changes_errors,
+    comment_mode_failures,
+    comment_mode_errors
 ]
+comment_modes_deprecated = {
+    comment_mode_create: comment_mode_always,
+    comment_mode_update: comment_mode_always
+}
 
 fail_on_mode_nothing = 'nothing'
 fail_on_mode_errors = 'errors'
@@ -80,6 +94,11 @@ def __init__(self,
         self._skipped_tests_before = set(skipped_tests_before) if skipped_tests_before is not None else None
         self._skipped_tests_current = set(skipped_tests_current) if skipped_tests_current is not None else None
 
+    @property
+    def has_changes(self) -> bool:
+        return (self.adds() is not None and self.removes() is not None and len(self.adds().union(self.removes())) > 0 or
+                self.skips() is not None and self.un_skips() is not None and len(self.skips().union(self.un_skips())) > 0)
+
     def adds(self) -> Optional[Set[str]]:
         if self._all_tests_before is None or self._all_tests_current is None:
             return None
@@ -701,7 +720,7 @@ def get_long_summary_with_digest_md(stats: UnitTestRunResultsOrDeltaResults,
         raise ValueError('stats must be UnitTestRunResults when no digest_stats is given')
     summary = get_long_summary_md(stats, details_url, test_changes, test_list_changes_limit)
     digest = get_digest_from_stats(stats if digest_stats is None else digest_stats)
-    return f'{summary}\n{digest_header}{digest}'
+    return f'{summary}\n{digest_header}{digest}\n'
 
 
 def get_case_messages(case_results: UnitTestCaseResults) -> CaseMessages: