initial implementation of a github actions reporter

pkgs/test/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 1.21.0-dev
+
+* Add a `github` reporter option for use with GitHub Actions.
+* Make the `github` test reporter the default when we detect we're running on
+  GitHub Actions.
+
 ## 1.20.2
 
 * Drop `dart2js-path` command line argument.

pkgs/test/README.md

@@ -3,6 +3,8 @@
 * [Writing Tests](#writing-tests)
 * [Running Tests](#running-tests)
   * [Sharding Tests](#sharding-tests)
+  * [Shuffling Tests](#shuffling-tests)
+  * [Selecting a Test Reporter](#selecting-a-test-reporter)
   * [Collecting Code Coverage](#collecting-code-coverage)
   * [Restricting Tests to Certain Platforms](#restricting-tests-to-certain-platforms)
   * [Platform Selectors](#platform-selectors)
@@ -209,6 +211,7 @@ the invocation to the `test` function, and are considered a match if
   on platform implementations.
 
 ### Sharding Tests
+
 Tests can also be sharded with the `--total-shards` and `--shard-index` arguments,
 allowing you to split up your test suites and run them separately. For example,
 if you wanted to run 3 shards of your test suite, you could run them as follows:
@@ -233,6 +236,19 @@ dart test --test-randomize-ordering-seed=random
 Setting `--test-randomize-ordering-seed=0` will have the same effect as not
 specifying it at all, meaning the test order will remain as-is.
 
+### Selecting a Test Reporter
+
+You can adjust the output format of the test results using the `--reporter=<option>`
+command line flag. The default format is the `compact` output format - a single
+line, continuously updated as tests are run. When running on the GitHub Actions CI
+however, the default changes to the `github` output format - a reporter customized
+for that CI/CD system. The available options for the `--reporter` flag are:
+
+- `compact`: a single, continuously updated line
+- `expanded`: a separate line for each update
+- `github`: a custom reporter for GitHub Actions
+- `json`: a machine-readable format; see https://dart.dev/go/test-docs/json_reporter.md
+
 ### Collecting Code Coverage
 
 To collect code coverage, you can run tests with the `--coverage <directory>`

pkgs/test/pubspec.yaml

@@ -1,5 +1,5 @@
 name: test
-version: 1.20.2
+version: 1.21.0-dev
 description: >-
   A full featured library for writing and running Dart tests across platforms.
 repository: https://github.com/dart-lang/test/blob/master/pkgs/test

pkgs/test/test/runner/github_reporter_test.dart

@@ -185,52 +185,6 @@ void main() {
         ::endgroup::
         🎉 2 tests passed.''');
     });
-
-    // todo: fix this
-    //   test('interleaves prints and errors', () {
-    //     return _expectReport('''
-    //       // This completer ensures that the test isolate isn't killed until all
-    //       // prints have happened.
-    //       var completer = Completer();
-    //       test('test', () {
-    //         scheduleMicrotask(() {
-    //           print("three");
-    //           print("four");
-    //           throw "second error";
-    //         });
-
-    //         scheduleMicrotask(() {
-    //           print("five");
-    //           print("six");
-    //           completer.complete();
-    //         });
-
-    //         print("one");
-    //         print("two");
-    //         throw "first error";
-    //       });
-
-    //       test('wait', () => completer.future);''', '''
-    //       +0: test
-    //       one
-    //       two
-    //       +0 -1: test [E]
-    //         first error
-    //         test.dart 24:11  main.<fn>
-
-    //       three
-    //       four
-    //         second error
-    //         test.dart 13:13  main.<fn>.<fn>
-    //         ===== asynchronous gap ===========================
-    //         dart:async       scheduleMicrotask
-    //         test.dart 10:11  main.<fn>
-
-    //       five
-    //       six
-    //       +0 -1: wait
-    //       +1 -1: Some tests failed.''');
-    //   });
   });
 
   group('skip:', () {

pkgs/test/test/runner/runner_test.dart

@@ -103,11 +103,11 @@ Running Tests:
                                       If not passed, do not randomize test case execution order.
 
 Output:
--r, --reporter                        Set how to print test results.
+-r, --reporter=<option>               Set how to print test results.
 
           [compact]                   A single line, updated continuously.
           [expanded] (default)        A separate line for each update.
-          [github]                    A custom reporter for GitHub Actions output.
+          [github]                    A custom reporter for GitHub Actions (the default reporter when running on GitHub Actions).
           [json]                      A machine-readable format (see https://dart.dev/go/test-docs/json_reporter.md).
 
     --file-reporter                   Enable an additional reporter writing test results to a file.

pkgs/test_core/lib/src/runner/configuration/args.dart

@@ -135,7 +135,8 @@ final ArgParser _parser = (() {
       help: 'Set how to print test results.',
       defaultsTo: defaultReporter,
       allowed: reporterDescriptions.keys.toList(),
-      allowedHelp: reporterDescriptions);
+      allowedHelp: reporterDescriptions,
+      valueHelp: 'option');
   parser.addOption('file-reporter',
       help: 'Enable an additional reporter writing test results to a file.\n'
           'Should be in the form <reporter>:<filepath>, '

pkgs/test_core/lib/src/runner/configuration/reporters.dart

@@ -45,7 +45,7 @@ final _allReporters = <String, ReporterDetails>{
               Directory(config.paths.single.testPath).existsSync(),
           printPlatform: config.suiteDefaults.runtimes.length > 1)),
   'github': ReporterDetails(
-      'A custom reporter for GitHub Actions output.',
+      'A custom reporter for GitHub Actions (the default reporter when running on GitHub Actions).',
       (config, engine, sink) => GithubReporter.watch(engine, sink,
           printPath: config.paths.length > 1 ||
               Directory(config.paths.single.testPath).existsSync())),

pkgs/test_core/lib/src/runner/reporter/github.dart

@@ -145,8 +145,6 @@ class GithubReporter implements Reporter {
       }
     }
     _sink.writeln(_helper.startGroup('$prefix $name$statusSuffix'));
-    // TODO: Note that printing messages and errors in this manner could display
-    // them in a different order than they were generated.
     for (var message in messages) {
       _sink.writeln(message.text);
     }
@@ -182,11 +180,11 @@ class GithubReporter implements Reporter {
 
 class _GithubHelper {
   // Char sets avilable at https://www.compart.com/en/unicode/.
-  static const String passed = '✅';
-  static const String skipped = '❎';
-  static const String failed = '❌';
-  static const String synthetic = '⏺';
-  static const String success = '🎉';
+  static const String passed = '\u2705';
+  static const String skipped = '\u274E';
+  static const String failed = '\u274C';
+  static const String synthetic = '\u23FA';
+  static const String success = '\u1F389';
 
   _GithubHelper();