Android tests failed when I using native commands in Patrol 2.0+

[hrubalski]

Cześć,

First of all, thanks for the Partol, I love it, you guys are the best!

About my issues: I didn't have any issues before Patrol 2.0. What I have now: The test works correctly, but if I use some native commands, I get errors in test results (but the whole flow of the test was correct). For me, it looks like Patrol throws some errors when trying to finish the test, but only when the test has some native commands. I do not have this issue in my test projects. Any ideas why?

void main() {
  patrolTest(
    'Authorization Test',
    nativeAutomation: true,
    ($) async {
      app.main();

      await $.pumpAndSettle();

      // This tap needed to activate app screen
      await $(#loginImageKey).tap();

      // find login title widget
      expect($(#loginTitleKey), findsOneWidget);
      // find login description widget
      expect($(#loginDescriptionKey), findsOneWidget);
      // find login image widget
      expect($(#loginImageKey), findsOneWidget);
      // find google sign in button
      expect($(#googleSignInButton), findsOneWidget);

      await $(#loginWithCredentialsButton).tap();

      await $(#emailInput).enterText('email@email.com');
      await $(#passwordInput).enterText('123456789');
      await $(#loginButton).tap();
      
      /// Without this line Android test will pass
      await $.native.denyPermission();

      expect($(#navigationBar), findsOneWidget);
    },
  );
}

Logs:

        : > Task :app:connectedDevDebugAndroidTest
        : Starting 1 tests on Pixel 7 - 13
        : 
        : replaced.fake.bundle.MainActivityTest > runDartTest[app_test][Pixel 7 - 13] FAILED 
        :       java.lang.AssertionError: Dart test failed: app_test
        : 
        : Tests on Pixel 7 - 13 failed: There was 1 failure(s).
        : 
        : > Task :app:connectedDevDebugAndroidTest FAILED

        FAILURE: Build failed with an exception.

        * What went wrong:
        Execution failed for task ':app:connectedDevDebugAndroidTest'.
        See the report at: index.html

        * Try:
        > Run with --stacktrace option to get the stack trace.
        > Run with --info or --debug option to get more log output.
        > Run with --scan to get full insights.

        * Get more help at https://help.gradle.org

        BUILD FAILED in 51s
        : 
        : Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
        : 
        : You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
        : 
        : See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings
        : 508 actionable tasks: 6 executed, 502 up-to-date
✗ Failed to execute tests of apk with entrypoint test_bundle.dart on device 192.168.0.107:43635 (Gradle test execution failed with code 1) (52.0s)
Exception: Gradle test execution failed with code 1
#0      AndroidTestBackend.execute.<anonymous closure> (package:patrol_cli/src/android/android_test_backend.dart:140:9)
<asynchronous suspension>
#1      DisposeScope.run (package:dispose_scope/src/dispose_scope.dart:46:7)
<asynchronous suspension>
#2      AndroidTestBackend.execute (package:patrol_cli/src/android/android_test_backend.dart:106:5)
<asynchronous suspension>
#3      TestCommand._execute (package:patrol_cli/src/commands/test.dart:232:7)
<asynchronous suspension>
#4      TestCommand.run (package:patrol_cli/src/commands/test.dart:168:23)
<asynchronous suspension>
#5      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#6      PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:310:18)
<asynchronous suspension>
#7      PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:254:18)
<asynchronous suspension>
#8      patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:72:20)
<asynchronous suspension>
#9      main (file:///Users/sonyboy/.pub-cache/hosted/pub.dev/patrol_cli-2.0.1/bin/main.dart:6:20)
<asynchronous suspension>

I'll be thankful for any help and ideas you can give to me!

[bartekpacia]

Siema :)

Sorry for the problems. It'd be helpful if you shared the screenshot of the index.html that's generated upon test failure. For some reason, the full path to that file not printed in your case:

* What went wrong:
Execution failed for task ':app:connectedDevDebugAndroidTest'.
See the report at: index.html

It should be in (assuming you're running the tests with the dev flavor):

<project_root>/build/app/reports/androidTests/connected/flavors/dev/index.html

Also, in case you missed it, we've got a migration guide 1.0 -> 2.0.

[hrubalski]

I know about this migration guide, I did everything step by step and verified it at least 10 times.

Here is the screenshot of the index.html

[bartekpacia]

Oof, that's not the logs I expected. It's time for adb logcat and grep for something like "EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK" (and paste here, ofc).

I'm curious what's the cause of this.

BTW, we do know error reporting is not great sometimes and we aim to fix that. See #951 and #1044 for more details.

[hrubalski]

@bartekpacia Thanks, solved!

Yes, we had EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK with PlatformException inside. Why? Because by some accident we were asking for notification permission twice.

Output:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════
The following PlatformException was thrown running a test:
2023-07-04 15:01:32.526 27857-27908 flutter  replaced.fake.bundle.dev  I  PlatformException(permissionRequestInProgress, Another permission request is already in progress,
2023-07-04 15:01:32.526 27857-27908 flutter  replaced.fake.bundle.dev  I  null, null)

Thank you!

[bartekpacia]

Glad to hear that :)

[todescli01]

I am having the same issue and can not see what the resolution was for this, can someone please share it.