How will UI Tests operate on Maui?

[Codelisk]

Can we reuse the Xamarin Forms Ui Tests for maui or will it be something different?

[de-preut]

@davidortinau could you please clarify this, so that we know if our "investment" in UI Tests running on AppCenter is still worth the effort?

[RwSgxx]

As with all other uncomfortable questions the Xamarin Team and the Visual Studio (the most unstable IDE) Team just wont respond, or even better, they will close the issue. Because Microsoft 💯

[listepo]

Any news? @jamesmontemagno maybe you have information?

[Xyncgas]

a lot people has said similar things @RwSgxx , but it's baseless accusation because of one bug or two, the bugs get fixed every day there's nothing wrong you can say about the fundamental decision these products are going which is what's important. The important thing for MAUI now is to produce stand alone executable, so Windows people can double click a MAUI.Blazor program and have it running instead of it not working after windows store no longer support it or censors it, in president xi's words : "先有诚意再说吧"

[keozx]

Automation is such an essential part of any development lifecycle, hope there is a priority to develop tooling for it, open sourcing UI Test would be a good start

[Xyncgas]

I would argue that UI testing is just a 'current thing', we are having other technology on the horizon, such as the concept of proofing the UI works during compiling through the type system (F* / Haskell / etc are developing techniques to verify more and more parts of your program during compilation)

[brunck]

Right now there are apps that run tests on devices (the "Device Tests") which you can find in the .NET MAUI solution. There are also thoughts about using Appium (XAppium?) as a potential replacement for Xamarin UI Test.

[lvendramini]

What about for testing hybrid Windows apps?

[cbaer-extron]

What "Device Test" app is there for Maui? I haven't found one. Use to use NUnit Xamarin Runners, but that is not supported in .NET Maui

[vitorelli]

What "Device Test" app is there for Maui? I haven't found one. Use to use NUnit Xamarin Runners, but that is not supported in .NET Maui

Please take a look: https://github.com/mattleibow/DeviceRunners

It works pretty well to run your unit test on devices.

[JohnHDev]

Up to and including preview 11, Xamarin UITest worked with .NET Maui on iOS. Preview 12 and above the TestCloud nuget can no longer be installed.
Xamarin UITest probably still works with Android though, I haven't tried.

[JohnHDev]

@johnathan-s yes, the test cloud nuget is not .net 6 compatible and needs to be recompiled. I have raised a ticket with the app center team, no ETA though.

[johnathan-s]

@JohnHDev thanks John. Any word on this as it appears RTM is coming soon?

[JohnHDev]

I haven’t heard anything more

[johnathan-s]

@JohnHDev any word? MAUI is GA and I still do not see anything from MS or otherwise.

[JohnHDev]

@johnathan-s I created an issue on the AppCenter repo back in April for this:
microsoft/appcenter#2416

It might be worth everyone that sees this, to ping on there as well. Had initial replies but nothing else for months.

[jeremy-bridges]

We've been using Appium to test our Xamarin.Forms app. We'd love to know if we can continue to trust in that stack as we move into the MAUI world. Will the control structure stay relatively unchanged? If not, what percentage of the Xam.Forms controls do we expect to break?

[Wynadorn]

Visual Studio App Center lists a number of supported test Frameworks
I'm also curious how this information applies to .NET MAUI.
https://docs.microsoft.com/en-us/appcenter/test-cloud/frameworks/

There's some discussion in the appcenter documentation repository (MicrosoftDocs/appcenter-docs#1259) about MAUI Support.

Personally I think having propper testing support for MAUI is essential to any serious development project.
Maybe solutions already exist, in which case the only thing really needed is further documentation.

[Wynadorn]

Additionally a Issue/Feature-request also exists on the appcenter repository requesting MAUI support (microsoft/appcenter#2360)

Strangely enough, as noted by @JohnHDev, there hasn't been a reply in the last couple months.

[Wynadorn]

The appcenter roadmap is concerning https://github.com/Microsoft/appcenter/wiki/Roadmap
See #315 (comment)

[rudyspano]

Tweets from the MAUI PM say that the AppCenter features compatibility for MAUI are WIP:
https://twitter.com/maddymontaquila/status/1537883335552139267?s=20&t=dJ2QllLIBROA4mo8a4XGDQ
https://twitter.com/maddymontaquila/status/1537778954630725632?s=20&t=PEXg9bxcGj-FtixaK4WmeQ

[Xyncgas]

Why test for UI when you can write unbreakable code via MVVM, all your UI ever gonna do is to display the view model, what could possibly go wrong on that

[Wynadorn]

🤡💻

[nickrandolph]

I can't tell whether you're being sarcastic or are serious..... either way, we need some decent UI test tooling for WinUI.... and the only reason I can imagine that we don't have them today, is that they don't exist... which is scary if you think about the number of bugs that ui testing catches....

[Xyncgas]

I remember pressing all the buttons myself on my phone when developing for Xamarin, I suppose if you don't want to test the UI after work, then having a script is easier

[RigtAnsw]

Hello,

We have written automated tests in Xamarin and we am wondering if we can reuse the Xamarin Ui Tests for .NET MAUI ?
Will there be a new tool for the mobile application to be able to automate manual tests after migration to .NET MAUI ?

[johnathan-s]

Hello,
I think we (those on this ticket) are following this ticket microsoft/appcenter#2416
JohnHDev has been very helpful to keep the MAUI Test Automation support on the radar for MS. (thank you @JohnHDev )

[johnathan-s]

Thanks John. I have pinged your issue and invite others to do the same.

…

On Sun, Jul 10, 2022 at 9:13 PM JohnHDev ***@***.***> wrote: @johnathan-s <https://github.com/johnathan-s> I created an issue on the AppCenter repo back in April for this: microsoft/appcenter#2416 <microsoft/appcenter#2416> It might be worth everyone that sees this, to ping on there as well. Had initial replies but nothing else for months. — Reply to this email directly, view it on GitHub <#315 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMUG7OK7RON25J4A5CNP72LVTKH47ANCNFSM4XWMCPBA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andrea-gavioli]

UI Tests are crucial for app development, in our company we used Xamarin UI Tests and now we're porting to .NET MAUI.
Should be great to have an official answer from the devs if App Center / UI Tests will be supported.
Thank you.

[johnathan-s]

Hello Andrea,
There was mention of app center in @jfversluis youtube channel in his "Use App Center Analytics, Crashlytics and more with .NET MAUI" video.
However, there has not been an ETA.
I have not seen if Appium has been tried with .net MAUI testing. (If others have tried Appium with .net MAUI, please reply.)

[jfversluis]

Thanks for the ping @johnathan-s!

A preview version is out right now for App Center in regards to analytics, crashlytics and distribution. You should be able to use that with .NET MAUI today.

As for UITest, version 4 of that package has been released with .NET 6 support, so also there work is done. I'm not 100% sure if that also means it works with .NET MAUI now, but at the very least you should be able to try!

It's all being worked on :)

[andrea-gavioli]

Thanks for the quick reply @johnathan-s and @jfversluis!
I will deeply test both App Center and UITest package with MAUI and I will come back with some feedback.

[JohnHDev]

@jfversluis there is still the Xamarin.TestCloud nuget that needs to be updated before Maui iOS projects can be tested. I submitted a request for that back in April I think, so no right you cannot UItest Maui iOS projects. You can with android however.
if there is anything you can do to help push that forward, we can’t really even consider migrating to Maui when we can’t migrate our ui tests at the same time.

[JohnHDev]

Also, UITest nuget 4.0.1 doesn’t work with the latest Xcode, been waiting for that for 2 months…

[NickDarvey]

It looks like it's on the backlog.

.NET MAUI UI Testing Strategy #11236

[Hackmodford]

@JohnHDev Do you have an example of this?

[Take-A-Byte]

Also, how can we test Windows and Mac?

[NickDarvey]

They're experimenting with using Appium to test MAUI itself. You might be able to try the same to write tests for macOS, Windows, iOS, and Android

See #14488, specific working versions of deps #15453, more tests #15191

[akhanalcs]

Is Appium any good?

[dedevXamarin]

It's supposed to be the best, but I found it flakey in actual testing. My tests would immediately fail 50% of the time, but it's possible I had some sort of configuration/server related issue. Xamarin was significantly more reliable for me.

[prasad210]

Any further update on which is the recommended UI testing framework for MAUI, since App Centre is retiring soon.

[joshardt]

This is not official but Appium and BrowserStack seems to be the best option right now.