VSTest task fails to execute tests in .NET Core 2.0 test project #5066
Comments
|
Possibly related: microsoft/vstest#596 The workaround from there of manually invoking a different vstest.console.exe and providing /Framework:FrameworkCore10 results in successfully executed tests (but prevents me actually using the VStest task). |
|
Same here, for some reason vstest task uses old version of test platform as described here |
|
@IvanAlekseev, @sandersaares - VSTest task cannot run .NET core tests as it uses the Test platform version 1. To run .NET core tests, we recommend using the .NET core task with the test command. @IvanAlekseev - Code coverage and other data collection is not supported in test platform v2 yet. |
|
@PBoraMSFT it is woking in VS Enterprise locally, and documentation mentions it should work from console https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md#working-with-code-coverage As you can see in logs posted above I was able to use TPv2 by specifying correct path and have coverage report created, but failed to upload because task is looking for it in different path. And there is clearly no coverage support for Can you please update VsTest task to use TPv2 and support .net core coverage. This is a real show stopper for us moving to .net core. |
|
Also here VS test team says that it is ready |
|
Please reopen this, seems clear to me that the mismatch paths could be the issue. |
|
Hi, We are fixing this issue as part of this https://github.com/Microsoft/vsts-agent/pull/1149/files Just reiterating this point - VSTest task cannot run .NET core tests as it uses the Test platform version 1. To run .NET core tests, we recommend using the .NET core task with the test command. |
|
Do I understand you right that when you say "we are fixing this" you mean that VSTest task will be fixed to be able to run .NET Core tests? That is the expected behavior but your later statement there confuses me. |
|
Actually, according to microsoft/vstest#579 (comment) VSTest can run .NET core tests by /Framework:FrameworkCore10 . I've tried this w/ one test, although it's still buggy and reporting 2 instead of 1 successful test, it is indeed ran by VSTest successfully. |
|
@nigurr can we please have more clarification here? Is it going to be fixed for VSTest task or we have to wait for another .net core release to support coverage in |
|
@IvanAlekseev - The VSTest task uses the test platform version 1. If using the vstest.console.exe of the Test platform v2 works for your scenarios, you may use it - but please note that unless the platform convergence happens, our ability to address any issues that may arise here will be limited. As for this specific issue related to code coverage, we are addressing it. As Nitin indicated earlier, this requires a fix on the agent and will require you to update the agent as the fix becomes available. |
|
@PBoraMSFT thanks for you reply. We are currently using hosted agents, so we wait for them to update. Is there any timeline for it and is there any specific place where you notify about updates? |
|
We will update on this thread. |
|
Today I've found that coverage is now displayed correctly for .net core project
This is what I had to do to make it work |
|
@IvanAlekseev i've replicated your settings and still have this error, have any idea what that might be? Params: Error: |
|
@tiagorosendo Maybe there is a problem with xunit runner. We are using MS Test v2, and it still works fine with settings mentioned above. |
|
@tiagorosendo - are you able to run the tests using vstest.console.exe independent of the task? |
|
@PBoraMSFT u can send me one example ? |
|
@tiagorosendo it is working fine with xunit. you may want to set the test assemblies to only the assemblies of your test project, then the testadapter.dll will be excluded. |
|
@chunliu thx, its works! but sonar qube throw one exception when i turn on code coverage =/ i will wait the sonarqube team fix this issue. |
|
Huge props to @IvanAlekseev for the work-around. We've got code coverage up and running using these settings. Here's the exact strings required (for easy copy/paste-ability) for anyone that needs them: Path to vstest.console.exe: Other console options: |
|
any plan to support .NETCore 2.0 unit tests? Because if I run
And it says it only supports FrameworkCore10. cc @PBoraMSFT |
|
@eriawan for .NETCore 2.0 I use /Framework:".NETCoreApp,Version=v2.0" which is picking up the tests and running them locally but having issues on VSTS hosted agent. |
|
/Framework:".NETCoreApp,Version=v2.0" fixed the issue for me with .NETCore 2.0 |
|
I actually get VSTS to run tests (.net core 2.0) but it fails with following error: 2018-06-01T11:06:36.1777961Z Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Failed to initialize client proxy: could not connect to test process. Any ideas what is causing this? |
|
@daliusd - you may want to open a new issue |
|
@bryanmacfarlane At the moment I solved this problem by using "dotnet test". I might consider opening new issue in the future if need will arise. Still I can answer all the question if someone from VSTS team is interested. |
|
Hi @daliusd - have you opened another ticket for the issue you describe above? If yes - please help me with the link. I have run into the same issue. |
|
I have not done anything on that. Feel free to do that. |
|
@daliusd - I've used a dotnet test step to get the tests executed. Have you done anything to get the code coverage working? |
|
@kiran-mahadev As I have said - nothing. Feel free to open new defect as suggested by @bryanmacfarlane |
|
I had the same problem. |
|
I can't get the code coverage working for me. All I see is in that tab is "Download Code Coverage results" here is my yaml: i also tried and here is the log for the vstest step: |
This is the piece I was missing to get rid of my error message of "Unable to find tests". My unit tests were running, but I was trying to figure out how to filter out the TestPlatform dlls. That Pipe for Multiline entry was the trick! But, to answer your question: Nothing is displayed under the "Code Coverage" tab except the link. If you want to see the display, go to the "Summary" tab and then click on the expansion arrow for "Code coverage succeeded" and you will see the progress bar for code coverage. |
|
@Antebios happy to help but following your instructions, this is all i see:
and just for reference, this is my tests tab: and code coverage tab:
100% of my code is definitely not covered. i'm looking for a more detailed view like in @IvanAlekseev's screenshot that shows blocks and lines covered. |
|
@joshbouganim - I'm sorry to say but that screenshot that @IvanAlekseev has is the "old" summary page. |
|
so the new code coverage page shows less and is worse? i find that hard to believe. is vstest the problem? .net core? |
|
@acesiddhu please look into this |
|
@joshbouganim the old UI used to show Block level data as well. we have removed that because we feel user generally needs and interpret Line level data. Dealing with Block is what we think is hard (specially for large code base). Having said that rest of the things remains as is, like showing line coverage data, ability to download coverage file. |
|
Currently my test framework is using .NET Core 2.1 and I do specify this in my runsettings. I am experiencing this same issue in Azure Dev Ops. Locally all I need is my Tests.dll in order to use vstest.console.exe or even dotnet test and it works. However, in Azure Dev Ops, **I build the project, copy just the bin folder for my Test project (We have a data, core & test project in one solution), then publish build artifact to a container. Using this container stored with our runsettings and bin folder/items, I run vstest and the error during test execution is ** "Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further." How hasnt this been fixed? I do have Microsoft.NET.Test.Sdk as a nuget reference for the project. Edit: My original comment, I was specifying /Framework: Framework10 Because I read that was a fix where I should have /Framework:.NETCoreApp,Version=v2.1. Now I am still getting the error mentioned here. |
|
@Huffman17 Did you specify the framework version in the "otherConsoleOptions" arguments? |
|
@Antebios Yes, I have specified it that way, I was specifying framework in the wrong format I guess, oddly enough. I get the error still "Unable to find tests for D:\a\r1\a\PATH\TO\DLL\My.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further."
in release:
Edit: I switched it back to my original comment. Currently I don't have the same issue where it would say im using NetCoreApp v1, But I still am getting the NQAP.ATC.Tests.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further. And it does not run the tests. |
We are experiencing the same issue as @joshbouganim and i'm a bit confused about the "things remain the same" part. Is there something that should be done so the coverage results would be shown on the coverage tab? Currently there's just the option to download .coverage file. |
Same here! I was expecting some awesome graphs =( |
so the new UI is the equivalent of the line metric from the old UI? |
@joshbouganim, yes that's correct |
|
Could VSTS just make it work with SonarQube? SonarQube is the best tool for not only Code Coverage but for defect idetifications, what VSTS did not even have. |
|
@CARROCON - ack. Coverage reporting in the web (without needing to download files for offline viewing in the IDE) is on the backlog. There are two parts to the .NET core problem - coverage collection on all platforms and reporting. Our first priority will be to enable collecting coverage data on Linux and then do the reporting bit. I am also locking the conversation here since the original issue is unrelated to the recent comments about coverage reporting - feel free to look through other related issues or open a new issue for this feedback. |
and others
If I provide a .NET Core 2.0 test project DLL as input to the VSTest task (version 2.0.70), I encounter the following error:
2017-08-21T07:42:45.2798678Z Warning: Unable to load types from the test source 'C:\Agent1\_work\34\s\Tests.NetCore\bin\Release\netcoreapp2.0\Tests.NetCore.dll'. Some or all of the tests in this source may not be discovered.Indeed, no tests from this assembly are actually executed. The same tests are executed without issues via VS 2017 test explorer.
VS 2017 update 3 and .NET Core SDK 2.0.0 are installed.
The text was updated successfully, but these errors were encountered: