New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filtering tests with any "PropertyAttribute" #779
Comments
Hello, yes it can be done given you are using at least version 3.16 of the nunit test adapter, and a relatively new version of vstest.console.exe (I'm using v16.3.0) ExampleUsing your example:
ExplanationAs of v 3.16, the NUnit adapter supports test selection using NUnit's own test selection language by including a 'Where' property in the runsettings file. This can be coupled with vstest's ability to provide runsettings arguments via the command line. Further readinghttps://blog.prokrams.com/2019/12/16/nunit3-filter-dotnet/ I hope that helps. |
I see. That's good it can be done. The advantage is, as I understand, that it can use full power of NUnit test selection language. The disadvantage I see it that it's specific for NUnit. It won't work with xUnit or MSTest as I understand. From this point of view, maybe it'd be still possible to pass those properties to vstest? As I understand, NUnit already passes some properties (e.g. "Category") as Traits to vstest. I hope that passing "Property" would not be a big change. |
Hopefully somebody else can correct me if I am wrong, but the NUnit adapter already passes test properties to VSTest as traits which is why the properties can be seen in the test explorer window in Visual Studio (I think the relevant code is the So it's a matter of whether vstest updates the spec for FYI the vstest-docs repo has a doc about test filters here. |
@shack05 Correct and thanks :-) |
Hmm so I should file something against vstest repo? Or should this issue be moved to vstest repo (if that's possible)? |
We can't move the issue to another org, so raise it at vstest, but we'll keep it open here. Once they have confirmed it is on their side, I'll mark this as External and VS Issue here. Nice if you add a link here too to the upcoming vstest issue. |
Just to confirm - I created this issue because guys from vstest said it must be implemented in test adapter (microsoft/testfx#731 (comment)). But what you're saying is that it is already implemented in test adapter but not yet supported in vstest "spec for /TestCaseFilter" ? |
@pkrukp @shack05 Can you verify if this fixes the issue for you: |
Hello @OsirisTerje, |
Hi,
vstest.console.exe has
/TestCaseFilter
switch (and inIVsTestConsoleWrapper
API there'sTestCaseFilter
inRunSettings
, indotnet test
it's--filter
AFAIK).But it seems it's not possible to run only tests which has specific PropertyAttribute. For example given following:
I would like to be able run something like this:
vstest.console.exe NUnitTestProject1.dll /TestCaseFilter:Bug=12345
. But it does not seems to work - no test is executed. Am I doing it wrong, or is there no support for this?According to microsoft/testfx#731 it must be supported by test adapter (it is supported by xUnit, not supported by MSTest).
If it is not supported by NUnit, I'd like to ask to add such support.
Please see the above issue for some more details and simple solution with 3 UT projects (mstest, xunit, nunit): ClassLibrary1.zip
I have tested this with vstest.console.exe from microsoft.testplatform.16.7.0 nuget package, with NUnit3TestAdapter 3.15.1.
The text was updated successfully, but these errors were encountered: