xUnit1016 warnings since 2.4.2 - can't find changelog

[304NotModified]

For some reason, we get xUnit1016 "MemberData must reference a public member" errors since xUnit 2.4.2 (and not in 2.4.1)

Those errors are correct, in the way that the MemberData where indeed not public.

But i'm confused;

• The private MemberData worked with xUnit 2.4.1 (locally and on the build server).
• We do we see this error since 2.4.2?
• I can't find a changelog or a milestone. So it this a new feature?

[bradwilson]

Private MemberData should never have worked in 2.4.1. The error message was poor, but it definitely fails:

Making it public allows the data to be run:

[adamralph]

@bradwilson the OP mentioned:

I can't find a changelog or a milestone.

Is there anything like this somewhere?

[bradwilson]

I did not put together release notes for the analyzers.

[adamralph]

To ensure I understand—does that mean xunit 2.4.2 contains only changes to analyzers?

[bradwilson]

No, I wrote release notes for 2.4.2 but nothing for xunit.analyzers, mostly because the person who was previously maintaining it never did (and the repo doesn't tag releases... something I should fix, honestly). But since it's been ~4 years since both xunit and xunit.analyzers were last shipped, a lot of fixes have gone into the analyzers code base that I wasn't tracking.

I should see if I can back-fill from the approximate last full release and come up with some release notes for 1.0.0 of xunit.analyzers.

[bradwilson]

@adamralph https://xunit.net/releases/analyzers/1.0.0

For what it's worth, the only change I could find from 0.10.0 to 1.0.0 for xUnit1016 was related to overridden names.

[MrKWatkins]

I've just hit a similar issue after upgrade. The member data method was internal. (Because it was returning an instance of a type being tested which was itself internal so couldn't be public.) We started getting xUnit1016 after upgrading to 2.4.2.

[MrKWatkins]

I think the difference is that private data source methods work under 2.4.1, but not properties. Try the following:

public sealed class XUnitTest
{
    private static readonly TheoryData<int> PropertyData = new TheoryData<int> { 1 };

    [Theory]
    [MemberData(nameof(PropertyData))]
    public void PropertyTest(int x) => Assert.Equal(1, x);

    private static TheoryData<int> MethodData() => new TheoryData<int> { 1 };

    [Theory]
    [MemberData(nameof(MethodData))]
    public void MethodTest(int x) => Assert.Equal(1, x);
}

Under 2.4.1 this gives me:

Microsoft (R) Test Execution Command Line Tool Version 17.1.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.90]     Blah.XUnitTest.PropertyTest [FAIL]
  Failed Blah.XUnitTest.PropertyTest [1 ms]
  Error Message:
   System.NotSupportedException : Specified method is not supported.


Failed!  - Failed:     1, Passed:     1, Skipped:     0, Total:     2, Duration: 8 ms - Blah.dll (net6.0)

[304NotModified]

Nice work!

[bradwilson]

Any private member data working is an unintentional bug. Public has always been the requirement.

[MrKWatkins]

I'm being hit by this heavily for internal properties/methods, not private. We have a lot of internal types that we test using this, making the types accessible to the tests via an InternalsVisibleToAttribute. Can you relax it so that internal properties/methods are supported for test case sources please? Internal test methods are supported after all; for example this runs fine in 2.4.2:

[Fact]
internal void Test() => Assert.True(true);

It would be really helpful if test case sources could be internal too. It would be consistent with the rules for tests and it would allow internal types to be passed into tests via test case sources and TheoryData.