Unable to set thread specific values in AssemblyInitialize since MSTest 3.3

[cremor]

Describe the bug

After updating my solution from MSTest 3.2.2 to 3.3.1 a few of my tests are failing on the build agent (but not on my local machine).
The failing tests seem to have one thing in common: They rely on a specific CultureInfo.CurrentCulture. The required culture is the one I'm using on my machine, but the build agent uses a different culture. But the CurrentCulture is set in my AssemblyInitialize method, and this worked previously.

Steps To Reproduce

    [TestClass]
    public static class Init
    {
        [AssemblyInitialize]
        public static void AssemblyInitialize(TestContext ctx)
        {
            CultureInfo.CurrentCulture = new CultureInfo("th-TH");
        }
    }

    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            Assert.AreEqual("th-TH", CultureInfo.CurrentCulture.Name);
        }
    }

Expected behavior

The test succeeds (it did with MSTest 3.2.2).

Actual behavior

The test fails. CurrentCulture is the culture of the machine which is running the test.

Additional context

A quote from @MarcoRossignoli in #2653 (comment)

I would not consider a breaking change too, MSTest doesn't give any guarantee for what I know(I don't see documentation around it) on the nature of the threads that are running the "tests method", we only guarantee that the execution order is the expected one. Tools/Tests should not take "dependency" on the runtime non documented/guarantee context.

We need the freedom to change it for more than one reason, timeout is one...but there're others that could come in future.

I see why you need this. And I also know that relying on a thread specific value is not good in tests. But now that I have such tests (because that worked previously), how can I solve this? Is there another place where I can initialize thread specific values for all tests?

And it would really have helped debugging this if the behaviour change was documented in the release notes!

AB#2051796

[Evangelink]

Hi @cremor,

We fixed this issue in v3.4 that will be released next week. I don't know if you could follow these instructions to try our feed and confirm this is correctly fixing your issue.

[cremor]

I've just tested version 3.4.0-preview.24253.3 and now even more tests are failing. All of the fails of 3.3.1 are still there, so 3.4.0 doesn't fix my problem.

Additionally I now get errors from WPF tests with messages like "System.InvalidOperationException: The calling thread cannot access this object because a different thread owns it." or "System.ArgumentException: Must create DependencySource on same Thread as the DependencyObject.".

The provided minimal sample also still fails with 3.4.0-preview.24253.3.

[Evangelink]

The provided minimal sample also still fails with 3.4.0-preview.24253.3.

Yes I can repro too. Setting DefaultThreadCurrentCulture instead of CurrentCulture solves this problem as it's setting the culture to all threads and not only current thread.

I will investigate what we can do more to "share context" between the threads.

Additionally I now get errors from WPF tests with messages like "System.InvalidOperationException: The calling thread cannot access this object because a different thread owns it." or "System.ArgumentException: Must create DependencySource on same Thread as the DependencyObject.".

All fixture methods were called synchronously on the app thread and only the test method was run with Task.Run or new Thread while now all fixtures are also created with a different thread.

[cremor]

Setting DefaultThreadCurrentCulture instead of CurrentCulture solves this problem as it's setting the culture to all threads and not only current thread.

Good idea, thanks. With that all my tests now pass with MSTest 3.3.1.

now all fixtures are also created with a different thread.

I create my WPF objects in TestInitialize and use them in TestMethod. Does this mean that this is not supported any more starting with MSTest 3.4.0? That would be quite annoying. I can see that sharing state between test methods is bad, but I'd consider TestInitialize a part of a test method.

[Evangelink]

@MarcoRossignoli shared this https://learn.microsoft.com/dotnet/api/system.threading.executioncontext.capture which will fix the problem of synchronization between threads when using static info.

I'll delay releasing 3.4 (few days) to do this fix. Thanks for the report @cremor

[testplatform-bot]

✅ Successfully linked to Azure Boards work item(s):

• User Story 2051796

[cremor]

I'll delay releasing 3.4 (few days) to do this fix.

What will be fixed? The issue I intially reported here or the one that only happens with the 3.4 preview?

[cremor]

I can confirm that both issues are fixed in 3.4.0-preview.24257.1, thanks!

[Evangelink]

Thank you so much for the confirmation! I'll create a different ticket for a similar bug when using timeout (this part requires a more complex fix). I'll link ticket to this one once created.