Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit tests fail on macOS (and probably on Linux too) #1021

Open
0xced opened this issue Jan 3, 2021 · 2 comments
Open

Unit tests fail on macOS (and probably on Linux too) #1021

0xced opened this issue Jan 3, 2021 · 2 comments

Comments

@0xced
Copy link
Contributor

0xced commented Jan 3, 2021

Here is how to reproduce:

$ cd src/Humanizer.Tests
$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.6
BuildVersion:	19G2021
$ dotnet --version
5.0.101
$ dotnet test --framework netcoreapp2.1
Microsoft (R) Build Engine version 16.8.0+126527ff1 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Humanizer -> ~/Projects/Humanizer/src/Humanizer/bin/Debug/netstandard2.0/Humanizer.dll
  Humanizer.Tests -> ~/Projects/Humanizer/src/Humanizer.Tests/bin/Debug/netcoreapp2.1/Humanizer.Tests.dll
Test run for ~/Projects/Humanizer/src/Humanizer.Tests/bin/Debug/netcoreapp2.1/Humanizer.Tests.dll (.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.8.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:06.33]     Years(days: 366, expected: "1 年") [FAIL]
  Failed Years(days: 366, expected: "1 年") [4 ms]
  Error Message:
   Assert.Equal() Failure
            ↓ (pos 2)
Expected: 1 年
Actual:   1 year
            ↑ (pos 2)
  Stack Trace:
     at Humanizer.Tests.Localisation.zhCN.TimeSpanHumanizeTests.Years(Int32 days, String expected) in ~/Projects/Humanizer/src/Humanizer.Tests.Shared/Localisation/zh-CN/TimeSpanHumanizeTests.cs:line 18

[...] Many more failed tests [...]

Failed!  - Failed:   102, Passed: 10366, Skipped:     0, Total: 10468, Duration: 38 s - ~/Projects/Humanizer/src/Humanizer.Tests/bin/Debug/netcoreapp2.1/Humanizer.Tests.dll (netcoreapp2.1)

I have noticed that the zh-CN resources are not copied into src/Humanizer.Tests/bin/Debug/netcoreapp2.1. All other localizations are there (af, ar, ... fr, fr-BE, ..., zh-Hans, zh-Hant) but there is no zh-CN directory containing a Humanizer.resources.dll file.

So I ran dotnet build -bl and examined the binary log with MSBuild Structured Log Viewer. It turns out that the AssignCulture task (which is run by the SplitResourcesByCulture target) miscategorizes the zh-CN culture which ends up in the ResxWithNoCulture item group instead of the ResxWithCulture item group where all other localized resources are.

I'm currently investigating why the AssignCulture task does not recognize zh-CN as a valid culture on macOS.

@0xced
Copy link
Contributor Author

0xced commented Jan 3, 2021

Here's the small program I wrote to diagnose the AssignCulture task.

assignculture.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="NSubstitute" Version="4.2.2" />
    <PackageReference Include="ReflectionMagic" Version="4.1.0" />
  </ItemGroup>

  <ItemGroup Condition="'$(ReferenceMsBuildNuGetPackages)' != 'true'">
    <Reference Include="/usr/local/share/dotnet/sdk/5.0.101/Microsoft.Build.Framework.dll" />
    <Reference Include="/usr/local/share/dotnet/sdk/5.0.101/Microsoft.Build.Tasks.Core.dll" />
    <Reference Include="/usr/local/share/dotnet/sdk/5.0.101/Microsoft.Build.Utilities.Core.dll" />
  </ItemGroup>

  <ItemGroup Condition="'$(ReferenceMsBuildNuGetPackages)' == 'true'">
    <PackageReference Include="Microsoft.Build.Tasks.Core" Version="16.8.0" />
  </ItemGroup>
  
</Project>

Program.cs

using System;
using System.Globalization;
using System.Linq;
using System.Runtime.InteropServices;
using Microsoft.Build.Framework;
using Microsoft.Build.Tasks;
using Microsoft.Build.Utilities;
using NSubstitute;
using ReflectionMagic;

try
{
    SimulateMsBuildInitialization();

    PrintAllChineseCultures();

    var task = RunAssignCultureTask("Resources.zh-CN.resx", "Resources.zh-Hans.resx");

    PrintAssignedFiles(task);
}
catch (Exception exception)
{
    Console.Error.WriteLine(exception);
}

static void SimulateMsBuildInitialization()
{
    var assembly = typeof(AssignCulture).Assembly;
    assembly.GetType("Microsoft.Build.Shared.AssemblyUtilities", throwOnError: true).AsDynamicType().Initialize();
    Console.WriteLine($"Running on {RuntimeInformation.FrameworkDescription} using {assembly} at {assembly.Location}");
    Console.WriteLine();
}

static void PrintAllChineseCultures()
{
    Console.WriteLine("*** All Chinese cultures ***");
    foreach (var culture in CultureInfo.GetCultures(CultureTypes.AllCultures).Where(e => e.Name.StartsWith("zh")))
    {
        Console.WriteLine($"[{culture.Name}] {culture.EnglishName}");
    }
    Console.WriteLine();
}

static AssignCulture RunAssignCultureTask(params string[] cultureNames)
{
    var buildEngine = Substitute.For<IBuildEngine>();
    buildEngine.LogMessageEvent(Arg.Do<BuildMessageEventArgs>(e => Console.WriteLine($"[{e.Importance}] {e.Message}")));
    var task = new AssignCulture
    {
        BuildEngine = buildEngine,
        Files = cultureNames.Select(e => new TaskItem(e)).Cast<ITaskItem>().ToArray(),
    };
    task.Execute();
    Console.WriteLine();
    return task;
}

static void PrintAssignedFiles(AssignCulture task)
{
    PrintTaskItems(task.AssignedFilesWithNoCulture, nameof(task.AssignedFilesWithNoCulture));
    PrintTaskItems(task.AssignedFilesWithCulture, nameof(task.AssignedFilesWithCulture));
}

static void PrintTaskItems(ITaskItem[] taskItems, string name)
{
    var itemSpecModifiers = typeof(AssignCulture).Assembly.GetType("Microsoft.Build.Shared.FileUtilities+ItemSpecModifiers", throwOnError: true).AsDynamicType();
    Console.WriteLine($"{name}: {taskItems.Length}");
    foreach (var taskItem in taskItems)
    {
        Console.WriteLine($"  {taskItem.ItemSpec}");
        foreach (var metadataName in taskItem.MetadataNames.Cast<string>().Where(e => !itemSpecModifiers.IsItemSpecModifier(e)))
        {
            var metadata = taskItem.GetMetadata(metadataName);
            if (!string.IsNullOrEmpty(metadata))
            {
                Console.WriteLine($"    {metadataName} = {metadata}");
            }
        }
    }
}

Here's the output when run with dotnet run on macOS:

Running on .NET 5.0.1-servicing.20575.16 using Microsoft.Build.Tasks.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a at assignculture/bin/Debug/net5.0/Microsoft.Build.Tasks.Core.dll

*** All Chinese cultures ***
[zh] Chinese
[zh-Hans] Chinese
[zh-Hans-CN] Chinese, Simplified (China mainland)
[zh-Hans-HK] Chinese, Simplified (Hong Kong)
[zh-Hans-MO] Chinese, Simplified (Macao)
[zh-Hans-SG] Chinese, Simplified (Singapore)
[zh-Hant] Chinese
[zh-Hant-CN] Chinese, Traditional (China mainland)
[zh-Hant-HK] Chinese, Traditional (Hong Kong)
[zh-Hant-MO] Chinese, Traditional (Macao)
[zh-Hant-TW] Chinese, Traditional (Taiwan)

[Low] Culture of "" was assigned to file "Resources.zh-CN.resx".
[Low] Culture of "zh-Hans" was assigned to file "Resources.zh-Hans.resx".

AssignedFilesWithNoCulture: 1
  Resources.zh-CN.resx
    WithCulture = false
    OriginalItemSpec = Resources.zh-CN.resx
AssignedFilesWithCulture: 1
  Resources.zh-Hans.resx
    Culture = zh-Hans
    WithCulture = true
    OriginalItemSpec = Resources.zh-Hans.resx

And here's the result when running on Windows with dotnet build /p:ReferenceMsBuildNuGetPackages=true && dotnet run --no-build:

Running on .NET 5.0.1-servicing.20575.16 using Microsoft.Build.Tasks.Core, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a at c:\Projects\Experiments\assignculture\bin\Debug\net5.0\Microsoft.Build.Tasks.Core.dll

*** All Chinese cultures ***
[zh] Chinese
[zh-Hans] Chinese
[zh-CN] Chinese (China)
[zh-Hans-HK] Chinese (Simplified, Hong Kong SAR)
[zh-Hans-MO] Chinese (Simplified, Macao SAR)
[zh-SG] Chinese (Singapore)
[zh-Hant] Chinese
[zh-HK] Chinese (Hong Kong SAR)
[zh-MO] Chinese (Macao SAR)
[zh-TW] Chinese (Taiwan)

[Low] Culture of "zh-CN" was assigned to file "Resources.zh-CN.resx".
[Low] Culture of "zh-Hans" was assigned to file "Resources.zh-Hans.resx".

AssignedFilesWithNoCulture: 0
AssignedFilesWithCulture: 2
  Resources.zh-CN.resx
    Culture = zh-CN
    WithCulture = true
    OriginalItemSpec = Resources.zh-CN.resx
  Resources.zh-Hans.resx
    Culture = zh-Hans
    WithCulture = true
    OriginalItemSpec = Resources.zh-Hans.resx

The difference, as we can see, is that zh-CN is only known to Windows.

I'm not sure yet what is the best solution to this problem.

0xced added a commit to 0xced/msbuild that referenced this issue Jan 3, 2021
The `Initialize()` method must be called before testing the `s_cultureInfoGetCultureMethod` static field.

This might go unnoticed if the `Initialize` method was called earlier through some other code path but the result of calling `CultureInfoHasGetCultures()` would be wrong if `Initialize()` was not called beforehand. For example, in the static constructor of `Microsoft.Build.Tasks.CultureInfoCache`.

Note: I stumbled on this when [diagnosing an issue][1] with the AssignCulture task.

[1]: Humanizr/Humanizer#1021 (comment)
@0xced
Copy link
Contributor Author

0xced commented Jan 4, 2021

Well, it turns out I just re-discovered a known MSBuild issue apparently: dotnet/msbuild#3897

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant