Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ProjectReferences Negotiate SetPlatform Metadata #6655

Merged
merged 74 commits into from
Jul 30, 2021

Conversation

benvillalobos
Copy link
Member

@benvillalobos benvillalobos commented Jul 6, 2021

Fixes #5338

Context

TargetFrameworks are determined dynamically at build time, this change sets a baseline for performing SetPlatform negotiation.

In a project setup like so:

A --> B --> C

A can build as x86
B as x86, x64
C as AnyCPU

Projects can tell ProjectReferences to build as their "most compatible" platform.

A would tell B to build as x86, and B would tell C to build AnyCPU.

The build can now negotiate what Platform a ProjectReference should build as. This negotiation happens in the _GetProjectReferencePlatformProperties target. It piggybacks off of the GetTargetFrameworks call that happens in _GetProjectReferenceTargetFrameworkProperties that now pulls $(Platforms) data from referenced projects. The GetCompatiblePlatform task performs the main negotiation. An optional PlatformLookupTable can be specified per ProjectReference, or in individual projects, or in a Directory.Build.props. This lookup table is in the form foo=bar;C=D and means (in English), "When some project is building with platform foo, and a ProjectReference can build with platform B, tell the referenced project to build as bar"

These are the rules by which the GetCompatiblePlatform task assigns a platform:

  1. Can the referenced project build with the same platform as the currently-building project? Use that.
  2. Does the ProjectReference item have a PlatformTranslationTable defined? Use that to find a mapping if possible
  3. Does the currently-building project have a PlatformTranslationTable defined? Use that to find a mapping if possible
  4. Can the child build as AnyCPU? Use that if possible.
  5. Can't determine the platform to pass along, log a warning and undefine platform.

Finally, the SetPlatform metadata is set or undefined per ProjectReference.

Changes Made

  • MODIFIED TARGETS

    • GetTargetFrameworks
      • Extracts $(Platforms) and $(IsVcxOrNativeProj) from ProjectReference Items
  • NEW TARGETS

    • _GetProjectReferencePlatformProperties
      • Calls GetPlatforms on referenced projects
      • Only runs when EnableDynamicPlatformResolution is true on the referencing project
      • Sets or undefines SetPlatform metadata on all referenced projects
  • New Task

    • GetCompatiblePlatform
      • Performs the negotiation
      • Can throw a warning when no compatible platform is found.

Testing

✔️

Notes

Reviewing as a whole is better for this PR.

To Do

  • Add codes/resources for all warnings
  • Add tests
  • Add docs

flow into TargetFramework logic.
…so don't modify any items if the task didn't run or there were no items added to ProjectsWithPlatformAssignment
…uild. Bring over CanMultiPlatform logic to CrossTargeting.targets
Continue instead of breaking if the platform lookup table is malformed,
other items in the table might be valid.

Remove CanMultiPlatform, not necessary if GetNearestPlatformTask can
figure that out.

Use IsVcxOrNativeProj when setting PlatformLookupTable
…PlatformTask. The use case is when a particular project wants its Win32 or AnyCPU platforms to map to a specific other platform
… check when setting default managed->unmanaged PlatformLookupTable
Next, every referenced project is required to define a `$(Platforms)` property. `$(Platforms)` is a semicolon-delimited list of platforms that project could build as. `<Platforms>x64;x86;AnyCPU</Platforms>`, for example.

Lastly, projects that contain `ProjectReference` items may need to define a `$(PlatformLookupTable)` property. `$(PlatformLookupTable)` is a semicolon-delimited list of mappings between projects. `<PlatformLookupTable>win32=x86</PlatformLookupTable>`, for example. This means that if the current project is building for `Win32`, it should build referenced projects using `x86` as the `Platform`. This is mostly relevant for references between managed and unmanaged projects.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way this paragraph reads makes me wonder if Win32=x86 set in a vcxproj will break P2Ps to other vcxproj's, which expect Win32 and not x86.
How would we resolve this for when a project references a blend of C# and C++ projects?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case the PlatformLookupTable is presumably defined globally in some Directory.Build.props, or in the cpp project that could reference c# and cpp projects. In the specific projectreference of cpp -> cpp, you can set PlatformLookupTable (or SetPlatform) metadata on that individual projectreference, and that metadata will take priority.

We could make this a cleaner experience but that would be in a future PR.

@@ -57,7 +57,8 @@ If implementing a project with an “outer” (determine what properties to pass
* It returns an item with the following metadata:
* `TargetFrameworks` indicating what TargetFrameworks are available in the project
* `TargetFrameworkMonikers` and `TargetPlatformMonikers` indicating what framework / platform the `TargetFrameworks` map to. This is to support implicitly setting the target platform version (for example inferring that `net5.0-windows` means the same as `net5.0-windows7.0`) as well as treating the `TargetFramework` values [as aliases](https://github.com/NuGet/Home/issues/5154)
* Boolean metadata for `HasSingleTargetFramework` and `IsRidAgnostic`.
* Boolean metadata for `HasSingleTargetFramework` and `IsRidAgnostic` and `IsVcxOrNativeProj`.
* `Platforms` indicating what `Platforms` are available for the project to build as.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the second use of the word platforms is as an english explanation rather than a recursive definition, so drop the identifier treatment.

Suggested change
* `Platforms` indicating what `Platforms` are available for the project to build as.
* `Platforms` indicating what platforms are available for the project to build as.

| Managed -> Managed | No | |
| Unmanaged -> Managed | **Yes** | |
| Managed -> Unmanaged | Optional | Uses default mapping: `AnyCPU=Win32;x86=Win32` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this default mapping still correct? I thought we agreed in the discussion that AnyCPU->Win32 should emit a warning at least, if not block, unless the user explicitly opted into that jump.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify here, when jumping from a parent that's AnyCPU to a child that is anything non-AnyCPU, a warning should be emitted>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good. But with the default table, an AnyCPU->Win32 jump looks permissible. I don't think I would expect a warning if an entry exists in the table. If it does, what would I do to suppress the warning if I needed to?

| Unmanaged -> Unmanaged | No | |
| Managed -> Managed | No | |
| Unmanaged -> Managed | **Yes** | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is a table required when C++ references a C# project? C++ is always arch-specific and C# will almost always have either AnyCPU or a compatible arch-specific platform. Couldn't a default mapping of win32=x86;x64=x64;arm=arm apply here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main confusion here was win32 being applicable to x86 and AnyCPU and not being entirely sure if either should be the default mapping from win32. Is win32->x86 more common?

Side note: x64=x64 isn't necessary because that's auto-detected in the task.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It goes back to my TFM analogy: where netstandard2.0 and net472 are both built by a child, a net472 parent project will pick the platform-specific one (net472) because if you offer something portable and a compatible specific target, the only reason the specific target would exist is because it is preferred over the portable one.
So ya, win32->x86 would be preferred over win32->anycpu by the same reasoning.
But of course for the (default) C# project that doesn't list x86 in its Platforms property, then the behavior would be win32->anycpu because of your "fallback to anycpu" logic. So your table for vcxproj->csproj references would just be win32=x86. And all other platform combination either may directly (e.g. x64=x64) or use the *=anycpu fallback.

documentation/ProjectReference-Protocol.md Outdated Show resolved Hide resolved
```xml
<ItemGroup>
<ProjectReference Include="B.csproj" PlatformLookupTable="Win32=AnyCPU">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this should happen automatically in most cases (as per my above comment), a more compelling sample would be a C# -> C++ reference I think. Of course in that case a default table should be able to resolve it without help, except in the case where C# only defines AnyCPU. In that case though, would the easiest thing be for the customer to set the table property, or just directly set the SetPlatform metadata?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When there's only one option, the customer should just set SetPlatform. It's when the parent project could build as multiple platforms and we're not sure which one its building as before the build has started.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that makes sense. But specifying Win32=AnyCPU here should only be necessary if C# offered an x86 option (that would fit the default mapping table) and the user didn't want that option taken but wanted AnyCPU to be used instead. This would be very strange, as why would the user define an x86 target platform and then not use it in an x86 process?

```xml
<ItemGroup>
<ProjectReference Include="B.csproj" PlatformLookupTable="Win32=AnyCPU">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to this point, the docs keep referring to PlatformLookupTable as a property, but this sample defines it as a ProjectReference item metadata. Is it one, the other, or both? Should the docs clarify this a bit more?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs definitely need a refresh 🙂

src/Tasks.UnitTests/GetCompatiblePlatform_Tests.cs Outdated Show resolved Hide resolved
src/Tasks.UnitTests/GetCompatiblePlatform_Tests.cs Outdated Show resolved Hide resolved
src/Tasks/GetCompatiblePlatform.cs Outdated Show resolved Hide resolved
src/Tasks/GetCompatiblePlatform.cs Outdated Show resolved Hide resolved

foreach (string s in stringTable.Split(MSBuildConstants.SemicolonChar, StringSplitOptions.RemoveEmptyEntries))
{
string[] keyVal = s.Split(MSBuildConstants.EqualsChar, StringSplitOptions.RemoveEmptyEntries);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does RemoveEmptyEntries buy you here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only unnecessary errors warnings in strange translation tables like A=Z;;B=G that are "technically" correct. I could see table entries added via properties that may or may not have values, but may be easier to include at once for convenience.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the semicolon split I'm totally on board. For the equals I'm still pretty skeptical.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to enforce proper formatting for individual entries. Will allow empty entries on splitting the equals.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowing empty entries complicates the invalid check and doesn't get us much benefit. Disallowing empty entries guarantees a=b at minimum, and keeps the invalid check simpler (length <=1 or > 2 then invalid). Otherwise its an extra two string null/empty checks, which would have been handled by string.Split

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it also silently handles super confusing cases like =x86========anycpu==, which I think is bad.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it also silently handles super confusing cases like =x86========anycpu==, which I think is bad.

Blarg, blarg I say! (Translation: you're right, will fix)

@@ -57,7 +57,8 @@ If implementing a project with an “outer” (determine what properties to pass
* It returns an item with the following metadata:
* `TargetFrameworks` indicating what TargetFrameworks are available in the project
* `TargetFrameworkMonikers` and `TargetPlatformMonikers` indicating what framework / platform the `TargetFrameworks` map to. This is to support implicitly setting the target platform version (for example inferring that `net5.0-windows` means the same as `net5.0-windows7.0`) as well as treating the `TargetFramework` values [as aliases](https://github.com/NuGet/Home/issues/5154)
* Boolean metadata for `HasSingleTargetFramework` and `IsRidAgnostic`.
* Boolean metadata for `HasSingleTargetFramework` and `IsRidAgnostic` and `IsVcxOrNativeProj`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of IsVcxOrNativeProj can we give a more crisp meaning? Like "SupportsPlatformNegotiation" or something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SupportsPlatformNegotiation is a bit too broad for this bool.

IsVcxOrNativeProj is used for:

  • Should SetPlatform be Platform= or PlatformTarget=?
  • Should we work around some Target Framework negotiation logic? (because it's conditioned against .vcxproj and .nativeproj)
  • Are we explicitly looking at a .vcxproj or .nativeproj from a managed project? (non .vcxproj/.nativeproj)

src/Tasks/Microsoft.Common.CurrentVersion.targets Outdated Show resolved Hide resolved
======================================================================================
-->

<!-- Managed projects need to have PlatformTarget set for SetPlatform negotiation. Default to $(Platform), which is AnyCPU by default. -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Managed projects need to have PlatformTarget set for SetPlatform negotiation.

Why?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, do you need this given this below

    <!-- Managed Platform "source of truth" is $(PlatformTarget). For cpp it's $(Platform) -->
    <PropertyGroup>
      <ParentPlatform>$(PlatformTarget)</ParentPlatform>
      <ParentPlatform Condition="'$(MSBuildProjectExtension)' == '.vcxproj' or '$(MSBuildProjectExtension)' == '.nativeproj'">$(Platform)</ParentPlatform>
    </PropertyGroup>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

There was a case I hit where PlatformTarget wasn't set when building a managed project.

Actually, do you need this given this below

That's why the above is needed. I suppose we could change it to:

    <PropertyGroup>
      <ParentPlatform>$(PlatformTarget)</ParentPlatform>
      <ParentPlatform Condition="'$(PlatformTarget)' == '' or '$(MSBuildProjectExtension)' == '.vcxproj' or '$(MSBuildProjectExtension)' == '.nativeproj'">$(Platform)</ParentPlatform>
    </PropertyGroup>


<Target Name="_GetProjectReferencePlatformProperties"
Condition="'$(EnableDynamicPlatformResolution)' == 'true'
and '$(BuildingInsideVisualStudio)' != 'true'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment this please

@benvillalobos
Copy link
Member Author

benvillalobos commented Jul 27, 2021

Notes from the meeting between @AArnott and @yuehuang010 and myself today:

✔️ We should construct the set of platforms a cpp or nativeproj can build as via its itemgroup labeled ProjectConfigurations, which has the Platform for each configuration.
❌ Whenever we're currently building as AnyCPU and "hop off" to any non-AnyCPU architecture, we should log a warning when it isn't a "user-specified" (lookup table) hop
- On second thought, the only way this could happen is if it came from a lookup table.
✔️ If the current project platform matches the referenced project's platform, that should always take priority.
✔️ AnyCPU should not map to Win32 by default

@benvillalobos
Copy link
Member Author

benvillalobos commented Jul 28, 2021

AnyCPU should not map to Win32 by default

@AArnott When a customer opts into this feature and has no PlatformLookupTable, a ProjectReference from a managed project to an unmanaged project won't "just work". You were pushing for a solution that had the feature do most of the work, is requiring a lookup table in this scenario acceptable? The answer must be yes since we agreed AnyCPU shouldn't map to win32 by default, just double checking.

This doesn't change the fact that if A was x86, it would see that B could be x86 and map to that by default.

@AArnott
Copy link
Contributor

AArnott commented Jul 28, 2021

This doesn't change the fact that if A was x86, it would see that B could be x86 and map to that by default.

We also want a C# x86 project to reference a C++ win32 project without any extra effort. So a default mapping of x86=win32 is desirable.

But yes, I think AnyCPU=win32 is not a good thing. Warn by default for such jumps, and make it easy to suppress the warning (by adding to the lookup table or adding SetPlatform metadata) for those that want that behavior.

Copy link
Contributor

@AArnott AArnott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this. This is gonna be awesome for building msbuild projects without a solution configuration!

documentation/ProjectReference-Protocol.md Outdated Show resolved Hide resolved
documentation/ProjectReference-Protocol.md Outdated Show resolved Hide resolved
documentation/ProjectReference-Protocol.md Outdated Show resolved Hide resolved
documentation/ProjectReference-Protocol.md Show resolved Hide resolved
If only set in one project, the `SetPlatform` metadata will carry forward to every consecutive project reference.

Next, every referenced project is required to define a `Platforms` property, where `Platforms` is a semicolon-delimited list of platforms that project could build as. For `.vcxproj` or `.nativeproj` projects, `Platforms` is constructed from the `ProjectConfiguration` items that already exist in the project. For managed SDK projects, the default is `AnyCPU`. Managed non-SDK projects need to define this manually.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Platforms is constructed from the ProjectConfiguration items that already exist in the project

Is this done at execution time instead of evaluation time then? Just curious because items can't usually be used to create properties.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this done at execution time instead of evaluation time then?

Right, because it's running in the GetTargetFrameworks target.

documentation/ProjectReference-Protocol.md Outdated Show resolved Hide resolved
@benvillalobos benvillalobos added the merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now. label Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ProjectReference should negotiate SetPlatform metadata similar to SetTargetFramework
9 participants