Support gathering additional arbitrary properties from referenced projects #5994

dsplaisted · 2020-12-31T03:04:33Z

This allows additional properties to be gathered from project references via the project reference protocol. This is in order to support generating an error when a self-contained Executable references a non-SelfContained Executable, or vice versa, as described in dotnet/sdk#15117.

The referenced project can declare what additional properties should be gathered with AdditionalTargetFrameworkInfoProperty items:

  <ItemGroup>
    <AdditionalTargetFrameworkInfoProperty Include="SelfContained"/>
    <AdditionalTargetFrameworkInfoProperty Include="_IsExecutable"/>
  </ItemGroup>

These items will then be included in the AdditionalPropertiesFromProject metadata on the _MSBuildProjectReferenceExistent items. This value will have PropertyName=PropertyValue combinations joined by semicolons, and those sets of properties for the different TargetFramework values will be joined by double semicolons. For example, a project multitargeted to two TargetFrameworks may have the following for the AdditionalPropertiesFromProject metadata:

SelfContained=true;_IsExecutable=true;;SelfContained=false;_IsExecutable=true

Finding the right set of properties from the referenced project will required looking up the index of the NearestTargetFramework in the TargetFrameworks, and using that index to select the set of properties in the AdditionalPropertiesFromProject metadata. For example:

string nearestTargetFramework = project.GetMetadata("NearestTargetFramework");
int targetFrameworkIndex = project.GetMetadata("TargetFrameworks").Split(';').ToList().IndexOf(nearestTargetFramework);
string projectAdditionalPropertiesMetadata = project.GetMetadata("AdditionalPropertiesFromProject").Split(new[] { ";;" }, StringSplitOptions.None)[targetFrameworkIndex];
Dictionary<string, string> projectAdditionalProperties = new(StringComparer.OrdinalIgnoreCase);
foreach (var propAndValue in projectAdditionalPropertiesMetadata.Split(';'))
{
    var split = propAndValue.Split('=');
    projectAdditionalProperties[split[0]] = split[1];
}

This is implemented in dotnet/sdk#15134.

If anyone has suggestions for a better separator than a double semicolon to separate the list of property/values for each TargetFramework, let me know.

…jects

Forgind

I'm not a fan of relying on ;; as a separator, since a lot of people add ; to the beginning or end of lists if they aren't sure whether a property is empty or not like $(P);$(Q);(R) if P and Q happen to be empty, and that would confuse parsing it. Perhaps something a little more complicated like (TF1);;(TF2);;(TF3)? It's also a bit annoying to have to do so much parsing. Makes me wish we had item metadata metadata 😉
How annoying would it be to have a collection of Items like:

<ItemGroup>
<Net472AdditionalPropertiesFromProject>
<...>
</Net472AdditionalPropertiesFromProject>
<Netcoreapp3.1AdditionalPropertiesFromProject>
<...>
</Netcoreapp3.1AdditionalPropertiesFromProject>
</ItemGroup>

? Is it even possible to pass that through a ProjectReference?

src/Tasks/Microsoft.Common.CurrentVersion.targets

dsplaisted · 2020-12-31T19:21:44Z

@Forgind I think it will handle empty properties and empty lists of properties correctly, though I'm not 100% sure.

Something like Net472AdditionalPropertiesFromProject won't work, as it needs to collect properties from whatever target frameworks the referenced project defines, which is an open set. See dotnet/sdk#15134 for how this would be used.

Forgind · 2020-12-31T21:14:27Z

This was my case:

<PropertyGroup>
<EmptyProperty Condition="false">x</EmptyProperty>
<NotEmpty Condition="true">value</NotEmpty>
<AlsoNotEmpty Condition="true">otherVal</AlsoNotEmpty>
</PropertyGroup>
...
<PropertyGroup>
<UsesProperties>$(AlsoNotEmpty);$(EmptyProperty);$(NotEmpty)</UsesProperties>
</PropertyGroup>
...

and then trying to pass that to another project. I'm not sure either, mostly because I get confused sometimes by item transforms and such. As I understand it, it would accept UsesProperties as a valid property (which it is) and add it to AdditionalPropertiesFromProject, then parsing would break the property in two, get confused, and silently use it incorrectly. I could be wrong.

dsplaisted · 2021-01-06T01:15:37Z

@cdmihai How would this interact with static graph?

rainersigwald · 2021-01-06T21:09:55Z

High-level questions (this implementation looks ok but I wonder if we can improve the overall design).

Would this be better as one-metadatum-per-TF? With names like AdditionalPropertiesFromProject_net472? That'd be somewhat harder to assemble but avoids (one layer of) the delimiting problem.
Another option would be to pass it back as an XML blob. That's what solutions do for similar reasons.

Those are both harder to implement with MSBuild logic and would suggest writing a new AggregateMultitargetingInformation task, but that'd probably be easy to implement and understand.

If we stick with delimited, maybe consider using one of the fancier unicode characters like INFORMATION SEPARATOR TWO?

cdmihai · 2021-01-06T23:33:34Z

Looks fine for static graph, as the msbuild task calling patterns haven't changed, just the data ferried by the calls, which is opaque to static graph builds.

cdmihai · 2021-01-06T23:36:10Z

Can you also update the protocol doc in this PR?

Forgind · 2021-01-22T23:57:58Z

Team triage:
@dsplaisted
Just checking whether the feedback on this was clear—we don't think this is mergeable as-is without one of the changes rainersigwald proposed. If it's just that there are other things higher on your priority list, carry on.

dsplaisted · 2021-01-26T06:51:36Z

@Forgind I've updated this to use Xml to pass the data around

Forgind

Some nits, but this looks good, thanks!

Forgind · 2021-01-26T16:49:51Z

src/Tasks/CombineTargetFrameworkInfoProperties.cs

@@ -0,0 +1,34 @@
+using Microsoft.Build.Framework;


copyright header

Forgind · 2021-01-26T16:49:59Z

src/Tasks/CombineXmlElements.cs

@@ -0,0 +1,34 @@
+using Microsoft.Build.Framework;


copyright header

Forgind · 2021-01-26T23:15:11Z

src/Tasks/CombineTargetFrameworkInfoProperties.cs

+
+            Result = root.ToString();
+
+            return true;


return !HasLoggedErrors? Same below.

Forgind · 2021-01-26T23:17:34Z

src/Tasks/CombineXmlElements.cs

+{
+    public class CombineXmlElements : TaskExtension
+    {
+        public string RootElementName { get; set; }


Required annotations for these?

Forgind · 2021-01-26T23:19:33Z

src/Tasks/Microsoft.Common.CurrentVersion.targets

@@ -1743,12 +1751,26 @@ Copyright (C) Microsoft Corporation. All rights reserved.
  <Target Name="GetTargetFrameworksWithPlatformForSingleTargetFramework"
        Returns="@(_TargetFrameworkInfo)">

+    <ItemGroup>
+      <_AdditionalTargetFrameworkInfoPropertyWithValue Include="@(AdditionalTargetFrameworkInfoProperty)">
+        <Value>$(%(AdditionalTargetFrameworkInfoProperty.Identity))</Value>


I didn't realize you could have a metadata reference as an identifier for a property. Neat!

cdmihai · 2021-01-27T02:08:37Z

Are the tests for this new flow in the sdk repo? If not, add some here?

dsplaisted · 2021-01-27T02:11:00Z

Are the tests for this new flow in the sdk repo? If not, add some here?

Yes, tests for this are in dotnet/sdk#15134

dsplaisted · 2021-01-29T20:12:03Z

I've applied the code review feedback and retargeted this to the master branch for VS 16.10.

ericstj · 2021-06-21T23:01:10Z

Looks like this broke WCF as it doesn't handle cases like portable-net45+win8+wp8

dsplaisted · 2021-06-22T01:09:31Z

Looks like this broke WCF as it doesn't handle cases like portable-net45+win8+wp8

@ericstj Do you have more details on how it broke? Is it something we need to fix? I don't know if we support the old PCL TFMs anymore.

ericstj · 2021-06-22T16:32:40Z

I filed #6603 to track the regression. It was due to an invalid XML character in the TargetFramework. I imagine a fix would be to make sure you escape the strings which seems like something you might want to do regardless.

I was able to help WCF find a workaround since they aren't actually building for PCLs (they are just trying to use CSProj + pack to correctly represent a package which overlaps with the PCL targeting packs). ericstj/wcf@66e68f2

… xml names such as including '+' (#6699) Fixes #6603 Context The ability to gather extra properties from referenced projects was added in #5994. It used XML transformations to couch that logic in a way that MSBuild could understand. Unfortunately, the XML-based logic assumed everything was a valid XML element. Ultimately, the goal of the change, at least as I understand it, was just to pass information. Changes Made TargetFrameworks had been XML elements, but some are not valid XML names. This was changed to being a property on the more generically named TargetFramework element. This also allows for escaping. Testing Verified that a simple case that had not previously worked now works and looked at what it produced in a debugger. Notes After this PR, the SDK can opt in. Then MSBuild can turn it on by default, and the SDK can stop specifying the property.

Support gathering additional arbitrary properties from referenced pro…

19284a3

…jects

dsplaisted mentioned this pull request Dec 31, 2020

Generate errors when referencing an Executable project with different SelfContained value dotnet/sdk#15134

Merged

dsplaisted requested review from rainersigwald, a team, Forgind and benvillalobos December 31, 2020 03:05

Forgind reviewed Dec 31, 2020

View reviewed changes

src/Tasks/Microsoft.Common.CurrentVersion.targets Outdated Show resolved Hide resolved

dsplaisted requested a review from cdmihai January 6, 2021 01:14

Use Xml for gathering additional properties from referenced projects

9ca4269

Forgind approved these changes Jan 26, 2021

View reviewed changes

dsplaisted added 2 commits January 26, 2021 18:42

Fix tasks when inputs not set

63cd06e

Apply code review feedback

8682eb1

dsplaisted changed the base branch from vs16.9 to master January 29, 2021 20:11

cdmihai approved these changes Jan 30, 2021

View reviewed changes

KirillOsenkov approved these changes Jan 30, 2021

View reviewed changes

Forgind added the merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now. label Feb 1, 2021

Forgind merged commit 83cd7d4 into dotnet:master Feb 3, 2021

Forgind mentioned this pull request Jul 21, 2021

Update schema for combining TargetFramework info to allow for invalid xml names such as including '+' #6699

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support gathering additional arbitrary properties from referenced projects #5994

Support gathering additional arbitrary properties from referenced projects #5994

dsplaisted commented Dec 31, 2020

Forgind left a comment •

edited

dsplaisted commented Dec 31, 2020

Forgind commented Dec 31, 2020 •

edited by dsplaisted

dsplaisted commented Jan 6, 2021

rainersigwald commented Jan 6, 2021

cdmihai commented Jan 6, 2021

cdmihai commented Jan 6, 2021

Forgind commented Jan 22, 2021

dsplaisted commented Jan 26, 2021

Forgind left a comment

Forgind Jan 26, 2021

Forgind Jan 26, 2021

Forgind Jan 26, 2021

Forgind Jan 26, 2021

Forgind Jan 26, 2021

cdmihai commented Jan 27, 2021

dsplaisted commented Jan 27, 2021

dsplaisted commented Jan 29, 2021

ericstj commented Jun 21, 2021

dsplaisted commented Jun 22, 2021

ericstj commented Jun 22, 2021

Support gathering additional arbitrary properties from referenced projects #5994

Support gathering additional arbitrary properties from referenced projects #5994

Conversation

dsplaisted commented Dec 31, 2020

Forgind left a comment • edited

Choose a reason for hiding this comment

dsplaisted commented Dec 31, 2020

Forgind commented Dec 31, 2020 • edited by dsplaisted

dsplaisted commented Jan 6, 2021

rainersigwald commented Jan 6, 2021

cdmihai commented Jan 6, 2021

cdmihai commented Jan 6, 2021

Forgind commented Jan 22, 2021

dsplaisted commented Jan 26, 2021

Forgind left a comment

Choose a reason for hiding this comment

Forgind Jan 26, 2021

Choose a reason for hiding this comment

Forgind Jan 26, 2021

Choose a reason for hiding this comment

Forgind Jan 26, 2021

Choose a reason for hiding this comment

Forgind Jan 26, 2021

Choose a reason for hiding this comment

Forgind Jan 26, 2021

Choose a reason for hiding this comment

cdmihai commented Jan 27, 2021

dsplaisted commented Jan 27, 2021

dsplaisted commented Jan 29, 2021

ericstj commented Jun 21, 2021

dsplaisted commented Jun 22, 2021

ericstj commented Jun 22, 2021

Forgind left a comment •

edited

Forgind commented Dec 31, 2020 •

edited by dsplaisted