Enable "portable" RID graph when targeting .NET 8 and higher #34279

dsplaisted · 2023-07-26T16:20:59Z

Resolves #34033

Projects targeting .NET 8 and higher will by default use the smaller, "portable" RuntimeIdentifier graph. They can opt in to the old, full RID graph by setting UseRidGraph to true. Likewise, projects targeting .NET 7 and earlier can use the trimmed-down RID graph by setting UseRidGraph to false (there may not be any scenario where this is needed though).

Additional runtime identifiers can be added to the portable RuntimeIdentifier graph when building the .NET SDK by defining AdditionalRuntimeIdentifier items in the redist.csproj project. This is to support source-built versions of the .NET SDK to add their own Runtime Identifiers. Here is an example of the format for these items:

 <ItemGroup>
   <AdditionalRuntimeIdentifier Include="ubuntu.14-x64" Imports="linux-x64" />
   <AdditionalRuntimeIdentifier Include="rhel-x64" Imports="linux-x64;linux" />
 </ItemGroup>

Forgind

Looks good to me! Just a few little comments 🙂

Forgind · 2023-07-26T17:05:46Z

src/Layout/toolset-tasks/UpdatePortableRuntimeIdentifierGraph.cs

+
+        public override bool Execute()
+        {
+            JToken json;


Should we have a "fast path" here for if AdditionalRuntimeIdentifiers is null? Then it should just be a copy, right?

Forgind · 2023-07-26T17:09:10Z

src/Layout/toolset-tasks/UpdatePortableRuntimeIdentifierGraph.cs

+                foreach (var rid in AdditionalRuntimeIdentifiers)
+                {
+                    var importedRids = rid.GetMetadata("Imports").Split(';');
+                    runtimes.Add(rid.ItemSpec, new JObject(new JProperty("#import", new JArray(importedRids))));


I think this is correct and updates json["runtimes"] rather than making a copy...but can we have a regression test for that? Or perhaps just add json["runtimes"] = runtimes at the end?

Forgind · 2023-07-26T17:12:33Z

src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.targets

@@ -48,6 +48,19 @@ Copyright (c) .NET Foundation. All rights reserved.
    <_GenerateSingleFileBundlePropertyInputsCache>$([MSBuild]::NormalizePath($(MSBuildProjectDirectory), $(_GenerateSingleFileBundlePropertyInputsCache)))</_GenerateSingleFileBundlePropertyInputsCache>
  </PropertyGroup>

+  <!-- For .NET 8 and higher, we will by default use a simplified "portable" RID graph -->
+  <PropertyGroup Condition="'$(UseRidGraph)' == ''">
+    <UseRidGraph Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp' AND $([MSBuild]::VersionGreaterThanOrEquals($(TargetFrameworkVersion), '8.0'))">false</UseRidGraph>


I was tripped up on this once and almost said this is incorrect when it's correct. UseRidGraph to me sounds like it means use the new thing, especially since the new thing is the "portable" rid graph...can we add portable to this name and flip its true/falseness? Or add full and not flip it?

tmds · 2023-07-26T21:18:30Z

This is to support source-built versions of the .NET SDK to add their own Runtime Identifiers

Is something needed (in addition to this PR) to use this during source-build?

cc @MichaelSimons

dsplaisted · 2023-07-27T01:33:55Z

This is to support source-built versions of the .NET SDK to add their own Runtime Identifiers

Is something needed (in addition to this PR) to use this during source-build?

cc @MichaelSimons

Yes, there will need to be logic during source-build to add an AdditionalRuntimeIdentifier item for the source-build specific RuntimeIdentifier.

tmds · 2023-07-27T21:10:02Z

Yes, there will need to be logic during source-build to add an AdditionalRuntimeIdentifier item for the source-build specific RuntimeIdentifier.

Will someone look into this? Should we create an issue to track it?

richlander · 2023-07-27T23:32:48Z

I assume this will make RC1. Yes?

dsplaisted · 2023-07-28T00:09:08Z

I assume this will make RC1. Yes?

Not necessarily, it somewhat depends on dotnet/runtime#89598

richlander · 2023-07-28T00:33:16Z

We should really push to make RC1. This change is pretty significant so it gets more concerning after that. Thoughts @elinor-fung?

ViktorHofer · 2023-07-31T10:10:44Z

I assume NuGet will use the minified graph as well for .NET 8 targeting projects with this change? How will it be able to find RID specific assets, i.e. runtimes/win/lib/net8.0/a.dll? Asking for dotnet/runtime packages which use RID specific assets and for APICompat / PackageValidation which uses NuGet APIs that require the current RID graph:

sdk/src/ApiCompat/Microsoft.DotNet.PackageValidation/Package.cs

Line 100 in dff005c

_conventions = new ManagedCodeConventions(s_runtimeGraph);

richlander · 2023-07-31T15:59:24Z

It can still find portable RIDs like linux, linux-musl, osx, and win. It won't be able to find alpine or debian.

dsplaisted · 2023-08-02T15:39:02Z

With #34349, these test failures may now be fixed.

/azp run

dsplaisted · 2023-08-02T15:40:35Z

/azp run

azure-pipelines · 2023-08-02T15:40:47Z

Azure Pipelines successfully started running 1 pipeline(s).

dsplaisted · 2023-08-04T00:34:57Z

I've merged this, so it should make it into RC1.

We have the following breaking change issue: dotnet/docs#36466

I think we should also have breaking change notices for the following:

The original runtime behavior change where looking up native assets
This SDK change where we use the portable RID graph by default

Do we have breaking change issues filed for these? Maybe we should have one issue that describes all of these related changes, and if not they should at least be linked somehow.

@richlander @baronfel

tmds · 2023-08-04T05:37:43Z

@dsplaisted is there an open issue for updating the graph to include the non-portable rid during source-build?

Fixes #90000 Follow-up on dotnet/sdk#34279

richlander · 2023-08-04T15:36:41Z

If there isn't one, can you create one @tmds ?

dsplaisted · 2023-08-04T15:48:21Z

dotnet/source-build#3578 is also related to these overall changes, so we could use that to track adding RIDs to the graph during source-build if we want.

elinor-fung · 2023-08-04T17:39:40Z

This is the breaking change notice for the runtime behaviour change for looking up RID-specific assets: https://learn.microsoft.com/dotnet/core/compatibility/deployment/8.0/rid-asset-list

I opened dotnet/docs#36527 for this SDK change. @dsplaisted could you fill it in with more details?

ghost · 2023-08-08T21:54:12Z

Added needs-breaking-change-doc-created label because this PR has the breaking-change label.

When you commit this breaking change:

Create and link to this PR and the issue a matching issue in the dotnet/docs repo using the breaking change documentation template, then remove this needs-breaking-change-doc-created label.
Ask a committer to mail the .NET Breaking Change Notification DL.

Tagging @dotnet/compat for awareness of the breaking change.

dsplaisted · 2023-08-08T22:04:10Z

I've updated the breaking change issue with the details on how to revert to the old build behavior.

mthalman · 2023-08-15T15:58:50Z

@dsplaisted - When you refer to redist.csproj as the location for where these AdditionalRuntimeIdentifier items are to be placed, where is that exactly? Is it this: https://github.com/dotnet/installer/blob/main/src/redist/redist.csproj?

dsplaisted · 2023-08-15T16:20:02Z

@dsplaisted - When you refer to redist.csproj as the location for where these AdditionalRuntimeIdentifier items are to be placed, where is that exactly? Is it this: https://github.com/dotnet/installer/blob/main/src/redist/redist.csproj?

@mthalman It actually needs to go in the project in the SDK repo: https://github.com/dotnet/sdk/blob/main/src/Layout/redist/redist.csproj

I would put it in or near the PublishPortableRuntimeIdentifierGraph target in https://github.com/dotnet/sdk/blob/main/src/Layout/redist/targets/GenerateLayout.targets, which is imported by the redist project.

dsplaisted requested review from AntonLapounov and a team as code owners July 26, 2023 16:20

dotnet-issue-labeler bot added Area-Infrastructure untriaged Request triage from a team member labels Jul 26, 2023

Forgind approved these changes Jul 26, 2023

View reviewed changes

MiYanni approved these changes Jul 27, 2023

View reviewed changes

dsplaisted mentioned this pull request Jul 28, 2023

Stop using computed RID for RUNTIME_IDENTIFIER property dotnet/runtime#89598

Merged

elinor-fung mentioned this pull request Aug 1, 2023

[Breaking change]: RuntimeInformation.RuntimeIdentifier returns platform for which the runtime was built dotnet/docs#36466

Closed

3 tasks

kasperk81 mentioned this pull request Aug 1, 2023

[main] Update dependencies from dotnet/runtime #34349

Merged

Enable "portable" RID graph

1df8f4f

dsplaisted force-pushed the use-portable-RID-graph branch from 83efef8 to 1df8f4f Compare August 3, 2023 16:50

dsplaisted added 2 commits August 3, 2023 15:34

Fix tests to account for portable RID graph

894644d

Fix test for non-portable RID

177651f

dsplaisted merged commit e626b7f into dotnet:main Aug 4, 2023
16 checks passed

ViktorHofer mentioned this pull request Aug 4, 2023

Delete the non portable RID graph in a future release of .NET dotnet/runtime#90000

Open

ViktorHofer added a commit to dotnet/runtime that referenced this pull request Aug 4, 2023

Remove Microsoft.NetCore.Platforms projects

1cfa553

Fixes #90000 Follow-up on dotnet/sdk#34279

elinor-fung mentioned this pull request Aug 4, 2023

[Breaking change]: Projects targeting .NET 8 and higher will by default use a smaller, portable RID graph. dotnet/docs#36527

Closed

3 tasks

dsplaisted added the breaking-change Using this label will notify dotnet/compat and trigger a request to file a compat bug label Aug 8, 2023

ghost added the needs-breaking-change-doc-created label Aug 8, 2023

tmds mentioned this pull request Aug 8, 2023

The non-portable rid needs to be added to the trimmed down rid graph dotnet/source-build#3584

Closed

jkotas mentioned this pull request Aug 14, 2023

NETSDK1094 errors in tests when using .NET 8 RC1 SDK dotnet/source-build#3591

Closed

elinor-fung mentioned this pull request Aug 15, 2023

Update scripts to set distro RID and properties to use it as TargetRid dotnet/installer#17185

Merged

marcpopMSFT mentioned this pull request Aug 15, 2023

[NETSDKE2E] [Regression]With 8.0.100-rc.1.23414.11 installed, the RID shows incorrectly on Linux Alpine3.18 Arm64 #34627

Closed

This was referenced Aug 24, 2023

Use RuntimeIdentifierGraphPath if available in runtime copy of Microsoft.NET.CrossGen.targets dotnet/runtime#91077

Merged

[release/8.0] Use RuntimeIdentifierGraphPath if available in runtime copy of Microsoft.NET.CrossGen.targets dotnet/runtime#91090

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable "portable" RID graph when targeting .NET 8 and higher #34279

Enable "portable" RID graph when targeting .NET 8 and higher #34279

dsplaisted commented Jul 26, 2023

Forgind left a comment

Forgind Jul 26, 2023

Forgind Jul 26, 2023

Forgind Jul 26, 2023

tmds commented Jul 26, 2023

dsplaisted commented Jul 27, 2023

tmds commented Jul 27, 2023

richlander commented Jul 27, 2023

dsplaisted commented Jul 28, 2023

richlander commented Jul 28, 2023

ViktorHofer commented Jul 31, 2023 •

edited

richlander commented Jul 31, 2023

dsplaisted commented Aug 2, 2023

dsplaisted commented Aug 2, 2023

azure-pipelines bot commented Aug 2, 2023

dsplaisted commented Aug 4, 2023

tmds commented Aug 4, 2023

richlander commented Aug 4, 2023

dsplaisted commented Aug 4, 2023

elinor-fung commented Aug 4, 2023

ghost commented Aug 8, 2023

dsplaisted commented Aug 8, 2023

mthalman commented Aug 15, 2023

dsplaisted commented Aug 15, 2023

Enable "portable" RID graph when targeting .NET 8 and higher #34279

Enable "portable" RID graph when targeting .NET 8 and higher #34279

Conversation

dsplaisted commented Jul 26, 2023

Forgind left a comment

Choose a reason for hiding this comment

Forgind Jul 26, 2023

Choose a reason for hiding this comment

Forgind Jul 26, 2023

Choose a reason for hiding this comment

Forgind Jul 26, 2023

Choose a reason for hiding this comment

tmds commented Jul 26, 2023

dsplaisted commented Jul 27, 2023

tmds commented Jul 27, 2023

richlander commented Jul 27, 2023

dsplaisted commented Jul 28, 2023

richlander commented Jul 28, 2023

ViktorHofer commented Jul 31, 2023 • edited

richlander commented Jul 31, 2023

dsplaisted commented Aug 2, 2023

dsplaisted commented Aug 2, 2023

azure-pipelines bot commented Aug 2, 2023

dsplaisted commented Aug 4, 2023

tmds commented Aug 4, 2023

richlander commented Aug 4, 2023

dsplaisted commented Aug 4, 2023

elinor-fung commented Aug 4, 2023

ghost commented Aug 8, 2023

dsplaisted commented Aug 8, 2023

mthalman commented Aug 15, 2023

dsplaisted commented Aug 15, 2023

ViktorHofer commented Jul 31, 2023 •

edited