Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Working around LibGitSharp2 not working on arm64? #3723

Closed
omajid opened this issue Aug 9, 2019 · 3 comments · Fixed by dotnet/core-setup#7696
Closed

Working around LibGitSharp2 not working on arm64? #3723

omajid opened this issue Aug 9, 2019 · 3 comments · Fixed by dotnet/core-setup#7696
Labels
arch-arm64 area-Infrastructure-installer

Comments

@omajid
Copy link
Member

omajid commented Aug 9, 2019

I am trying to build core-setup (commit e4c55203) on arm64.

$ cat /etc/os-release 
NAME="Red Hat Enterprise Linux"
VERSION="8.0 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.0"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.0 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.0:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.0
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.0"

Aside from a number of other issues, ./build.sh gives me this error:

/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error : The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Core.NativeMethods.git_buf_free(GitBuf buf) [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Core.Proxy.git_buf_free(GitBuf buf) [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Core.Handles.GitBuf.Dispose() [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Core.Proxy.ConvertPath(Func`2 pathRetriever) [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Core.Proxy.git_repository_discover(FilePath start_path) [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at LibGit2Sharp.Repository.Discover(String startingPath) [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at Microsoft.Build.Tasks.Git.GitOperations.LocateRepository(String directory) in /_/src/Microsoft.Build.Tasks.Git.Operations/GitOperations.cs:line 26 [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.build.tasks.git/1.0.0-beta2-18618-05/build/Microsoft.Build.Tasks.Git.targets(20,5): error :    at Microsoft.Build.Tasks.Git.RepositoryTasks.LocateRepository(LocateRepository task) in /_/src/Microsoft.Build.Tasks.Git.Operations/RepositoryTasks.cs:line 58 [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]
/home/omajid/core-setup/.packages/microsoft.sourcelink.common/1.0.0-beta2-18618-05/build/Microsoft.SourceLink.Common.targets(53,5): error : Source control information is not available - the generated source link is empty. [/home/omajid/core-setup/tools-local/tasks/core-setup.tasks.csproj]

I know there are multiple fixes (some in the works) to get this working on arm64:

Can someone help me get one of these fixes to be consumed by core-setup?
@omajid omajid changed the title Using a LibGitSharp2 that works on arm64? Working around LibGitSharp2 not working on arm64? Aug 9, 2019
@dagood
Copy link
Member

dagood commented Aug 9, 2019

The version comes from here:

https://github.com/dotnet/core-setup/blob/e4c5520372fe36552a67ee1d51c1c08fba9aa316/eng/Versions.props#L89-L90

I'd kind of expect that to come from Arcade, so the best solution might be to remove that and use an Arcade default... but maybe doing a one-time upgrade is fine as a fix for this particular problem?

omajid referenced this issue in omajid/dotnet-core-setup Aug 13, 2019
@omajid
Partially enable hosted builds on arm64
adf9950 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide whether to default to
  x64 or to switch to arm64 when TargetArchitecture is not specified

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need filter the list of
  found libclrjit.so files to pick the normal-RID (eg, linux-arm64)
  libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/5.0.0-alpha1.19403.2/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/5.0.0-alpha1.19403.2/runtimes/x64_arm64/native/libclrjit.so

This commit does not fix the issue where sourcelink is broken on arm64
and so the build fails because of missing information about sources. See
https://github.com/dotnet/core-setup/issues/7653 for more information
about that.
omajid referenced this issue in omajid/dotnet-core-setup Aug 13, 2019
@omajid
Enable hosted builds on arm64
3dc5d66 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide whether to default to
  x64 or to switch to arm64 when TargetArchitecture is not specified

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to filter the list of
  found libclrjit.so files to pick the normal-RID (eg, linux-arm64)
  libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
omajid referenced this issue in omajid/dotnet-core-setup Aug 13, 2019
@omajid
Enable hosted builds on arm64
2b71679 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide whether to default to
  x64 or to switch to arm64 when TargetArchitecture is not specified

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to filter the list of
  found libclrjit.so files to pick the normal-RID (eg, linux-arm64)
  libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
omajid referenced this issue in omajid/dotnet-core-setup Aug 13, 2019
@omajid
Enable hosted builds on arm64
9b7c172 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide whether to default to
  x64 or to switch to arm64 when TargetArchitecture is not specified

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to filter the list of
  found libclrjit.so files to pick the normal-RID (eg, linux-arm64)
  libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
omajid referenced this issue in omajid/dotnet-core-setup Aug 13, 2019
@omajid
Enable hosted builds on arm64
b957d52 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide whether to default to
  x64 or to switch to arm64 when TargetArchitecture is not specified

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to filter the list of
  found libclrjit.so files to pick the normal-RID (eg, linux-arm64)
  libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
omajid referenced this issue in omajid/dotnet-core-setup Aug 14, 2019
@omajid
Enable hosted builds on arm64
6a9b9ba 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide the HostArch. Use that
  as the default value of TargetArchitecture, unless another value was
  provided by the build configuration.

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to pick the normal-RID
  (eg, linux-arm64) libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
dagood referenced this issue in dotnet/core-setup Aug 15, 2019
@omajid @dagood
Enable hosted builds on arm64 (#7696)
4b82b1e 
To enable building core-setup on arm64 (hosted, not cross compiled), we
need to do a few things:

- Set the right TargetArchitecture

  Use the currently running architecture to decide the HostArch. Use that
  as the default value of TargetArchitecture, unless another value was
  provided by the build configuration.

- Use the right coreclr JIT

  If we are cross-compiling, we need to use the x86_arm64 libclrjit.so.
  But if we are building on an arm64 host, we need to pick the normal-RID
  (eg, linux-arm64) libclrjit.so from the two:

    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/linux-arm64/native/libclrjit.so
    ./.packages/transport.runtime.linux-arm64.microsoft.netcore.jit/###/runtimes/x64_arm64/native/libclrjit.so

- Use a version of SourceLink that supports arm64

  We need to upgrade SourceLink to a version that contains
  dotnet/sourcelink#288. This commit just
  updates it to the latest version.

Fixes #7653
@msftgits msftgits transferred this issue from dotnet/core-setup Jan 30, 2020
@xiangzhai
Copy link
Contributor

Hi,

How to support MIPS64 bootstrap?

$ ./build.sh 
Commencing CoreCLR Repo build
__DistroRid: linux-mips64
__RuntimeId: linux-mips64
Setting up directories for build
Checking prerequisites...
/home/loongson/zhaixiang/coreclr-mips64-dev/eng/common/tools.sh: line 359: /pipeline-logging-functions.sh: No such file or directory
/home/loongson/zhaixiang/coreclr-mips64-dev/eng/common/tools.sh: line 359: /pipeline-logging-functions.sh: No such file or directory
Laying out dynamically generated EventSource classes
Adding CMake flags to build native tests for Linux.mips64.Debug
Commencing build of CoreCLR component for Linux.mips64.Debug in /home/loongson/zhaixiang/coreclr-mips64-dev/bin/obj/Linux.mips64.Debug
/home/loongson/zhaixiang/coreclr-mips64-dev
  Restore completed in 612.39 ms for /home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj.
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error : The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Core.NativeMethods.git_buf_free(GitBuf buf) [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Core.Proxy.git_buf_free(GitBuf buf) [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Core.Handles.GitBuf.Dispose() [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Core.Proxy.ConvertPath(Func`2 pathRetriever) [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Core.Proxy.git_repository_discover(FilePath start_path) [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at LibGit2Sharp.Repository.Discover(String startingPath) [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at Microsoft.Build.Tasks.Git.GitOperations.LocateRepository(String directory) in /_/src/Microsoft.Build.Tasks.Git.Operations/GitOperations.cs:line 26 [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.build.tasks.git/1.0.0-beta2-19270-01/build/Microsoft.Build.Tasks.Git.targets(16,5): error :    at Microsoft.Build.Tasks.Git.RepositoryTasks.LocateRepository(LocateRepository task) in /_/src/Microsoft.Build.Tasks.Git.Operations/RepositoryTasks.cs:line 58 [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]
/home/loongson/zhaixiang/coreclr-mips64-dev/.packages/microsoft.dotnet.arcade.sdk/1.0.0-beta.19319.25/tools/Version.targets(53,5): error : SourceRevisionId is not set, which means the SourceLink targets are not included in the build. Those are needed to produce a correct sha for our build outputs. [/home/loongson/zhaixiang/coreclr-mips64-dev/eng/empty.csproj]

Build FAILED.

There are MIPS64 EA SDKs: https://github.com/gsvm/loongson-dotnet/releases

Thanks,
Leslie Zhai

@xiangzhai
Copy link
Contributor

Workaround for MIPS64:

diff --git a/global.json b/global.json
index 34fb77b..47b3d91 100644
--- a/global.json
+++ b/global.json
@@ -1,15 +1,16 @@
 {
   "tools": {
-    "dotnet": "3.0.101"
+    "dotnet": "3.1.106"
   },
   "native-tools": {
-    "cmake": "3.11.1",
+    "cmake": "3.14.2",
     "python": "2.7.15"
   },
   "msbuild-sdks": {
-    "Microsoft.DotNet.Arcade.Sdk": "1.0.0-beta.19319.25",
-    "Microsoft.DotNet.Helix.Sdk": "2.0.0-beta.19319.25",
+    "Microsoft.DotNet.Arcade.Sdk": "5.0.0-beta.19562.5",
+    "Microsoft.DotNet.Helix.Sdk": "5.0.0-beta.19562.5",
     "Microsoft.Build.NoTargets": "1.0.53",
-    "Microsoft.Build.Traversal": "2.0.2"
+    "Microsoft.Build.Traversal": "2.0.2",
+    "Microsoft.NET.Sdk.IL": "5.0.0-alpha1.19413.7"
   }
 }

Cheers,
Leslie Zhai

@dotnet dotnet locked as resolved and limited conversation to collaborators Dec 12, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-arm64 area-Infrastructure-installer
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable hosted builds on arm64 omajid/dotnet-core-setup
3 participants
@omajid @xiangzhai @dagood