Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The gcc and g++ versions used to generate the wheel package uploaded to pypi in both x86 and aarch64 architecture #1415

Open
FFFrog opened this issue Jun 6, 2023 · 5 comments

Comments

@FFFrog
Copy link

FFFrog commented Jun 6, 2023

Hi, everyone.

In order to avoid the negative impact of gcc compatibility, so we want to learn about the the gcc and g++ versions used to generate the wheel package uploaded to pypi in both x86 and aarch64 architecture and try to keep consistent with community.

And we recently unpack the release wheel of torch from pypi.org and found that the gcc version in the aarch environment is 10.2.1, as shown below. In the same way, we also got the version of gcc in the x86 environment is 9.3.1.

However, the .comment section in the elf is not correct always, so please help to confirm whether the versions are correct, thank you a lot.

What needs to be added is that we found the docker image for building x86, but we did not find the build environment for aarch64.

Looking forward to your relpy.

@FFFrog
Copy link
Author

FFFrog commented Jun 6, 2023

@ptrblck, sorry to @you again, please take a loot at this, thank you a lot.

@ptrblck
Copy link
Collaborator

ptrblck commented Jun 6, 2023

What needs to be added is that we found the docker image for building x86, but we did not find the build environment for aarch64.

I'm not involved in the ARM builds, but would guess these images are used:

# AMI images for us-east-1, change the following based on your ~/.aws/config
os_amis = {
'ubuntu18_04': "ami-078eece1d8119409f", # login_name: ubuntu
'ubuntu20_04': "ami-052eac90edaa9d08f", # login_name: ubuntu
'ubuntu22_04': "ami-0c6c29c5125214c77", # login_name: ubuntu
'redhat8': "ami-0698b90665a2ddcf1", # login_name: ec2-user
}
ubuntu18_04_ami = os_amis['ubuntu18_04']

@FFFrog
Copy link
Author

FFFrog commented Jun 6, 2023

What needs to be added is that we found the docker image for building x86, but we did not find the build environment for aarch64.

I'm not involved in the ARM builds, but would guess these images are used:

# AMI images for us-east-1, change the following based on your ~/.aws/config
os_amis = {
'ubuntu18_04': "ami-078eece1d8119409f", # login_name: ubuntu
'ubuntu20_04': "ami-052eac90edaa9d08f", # login_name: ubuntu
'ubuntu22_04': "ami-0c6c29c5125214c77", # login_name: ubuntu
'redhat8': "ami-0698b90665a2ddcf1", # login_name: ec2-user
}
ubuntu18_04_ami = os_amis['ubuntu18_04']

Thanks a lot, we can split this question into two parts:

  1. Decompress the torch wheel package from pypi, and then get the version of gcc according to the .comment section of the dynamic library in whl. Is this method ok? I personally think that the community should not actively modify the .comment section through tools like patchelf.
  2. Is it possible to confirm that the gcc in the x86 environment is 9.3.1?

The two questions are very important to us, thank you.

@FFFrog
Copy link
Author

FFFrog commented Jun 7, 2023

@ptrblck , sorry to @you again, can you help answer the above two questions? Thank you a lot.

@weiwangudel
Copy link

You could take a look at the pytorch hud page https://hud.pytorch.org/hud/pytorch/pytorch/v2.0.1 It stores the github actions that built the binaries. You could then choose one workflow, e.g. https://github.com/pytorch/pytorch/actions/runs/4786980664/jobs/8511609240 and view the raw logs: https://pipelines.actions.githubusercontent.com/serviceHosts/7d146c05-69c3-4c20-a0e7-818111670117/_apis/pipelines/1/runs/2882401/signedlogcontent/22?urlExpires=2023-07-02T20%3A20%3A26.3734274Z&urlSigningMethod=HMACV1&urlSignature=tzb%2Bk%2FfRqlCrm19tr%2FDOP0AS%2Fj91SYQbvt8y5OlQqmA%3D
I could see "2023-04-24T13:47:20.3764365Z -- GCC 9.3.1: Adding gcc and gcc_s libs to link line" in the raw log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants