Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JIT] Imbue stringbuf with C locale #79929

Closed
wants to merge 1 commit into from

Conversation

malfet
Copy link
Contributor

@malfet malfet commented Jun 21, 2022

To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

Fixes #79583

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jun 21, 2022

🔗 Helpful links

❌ 5 New Failures, 2 Base Failures

As of commit 05f28af (more details on the Dr. CI page):

Expand to see more

🕵️ 5 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages

See GitHub Actions build linux-binary-manywheel / manywheel-py3_10-cpu-test (1/5)

Step: "Checkout PyTorch" (full log | diagnosis details | 🔁 rerun)

2022-06-21T06:42:09.6591201Z ##[error]An error ...torch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:42:09.6555232Z   PYTORCH_FINAL_PACKAGE_DIR: /artifacts
2022-06-21T06:42:09.6555426Z   PYTORCH_ROOT: /pytorch
2022-06-21T06:42:09.6555644Z   SHA1: 05f28af9c0574e402d3ce38cd2ea30bb7bd69d2c
2022-06-21T06:42:09.6555851Z   SKIP_ALL_TESTS: 1
2022-06-21T06:42:09.6556054Z   PACKAGE_TYPE: manywheel
2022-06-21T06:42:09.6556226Z   DESIRED_CUDA: cpu
2022-06-21T06:42:09.6556406Z   GPU_ARCH_TYPE: cpu
2022-06-21T06:42:09.6556623Z   DOCKER_IMAGE: pytorch/manylinux-builder:cpu
2022-06-21T06:42:09.6556839Z   DESIRED_PYTHON: 3.10
2022-06-21T06:42:09.6557006Z ##[endgroup]
2022-06-21T06:42:09.6591201Z ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/pytorch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:42:09.6605910Z ##[group]Run # Ensure the working directory gets chowned back to the current user
2022-06-21T06:42:09.6606248Z �[36;1m# Ensure the working directory gets chowned back to the current user�[0m
2022-06-21T06:42:09.6606566Z �[36;1mdocker run --rm -v "$(pwd)":/v -w /v "${ALPINE_IMAGE}" chown -R "$(id -u):$(id -g)" .�[0m
2022-06-21T06:42:09.6616789Z shell: /usr/bin/bash -e {0}
2022-06-21T06:42:09.6616974Z env:
2022-06-21T06:42:09.6617198Z   ALPINE_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine
2022-06-21T06:42:09.6617452Z   ANACONDA_USER: pytorch
2022-06-21T06:42:09.6617656Z   AWS_DEFAULT_REGION: us-east-1
2022-06-21T06:42:09.6617858Z   BINARY_ENV_FILE: /tmp/env
2022-06-21T06:42:09.6618069Z   BUILD_ENVIRONMENT: linux-binary-manywheel

See GitHub Actions build linux-binary-manywheel / manywheel-py3_9-cpu-test (2/5)

Step: "Checkout PyTorch" (full log | diagnosis details | 🔁 rerun)

2022-06-21T06:41:57.9477899Z ##[error]An error ...torch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:41:57.9441590Z   PYTORCH_FINAL_PACKAGE_DIR: /artifacts
2022-06-21T06:41:57.9441810Z   PYTORCH_ROOT: /pytorch
2022-06-21T06:41:57.9442030Z   SHA1: 05f28af9c0574e402d3ce38cd2ea30bb7bd69d2c
2022-06-21T06:41:57.9442250Z   SKIP_ALL_TESTS: 1
2022-06-21T06:41:57.9442474Z   PACKAGE_TYPE: manywheel
2022-06-21T06:41:57.9442659Z   DESIRED_CUDA: cpu
2022-06-21T06:41:57.9442846Z   GPU_ARCH_TYPE: cpu
2022-06-21T06:41:57.9443077Z   DOCKER_IMAGE: pytorch/manylinux-builder:cpu
2022-06-21T06:41:57.9443288Z   DESIRED_PYTHON: 3.9
2022-06-21T06:41:57.9443475Z ##[endgroup]
2022-06-21T06:41:57.9477899Z ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/pytorch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:41:57.9493460Z ##[group]Run # Ensure the working directory gets chowned back to the current user
2022-06-21T06:41:57.9493810Z �[36;1m# Ensure the working directory gets chowned back to the current user�[0m
2022-06-21T06:41:57.9494154Z �[36;1mdocker run --rm -v "$(pwd)":/v -w /v "${ALPINE_IMAGE}" chown -R "$(id -u):$(id -g)" .�[0m
2022-06-21T06:41:57.9504155Z shell: /usr/bin/bash -e {0}
2022-06-21T06:41:57.9504333Z env:
2022-06-21T06:41:57.9504578Z   ALPINE_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine
2022-06-21T06:41:57.9504837Z   ANACONDA_USER: pytorch
2022-06-21T06:41:57.9505035Z   AWS_DEFAULT_REGION: us-east-1
2022-06-21T06:41:57.9505243Z   BINARY_ENV_FILE: /tmp/env
2022-06-21T06:41:57.9505475Z   BUILD_ENVIRONMENT: linux-binary-manywheel

See GitHub Actions build linux-binary-manywheel / manywheel-py3_7-cpu-test (3/5)

Step: "Checkout PyTorch" (full log | diagnosis details | 🔁 rerun)

2022-06-21T06:42:46.0430736Z ##[error]An error ...torch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:42:46.0394981Z   PYTORCH_FINAL_PACKAGE_DIR: /artifacts
2022-06-21T06:42:46.0395187Z   PYTORCH_ROOT: /pytorch
2022-06-21T06:42:46.0395404Z   SHA1: 05f28af9c0574e402d3ce38cd2ea30bb7bd69d2c
2022-06-21T06:42:46.0395598Z   SKIP_ALL_TESTS: 1
2022-06-21T06:42:46.0395821Z   PACKAGE_TYPE: manywheel
2022-06-21T06:42:46.0395996Z   DESIRED_CUDA: cpu
2022-06-21T06:42:46.0396175Z   GPU_ARCH_TYPE: cpu
2022-06-21T06:42:46.0396394Z   DOCKER_IMAGE: pytorch/manylinux-builder:cpu
2022-06-21T06:42:46.0396599Z   DESIRED_PYTHON: 3.7
2022-06-21T06:42:46.0396780Z ##[endgroup]
2022-06-21T06:42:46.0430736Z ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/pytorch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:42:46.0445379Z ##[group]Run # Ensure the working directory gets chowned back to the current user
2022-06-21T06:42:46.0445715Z �[36;1m# Ensure the working directory gets chowned back to the current user�[0m
2022-06-21T06:42:46.0446029Z �[36;1mdocker run --rm -v "$(pwd)":/v -w /v "${ALPINE_IMAGE}" chown -R "$(id -u):$(id -g)" .�[0m
2022-06-21T06:42:46.0456273Z shell: /usr/bin/bash -e {0}
2022-06-21T06:42:46.0456452Z env:
2022-06-21T06:42:46.0456675Z   ALPINE_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine
2022-06-21T06:42:46.0456927Z   ANACONDA_USER: pytorch
2022-06-21T06:42:46.0457125Z   AWS_DEFAULT_REGION: us-east-1
2022-06-21T06:42:46.0457319Z   BINARY_ENV_FILE: /tmp/env
2022-06-21T06:42:46.0457526Z   BUILD_ENVIRONMENT: linux-binary-manywheel

See GitHub Actions build windows-binary-wheel / wheel-py3_9-cpu-test (4/5)

Step: "Checkout PyTorch" (full log | diagnosis details | 🔁 rerun)

2022-06-21T07:12:23.9185977Z ##[error]The proce...ram Files\Git\cmd\git.exe' failed with exit code 1
2022-06-21T07:12:23.2717452Z Synchronizing submodule url for 'third_party/zstd'
2022-06-21T07:12:23.3092461Z [command]"C:\Program Files\Git\cmd\git.exe" -c protocol.version=2 submodule update --init --force --depth=1 --recursive
2022-06-21T07:12:23.8144800Z Cloning into 'C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/eigen'...
2022-06-21T07:12:23.8145626Z ##[error]fatal: unable to access 'https://gitlab.com/libeigen/eigen.git/': The requested URL returned error: 500
2022-06-21T07:12:23.8146703Z ##[error]fatal: clone of 'https://gitlab.com/libeigen/eigen.git' into submodule path 'C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/eigen' failed
2022-06-21T07:12:23.8147323Z Failed to clone 'third_party/eigen'. Retry scheduled
2022-06-21T07:12:23.8978642Z Cloning into 'C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/eigen'...
2022-06-21T07:12:23.8979513Z ##[error]fatal: unable to access 'https://gitlab.com/libeigen/eigen.git/': The requested URL returned error: 500
2022-06-21T07:12:23.8980607Z ##[error]fatal: clone of 'https://gitlab.com/libeigen/eigen.git' into submodule path 'C:/actions-runner/_work/pytorch/pytorch/pytorch/third_party/eigen' failed
2022-06-21T07:12:23.8981245Z Failed to clone 'third_party/eigen' a second time, aborting
2022-06-21T07:12:23.9185977Z ##[error]The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 1
2022-06-21T07:12:23.9344234Z ##[group]Run .github\scripts\wait_for_ssh_to_drain.ps1
2022-06-21T07:12:23.9344569Z �[36;1m.github\scripts\wait_for_ssh_to_drain.ps1�[0m
2022-06-21T07:12:23.9366314Z shell: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command ". '{0}'"
2022-06-21T07:12:23.9366590Z env:
2022-06-21T07:12:23.9366834Z   ALPINE_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine
2022-06-21T07:12:23.9367076Z   ANACONDA_USER: pytorch
2022-06-21T07:12:23.9367278Z   AWS_DEFAULT_REGION: us-east-1
2022-06-21T07:12:23.9367511Z   BUILD_ENVIRONMENT: windows-binary-wheel
2022-06-21T07:12:23.9368086Z   GITHUB_TOKEN: ***
2022-06-21T07:12:23.9368262Z   PR_NUMBER: 

See GitHub Actions build linux-binary-manywheel / manywheel-py3_8-cpu-test (5/5)

Step: "Checkout PyTorch" (full log | diagnosis details | 🔁 rerun)

2022-06-21T06:43:18.4075832Z ##[error]An error ...torch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:43:18.4038702Z   PYTORCH_FINAL_PACKAGE_DIR: /artifacts
2022-06-21T06:43:18.4038940Z   PYTORCH_ROOT: /pytorch
2022-06-21T06:43:18.4039190Z   SHA1: 05f28af9c0574e402d3ce38cd2ea30bb7bd69d2c
2022-06-21T06:43:18.4039430Z   SKIP_ALL_TESTS: 1
2022-06-21T06:43:18.4039662Z   PACKAGE_TYPE: manywheel
2022-06-21T06:43:18.4039867Z   DESIRED_CUDA: cpu
2022-06-21T06:43:18.4040080Z   GPU_ARCH_TYPE: cpu
2022-06-21T06:43:18.4040329Z   DOCKER_IMAGE: pytorch/manylinux-builder:cpu
2022-06-21T06:43:18.4040566Z   DESIRED_PYTHON: 3.8
2022-06-21T06:43:18.4040778Z ##[endgroup]
2022-06-21T06:43:18.4075832Z ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/ec2-user/actions-runner/_work/pytorch/pytorch/pytorch/'. No such file or directory
2022-06-21T06:43:18.4091510Z ##[group]Run # Ensure the working directory gets chowned back to the current user
2022-06-21T06:43:18.4091880Z �[36;1m# Ensure the working directory gets chowned back to the current user�[0m
2022-06-21T06:43:18.4092225Z �[36;1mdocker run --rm -v "$(pwd)":/v -w /v "${ALPINE_IMAGE}" chown -R "$(id -u):$(id -g)" .�[0m
2022-06-21T06:43:18.4102787Z shell: /usr/bin/bash -e {0}
2022-06-21T06:43:18.4103001Z env:
2022-06-21T06:43:18.4103257Z   ALPINE_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine
2022-06-21T06:43:18.4103541Z   ANACONDA_USER: pytorch
2022-06-21T06:43:18.4103769Z   AWS_DEFAULT_REGION: us-east-1
2022-06-21T06:43:18.4103984Z   BINARY_ENV_FILE: /tmp/env
2022-06-21T06:43:18.4104238Z   BUILD_ENVIRONMENT: linux-binary-manywheel

🚧 2 fixed upstream failures:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Jun 21, 2022
To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:
```cpp

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

```
@malfet malfet force-pushed the malfet/ir-imbue-with-C-locale branch from 17db387 to 05f28af Compare June 21, 2022 06:04
@malfet malfet added the ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR label Jun 21, 2022
@malfet
Copy link
Contributor Author

malfet commented Jun 21, 2022

pip installing https://gha-artifacts.s3.amazonaws.com/pytorch/pytorch/2533367977/manywheel-py3_7-cuda11_6/torch-1.13.0.dev20220621%2Bcu116-cp37-cp37m-linux_x86_64.whl shows that problem is indeed gone

@malfet
Copy link
Contributor Author

malfet commented Jun 21, 2022

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here

@malfet malfet added this to the 1.12.0 milestone Jun 21, 2022
@pytorchmergebot
Copy link
Collaborator

Merge failed due to Refusing to merge as mandatory check(s) pull failed for rule superuser
Raised by https://github.com/pytorch/pytorch/actions/runs/2535856174

@malfet
Copy link
Contributor Author

malfet commented Jun 21, 2022

@pytorchbot merge -f

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a merge job. Check the current status here

@github-actions
Copy link

Hey @malfet.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@malfet malfet deleted the malfet/ir-imbue-with-C-locale branch June 21, 2022 13:58
@malfet malfet added release notes: jit release notes category topic: bug fixes topic category labels Jun 21, 2022
atalman pushed a commit to atalman/pytorch that referenced this pull request Jun 21, 2022
To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:
```cpp

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

```

Fixes pytorch#79583

Pull Request resolved: pytorch#79929
Approved by: https://github.com/davidberard98
facebook-github-bot pushed a commit that referenced this pull request Jun 22, 2022
Summary:
To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:
```cpp

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

```

Fixes #79583

Pull Request resolved: #79929
Approved by: https://github.com/davidberard98

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/4b6ba340e211503b758ea6c3c173e1e7bbd9c11c

Reviewed By: atalman

Differential Revision: D37316875

Pulled By: malfet

fbshipit-source-id: d3900f532930850bd4973274c3274cf31d8b8d09
atalman added a commit that referenced this pull request Jun 22, 2022
To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:
```cpp

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

```

Fixes #79583

Pull Request resolved: #79929
Approved by: https://github.com/davidberard98

Co-authored-by: Nikita Shulga <nshulga@fb.com>
miladm pushed a commit to miladm/pytorch that referenced this pull request Jun 27, 2022
To prevent 12345 become "12,345" if locale is not "C", as shown in the
following example:
```cpp

int main() {
  std::locale::global(std::locale("en_US.utf-8"));
  std::stringstream ss;
  ss << "12345 in " << std::locale().name()  << " locale is " << 12345 ;
  ss.imbue(std::locale("C"));
  ss << " but in C locale is " << 12345;
  std::cout << ss.str() << std::endl;
}

```

Fixes pytorch#79583

Pull Request resolved: pytorch#79929
Approved by: https://github.com/davidberard98
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR cla signed Merged oncall: jit Add this issue/PR to JIT oncall triage queue release notes: jit release notes category topic: bug fixes topic category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Torchscript Serialisation is broken for numbers over 1000 if global locale is set
4 participants