testscript: fix ptyName() returning /dev/pts/4294967296 on s390x #246
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use uint32 instead of uint (64-bit in Go on s390x) to store the return value of the TIOCGPTN syscall. This is to avoid the 32-bit value from being stored into a 64-bit buffer and get left-shifted by 32 when dereferencing, turning what should be e.g. /dev/pts/1 to /dev/pts/4294967296 on big-endian architectures such as s390x.
Special thanks to the explanation and a similar bug fix provided at containerd/console#51
Fixes the following TestScripts/pty error on s390x:
discovered by Debian autopkgtest CI for golang-github-rogpeppe-go-internal/1.12.0-1;
see log at https://ci.debian.net/packages/g/golang-github-rogpeppe-go-internal/testing/s390x/43022768/#L3047
(Hmm... I guess /dev/pts/21474836480 should have been /dev/pts/5, as
5 << 32 = 21474836480
.)Special thanks also to @kapouer for providing an interim workaround in golang-github-rogpeppe-go-internal/1.12.0-2 for Debian as 1.12.0-1 was blocked from entering "trixie (testing)" for 16 days due to test error on s390x.