Skip to content

Commit

Permalink
testscript: fix ptyName() returning /dev/pts/4294967296 on s390x
Browse files Browse the repository at this point in the history
Use uint32 instead of uint (64-bit in Go on s390x) to store the return
value of the TIOCGPTN syscall.  This is to avoid the 32-bit value from
being stored into a 64-bit buffer and get left-shifted by 32 when
dereferencing, turning what should be /dev/pts/1 to /dev/pts/4294967296
on big-endian architectures such as s390x.

Special thanks to the explanation and a similar bug fix provided at
containerd/console#51
  • Loading branch information
anthonyfok authored and mvdan committed Feb 16, 2024
1 parent 2c88e7f commit 8e66eb9
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion testscript/internal/pty/pty_linux.go
Expand Up @@ -8,7 +8,7 @@ import (
)

func ptyName(f *os.File) (string, error) {
var out uint
var out uint32
err := ioctl(f, "TIOCGPTN", syscall.TIOCGPTN, uintptr(unsafe.Pointer(&out)))
if err != nil {
return "", err
Expand Down

0 comments on commit 8e66eb9

Please sign in to comment.