s390x support #990

Nayana-ibm · 2020-11-06T13:49:21Z

I was building/executing tests for vault. There are some tests makes use of routines from https://github.com/shirou/gopsutil/blob/master/cpu/cpu_linux.go#L116 to read /proc/cpuinfo

I could build vault from source however thetwo tests are failing for s390x architecture ( with an error: Expected non-nil CPU value) as this arch's /proc/cpuinfo has different contents than intel x86.

Is there any plan to support s390x arch for https://github.com/shirou/gopsutil ?
Please let me know.

Lomanic · 2020-11-06T17:51:55Z

Could you please paste the content of /proc/cpuinfo from a s390x machine maybe? We can only try supporting that architecture if we have sufficient info. Please also share the result of lscpu to compare.

Nayana-ibm · 2020-11-09T07:03:52Z

Thanks @Lomanic for considering s390x support. Here are the contents from /proc/cpuinfo and lscpu from a s390x m/c:
4 processor m/c:
/proc/cpuinfo

vendor_id       : IBM/S390
# processors    : 4
bogomips per cpu: 3241.00
max thread id   : 0
features	: esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs vxe2 vxp sort dflt 
facilities      : 0 1 2 3 4 6 7 8 9 10 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 30 31 32 33 34 35 36 37 38 40 41 42 43 44 45 47 48 49 50 51 52 53 54 57 58 59 60 61 64 69 71 73 74 75 76 77 78 80 81 82 129 130 131 133 134 135 138 139 146 147 148 150 151 152 155 156
cache0          : level=1 type=Data scope=Private size=128K line_size=256 associativity=8
cache1          : level=1 type=Instruction scope=Private size=128K line_size=256 associativity=8
cache2          : level=2 type=Data scope=Private size=4096K line_size=256 associativity=8
cache3          : level=2 type=Instruction scope=Private size=4096K line_size=256 associativity=8
cache4          : level=3 type=Unified scope=Shared size=262144K line_size=256 associativity=32
cache5          : level=4 type=Unified scope=Shared size=983040K line_size=256 associativity=60
processor 0: version = FF,  identification = 0618E8,  machine = 8561
processor 1: version = FF,  identification = 0618E8,  machine = 8561
processor 2: version = FF,  identification = 0618E8,  machine = 8561
processor 3: version = FF,  identification = 0618E8,  machine = 8561

cpu number      : 0
cpu MHz dynamic : 5200
cpu MHz static  : 5200

cpu number      : 1
cpu MHz dynamic : 5200
cpu MHz static  : 5200

cpu number      : 2
cpu MHz dynamic : 5200
cpu MHz static  : 5200

cpu number      : 3
cpu MHz dynamic : 5200
cpu MHz static  : 5200

lscpu

Architecture:        s390x
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Big Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s) per book:  1
Book(s) per drawer:  1
Drawer(s):           4
NUMA node(s):        1
Vendor ID:           IBM/S390
Machine type:        8561
CPU dynamic MHz:     5200
CPU static MHz:      5200
BogoMIPS:            3241.00
Hypervisor:          KVM/Linux
Hypervisor vendor:   KVM
Virtualization type: full
Dispatching mode:    horizontal
L1d cache:           128K
L1i cache:           128K
L2d cache:           4096K
L2i cache:           4096K
L3 cache:            262144K
L4 cache:            983040K
NUMA node0 CPU(s):   0-3
Flags:               esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs vxe2 vxp sort dflt

1 cpu vm:
/proc/cpuinfo

vendor_id       : IBM/S390
# processors    : 1
bogomips per cpu: 17513.00
max thread id   : 0
features        : esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs sie
facilities      : 0 1 2 3 4 6 7 8 9 10 12 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 30 31 32 33 34 35 36 37 38 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 57 58 59 60 69 70 71 72 73 74 75 76 77 80 81 82 128 129 130 131 133 134 135 146 147 156 168
cache0          : level=1 type=Data scope=Private size=128K line_size=256 associativity=8
cache1          : level=1 type=Instruction scope=Private size=128K line_size=256 associativity=8
cache2          : level=2 type=Data scope=Private size=4096K line_size=256 associativity=8
cache3          : level=2 type=Instruction scope=Private size=2048K line_size=256 associativity=8
cache4          : level=3 type=Unified scope=Shared size=131072K line_size=256 associativity=32
cache5          : level=4 type=Unified scope=Shared size=688128K line_size=256 associativity=42
processor 0: version = FF,  identification = 141DB7,  machine = 3906

cpu number      : 0
cpu MHz dynamic : 4166
cpu MHz static  : 5208

lscpu

Architecture:        s390x
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Big Endian
CPU(s):              1
On-line CPU(s) list: 0
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s) per book:  1
Book(s) per drawer:  1
Drawer(s):           1
NUMA node(s):        1
Vendor ID:           IBM/S390
Machine type:        3906
CPU dynamic MHz:     4166
CPU static MHz:      5208
BogoMIPS:            17513.00
Hypervisor:          z/VM 7.1.0
Hypervisor vendor:   IBM
Virtualization type: full
Dispatching mode:    horizontal
L1d cache:           128K
L1i cache:           128K
L2d cache:           4096K
L2i cache:           2048K
L3 cache:            131072K
L4 cache:            688128K
NUMA node0 CPU(s):   0
Flags:               esan3 zarch stfle msa ldisp eimm dfp edat etf3eh highgprs te vx vxd vxe gs sie

Nayana-ibm · 2020-11-12T09:09:23Z

@Lomanic Can you please tel me if we have any update on this.
Also I could see s390x support for host/host_linux_s390x.go already present

Nayana-ibm · 2021-01-08T09:06:44Z

@Lomanic Any update on this , please?

Lomanic · 2021-01-08T19:46:59Z

See #1004 (comment), I added some mocks for #881 in https://github.com/Lomanic/gopsutil/tree/issue881, which will also enable this issue #990 to be handled without regressions for untouched platforms (386 and amd64 namely). I have to get back to it.

Nayana-ibm · 2021-06-29T12:58:22Z

@Lomanic could you please let us know if support to s390x arch is added?

zhsj · 2022-07-02T07:30:39Z

Currently there are two tests are failing on s390x.

=== RUN   TestCPUCountsAgainstLscpu
    cpu_linux_test.go:73: missing info from lscpu: threadsPerCore=1 coresPerSocket=1 sockets=0
    cpu_linux_test.go:88: expected 0, got 10
    cpu_linux_test.go:91: expected 0, got 10
--- FAIL: TestCPUCountsAgainstLscpu (0.07s)

=== RUN   Test_Connections
    process_test.go:565: expected 1 server connection, have 0.
        Details: [{"fd":7,"family":2,"type":1,"localaddr":{"ip":"1.0.0.127","port":34369},"remoteaddr":{"ip":"0.0.0.0","port":0},"status":"LISTEN","uids":[1000,1000,1000,1000],"pid":4267} {"fd":9,"family":2,"type":1,"localaddr":{"ip":"1.0.0.127","port":34369},"remoteaddr":{"ip":"1.0.0.127","port":57012},"status":"ESTABLISHED","uids":[1000,1000,1000,1000],"pid":4267} {"fd":8,"family":2,"type":1,"localaddr":{"ip":"1.0.0.127","port":57012},"remoteaddr":{"ip":"1.0.0.127","port":34369},"status":"ESTABLISHED","uids":[1000,1000,1000,1000],"pid":4267}]
--- FAIL: Test_Connections (0.00s)

Test_Connections looks like indeed a bug, since the code assumes little endian.

gopsutil/net/net_linux.go

Lines 724 to 726 in 7a094df

    
           // Assumes this is little_endian 
        
           if family == syscall.AF_INET { 
        
           	ip = net.IP(ReverseWithContext(ctx, decoded))

shirou · 2022-07-02T13:41:51Z

I have opened #1322. Could you test by using that PR? (sorry, I don't have a s390x)

Nayana-ibm · 2022-07-04T06:13:08Z

@shirou Could you please share exact steps that I can follow to build and test on s390x?

zhsj · 2022-07-04T06:17:03Z

@shirou Could you please share exact steps that I can follow to build and test on s390x?

I think you can just run go test ./... in this repository.

Nayana-ibm · 2022-07-04T06:36:49Z

@zhsj got below result :

$ go test ./...

?       github.com/shirou/gopsutil/v3   [no test files]
--- FAIL: TestCPUCountsAgainstLscpu (0.00s)
    cpu_linux_test.go:73: missing info from lscpu: threadsPerCore=1 coresPerSocket=1 sockets=0
    cpu_linux_test.go:88: expected 0, got 8
    cpu_linux_test.go:91: expected 0, got 8
--- FAIL: TestCpuInfo (0.00s)
    cpu_test.go:120: could not get CPU Info
FAIL
FAIL    github.com/shirou/gopsutil/v3/cpu       0.067s
ok      github.com/shirou/gopsutil/v3/disk      (cached)
ok      github.com/shirou/gopsutil/v3/docker    (cached)
ok      github.com/shirou/gopsutil/v3/host      (cached)
ok      github.com/shirou/gopsutil/v3/internal/common   (cached)
ok      github.com/shirou/gopsutil/v3/load      (cached)
--- FAIL: TestSwapDevices (0.00s)
    mem_test.go:123: SwapDevices() -> []
    mem_test.go:126: no swap devices found. [this is expected if the host has swap disabled]
FAIL
FAIL    github.com/shirou/gopsutil/v3/mem       0.008s
ok      github.com/shirou/gopsutil/v3/net       0.029s
ok      github.com/shirou/gopsutil/v3/process   2.146s
FAIL

shirou · 2022-07-09T01:49:53Z

(perhaps cpuinfo is still problem. so re-opened this PR)

Nayana-ibm · 2023-06-30T19:01:00Z

#1414 fixes this issue.
Closing.

jeffweiss · 2023-06-30T19:59:02Z

@Nayana-ibm I added this support so that we could get the DataDog agent running on our IBM Z.

Lomanic added os:linux package:cpu labels Nov 9, 2020

Lomanic mentioned this issue Dec 1, 2020

[mem][linux] Fix #1002 only try to parse /proc/meminfo numeric values on fields we're interested in #1004

Merged

shirou mentioned this issue Jul 2, 2022

fix(net,linux): fix decodeaddress if Big Endian #1322

Merged

shirou closed this as completed in #1322 Jul 9, 2022

shirou reopened this Jul 9, 2022

jeffweiss mentioned this issue Feb 7, 2023

get tests passing on s390x #1414

Merged

Nayana-ibm closed this as completed Jun 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

s390x support #990

s390x support #990

Nayana-ibm commented Nov 6, 2020

Lomanic commented Nov 6, 2020

Nayana-ibm commented Nov 9, 2020 •

edited

Nayana-ibm commented Nov 12, 2020

Nayana-ibm commented Jan 8, 2021

Lomanic commented Jan 8, 2021

Nayana-ibm commented Jun 29, 2021

zhsj commented Jul 2, 2022 •

edited

shirou commented Jul 2, 2022 •

edited

Nayana-ibm commented Jul 4, 2022

zhsj commented Jul 4, 2022

Nayana-ibm commented Jul 4, 2022 •

edited

shirou commented Jul 9, 2022

Nayana-ibm commented Jun 30, 2023

jeffweiss commented Jun 30, 2023

s390x support #990

s390x support #990

Comments

Nayana-ibm commented Nov 6, 2020

Lomanic commented Nov 6, 2020

Nayana-ibm commented Nov 9, 2020 • edited

Nayana-ibm commented Nov 12, 2020

Nayana-ibm commented Jan 8, 2021

Lomanic commented Jan 8, 2021

Nayana-ibm commented Jun 29, 2021

zhsj commented Jul 2, 2022 • edited

shirou commented Jul 2, 2022 • edited

Nayana-ibm commented Jul 4, 2022

zhsj commented Jul 4, 2022

Nayana-ibm commented Jul 4, 2022 • edited

shirou commented Jul 9, 2022

Nayana-ibm commented Jun 30, 2023

jeffweiss commented Jun 30, 2023

Nayana-ibm commented Nov 9, 2020 •

edited

zhsj commented Jul 2, 2022 •

edited

shirou commented Jul 2, 2022 •

edited

Nayana-ibm commented Jul 4, 2022 •

edited