Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mem.VirtualMemory(): wrong page size on darwin arm64 #1334

Merged
merged 1 commit into from Jul 29, 2022

Conversation

kyz
Copy link
Contributor

@kyz kyz commented Jul 25, 2022

mem.VirtualMemory() returns values that are 4x larger/smaller than they should be on darwin arm64.

Generally speaking, unix.Getpagesize() returns the correct page size on macOS; 4K on Intel Macs and 16K on M1 Macs.

However, that is not the unit that host_statistics() values are measured in; they are kernel memory pages which are vm_kernel_page_size bytes long.

You can see this in the source code of vm_stat, which is the executable that the non-CGO version of the darwin VM stats runs and parses.

https://github.com/unofficial-opensource-apple/system_cmds/blob/master/vm_stat.tproj/vm_stat.c#L126-L132

printf("Mach Virtual Memory Statistics: (page size of %llu bytes)\n", (mach_vm_size_t)vm_kernel_page_size);

This PR replaces unix.Getpagesize() with C.vm_kernel_page_size

@kyz
Copy link
Contributor Author

kyz commented Jul 25, 2022

  • Running an x86_64 executable on an Intel Mac: unix.Getpagesize() returns 4096 (correct), C.vm_kernel_page_size returns 4096 (correct)
  • Running an x86_64 executable on an M1 Mac (under emulation): unix.Getpagesize() returns 4096 (incorrect), C.vm_kernel_page_size returns 16384 (correct)
  • Running an arm64 executable on an M1 Mac natively: unix.Getpagesize() returns 16384 (correct), C.vm_kernel_page_size returns 16384 (correct)

@shirou
Copy link
Owner

shirou commented Jul 25, 2022

Thank you for the PR!

Running an x86_64 executable on an M1 Mac (under emulation): unix.Getpagesize() returns 4096 (incorrect)

Is this pattern the only incorrect? and "Emulation" means Rosetta 2?

@kyz
Copy link
Contributor Author

kyz commented Jul 25, 2022

That is the case. The built-in emulation via Rosetta 2

Copy link
Owner

@shirou shirou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have confirmed this short code.

package main

/*
#include <mach/mach_host.h>
#include <mach/vm_page_size.h>
*/
import "C"

import (
        "fmt"
        "golang.org/x/sys/unix"
)

func main() {
        cPageSize := uint64(C.vm_kernel_page_size)
        uPageSize := unix.Getpagesize()

        fmt.Printf("C=%d, unix=%d\n", cPageSize, uPageSize)
}

using

[golang.org/x/sys](http://golang.org/x/sys) v0.0.0-20220728004956-3c1f35247d10 

on Mac 11 i7:

C=4096, unix=4096

on M1 mac (I run on scaleway.com. Thank you for the sponsors!)

GOOS=darwin GOARCH=amd64 /usr/local/go/bin/go run main.go
C=16384, unix=4096

GOOS=darwin GOARCH=arm64 /usr/local/go/bin/go run main.go
C=16384, unix=16384

According to current code, unix.Getpagesize uses sysctl hw.pagesize. And Rosetta 2 uses 4kb. I think that means in Rosetta 2, it is rewritten including sysctl, but C.vm_kernel_page_size is still the same.

Now, as gopsutil, which is the correct value ?
As a gopsutil, I think we should use C.vm_kernel_page_size as your PR because we want to know memory on a host, even if it works in the Rosetta 2.

So let me merge this great PR. Thank you so much!

@shirou shirou merged commit 190dfee into shirou:master Jul 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants