Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

l3_misses and l3_accesses perf events unable to retrieve on AMD host #3492

Open
yunzhao-cf opened this issue Mar 5, 2024 · 1 comment
Open

Comments

@yunzhao-cf
Copy link

yunzhao-cf commented Mar 5, 2024

Hello,

I am encountering an issue where cAdvisor's perf is unable to retrieve the perf events "l3_misses" and "l3_accesses". The following error message is returned, suggesting a failure in the pfm_get_os_event_encoding() call:

E0305 17:49:03.940537 892708 collector_libpfm.go:203] Cannot count perf event group [l3_misses l3_accesses]: cannot create config from perf event: unable to transform event name l3_misses to perf_event_attr: -4

Checking the libpfm and '-4' is for "event not found": https://github.com/s-kanev/misc/blob/fc38c60e84d20f5aceb8ed0c70478b45b92f48b1/perf_harness/libpfm/perfmon/pfmlib.h#L359

However, these events can be successfully monitored using the perf CLI tool:

 ~$sudo perf stat -a -e l3_misses,l3_accesses -- sleep 1

 Performance counter stats for 'system wide':

     1,122,395,814      l3_misses                                                          
     4,398,985,531      l3_accesses                                                        

       1.005054759 seconds time elapsed

I attempted to manually specify the event configuration based on the output from perf:

 ~$sudo perf stat -I 5000 -vvv -e l3_misses
Using CPUID AuthenticAMD-23-31-0
Attempting to add event pmu 'amd_l3' with 'l3_misses,' that may result in non-fatal errors
After aliases, add event pmu 'amd_l3' with 'event,umask,' that may result in non-fatal errors
l3_misses -> amd_l3/event=0x4,umask=0x1/
Control descriptor is not initialized
------------------------------------------------------------
perf_event_attr:
  type                             11
  size                             128
  config                          0x104
  sample_type               IDENTIFIER
  read_format                TOTAL_TIME_ENABLED|TOTAL_TIME_RUNNING
  disabled                      1
  inherit                          1
  exclude_guest            1

I then defined this configuration in a custom events file for cAdvisor:

    "custom_events": [
      {
        "type": 11,
        "config": [
          "0x104"
        ],
        "name": "l3_misses"
      }
    ]

However, no data is generated when using this custom configuration, and no error message is reported.

The environment is running on a kernel version 6.1 with an AMD EPYC 7642 48-Core Processor.

@yunzhao-cf
Copy link
Author

Verified with a simple libpfm4 code, and it shows these perf events are not supported yet on AMD nodes: https://sourceforge.net/p/perfmon2/mailman/perfmon2-devel/thread/CAAnvDSxwyyguSk56y87mu2f1OS4J0NQdiaTDsavUEkRWGHvL9Q%40mail.gmail.com/#msg58747559

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant