cross compile ebpf programs

[Yinwhe]

Due to some reasons, I got to cross compile a ebpf program on my x86 host and run it on the arm64 platforms (A rather simple test platform with busybox as FS, thus no lib support). I locally build libelf, libz, libbpf and generate libelf.a, libz.a, libbpf.a and related header files. Then I use theses files to try to statically compile a ebpf program. But when I run it on my arm platform, it failed with exit code -22. Anyone know how to cross compile ebpf programs and run it on another platform with no lib support ?

I detailed my problem on stack overflow. I also post my origin questions here:

I'm working on some kernel modules development on linux, and I set up a environments as this: run linux on Arm Fixed Virtual Platforms (I need some features and cannot use qemu) with busybox as fs. However, my development involves debugging ebpf system, and thus I wish to run some simple ebpf program on the test platforms, but I found it hard. I tried cross and static compilations but when I run it, still failed. I wish some one could help me.

My test ebpf program is quite easy, I post it here:

// bpf

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

SEC("tracepoint/syscalls/sys_enter_execve")
int bpf_prog(void *ctx) {
    bpf_printk("Hello from eBPF with Skeleton!");
    return 0;
}

char _license[] SEC("license") = "GPL";

// loader (some print codes omitted)

#include <stdio.h>
#include <stdlib.h>
#include "bpf_test.skel.h"

int main(int argc, char **argv)
{
    struct bpf_test_bpf *skel;
    int err;

    skel = bpf_test_bpf__open_and_load();
    bpf_test_bpf__attach(skel);

    printf("BPF skeleton loaded and attached successfully. Press enter to exit...\n");
    getchar();

    bpf_test_bpf__destroy(skel);
    return 0;
}

As for cross compilations, I refer to this post. Briefly to say, I locally build libelf, libz, libbpf and generate libelf.a, libz.a, libbpf.a and related header files in a directory /opt/arm/tools . The linux header files of the test kernel are also installed in /opt/arm/linux . The directory structure show like this:

linux
└── include
    ├── asm
    ├── asm-generic
    ├── drm
    ├── linux
    ├── misc
    ├── mtd
    ├── rdma
    ├── scsi
    ├── sound
    ├── video
    └── xen
tools
├── include
│   ├── bpf
│   ├── gelf.h
│   ├── libelf.h
│   ├── nlist.h
│   ├── zconf.h
│   └── zlib.h
├── lib
│   ├── libelf.a
│   ├── libz.a
│   └── pkgconfig
├── lib64
│   ├── libbpf.a
│   ├── libbpf.so -> libbpf.so.1.0.0
│   ├── libbpf.so.1 -> libbpf.so.1.0.0
│   ├── libbpf.so.1.0.0
│   └── pkgconfig
└── share
    └── man

I cross build the ebpf:

clang
  -I/opt/arm/tools/include \
  -I/opt/arm/linux/include \
  -g -O2 -Wall -target bpf -D__TARGET_ARCH_aarch64 \
  -c bpf_test.bpf.c \
  -o bpf_test.bpf.o -static

bpftool gen skeleton bpf_test.bpf.o > bpf_test.skel.h

aarch64-linux-gnu-gcc loader.c \
  -I/opt/arm/tools/include \
  -I/opt/arm/linux/include \
  /opt/arm/tools/lib64/libbpf.a \
  /opt/arm/tools/lib/libz.a \
  /opt/arm/tools/lib/libelf.a \
  -o loader -static

However, when run on the test platform, I got:

libbpf: prog 'bpf_prog': BPF program load failed: Invalid argument
libbpf: failed to load program 'bpf_prog'
libbpf: failed to load object 'bpf_test_bpf'
libbpf: failed to load BPF skeleton 'bpf_test_bpf': -22
Failed to open and load BPF skeleton

I just don't know how to fix it, this really drive me mad. Is there any way to run ebpf on such a simple env ? By the way, since I focus on kernel module, and I'm not familiar with FVP, thus I don't think install a better fs is feasible for me. Could someone help me ?