Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull BTF information if missing using BTFHub #1131

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

kakkoyun
Copy link
Member

@kakkoyun kakkoyun commented Dec 14, 2022

This PR enables the agent to run in a valid CO:RE fashion. By these changes, the eBPF programs will be able to run flawlessly for different Kernel versions and different architectures (even though we only support x86 right now, soon we will add ARM64 support).

To be able to achieve this goal:

  • It downloads the supported Kernel BTF definitions from BTF Hub during builds time, embeds them to the binary, and loads those definitions in the case of missing BTF definitions for a kernel on a running host. The host's BTF definition is always prioritized.
  • It converts all the helper functions with CO:RE equivalents.

Moreover, PR makes changes to the build pipeline to support building multiple eBPF objects files and consuming them to generate necessary BTF files using different definitions for corresponding Kernel versions.

Signed-off-by: Kemal Akkoyun kakkoyun@gmail.com

Fixes #715

Related/Blocker for: #1209 #697 #1114

TODO:

  • Add CI cache for "btfhub-archive" downloads
  • Cross-generate BTF for arm/x86
  • Support multiple eBPF object files
  • Add missing CI tooling: bpftool, ?, ...
  • Check if the CI host machines are BTF enabled
  • Modify the eBPF program to read from the task struct
    • PT_REGS
    • (interim or test only) task_struct->PID
  • Testing (using temporary task_struct->PID)
    • Test on x86
    • Test on CI
    • Test on arm64 (with lima or making CI work on ARM)
    • Check error cases when BTF files are missing

@kakkoyun kakkoyun changed the title Add BTF support Add CO:RE BTF support Dec 14, 2022
@kakkoyun kakkoyun force-pushed the btf_support branch 3 times, most recently from e36e6cb to 804bfd3 Compare December 14, 2022 17:15
@v-thakkar v-thakkar self-requested a review December 15, 2022 07:07
@kakkoyun kakkoyun force-pushed the btf_support branch 5 times, most recently from 46cdd68 to d42dcb6 Compare December 15, 2022 16:22
@kakkoyun kakkoyun changed the title Add CO:RE BTF support Add CO:RE BTF support using BTFHub Dec 15, 2022
@javierhonduco javierhonduco mentioned this pull request Dec 16, 2022
@javierhonduco javierhonduco changed the title Add CO:RE BTF support using BTFHub Pull BTF information if missing using BTFHub Dec 16, 2022
@kakkoyun kakkoyun force-pushed the btf_support branch 2 times, most recently from 2ed4414 to 678140e Compare December 21, 2022 13:39
@kakkoyun
Copy link
Member Author

kakkoyun commented Jan 5, 2023

I'll pick this up after merging #1142. We need to add tests.

kakkoyun and others added 9 commits March 22, 2023 14:13
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
kakkoyun and others added 13 commits March 22, 2023 14:14
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Generate dummy files for pre-commit

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
pre-commit-ci-lite bot and others added 2 commits March 22, 2023 13:28
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
@kakkoyun
Copy link
Member Author

kakkoyun commented Jul 3, 2023

After the changes in #1805, this will be easier to test.
I will start a new PR soon rather than lose my hair while rebasing this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CO:RE Support: Add support for embedding and loading BTF for the running host
2 participants