bindgen wanted features & bugfixes

[ojeda]

See as well the "rust-for-linux" label in bindgen: https://github.com/rust-lang/rust-bindgen/labels/rust-for-linux.

Features that we would like to see:

• Support for a GCC-based backend.

  • Issue: Support for a GCC-based backend rust-lang/rust-bindgen#1949.

• Support unsafe_op_in_unsafe_fn.

  • Issue: Support unsafe_op_in_unsafe_fn rust-lang/rust-bindgen#2063.
  • PR: Wrap unsafe function's bodies in unsafe blocks rust-lang/rust-bindgen#2266.

• Support _Noreturn, [[noreturn]], __attribute__((noreturn))

  • Issue: Support _Noreturn, [[noreturn]], __attribute__((noreturn)) rust-lang/rust-bindgen#2094.
  • PR: Handle no return attributes rust-lang/rust-bindgen#2259.

• Support non-trivial object-like macros that evaluate to a numeric value when fully expanded.

  • For instance:
    • An expression, e.g. #define A ((int)1).
    • Macros using function-like macros, e.g. Linux ioctl #defines using _IO*.
  • Issue: Improve macro defined constants to recognize #define CONSTANT ((int) 1) rust-lang/rust-bindgen#316 (expressions).
  • Issue: Functional C macros are not expanded. rust-lang/rust-bindgen#753 (using other macros).
  • PR ("one pass per macro", i.e. option 1 below): Workaround for expansion of function-like macros rust-lang/rust-bindgen#2779 (0.70.0?).
  • PR (follow up fixes): Switch to single precompiled header in macro fallback rust-lang/rust-bindgen#2823 (0.70.0?).
  • Meta-issue: Preprocessing meta-issue rust-lang/rust-bindgen#2426.
  • Zulip: https://rust-for-linux.zulipchat.com/#narrow/stream/288089-General/topic/bindgen.20issue.20.23753.20-.20function.20like.20macros/near/420514477.
  • Cc: @jbaublitz who has been following/helping with this issue since the beginning and potentially interested in working on it.
  • Status:

    I met with @emilio a few days ago (early February 2024), and we discussed this for a bit, including the bindgen-side solution/workaround of emitting extra code so that we can fetch the value.

    The variations we discussed were "one pass per macro" (slow, but could work without user input), "one extra pass for all macros" (this would be fast, but would require the user to pass e.g. a regex of matching macros to evaluate that are "known to be constants" already); as well as the possibility of "appending the macros later on" (i.e. add them into a context or similar, which may be possible or not with the current API; we should also check e.g. what Clang-Repl/Cling does here).

    So, in summary, bindgen would be happy with a workaround. But solving it in LLVM is, of course, the best long-term, so that would be great too (ideally both, so that we can use the feature before projects like the kernel have a minimum LLVM version >= 19).

    For the LLVM side, @AaronBallman (the Clang maintainer) told me he would be happy to review our changes if we sent them, in order to help bindgen, so that is great if someone wants to work on it.

    • The first option, i.e. "one pass per macro" was prototyped by @jbaublitz.

• Support static inline functions in headers (by generating a wrapper).

  • Currently we do it manually. bindgen currently (0.64.0) has experimental support via --wrap-static-fns --experimental.
  • Netdev would like it, e.g. skbuff.h has 300+ static inlines: https://lore.kernel.org/rust-for-linux/f28d6403-d042-4ffb-9872-044388d0f9d9@lunn.ch/
  • Issue: Generate C code to export static inline functions rust-lang/rust-bindgen#1090.
  • PR: Generate extern wrappers for inlined functions rust-lang/rust-bindgen#2335.
  • Issue: Interaction between wrapped statics and redefined function rust-lang/rust-bindgen#2420.

• Support typedefs shadowing types e.g. typedef struct T *T;.

  • In other words, avoid conflating C name spaces.
  • Seen in the past e.g. for maple_*nodes (Build fail on linux-next (next-20220629 tag): the name maple_enode is defined multiple times #795).
  • Issue: Multiple definitions when a pointer gets typedef’d to the same name as a struct rust-lang/rust-bindgen#2227.
  • PR: Handle the const struct * and struct * patterns rust-lang/rust-bindgen#2304.

• Take advantage of feature(core_ffi_c) and feature(c_size_t).

  • This avoids having to manually specify --ctypes-prefix core::ffi.
  • Issue: Take advantage of feature(core_ffi_c) (and possibly feature(c_size_t)) rust-lang/rust-bindgen#2231.
  • PR: use #[feature(core_ffi_c)] when available rust-lang/rust-bindgen#2267.

• A way to query the Clang version being used.

  • Issue: Querying bindgen for LLVM version being used rust-lang/rust-bindgen#2138.
  • PR: options: Implement --version manually and print clang version on --version --verbose rust-lang/rust-bindgen#2140.

• Support aligned members (including structs with bitfields) inside packed structs.

  • i.e. avoid generating #[repr(C, align(...)] inside an #[repr(C, packed)].
  • Seen for struct alt_instr: https://lore.kernel.org/all/202212272003.rgQDX8DQ-lkp@intel.com/
  • Also supporting aligned+packed structs.
  • Issue: packed type cannot transitively contain a #[repr(align)] type rust-lang/rust-bindgen#2179.
  • Issue: structure layout using __aligned__ attribute is incorrect rust-lang/rust-bindgen#867.
  • Issue: handle more cases of packed and aligned structs when it's safe/correct rust-lang/rust-bindgen#2725.
  • Related issue: E0587 error on packed and aligned structures from C rust-lang/rust-bindgen#1538.
  • Rust issue: E0587 error on packed and aligned structures from C rust-lang/rust#59154.
  • PR: Consider alignment contribution from bitfields rust-lang/rust-bindgen#2388.
  • PR: try to avoid #[repr(packed)] when align is needed rust-lang/rust-bindgen#2734.
  • PR (RFC alternative 1): [RFC, V1] try to exclude packed attr for types that contain aligned types rust-lang/rust-bindgen#2769.
  • PR (RFC alternative 2): [RFC, V2] exclude packed attr for types that contain aligned types when possible rust-lang/rust-bindgen#2770.

• Support a new "enum variant" kind, similar to rustified_enum, that provides a safe method to transform a C enum to Rust.

  • The goal is to ensure at compile-time that, if the C side enum adds a variant, the Rust side (i.e. the conversion method) will be updated properly too.
  • From: https://rust-for-linux.zulipchat.com/#narrow/stream/288089-General/topic/Enum.20handling/near/392620696.
  • Issue: Add enum option constified_enum + rustified_enum with conversions rust-lang/rust-bindgen#2646.

• Raw pointer bitfield accessors.

  • Issue: bitfields raw pointer accessors rust-lang/rust-bindgen#2674.

• Compile-time layout tests.

  • We are planning to run the layout tests as e.g. KUnit tests, but if they were compile-time ones, it would be simpler.
  • Issue: Feature request: compile-time layout tests rust-lang/rust-bindgen#2786.
  • PR: Compile time layout tests rust-lang/rust-bindgen#2787 (0.xx.x).

---

Bugs that we would like to see fixed:

• C javadoc comments are not Markdown-escaped, triggering rustdoc warnings.

  • Issue: C javadoc comments are not Markdown-escaped, triggering rustdoc warnings rust-lang/rust-bindgen#2057.

• Attribute btf_type_tag makes constness of pointers disappear.

  • It will allow to undo the f6bb7cb workaround.
  • Should be fixed in 0.61.
  • Issue: Attribute btf_type_tag makes constness of pointers disappear  rust-lang/rust-bindgen#2244.
  • PR: ty: Use canonical type to access pointee. rust-lang/rust-bindgen#2245.

• "Unable to find libclang" problems (rust compiler not found #934), possibly related to Clang 15.

  • Issue: Unable to find libclang: "the libclang shared library rust-lang/rust-bindgen#2360.
  • PR (may or may not solve our issue): Bump clang-sys version rust-lang/rust-bindgen#2368.

• --wrap-static-fns generates code that returns a call to a void function.

  • Issue: --wrap-static-fns generates code that returns a call to a void function rust-lang/rust-bindgen#2485.
  • PR: Do not return when calling void functions  rust-lang/rust-bindgen#2497.

• --wrap-static-fns generates () for empty parameter lists.

  • Issue: --wrap-static-fns generates () for empty parameter lists rust-lang/rust-bindgen#2484.
  • Should be fixed in 0.65.1.

• Fix the support for linking libclang statically.

  • Zulip: https://rust-for-linux.zulipchat.com/#narrow/stream/291565-Help/topic/Build.20errors/near/391197677.
  • PR: Fix bindgen-cli support link static libclang rust-lang/rust-bindgen#2644.

• BindgenUnion can get incorrect alignment on i686.

  • Issue: BindgenUnion can get incorrect alignment on i686 rust-lang/rust-bindgen#2773.

[pvdrz]

Hi everyone!

I have opened some PRs on bindgen adding some features you needed:

• Handle no return attributes rust-lang/rust-bindgen#2259
• Wrap unsafe function's bodies in unsafe blocks rust-lang/rust-bindgen#2266
• use #[feature(core_ffi_c)] when available rust-lang/rust-bindgen#2267

I'd be happy to hear if the PRs here actually provide what you need.

[alex]

These look spot on to me. Thanks for your efforts!

[ojeda]

Added links above, thanks a lot @pvdrz! :)

[dwlsalmeida]

@ojeda Hey Miguel, is there a workaround in place for this?

Support function-like macros that use other macros and that evaluate to a numeric value when fully expanded (e.g. for Linux ioctl #defines using _IO*).

I am trying to write a v4l2 driver and one of the issues I found was that the pixel formats bindings are not generated by bindgen because the V4L2_PIX_FMT_* stuff is defined using the v4l2_fourcc macro

[ojeda]

For something like pixel formats that are stable (given they are in uapi, right?), I guess the easiest is re-implementing them on the Rust side, especially if you want to give them a proper type etc. anyway. Probably using a small temporary script to perform the bulk of the conversion.

Otherwise, if performance is not too much of a concern (even if cross-language LTO could reduce the cost at some point), one alternative is "executing" the macro at runtime by providing a C function that does it for the Rust side. In your case this one is likely not interesting because the important bits are the parameters to the macro, rather than the macro itself.

[bjorn3]

For the drm fourcc pixel format codes, the drm-fourcc crate has some code which fetches all #define from the header, generates a constant for each #define in a new header and then runs bindgen on the generated header: https://github.com/danielzfranklin/drm-fourcc-rs/blob/main/build.rs

[dwlsalmeida]

One question though, suppose I use the approach suggested by @bjorn3

Do I commit the bindings I generate? Asking because apparently bindings_generated.rs is not part of the kernel per-se and is only generated during build time.

[bjorn3]

I think the bindings should be generated on the fly to ensure they stay up to date, just like bindings_generated.rs.

[dwlsalmeida]

well, in which case I suppose that the code to generate them (i.e. something similar to the code in the drm-fourcc crate) would have to be integrated into the kernel build system somehow?

[bjorn3]

Indeed. There is already infrastructure to run code written in rust. See for example scripts/generate_rust_target.rs.

[dwlsalmeida]

Makes sense, thanks for the pointers!

[ojeda]

It depends. If these constants are strictly stable (unlike many kernel APIs that you may see in bindings_generated.rs), you have the possibility to commit them, but in that case I would recommend writing directly in Rust what you need (rather than raw bindings that then you wrap). This is why I mentioned above the stability. You could also have a test to verify both sides match, though that could be a one-off thing.

If you go the generated route (which many times is the only way), then indeed, you could use a host program (it does not necessarily need to be written in Rust, but as @bjorn3 mentions, the support is there, so I would recommend it -- though currently you will only have the Rust standard library available, e.g. no regex). This approach has the drawback of adding complexity (the host program) and, depending on how you implement/parse the C header, you may introduce constraints on the C side. Another similar option is a proc macro, but that is slower (triggered every time you recompile the kernel crate, which would be unneeded -- you only need to do it once) and non-"pure" (in the "do not use the environment from a proc macro" sense).

Another angle to think about is whether new constants will appear frequently or not.