Use a DWARF version consistent with rustc #694

glandium · 2022-07-07T04:54:54Z

Rustc defaults to DWARF-2 on some targets, and DWARF-4 on others.
However using -g with the C compiler yields whatever default version the
C compiler prefers.

One side effect is that the DWARF debug info shipped in some libraries
with rustc itself (e.g. libcompiler_builtins and others) have recently
switched to DWARF-5 as a side effect of upgrading the clang version
used on rustc CI. (rust-lang/rust#98746)

Ideally, the preferred DWARF version would be given by the rust compiler
and/or cargo, but that's not the case at the moment, so the next best
thing is something that aligns with the current defaults, although
work in under way to add a rustc flag that would allow to pick the
preferred DWARF version (rust-lang/rust#98350)

glandium · 2022-07-07T04:59:54Z

The CI failures don't seem to be my doing.

dot-asm · 2022-07-09T20:22:13Z

The CI failures don't seem to be my doing.

Right, see #677.

dot-asm · 2022-07-09T20:37:58Z

src/lib.rs

@@ -2771,6 +2775,23 @@ impl Build {
        })
    }

+    fn get_dwarf_version(&self) -> Option<u32> {


I would argue that it would be more than appropriate to add a commentary stating that the version choice reflects rustc defaults as of version X through Y. Idea is that whenever defaults change, there would be a hint that corresponding adjustment would be needed.

dot-asm · 2022-07-10T20:55:53Z

On a related note, mingw32 seems to suffer from the same problem. Or at least it appears to be a reasonable conclusion to reach based on the facts that a) attempt to use latest mingw toolchain renders debug-build executables non-executable (see #677 (comment)), and b) imposing -gdwarf-2 flags on C and C++ compilers mitigates the problem (see #677 (comment)).

mati865 · 2022-07-10T23:40:41Z

Depending on toolchain version you will end up with one of these Dwarf versions: 2, 4, 5 when targeting 32-bit mingw-w64.
Toolchain used to build Rust is rather ancient one so I'm fairly sure it uses Dwarf-2 by default. How would potential upgrade path for it look like if we choose one version? I.e. should upgrade of mingw-w64 toolchain also change Dwarf version in the spec?

dot-asm · 2022-07-12T11:15:42Z

src/lib.rs

+            || target.contains("freebsd")
+            || target.contains("netbsd")
+            || target.contains("openbsd")
+            || target.contains("windows-gnu")


Just in case. The original remark was specifically about mingw32, the 32-bit target. The 64-bit one was not observed to fail. And the thing is that "windows-gnu" would cover both 32- and 64-bit variants. At the same time the flag appears to be recognized by the 64-bit toolchain and causes no run-time problems. In other words, even though this line doesn't pinpoint the specific problem, it doesn't seem to incur any collateral damage. If it's acceptable is up to the actual cc-rs maintainers.

I have no idea what will happen on 64-bit targets because they typically use SEH instead of DWARF.
I worry it can work with GNU tools but introduce bugs like rust-lang/rust#99143 with LLVM.

64-bit targets because they typically use SEH instead of DWARF.

Note that the 32-bit problem is not really/necessarily about exception handling. Because it was specifically debug-build binaries, ones packed with debugging information, that were rendered unexecutable. This is not to say that I'm saying that the 64-bit toolchain uses DWARF for debugging symbols. Because I'm not, as I don't really know. But I can confirm that -gdwarf appears to be an acceptable option. It can just as well be ignored, but the binaries are executable, and that's what counts for the moment.

it can work with GNU tools but introduce bugs ... with LLVM.

But it won't be affected by this target.contains. Right? So that if anything, you should double-check and report back. Or keep this in mind and suggest a workaround later on.

I have no idea what will happen on 64-bit targets because they typically use SEH instead of DWARF. I worry it can work with GNU tools but introduce bugs like rust-lang/rust#99143 with LLVM.

Exceptions handling and Debug info are more or less orthogonal. -g defaults to DWARF on mingw targets for both GCC and clang. I'm not sure what rust was doing for those targets before #99143, though.

dot-asm · 2022-07-12T11:19:27Z

For reference, CUDA failure appears to be transient, it failed to download package index files. They suggest it can be side effect of "Mirror sync in progress".

dot-asm · 2022-07-12T11:29:41Z

How would potential upgrade path for it look like if we choose one version?

Not that I can offer any real answers here, but what would be definitely helpful if linker was producing working executable files even when faced with mismatching DWARF versions. One can even wonder if it would be possible to classify it as a GNU ld bug. More specifically such an ungraceful failure mentioned above...

mati865 · 2022-07-12T15:29:07Z

One can even wonder if it would be possible to classify it as a GNU ld bug.

IMO it's surely a bug in GNU ld but it's not clear if we can do anything other than workarounds here.

glandium · 2022-08-09T05:35:17Z

ping?

thomcc · 2022-11-10T16:06:41Z

Can you rebase? The CI failures should be fixed now.

thomcc

This largely looks fine. I'd really prefer long-term this information come from cargo, via an env var or something. That seems necessary to do the right thing here if rustc allows setting this via a flag, for example (Well, I suppose we could parse rustflags...)

I won't block on this, though.

My one concern is if folks are using compilers that don't support this argument, then this might cause issues. I'm not sure if such compilers exist, but I suppose we'll find out...

Rustc defaults to DWARF-2 on some targets, and DWARF-4 on others. However using -g with the C compiler yields whatever default version the C compiler prefers. One side effect is that the DWARF debug info shipped in some libraries with rustc itself (e.g. libcompiler_builtins and others) have recently switched to DWARF-5 as a side effect of upgrading the clang version used on rustc CI. (rust-lang/rust#98746) Ideally, the preferred DWARF version would be given by the rust compiler and/or cargo, but that's not the case at the moment, so the next best thing is something that aligns with the current defaults, although work in under way to add a rustc flag that would allow to pick the preferred DWARF version (rust-lang/rust#98350)

glandium · 2022-11-10T20:40:00Z

Rebased.

thomcc · 2022-11-10T23:03:24Z

Thanks

thomcc · 2022-11-20T23:01:42Z

This is in 1.0.77.

One concern I have here is that in rust-lang/rust#104385 we'll be updating the DWARF version to DWARF-4 on Apple targets.

@glandium, are you familiar with how this will impact this patch? Is there an issue with cc using DWARF-2 when the rust compiler uses DWARF-4? I'd assume there isn't, but figured you'd know.

If there is, I suppose cc will need do version-detection on rustc to handle this case... Which is gross, but is not exactly technically challenging.

glandium · 2022-11-21T20:33:10Z

Mixing versions shouldn't cause problems. We've been mixing DWARF-2 and DWARF-4 on mac builds of Firefox for a very long time (DWARF-2 from rust, DWARF-4 from clang). DWARF-5 was only causing problems because not all tooling supports it.
Ideally, though, cc-rs would get the information from rustc somehow.

thomcc · 2022-11-21T20:40:53Z

Mixing versions shouldn't cause problems. We've been mixing DWARF-2 and DWARF-4 on mac builds of Firefox for a very long time (DWARF-2 from rust, DWARF-4 from clang). DWARF-5 was only causing problems because not all tooling supports it.

That's what I thought, but wanted to confirm, thanks.

Ideally, though, cc-rs would get the information from rustc somehow.

Yeah, I agree. Probably through cargo, although that's a bit of a headache for cargo.

glandium force-pushed the dwarf-version branch from 6d88dd7 to cc61cdd Compare July 7, 2022 05:01

dot-asm reviewed Jul 9, 2022

View reviewed changes

dot-asm mentioned this pull request Jul 10, 2022

Update CUDA Linux GPG repository key [and compiler version]. #677

Merged

glandium force-pushed the dwarf-version branch 2 times, most recently from 02d6087 to 932a55c Compare July 12, 2022 02:00

dot-asm reviewed Jul 12, 2022

View reviewed changes

apiraino mentioned this pull request Oct 26, 2022

libcompiler-builtins contains DWARF5 debuginfo in 1.62.0 rust-lang/rust#98746

Open

thomcc approved these changes Nov 10, 2022

View reviewed changes

thomcc mentioned this pull request Nov 10, 2022

provide way to users to opt into DWARF5 on stable Rust rust-lang/rust#103057

Open

glandium force-pushed the dwarf-version branch from 932a55c to 380d5d8 Compare November 10, 2022 20:39

thomcc merged commit 0fff25a into rust-lang:main Nov 10, 2022

glandium deleted the dwarf-version branch November 11, 2022 02:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use a DWARF version consistent with rustc #694

Use a DWARF version consistent with rustc #694

glandium commented Jul 7, 2022 •

edited

glandium commented Jul 7, 2022

dot-asm commented Jul 9, 2022

dot-asm Jul 9, 2022

dot-asm commented Jul 10, 2022

mati865 commented Jul 10, 2022

dot-asm Jul 12, 2022

mati865 Jul 12, 2022

dot-asm Jul 12, 2022

glandium Jul 13, 2022

dot-asm commented Jul 12, 2022

dot-asm commented Jul 12, 2022

mati865 commented Jul 12, 2022

glandium commented Aug 9, 2022

thomcc commented Nov 10, 2022

thomcc left a comment

glandium commented Nov 10, 2022

thomcc commented Nov 10, 2022

thomcc commented Nov 20, 2022

glandium commented Nov 21, 2022

thomcc commented Nov 21, 2022

Use a DWARF version consistent with rustc #694

Use a DWARF version consistent with rustc #694

Conversation

glandium commented Jul 7, 2022 • edited

glandium commented Jul 7, 2022

dot-asm commented Jul 9, 2022

dot-asm Jul 9, 2022

Choose a reason for hiding this comment

dot-asm commented Jul 10, 2022

mati865 commented Jul 10, 2022

dot-asm Jul 12, 2022

Choose a reason for hiding this comment

mati865 Jul 12, 2022

Choose a reason for hiding this comment

dot-asm Jul 12, 2022

Choose a reason for hiding this comment

glandium Jul 13, 2022

Choose a reason for hiding this comment

dot-asm commented Jul 12, 2022

dot-asm commented Jul 12, 2022

mati865 commented Jul 12, 2022

glandium commented Aug 9, 2022

thomcc commented Nov 10, 2022

thomcc left a comment

Choose a reason for hiding this comment

glandium commented Nov 10, 2022

thomcc commented Nov 10, 2022

thomcc commented Nov 20, 2022

glandium commented Nov 21, 2022

thomcc commented Nov 21, 2022

glandium commented Jul 7, 2022 •

edited