Some C++ symbols emitted by clang-14 cannot be demangled

[gabrielesvelto]

Environment

Symbolic 10.0.0

Steps to Reproduce

symbolic-demangle seems to be unable to demangle any of the symbols in this list. They are taken from a Firefox build and have been generated with clang 14.0.5

Expected Result

The symbols are demangled.

Actual Result

The symbols aren't demangled. That being said I'm not sure if this is an issue with symbolic-demangle itself or if clang emitted buggy demangled symbols... it happened in the past already.

[Swatinem]

I was briefly looking at some of the simpler identifiers here.

Essentially it all boils down to https://github.com/gimli-rs/cpp_demangle not supporting them.

For some of these and looking at the llvm-cxxfilt output:

> llvm-cxxfilt '__Z14WasmMemoryFillIPhPDoFPvS1_imEjEiP9JSContextT_mT1_jS7_T0_'
int WasmMemoryFill<unsigned char*, void* (*)(void*, int, unsigned long) noexcept, unsigned int>(JSContext*, unsigned char*, unsigned long, unsigned int, unsigned int, unsigned int, void* (*)(void*, int, unsigned long) noexcept)

A bit unclear, maybe the noexcept part that is not supported?

> llvm-cxxfilt "__ZL21CreateWasmConstructorIN2js13WasmTagObjectEXadsoKcL_ZL11WasmTagNameEEEEP8JSObjectP9JSContext10JSProtoKey"
JSObject* CreateWasmConstructor<js::WasmTagObject, &(WasmTagName.<char const at offset 0>)>(JSContext*, JSProtoKey)

> llvm-cxxfilt "__ZN3hwy13FunctionCacheIvJEE13ChooseAndCallIXadsoKPFvvEL_ZN3jxlL42BenchmarkFastIDCT32x32HighwayDispatchTableEEEEEEvv"
void hwy::FunctionCache<void>::ChooseAndCall<&(jxl::BenchmarkFastIDCT32x32HighwayDispatchTable.<void (* const)() at offset 0>)>()

For these two, I believe the .<X at offset Y> is something that is not supported.

Its a bit hard to find out what exactly it is unhappy with. I’m not quite sure what it would take to contribute this upstream, I found the code extremely hard to work with last time I contributed something. Also, the whole name mangling is way too underdocumented / underspecified.

[steven-michaud]

I just cloned https://github.com/gimli-rs/cpp_demangle, built it (using cargo build --release), and installed a c++filt clone of it locally (using cargo install cpp_demangle --example cppfilt). It works with all the non-demangled symbols from https://bugzilla.mozilla.org/show_bug.cgi?id=1795461#c0.

So I don't think this bug can be blamed on cpp_demangle. I expect, rather, that it's being given the wrong input.

By the way, I did my tests with and without the "extra" underscore that all symbols have in binaries' symbol tables. cppfilt worked both ways. But it did fail if I removed all leading underscores.

[steven-michaud]

But cppfilt does fail with the examples you gave above, @Swatinem. And c++filt does work with those examples.

So maybe we have two bugs -- one that (sometimes) gives the wrong input to cpp_demangle, and another that's a problem with cpp_demangle itself.

[steven-michaud]

@gabrielesvelto, do you know if there's a (reasonably simple) way to do a local Firefox build with tools from a different version of LLVM? Say 15.0.3, which is the current version? It'd be interesting to see if that made any difference here.

Edit: I think I've figured this out, and am working on it.

Edit: No, building with the tools from LLVM 15.0.3 doesn't seem to have made any difference.

[Swatinem]

maybe not relevant for your usecase, but we are using a patched version of cpp_demangle ourselves which might be a bit behind master.

Either way, there doesn’t seem to be any significant changes between master and the last released version either.

Can you provide a filtered down list of identifiers that work with upstream cpp_demangle but fail in symbolic? I doubt we are doing anything special, except maybe different size/recursion limits

[gabrielesvelto]

Can you provide a filtered down list of identifiers that work with upstream cpp_demangle but fail in symbolic? I doubt we are doing anything special, except maybe different size/recursion limits

I can do that, but it's proving funny because some of these symbols cause the latest version of cpp_demangle to panic.

[gabrielesvelto]

All of the 1472 symbols in the list I attached fail on symbolic. The last version of cpp_demangle is capable of demangling 1385 of those symbols correctly. It fails on 78 of them with an error and panics on 9.

[gabrielesvelto]

I filed gimli-rs/cpp_demangle#272 and gimli-rs/cpp_demangle#273 to address the symbols that cpp_demangle does not process correctly, for the rest updating symbolic to use version 0.3.5 should provide significant improvements for us.

[Swatinem]

updating symbolic to use version 0.3.5 should provide significant improvements for us.

That is a semver compatible range, and we don’t have any lockfiles in symbolic directly. So a simple cargo update should solve it on your end?

[Swatinem]

I believe this is completely upstream work to do in https://github.com/gimli-rs/cpp_demangle which I believe is already being tackled, so there is nothing really to do on our end.

[gabrielesvelto]

I can confirm that this has been fixed upstream and we've been using the fix via symbolic w/o encountering any other issues.