elfwriter: Unsupported compression types

[javierhonduco]

Using our testing binary from https://github.com/parca-dev/parca-demo/tree/main/cpp and running it with make -C cpp clang-with-fp && docker run -it parca-demo:cpp-clang-with-fp we can see that we failed to write sections in the extracted elf file, this is

level=debug ts=2022-07-05T10:35:15.213545734Z caller=debuginfo.go:134 component=debuginfo msg="failed to extract debug information" err="failed to write ELF file: failed to write sections: unsupported compression type" path=/proc/1371721/root/usr/lib/debug/.build-id/ea/58e052e9e312bdca11ac2f9e72b1ebc9e5874c.debug

I believe this is what's causing some frames (libc's in this case) to not get symbolised #533

cc @kakkoyun

[javierhonduco]

Do we want to extract the debug information in the case of having a separate DWARF already?

parca-agent/pkg/debuginfo/debuginfo.go

Line 116 in 6b67638

if err := di.Extract(ctx, buf, src); err != nil {

[javierhonduco]

With

diff --git a/pkg/elfwriter/elfwriter.go b/pkg/elfwriter/elfwriter.go
index ff9d28b..517a741 100644
--- a/pkg/elfwriter/elfwriter.go
+++ b/pkg/elfwriter/elfwriter.go
@@ -812,6 +812,11 @@ func (w *Writer) compressionHeader(s *elf.Section) *compressionInfo {
        default:
                w.err = fmt.Errorf("unknown ELF class: %v", w.fhdr.Class)
        }
+
+       if c.compressionType != 0 && c.compressionType != elf.COMPRESS_ZLIB {
+               fmt.Println("... Section name=", s.Name, "compression type", c.compressionType, "at addr", fmt.Sprintf("%x", w.here()))
+       }
+
        return c
 }
 

we can see we are getting bogus compressionType values

[...]
debug
level=debug ts=2022-07-07T12:14:19.597557778Z caller=systemd.go:89 discovery=systemd msg="running systemd manager" units=1
... Section name= .debug_abbrev compression type COMPRESS_ZLIB+25332225 at addr 64ac4
... Section name= .debug_line compression type COMPRESS_ZLIB+17432 at addr 64ac6
... Section name= .debug_abbrev compression type COMPRESS_ZLIB+620826881 at addr 3927e8
... Section name= .debug_str compression type COMPRESS_LOPROC+41377631 at addr 6e488
level=debug ts=2022-07-07T12:14:20.596719875Z caller=systemd.go:89 discovery=systemd msg="running systemd manager" units=1
level=debug ts=2022-07-07T12:14:20.596815938Z caller=manager.go:87 msg="reconciling targets"
level=debug ts=2022-07-07T12:14:20.613020241Z caller=write_client.go:108 msg="batch write client sent profiles" count=1
... Section name= .debug_ranges compression type COMPRESS_ZLIB+496 at addr 7aa5c
level=debug ts=2022-07-07T12:14:21.100147677Z caller=debuginfo.go:117 component=debuginfo msg="failed to extract debug information" err="failed to write ELF file: failed to write sections: unsupported compression type" buildID=4587364908de169dec62ffa538170118c1c3a078 path=/proc/1371721/root/usr/lib/debug/.build-id/45/87364908de169dec62ffa538170118c1c3a078.debug
[...]

[javierhonduco]

A hypothesis I have been exploring without much success so far is that perhaps the FileSize is wrong for compressed sections. We are currently setting it to:

parca-agent/pkg/elfwriter/elfwriter.go

Line 565 in 6b67638

sec.FileSize = uint64(w.here()) - sec.Offset

and that Offset might be wrong, perhaps we need to use compressionOffset, so I tried that

diff --git a/pkg/elfwriter/elfwriter.go b/pkg/elfwriter/elfwriter.go
index ff9d28b..2138cf1 100644
--- a/pkg/elfwriter/elfwriter.go
+++ b/pkg/elfwriter/elfwriter.go
@@ -554,17 +554,17 @@ func (w *Writer) writeSections() {
                        if sec.Type == elf.SHT_NULL {
                                continue
                        }
-                       // TODO(kakkoyun): Next iterations: Compress DWARF sections when enabled.
-                       // if w.debugCompressionEnabled {}
+
                        r := sec.Open()
-                       if sec.Flags&elf.SHF_COMPRESSED != 0 {
-                               w.writeCompressedFrom(r, w.compressionHeader(sec))
+                       if sec.Flags&elf.SHF_COMPRESSED == 0 {
+                               sec.Size = sec.FileSize
+                               w.writeFrom(r)
+                       } else {
+                               header := w.compressionHeader(sec)
+                               w.writeCompressedFrom(r, header)
+                               sec.FileSize = uint64(w.here()) - uint64(header.compressionOffset)
                        }
-                       w.writeFrom(r)
                }
-               sec.FileSize = uint64(w.here()) - sec.Offset
-               // Unless the section is not compressed, the Size and FileSize is the same.
-               sec.Size = sec.FileSize
        }
 
        // Start writing the section header table.
@@ -812,6 +812,11 @@ func (w *Writer) compressionHeader(s *elf.Section) *compressionInfo {
        default:
                w.err = fmt.Errorf("unknown ELF class: %v", w.fhdr.Class)
        }
+
+       if c.compressionType != 0 && c.compressionType != elf.COMPRESS_ZLIB {
+               fmt.Println("... Section name=", s.Name, "compression type", c.compressionType, "at addr", fmt.Sprintf("%x", w.here()))
+       }
+
        return c
 }

but unfortunately this did not do it

[kakkoyun]

We actually need to implement a compression writer. AFAIR, it was left as future work. Sorry if I haven't opened the issue for that.
Let's keep this one to track the progress. Go 1.19 will have compressed DWARF sections by default.