SIGSEGV from String#replace in rubyzip test suite

[jdleesmiller]

Thank you for taking the time to let us know that you're having an issue with Rubinius. We apologize for the inconvenience.

Please respond to the following questions to help us fix the issue. Please leave the form contents in place. If a question isn't relevant, please respond with N/A.

Please include a link to a gist (https://gist.github.com/) for terminal output, including backtraces, rather than pasting them inline in the issue.

1. What command did you run?

The test suite for rubyzip: https://travis-ci.org/rubyzip/rubyzip/jobs/509599516

2. What behavior did you expect?

I expected test suite to run.

3. What behavior did you get instead?

I observed SIGSEGV. Please see https://travis-ci.org/rubyzip/rubyzip/jobs/509599516 for full output.

4. What version of Rubinius?

Note that we accept issue reports for the current release of Rubinius. That version can be found at https://rubinius.com/install/. If there is a reason that you cannot use the current version, other than not having installed the current version, please include that reason in your issue.

Output of rbx -v: rubinius 3.107 (2.3.1 387c4887 2018-07-15 5.0.0git-929163d) [x86_64-linux-gnu]

5. What version of operating system?

Output of uname -a: Not available, as it's on Travis CI, but it says kernel 4.4.0-101-generic.

6. What is your operating system distribution, if your operating system has more than one?

My distribution is Ubuntu 14.04.5 LTS .

7. How did you build your version of Rubinius?

• I used RVM.
• I used ruby-build.
• I used ruby-install.
• I built manually from a tarball.
• I built manually from a git clone.
• I installed a binary.

8. Does this issue involve proprietary code?

• Yes, this issue involves proprietary code that I cannot share.
• Yes, this issue involves proprietary code, but I am able to share it under certain conditions.
• No, this issue doesn't involve proprietary code.

9. Are you able to help us debug the issue?

• Yes, I'm able to help debug, including running commands under lldb.
• No, I'm not able help debug or I don't have time to help.

[brixen]

@jdleesmiller thanks for opening an issue for this. Rubinius 4.3 runs the tests without any segv, but there are several errors like this that I need to investigate:

ZipFileTest#test_compound1:
RuntimeError: RString does not have pinned ByteArray

A bunch of stuff was changed with how object memory headers work, so these errors may be Rubinius issues (probably), or they may be issues with assumptions about the C-API that Rubinius won't be able to support.

If you want to switch to Rubinius 4.x, you should be able to use rbx-4 in your .travis.yml (see https://travis-ci.org/rubinius/travis-canary/jobs/554915953). Only the latest version of Rubinius is supported, so rbx-4 should install whatever is the latest.

[brixen]

@jdleesmiller I ran 4.3 in a loop and see a segv, so I'm going to re-open this so I can investigate. I still recommend using 4.x on Travis.

[brixen]

@jdleesmiller I have a bit more info if you're curious... one of the recent changes I've made is making the garbage collector recursive, so that during collection, if a segv like this happens, I can see the linked chain of objects in this subgraph. In one of the segvs, a ByteArray object (which is the memory for a String) has a _klass_ pointer that is corrupt. But it's not randomly corrupt:

_klass_ = 0x000000010bca5800 // the ByteArray klass pointer

object_ = 0x000000010bca5848 // the actual ByteArray klass address

This ByteArray instance is in a String that is a method_id in a MethodTable of a Class instance in the core library that's loaded during boot and has been through multiple GC cycles to get to this point.

So this looks like something is writing into a rather random location in memory. If the ByteArray were in a String that was being read or written by the C-API, I'd say that would be a big red flag, but it's not.

I'll look at a few more segv instances running these tests and see if I can learn anything more.

Thanks again for opening the issue.

[jdleesmiller]

Glad it helped!

I've put CI onto rbx-4. rubyzip/rubyzip#399

I should add that rbx-4 no longer seems to install on Travis with the default config, because the default config has now moved from Ubuntu Trusty to Ubuntu Xenial.

It looks like https://github.com/rubinius/travis-canary sets dist: trusty so is still working there, but others may end up on xenial, where it appears not to work. (And eventually we'll need to move to xenial.)

[brixen]

@jdleesmiller ah, thanks for letting me know about Xenial. We used trusty way back to get on the newest version when Travis default was behind, and now we are behind. 😢 I'll make sure it works on Xenial.

[brixen]

@jdleesmiller I'm building Xenial binaries, but it appears that RVM has some issue installing them on Travis 😞 https://travis-ci.org/rubinius/travis-canary/jobs/558292568

[brixen]

Much of the internals of Rubinius have been completely or mostly rewritten in the past couple years. This includes the garbage collector, concurrency facilities, Fibers, much of the instruction set, and a migration away from "primitive" functions that implement Ruby features.

Since a number of segfaults or process hangs have occurred in these features over time, this issue may be fixed.

The focus for Rubinius in the near term is on the following capabilities:

1. Instruction set
2. Debugger
3. Profiler
4. Just-in-time compiler
5. Concurrency
6. Garbage collector

Contributions in the form of PRs for any of the areas of focus above are appreciated. Once these capabilities are more robust, it will be possible to more efficiently debug and fix any process crashes.

Other than these core capabilities, PRs to fix any specific issue are always welcome.