update travis matrix (add truffleruby, remove rubinius)

[iliabylich]

No description provided.

[bbatsov]

Nice! I didn't even know that Rbx was still being developed.

[iliabylich]

looks like rbx 5.0 has been released a week ago 😄 However, I wasn't able to find any announcements or articles about it (maybe they are still WIP)

[iliabylich]

@eregon It takes ~15 minutes to run a test suite (for MRI 2.7 it's only 30s). Is there any way to make it faster?

[eregon]

I'll take a look.

[eregon]

An easy gain that makes it about 2x faster is to not run with COVERAGE=1.
That can be done by swapping bundle exec rake test_cov with bundle exec rake test.
Locally, that reduce the time from 250s specs 5min28s total to 135s specs 2min24s total.

Coverage is usually not that expensive, but here there is a custom :line TracePoint and it seems TruffleRuby splits & compiles it many times which is really slow (something to fix).

BTW on MRI I also see that COVERAGE=1 is >2x slower (11.3s with, 3.5s without for specs), but of course it doesn't matter with those times.

[eregon]

Something else that also speeds it up about 2x is to use the latency Mode, which improves warmup (i.e., JIT faster) by reducing splitting and inlining (at the cost of peak performance, but rarely relevant for tests).

TRUFFLERUBYOPT=--engine.Mode=latency bundle exec rake test

gives me locally 68s specs 1min17s total.

For the record,
initial TravisCI run: 617s specs https://travis-ci.org/github/whitequark/parser/jobs/690086457#L439
rake test: 290s specs https://travis-ci.org/github/whitequark/parser/jobs/690130316

[iliabylich]

@eregon Thanks! I'll play with locally. Compiling parsers using racc is very slow atm. Also, does it make sense to use JIT at all? I mean, we run different codepaths on purpose, is it possible to completely disable runtime optimizations?

[eregon]

Compiling parsers using racc is very slow atm.

It takes about 5s per file with TRUFFLERUBYOPT=--engine.Mode=latency and 10s without for me locally.

Also, does it make sense to use JIT at all? I mean, we run different codepaths on purpose, is it possible to completely disable runtime optimizations?

Yes, and in fact it's significantly slower with the JIT disabled. It's always useful to e.g. speculate that Integer#+ doesn't overflow, etc.

[iliabylich]

Yes, I've got similar results locally (16s -> 8s). Also, I've noticed that using pure ruby version of racc saves extra second, but there's no clean way to enforce it. Thanks a lot, I'll add this option to CI config.

Side question, is it applicable to all pure/ext libraries like json/ext vs json/pure?

[eregon]

Interesting, how did you force using the Ruby version of racc?

Side question, is it applicable to all pure/ext libraries like json/ext vs json/pure?

No, it depends. Sometimes the Ruby version is faster and sometimes the C extension is faster.

[iliabylich]

well, there's no simple way to do it. Here - https://github.com/ruby/racc/blob/master/lib/racc/parser.rb#L197 - I added raise LoadError to pretend that there's no compiled cparse.[so|bundle|dll] and to come to the rescue branch that configures racc to use pure ruby version. At the end you should see the difference in the output of

$ racc --runtime-version
racc runtime version 1.4.15 (rev. $Id: 2bd697d1146b280d8d3878e21e556ef769484a37 $); c core version 1.4.15 (rev. 1.8)

# vs

$ racc --runtime-version
racc runtime version 1.4.15 (rev. $Id: 2bd697d1146b280d8d3878e21e556ef769484a37 $); ruby core version 1.4.15 (rev. 2bd697d1146b280d8d3878e21e556ef769484a37)

also, IIRC racc ... uses parsefilegenerator.rb, so 1s is spent to load C version.