jt test fast fails with JVM CE environment

[nirvdrum]

In local development I run with JT_ENV=jvm-ce. I recently pulled from master and found that I had 12 test failures that I didn't expect. I thought maybe that I had some stale data, but running jt rebuild && jt test fast didn't fix the problem. Then I tried running again with JT_ENV=jvm and the test suite passed.

I only see the problem on macOS ARM64. I tested on Linux x86_64 and did not see the problem. I haven't bisected, but I think it's likely that #3508 is when the issue was introduced.

$ /Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm-ce/bin/ruby --vm.ea --vm.esa --experimental-options --core-load-path=/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/src/main/ruby/truffleruby --vm.XX:-UseJVMCICompiler --experimental-options --engine.Compilation=false --engine.Splitting=false --polyglot /Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/mspec/bin/mspec-run -B spec/truffleruby.mspec --excl-tag fails --excl-tag slow
truffleruby 24.1.0-dev-72f94f4c, like ruby 3.2.2, GraalVM CE JVM [aarch64-darwin]
                                                                                             
1)
Inline caching for dynamically-created Regexp works for Regexp.new FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:29: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:28:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
2)
Inline caching for dynamically-created Regexp works for Regexp.union with 1 argument FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:55: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:51:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
3)
Inline caching for dynamically-created Regexp works for Regexp.union with multiple arguments FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:75: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:74:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
4)
Inline caching for dynamically-created Regexp works for String#scan FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:107: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:106:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
5)
Inline caching for dynamically-created Regexp works for String#sub FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:151:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
6)
Inline caching for dynamically-created Regexp works for String#sub! FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:181:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
7)
Inline caching for dynamically-created Regexp works for String#gsub FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:211:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
8)
Inline caching for dynamically-created Regexp works for String#gsub! FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:241:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
9)
Inline caching for dynamically-created Regexp works for String#match FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:264:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
10)
Inline caching for dynamically-created Regexp works for String#match? FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:287:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
11)
Inline caching for dynamically-created Regexp works for Symbol#match FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:310:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
                                                                                             
12)
Inline caching for dynamically-created Regexp works for Symbol#match? FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:333:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
[- | ==================100%================== | 00:00:00]     12F      0E 

[eregon]

Do you have JT_SPECS_COMPILATION or JT_SPECS_SPLITTING set in the environment maybe?
These specs do seem to pass fine in our darwin-aarch64 CI.

[nirvdrum]

I have JT_SPECS_COMPILATION enabled. When I remove that, the specs pass even on JVM CE.

[eregon]

OK, that's kind of expected then since JT_SPECS_COMPILATION=false does --engine.Splitting=false (unless JT_SPECS_SPLITTING=true).
So we should check that condition in the guard then.

Ideally we would check specifically that splitting is enabled.
I think that might be possible by calling DirectCallNode#isCallTargetCloned() on something we know we always-split like Array#[].
We could create a dummy DirectCallNode e.g. in CoreLibrary, call cloneCallTarget() and then if isCallTargetCloned() returns true it should mean splitting is enabled.

@andrykonchin Could you look into that?