Unable to install grpc gem with JRuby

[blowmage]

The glcoud-ruby project has an issue opened by a user (googleapis/google-cloud-ruby#712) that is unable to install the latest version due to the dependencies on the grpc and google-protobuf gems not installing on jruby. I am not able to get the grpc gem to install on my mac using versions 9.1.1.0 or 9.0.0.0. I have the latest libgrpc installed, using both homebrew and built from source.

Here is the install on mac:

$ ruby -v
jruby 9.1.1.0 (2.3.0) 2016-05-19 fe84e89 Java HotSpot(TM) 64-Bit Server VM 25.91-b14 on 1.8.0_91-b14 +jit [darwin-x86_64]

$ gem install --ignore-dependencies grpc
Fetching: grpc-0.14.1.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing grpc:
    ERROR: Failed to build gem native extension.

    current directory: /Users/blowmage/.gem/repos/jruby-9.1.1.0-grpc/gems/grpc-0.14.1/src/ruby/ext/grpc
/Users/blowmage/.rubies/jruby-9.1.1.0/bin/jruby -r ./siteconf20160524-43017-endf8.rb extconf.rb
NotImplementedError: C extensions are not supported
    <top> at /Users/blowmage/.rubies/jruby-9.1.1.0/lib/ruby/stdlib/mkmf.rb:1
  require at org/jruby/RubyKernel.java:944
   (root) at /Users/blowmage/.rubies/jruby-9.1.1.0/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
    <top> at extconf.rb:30

extconf failed, exit code 1

Gem files will remain installed in /Users/blowmage/.gem/repos/jruby-9.1.1.0-grpc/gems/grpc-0.14.1 for inspection.
Results logged to /Users/blowmage/.gem/repos/jruby-9.1.1.0-grpc/extensions/universal-java-1.8/2.3.0/grpc-0.14.1/gem_make.out

Strangly enough, it does install on linux! Here is that install:

$ ./bin/jruby -v
jruby 9.1.1.0 (2.3.0) 2016-05-19 fe84e89 OpenJDK 64-Bit Server VM 24.95-b01 on 1.7.0_95-b00 +jit [linux-x86_64]

$ gem install --ignore-dependencies grpc
Building native extensions.  This could take a while...

[murgatroid99]

As far as I know, we haven't done any work to get the gem working on jruby, and frankly, I'm surprised that it work at all.

[blowmage]

Well, I said it installed, not worked. 😉 We are also blocked by the fact that google-protobuf fails to install.

[murgatroid99]

The primary issue here is that the library is built on a C extension which uses a variety of Ruby C APIs. I don't think that is supposed to work with jruby.

[raggi]

I believe that sufficiently modern jruby supports C extensions with sufficiently modern versions of OpenJDK

[blowmage]

Ahh, well that explains that!

[apolcyn]

@raggi looking into this, but as far as I can see jruby support for the mri-c extensions was tried earlier on but deprecated in jruby 1.7.0

[JasonLunn]

My understanding is that support for native C extensions are back in JRuby 9000, which is probably all anyone needs to worry about any more as the 1.7.x series is being end-of-lifed at the end of this year.

[teodor-pripoae]

Hi,

Any update on this issue ?

[JasonLunn]

Is the C extensions issue still blocking progress on this? Has a ruby wrapper around the Java client jar been considered?

[apolcyn]

@JasonLunn at least in the near-future I unfortunately it's not currently on the road map. With the large amount of c-extension code in the gem, wrapping the java client should be pretty involved - basically a reimplementation.
The truffle solution interpreted-C-extension might be an option but I'm not sure about the consequences - the core functionality of the grpc-ruby is pretty much all a C-library.

[JasonLunn]

@apolcyn: Assume for the purposes of argument that I was willing to try to create such a wrapper around grpc-java, since I'm not able to switch to truffle in production any time soon. Would you want the PR to be submitted here, since this repository seems to own the Ruby gem, or would you want the PR submitted to the grpc-java? The goal in either case being not to create a new grpc-java gem that only works on JRuby, but to create a version of the grpc gem that has a platform of java?

[apolcyn]

I think this depends mostly on:

1. Are you looking for a drop-in replacement of the current grpc-ruby that can be used on the java ruby platform? E.g., API-compatible with grpc-ruby, don't have to regenerate grpc-related .proto files?

• In this case this should probably go into grpc-ruby repo, as an addition to the grpc-ruby project. But this would also have to fit in to the same testing setup as grpc.

2. Is a grpc implementation that just works on jruby good enough? Has wrapping the grpc-java client from within jruby "app-level" code been considered?
3. The main work that the grpc-ruby C extensions do is wrap the grpc C-core library. This could potentially also be wrapped with JNI, and then possibly a lot of the grpc-ruby "pure-ruby" code could be reused in that way.

For 1) and 2)3), these should be introducing fairly big changes to the grpc-ruby project. Just to save going down the wrong direction, could you please provide a proposal before the PR?

[apolcyn]

note though that an addition to grpc-ruby like this might be difficult to bring in, just because of the overall complexity it might add.

[teodor-pripoae]

Or you can use extract the C extension + some light wrapper code into a grpc-ruby-ext gem which grpc gem depends on only for CRuby platform. For java platform it can depend on grpc-java-ext which implements the same API wrapper and can live in the grpc-java codebase.

I'm working on a project which uses Google Cloud APIs and I would like to use run on JRuby, too. It would be nice if the JRuby version would support the same grpc API without any code change.

[apolcyn]

also @JasonLunn if a plan was created it would probably fit into grpc project-wide proposals. see https://github.com/grpc/proposal

[JasonLunn]

@apolcyn - what I'm looking for is the ability for Ruby dependency management tools like bundler to Do the Right Thing™️ when higher level gems (like google-cloud-datastore for example) express a dependency on the grpc gem under a JRuby interpreter. From the perspective of the Gemspec, the approach that the Google Protobuf took is near ideal for my purposes: when executed on the JRuby interpreter, include the jar file rather than the C implementation files. In Protobuf's case, the job of getting your hands on the jar is a little bit more straightforward as it is built from the same repo as the gem, but there already exist tools to leverage Maven dependencies from JRuby rather than store the jar binary alongside the ruby source. From there it is about building a shim over the Java implementation (to the extent necessary) to provide Ruby API compatibility.

[JasonLunn]

I took a stab at writing a proposal, @apolcyn - see the PR

Any feedback would be appreciated.

CC: @teodor-pripoae @blowmage

[muxi]

@apolcyn what is the status of this issue?

[apolcyn]

I'll close this since I don't think there will be any action on this from grpc team. If there's progress on this, I suspect it would come as an open source contribution; (I believe @JasonLunn has made some progress here).