Unable to build googleapis for ruby

[blowmage]

Hey all, I've been poking around protobuf for a couple months now and it looks really exciting!

I am unable to generate ruby files for the google/googleapis project. When I have a clean install of grpc and protobuf (meaning the /usr/local/bin/grpc_ruby_plugin file was created successfully) I can attempt to generate the ruby protobuf files, but I run into the following error:

~/googleapis$ make OUTPUT=./output LANGUAGE=ruby
mkdir -p ./output
protoc --proto_path=.:/usr/local/include --ruby_out=./output --grpc_out=./output --plugin=protoc-gen-grpc=/usr/local/bin/grpc_ruby_plugin google/rpc/code.proto
...
mkdir -p ./output
protoc --proto_path=.:/usr/local/include --ruby_out=./output --grpc_out=./output --plugin=protoc-gen-grpc=/usr/local/bin/grpc_ruby_plugin /usr/local/include/google/protobuf/compiler/plugin.proto
--ruby_out: google/protobuf/compiler/plugin.proto: Can only generate Ruby code for proto3 .proto files.
Please add 'syntax = "proto3";' to the top of your .proto file.

make: *** [/usr/local/include/google/protobuf/compiler/plugin.pb.cc] Error 1

It turns out the google/protobuf/compiler/plugin.proto and google/protobuf/descriptor.proto files are still using the proto2 syntax, and the ruby generator only understands proto3. Is there an ETA for these files migrating to proto3?

[xfxyjwf]

These files will remain in proto2 syntax and there is no plan to migrate them to proto3. I think the Makefile in googleapis repo is flawed. It shouldn't try to run protoc on descriptor.proto and plugin.proto. Generated code of these two protos are already integrated to protobuf runtime lib.

[blowmage]

Those two proto files are used in googleapis and are expected to be output when running protoc. For example, the following happens when requiring the generated Pub/Sub ruby code:

$ irb -I .
2.2.1 :001 > require "google/pubsub/v1/pubsub_services"
/home/vagrant/.rvm/gems/ruby-2.2.1/gems/grpc-0.11.0/lib/grpc/generic/client_stub.rb:173: warning: circular argument reference - parent
LoadError: cannot load such file -- google/protobuf/descriptor
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /vagrant/lib/google/api/annotations.rb:7:in `<top (required)>'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /vagrant/lib/google/pubsub/v1/pubsub.rb:6:in `<top (required)>'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /vagrant/lib/google/pubsub/v1/pubsub_services.rb:5:in `<top (required)>'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from (irb):1
    from /home/vagrant/.rvm/rubies/ruby-2.2.1/bin/irb:11:in `<main>'

I'm not sure how this files are "already integrated to the protobuf runtime". After that Pub/Sub file is loaded both the grpc and google-protobuf gems have also been loaded. Loading grpc and google-protobuf first doesn't change the behavior. The generated ruby code is looking to load both google/protobuf/descriptor and google/protobuf/compiler/plugin, which to me looks like they are also expected to be generated. Additionally, there are other proto3 files that are also missing, like google/protobuf/empty, because protoc can't generate them because of the error when running.

My guess is that since protobuf supports both proto2 and proto3 in other languages they never run into this. But since protobuf only supports proto3 in ruby, not being able to generate those files this is a major blocker for anyone wanting to use access the google APIs with gRPC in ruby.

[xfxyjwf]

descriptor.proto and other protos under google/protobuf package are part of protobuf library. Their generated code (e.g., google/protobuf/descriptor.rb) should have been included in google-protobuf gem. This hasn't been fixed in ruby though.

[blowmage]

Is there an issue for getting this code in the google-protobuf gem I can follow?

[blowmage]

I created a new issue that might supplant this one.

[haberman]

The most recent Ruby packages include well-known types.