Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buf build error on bazel external dependency #2851

Open
cyrilhuang opened this issue Mar 28, 2024 · 4 comments
Open

buf build error on bazel external dependency #2851

cyrilhuang opened this issue Mar 28, 2024 · 4 comments

Comments

@cyrilhuang
Copy link

We're using buf to do some protobuf breaking detections. Our repo is developing on bazel. Wewant to first buf build to generate image, then to compare reulsts based on image.
In one of our proto_library, we depends on external dependency (which is not available on buf remote registry), and import that .proto file in our own code.
This proto definition can be found in bazel-out/../../../external/riegeli. bazel will call protoc -I to automatically fix compiling issue.

However, under buf, we cannot add that include path to buf cli/buf.yaml. One official solution is that to use buf.work.yaml. However, in our repo, all proto files are on root, and . is not allowed in buf.work.yaml, therefore creating a buf.yaml in riegeli seems not possible here.
example.proto:5:8:read riegeli/records/records_metadata.proto: file does not exist

One temporary solution is to copy that directory into our root directory, then call buf build will work. But I'm wondering what might be the official way to resolve it.

buf version:
1.29.0

@bufdev
Copy link
Member

bufdev commented Apr 1, 2024

You'll have to provide us a specific, reproducible example, and then we'd be happy to investigate. Let us know!

@cyrilhuang
Copy link
Author

cyrilhuang commented Apr 2, 2024

@bufdev I'll given a minimum structure of our codebase, please help take a look, thanks
In the following example, we can run bazel build //b:proto, since bazel downloads external dependency to local, and add include path to corresponding protoc arg

WORKSPACE

workspace(name = "example")

load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(
    name = "riegeli",
    commit = "904c0c263b8632265103f0066c168a92c7713b07",
    remote = "ssh://git@github.com:google/riegeli.git",
)

a/BUILD

package(default_visibility = ["//visibility:public"])

proto_library(
    name = "config",
    srcs = ["config.proto"],
)

a/config.proto

syntax = "proto3";
package a;
message config {}

b/BUILD

package(default_visibility = ["//visibility:public"])

proto_library(
    name = "config",
    srcs = ["config.proto"],
    deps = [
        "//a:config",
        "@riegeli//riegeli/records:records_metadata_proto",
    ]
)

b/config.proto

syntax = "proto3";
package b;

import "a/config.proto";
import "riegeli/records/records_metadata.proto";

@bufdev
Copy link
Member

bufdev commented Apr 3, 2024

What we're looking for:

  • The code, in a consumable manner such as a temporary github repository, that we can inspect and build ourselves
  • What you see
  • What you expected to see

If you can give us something that we can run ourselves, we're happy to look into it.

@cyrilhuang
Copy link
Author

@bufdev I put the above example in https://github.com/cyrilhuang/buf_example/tree/main,

In this repo, we can build all proto files with bazel by running:

bazel build //...

However, when running buf, build failed due to cannot find external files:

buf build --disable-symlinks
b/config.proto:5:8:read riegeli/records/records_metadata.proto: file does not exist

The mnissing file can be found in bazel-buf_example/external/riegeli/, which bazel added during compiling proto files (buf_example is the repo name).
bazel-buf_example/external/riegeli/riegeli/records/records_metadata.proto

I'm wondering how to deal with external dependency not available on buf remote registry like riegeli, thanks.

bazel version:7.1.1
buf version: 1.30.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants