Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go get error on github.com/hashicorp/vault/command - undefined field or method for joinReq type #10922

Closed
gregfolker opened this issue Feb 15, 2021 · 8 comments

Comments

@gregfolker
Copy link

gregfolker commented Feb 15, 2021

Describe the bug
Attempting to download the module github.com/hashicorp/vault results in the following error with github.com/hashicorp/vault/command

$ go get github.com/hashicorp/vault@latest
go: downloading github.com/apple/foundationdb/bindings/go v0.0.0-20190411004307-cd5c9d91fad2
go: downloading github.com/sasha-s/go-deadlock v0.2.0
go: downloading google.golang.org/appengine v1.6.5
go: downloading golang.org/x/tools v0.0.0-20200616133436-c1934b75d054
go: downloading github.com/StackExchange/wmi v0.0.0-20190523213315-cbe66965904d
go: downloading github.com/jstemmer/go-junit-report v0.9.1
go: downloading golang.org/x/lint v0.0.0-20200302205851-738671d3881b
go: downloading honnef.co/go/tools v0.0.1-2020.1.3
go: downloading github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5
go: downloading github.com/go-ole/go-ole v1.2.4
go: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go: downloading golang.org/x/mod v0.3.0
# github.com/hashicorp/vault/command
../../go/pkg/mod/github.com/hashicorp/vault@v1.6.2/command/operator_raft_join.go:189:10: joinReq.AutoJoin undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoin)
../../go/pkg/mod/github.com/hashicorp/vault@v1.6.2/command/operator_raft_join.go:190:10: joinReq.AutoJoinScheme undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoinScheme)
../../go/pkg/mod/github.com/hashicorp/vault@v1.6.2/command/operator_raft_join.go:191:10: joinReq.AutoJoinPort undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoinPort)

Exact same issue was reported here but closed due to no response from the submitter as #10405 - I am providing as much info as I can that was requested from this original ticket submission

To Reproduce
Steps to reproduce the behavior:

  1. Run go get github.com/hashicorp/vault@latest
  2. See error

Expected behavior
No error should be observed; Module should be downloaded as expected

Environment:
Issue is observed in multiple environments on multiple systems

  • Server Operating System/Architecture: CentOS Linux release 8.3.2011
  • Go Version: go1.14.12 linux/amd64
  • go env
    GO111MODULE="on"
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/home/greg/.cache/go-build"
    GOENV="/home/greg/.config/go/env"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOINSECURE=""
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/home/greg/go"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/usr/lib/golang"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="gcc"
    CXX="g++"
    CGO_ENABLED="1"
    GOMOD="/home/greg/dev/securesigntool/go.mod"
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build986183306=/tmp/go-build -gno-record-gcc-switches"
  • uname -pisorv && lscpu
    Linux 4.18.0-240.10.1.el8_3.x86_64 Initial Website Import #1 SMP Mon Jan 18 17:05:51 UTC 2021 x86_64 x86_64 GNU/Linux
    Architecture: x86_64
    CPU op-mode(s): 32-bit, 64-bit
    Byte Order: Little Endian
    CPU(s): 40
    On-line CPU(s) list: 0-39
    Thread(s) per core: 2
    Core(s) per socket: 10
    Socket(s): 2
    NUMA node(s): 2
    Vendor ID: GenuineIntel
    CPU family: 6
    Model: 85
    Model name: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
    Stepping: 4
    CPU MHz: 1186.899
    CPU max MHz: 3000.0000
    CPU min MHz: 800.0000
    BogoMIPS: 4400.00
    Virtualization: VT-x
    L1d cache: 32K
    L1i cache: 32K
    L2 cache: 1024K
    L3 cache: 14080K
    NUMA node0 CPU(s): 0-9,20-29
    NUMA node1 CPU(s): 10-19,30-39
    Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_pkg_req pku ospke md_clear flush_l1d

Additional info
go clean -modcache does not resolve the issue

@HridoyRoy
Copy link
Contributor

Hi @gregfolker , so far I have been unable to replicate this failure locally. Also, I'm not sure if we currently support including vault as a dependency, but I will inquire and report back.

Thanks for filing this issue!

@gregfolker
Copy link
Author

Hi @HridoyRoy - thank you for the reply and investigating

An older version of Vault has been working fine as a module and dependency to the same project for some time now. v1.3.2 - I've been working on porting this tool to another source, which is where I started hitting this issue. Due to other dependencies I ended up just trying to get the latest version of Vault (If you're unable to repro locally, make sure you're getting the latest version? I can repro this on multiple systems. go get github.com/hashicorp/vault@latest)

The only CL I can find that recently touched the file where the error is occuring is #10095 - It looks like the AutoJoin support was added with Vault v1.6.x? Is it possible some backwards compatibility may be broken because of this change?

Thanks!
Greg

@gregfolker
Copy link
Author

gregfolker commented Feb 23, 2021

I got to play around with this more today. I'm able to download the module successfully and build the code if I manually modify the file with the problem, which appears to be in the api/ submodule of vault

From looking at this file at the head of master: https://github.com/hashicorp/vault/blob/master/api/sys_raft.go

Updated the file located at vendor/github.com/hashicorp/vault/api/sys_raft.go to have the structure fields (Previously, it did not)

This leads me to believe somehow my submodule for api/ was out of sync with vault. Any idea how this could happen or how I can prevent it in the future?

@gregfolker
Copy link
Author

gregfolker commented Mar 8, 2021

Documentation for the vault/api submodule shows the type RaftJoinRequest as not having these fields defined here https://pkg.go.dev/github.com/hashicorp/vault/api?utm_source=godoc#RaftJoinRequest

This is at what the documentation indicates is the latest version, v1.0.4

These fields are defined at the head of master: https://github.com/hashicorp/vault/blob/master/api/sys_raft.go - but I don't see any release past v1.0.4

Attempting to run go get github.com/hashicorp/vault/api@latest locally results in the following error:

go: github.com/hashicorp/vault/api latest => v1.0.4 can't load package: package github.com/hashicorp/vault/api: ambiguous import: found package github.com/hashicorp/vault/api in multiple modules: github.com/hashicorp/vault v1.1.5 (/home/greg/go/pkg/mod/github.com/hashicorp/vault@v1.1.5/api) github.com/hashicorp/vault/api v1.0.4 (/home/greg/go/pkg/mod/github.com/hashicorp/vault/api@v1.0.4)

It looks like I have some older version of Vault (v1.1.5) cached locally that may have previously been working with this version of vault/api@v1.0.4 ? And it's failing with a newer version of Vault that expects the vault/api@latest to have the definitions for these new fields

Running go get github.com/hashicorp/vault@latest still results in this error for me:

go: github.com/hashicorp/vault latest => v1.6.3 github.com/hashicorp/vault/command ../../pkg/mod/github.com/hashicorp/vault@v1.6.3/command/operator_raft_join.go:189:10: joinReq.AutoJoin undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoin) ../../pkg/mod/github.com/hashicorp/vault@v1.6.3/command/operator_raft_join.go:190:10: joinReq.AutoJoinScheme undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoinScheme) ../../pkg/mod/github.com/hashicorp/vault@v1.6.3/command/operator_raft_join.go:191:10: joinReq.AutoJoinPort undefined (type *"github.com/hashicorp/vault/api".RaftJoinRequest has no field or method AutoJoinPort)

Thanks!
Greg

Edit:

Since the problem appears to be with importing vault/api@latest I went ahead and tried to run the following command:

go get github.com/hashicorp/vault/api@v1.0.4

From the error below, it looks like this correctly tries to import github.com/hashicorp/vault v1.6.2 as a dependency, but after trying to import vault/sdk it tries to import github.com/hashicorp/vault v1.1.5 - would this indicate that vault/sdk depends on an older version of vault that did not support these structure fields in the API? Again, it looks like these definitions were added in v1.6.x

Output of go get github.com/hashicorp/vault/api@1.0.4:

go: downloading github.com/hashicorp/vault v1.6.2
go: downloading github.com/hashicorp/vault/sdk v0.1.14-0.20210127182440-8477cfe632c0
go: downloading github.com/hashicorp/vault v1.1.5

can't load package: package github.com/hashicorp/vault/api: ambiguous import: found package github.com/hashicorp/vault/api in multiple modules:
github.com/hashicorp/vault v1.1.5 (/home/greg/go/pkg/mod/github.com/hashicorp/vault@v1.1.5/api)
github.com/hashicorp/vault/api v1.0.4 (/home/greg/go/pkg/mod/github.com/hashicorp/vault/api@v1.0.4)

Prior to this I manually removed $GOPATH/pkg/mod/github.com/hashicorp/vault@1.1.5 but it looks like that particular go get command is importing it again when I try to build

@pmmukh
Copy link
Contributor

pmmukh commented Sep 2, 2021

We don't really support building Vault from source, as that can often run into conflicts when importing both vault, along with the api and sdk modules, as you've run into here. We understand you may have your reasons to do this, but we don't offer much in the way of guarantees that this will work. As such, I'm going to go ahead and close this issue out.

@pmmukh pmmukh closed this as completed Sep 2, 2021
@gregfolker
Copy link
Author

gregfolker commented Sep 2, 2021

Is it considered building from source if I'm just trying to import vault as a module into another project? The issue was reproducible by just running go get, no build/make commands are involved. If there's a correct way to import vault's 'api/' tree, I'd love to know how. Access to exported methods and types is all that's really required, and I shouldn't have to build vault from source to accomplish that

@pmmukh
Copy link
Contributor

pmmukh commented Sep 2, 2021

Ah sorry i misspoke there, no that is not considered building from source, but also yeah we don't really support people directly importing the vault module in their code ( ergo trying to run go get github.com/hashicorp/vault@latest ).

@gregfolker
Copy link
Author

Thank you for the quick response and clarification! If importing vault as a module is not officially supported I understand

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants