unrecognized import path "vbom.ml/util"

[Spredzy]

Bug Report

When running make install, the execution fails with the following trace

#> make install 
which: no controller-gen in (/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/.local/bin:/home/spredzy/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/spredzy/.fzf/bin:/home/spredzy/bin:/home/spredzy/bin)
which: no kustomize in (/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/.local/bin:/home/spredzy/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/spredzy/.fzf/bin:/home/spredzy/bin:/home/spredzy/bin)
Makefile:125: warning: overriding recipe for target 'test'
Makefile:30: warning: ignoring old recipe for target 'test'
go: creating new go.mod: module tmp
go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0
/home/spredzy/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go: creating new go.mod: module tmp
go get: sigs.k8s.io/kustomize/kustomize/v3@v3.5.4 requires
        sigs.k8s.io/kustomize/cmd/kubectl@v0.0.3 requires
        k8s.io/kubectl@v0.0.0-20191219154910-1528d4eea6dd requires
        vbom.ml/util@v0.0.0-20160121211510-db5cfe13f5cc: unrecognized import path "vbom.ml/util": https fetch: Get "https://vbom.ml/util?go-get=1": dial tcp: lookup vbom.ml on [fd0f:ee:b0::1]:53: no such host
make: *** [Makefile:96: kustomize] Error 1

I have found this issue opened 5 days ago kubernetes/kubectl#925, and the corresponding fix merged a day later kubernetes/kubernetes#94451

I suspect this will need to be brought up to the kustomize team so they update their kubectl deps, and the operator-sdk team will in turn update their kustomize version requirements.

What did you do?

Followed the official documentation (https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/) until the make install step

What did you expect to see?

Not an error.

What did you see instead? Under which circumstances?

#> make install 
which: no controller-gen in (/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/.local/bin:/home/spredzy/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/spredzy/.fzf/bin:/home/spredzy/bin:/home/spredzy/bin)
which: no kustomize in (/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/Projects/virtualenvs/product-docs/bin:/home/spredzy/.local/bin:/home/spredzy/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/spredzy/.fzf/bin:/home/spredzy/bin:/home/spredzy/bin)
Makefile:125: warning: overriding recipe for target 'test'
Makefile:30: warning: ignoring old recipe for target 'test'
go: creating new go.mod: module tmp
go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.3.0
/home/spredzy/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go: creating new go.mod: module tmp
go get: sigs.k8s.io/kustomize/kustomize/v3@v3.5.4 requires
        sigs.k8s.io/kustomize/cmd/kubectl@v0.0.3 requires
        k8s.io/kubectl@v0.0.0-20191219154910-1528d4eea6dd requires
        vbom.ml/util@v0.0.0-20160121211510-db5cfe13f5cc: unrecognized import path "vbom.ml/util": https fetch: Get "https://vbom.ml/util?go-get=1": dial tcp: lookup vbom.ml on [fd0f:ee:b0::1]:53: no such host
make: *** [Makefile:96: kustomize] Error 1

Environment

Operator type:

/language go

Kubernetes cluster type:

N/A

$ operator-sdk version

operator-sdk version: "v1.0.0", commit: "d7d5e0cd6cf5468bb66e0849f08fda5bf557f4fa", kubernetes version: "v1.18.2", go version: "go1.13.11 linux/amd64", GOOS: "linux", GOARCH: "amd64"

$ go version (if language is Go)

go version go1.14.6 linux/amd64

$ kubectl version

N/A

Possible Solution

Ping the team in charge of kustomize so they update their requirements and the operator-sdk team can in turn update theirs.

Additional context

kubernetes/kubectl#925

[camilamacedo86]

HI @Spredzy,

Also, I am unable to reproduce your issue.

The go version required is 1.13.x. Could you please check it with this version?
Also, by looking :

Makefile:125: warning: overriding recipe for target 'test'
Makefile:30: warning: ignoring old recipe for target 'test'

Shows that instead of you replace the test target as described in the doc you added it. Could you please check?

[Spredzy]

The go version required is 1.13.x. Could you please check it with this version?

Using this version make the make install pass. Thanks @camilamacedo86 for quick answer. Closing.

[camilamacedo86]

Really thank you for let us know.

[imunhatep]

I'm curious why not to update to go 1.14/1.15? is there any specific reason?

[mhrivnak]

FWIW: I think this only succeeds on 1.13 because, from release notes: "As of Go 1.13, the go command by default downloads and authenticates modules using the Go module mirror and Go checksum database run by Google."

So for now, successful builds are depending on that continuing to be available in Google's module cache or some other cache. It may work, but it's a dependency to be aware of.

[joelanford]

So this issue isn't go version dependent. It is GOPROXY dependent.

I just cleaned my local modcache, set GOPROXY=direct and attempted to compile from source with go 1.13, 1.14, and 1.15. All fail with:

$ GOPROXY=direct make install
go: k8s.io/kubectl@v0.18.2 requires
	vbom.ml/util@v0.0.0-20160121211510-db5cfe13f5cc: unrecognized import path "vbom.ml/util" (https 
fetch: Get https://vbom.ml/util?go-get=1: dial tcp: lookup vbom.ml: No address associated with 
hostname)

Setting GOPROXY=https://proxy.golang.org,direct gets things working for all go versions since 1.13.

Different distros and installation methods set the default GOPROXY differently. See discussion here.

So to build SDK from source, it is currently required to set GOPROXY to include a proxy that serves the now-defunct vbom.ml/util package. To be fair, one of the stated goals of module proxies is to solve this exact problem.

From the proxy.golang.com FAQ:

I removed a bad release from my repository but it still appears in the mirror, what should I do?

Whenever possible, the mirror aims to cache content in order to avoid breaking builds for people that depend on your package, so this bad release may still be available in the mirror even if it is not available at the origin. The same situation applies if you delete your entire repository. We suggest creating a new version and encouraging people to use that one instead.

---

For the real solution, see kubernetes/kubectl#925

We can likely use a replace to fix this until this fix propagates through all of our dependencies:

replace vbom.ml/util => github.com/fvbommel/util v0.0.0-20160121211510-db5cfe13f5cc

Re-opening so we can keep track that we need to fix this.

[mhrivnak]

I reached out to the maintainer, and he was able to get the redirect up and running again for vbom.ml/util. This problem should be resolved for now, and the new import path should be picked up naturally whenever we rebase onto k8s 1.20.

[estroz]

Given @mhrivnak's comment, this is resolved.

[graham924]

So this issue isn't go version dependent. It is GOPROXY dependent.

I just cleaned my local modcache, set GOPROXY=direct and attempted to compile from source with go 1.13, 1.14, and 1.15. All fail with:

$ GOPROXY=direct make install
go: k8s.io/kubectl@v0.18.2 requires
	vbom.ml/util@v0.0.0-20160121211510-db5cfe13f5cc: unrecognized import path "vbom.ml/util" (https 
fetch: Get https://vbom.ml/util?go-get=1: dial tcp: lookup vbom.ml: No address associated with 
hostname)

Setting GOPROXY=https://proxy.golang.org,direct gets things working for all go versions since 1.13.

Different distros and installation methods set the default GOPROXY differently. See discussion here.

So to build SDK from source, it is currently required to set GOPROXY to include a proxy that serves the now-defunct vbom.ml/util package. To be fair, one of the stated goals of module proxies is to solve this exact problem.

From the proxy.golang.com FAQ:

I removed a bad release from my repository but it still appears in the mirror, what should I do?
Whenever possible, the mirror aims to cache content in order to avoid breaking builds for people that depend on your package, so this bad release may still be available in the mirror even if it is not available at the origin. The same situation applies if you delete your entire repository. We suggest creating a new version and encouraging people to use that one instead.

For the real solution, see kubernetes/kubectl#925

We can likely use a replace to fix this until this fix propagates through all of our dependencies:

replace vbom.ml/util => github.com/fvbommel/util v0.0.0-20160121211510-db5cfe13f5cc

Re-opening so we can keep track that we need to fix this.

it's useful